Search Results for author:
Found 45 papers, 18 papers with code
Virtual Pets: Animatable Animal Generation in 3D Scenes
Toward unlocking the potential of generative models in immersive 4D experiences, we introduce Virtual Pet, a novel pipeline to model realistic and diverse motions for target animal species within a 3D environment.
Putting the Object Back into Video Object Segmentation
We present Cutie, a video object segmentation (VOS) network with object-level memory reading, which puts the object representation from memory back into the video object segmentation result.
Ranked #1 on
Semi-Supervised Video Object Segmentation
on MOSE
Tracking Anything with Decoupled Video Segmentation
To 'track anything' without training on video data for every individual task, we develop a decoupled video segmentation approach (DEVA), composed of task-specific image-level segmentation and class/task-agnostic bi-directional temporal propagation.
Ranked #1 on
Unsupervised Video Object Segmentation
on DAVIS 2016 val
(using extra training data)
Open-Vocabulary Video Segmentation
Open-World Video Segmentation
+7
SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generation
To enable interactive generation, our method supports a variety of input modalities that can be easily provided by a human, including images, text, partially observed shapes and combinations of these, further allowing to adjust the strength of each input.
RGB-Only Reconstruction of Tabletop Scenes for Collision-Free Manipulator Control
We present a system for collision-free control of a robot manipulator that uses only RGB views of the world.
On the Importance of Gradient Norm in PAC-Bayesian Bounds
Generalization bounds which assess the difference between the true risk and the empirical risk, have been studied extensively.
Joint Forecasting of Panoptic Segmentations with Difference Attention
For this, panoptic segmentations have been studied as a compelling representation in recent work.
MuMuQA: Multimedia Multi-Hop News Question Answering via Cross-Media Knowledge Extraction and Grounding
Specifically, the task involves multi-hop questions that require reasoning over image-caption pairs to identify the grounded visual object being referred to and then predicting a span from the news body text to answer the question.
Perceptual Score: What Data Modalities Does Your Model Perceive?
To study and quantify this concern, we introduce the perceptual score, a metric that assesses the degree to which a model relies on the different subsets of the input features, i. e., modalities.
CoVA: Context-aware Visual Attention for Webpage Information Extraction
To address this challenge we propose to reformulate WIE as a context-aware Webpage Object Detection task.
Ranked #1 on
Webpage Object Detection
on CoVA
(using extra training data)
Interpretation of Emergent Communication in Heterogeneous Collaborative Embodied Agents
We show that the emergent communication can be grounded to the agent observations and the spatial structure of the 3D environment.
The Surprising Effectiveness of Visual Odometry Techniques for Embodied PointGoal Navigation
It is fundamental for personal robots to reliably navigate to a specified goal.
Ordered Attention for Coherent Visual Storytelling
OIA models interactions between the sentence-corresponding image and important regions in other images of the sequence.
GridToPix: Training Embodied Agents with Minimal Supervision
While deep reinforcement learning (RL) promises freedom from hand-labeled data, great successes, especially for Embodied AI, require significant work to create supervision via carefully shaped rewards.
Panoptic Segmentation Forecasting
Following this decomposition, we introduce panoptic segmentation forecasting.
Removing Bias in Multi-modal Classifiers: Regularization by Maximizing Functional Entropies
However, regularization with the functional entropy is challenging.
Ranked #3 on
Visual Question Answering (VQA)
on VQA-CP
A Contrastive Learning Approach for Training Variational Autoencoder Priors
To tackle this issue, we propose an energy-based prior defined by the product of a base prior distribution and a reweighting factor, designed to bring the base closer to the aggregate posterior.
Ranked #6 on
Image Generation
on CelebA 256x256
(FID metric)
Bridging the Imitation Gap by Adaptive Insubordination
However, we show that when the teaching agent makes decisions with access to privileged information that is unavailable to the student, this information is marginalized during imitation learning, resulting in an "imitation gap" and, potentially, poor results.
A Cordial Sync: Going Beyond Marginal Policies for Multi-Agent Embodied Tasks
Autonomous agents must learn to collaborate.
The 1st Agriculture-Vision Challenge: Methods and Results
The first Agriculture-Vision Challenge aims to encourage research in developing novel and effective algorithms for agricultural pattern recognition from aerial images, especially for the semantic segmentation task associated with our challenge dataset.
Disentangling Controllable Object through Video Prediction Improves Visual Reinforcement Learning
In many vision-based reinforcement learning (RL) problems, the agent controls a movable object in its visual field, e. g., the player's avatar in video games and the robotic arm in visual grasping and manipulation.
Agriculture-Vision: A Large Aerial Image Database for Agricultural Pattern Analysis
To encourage research in computer vision for agriculture, we present Agriculture-Vision: a large-scale aerial farmland image dataset for semantic segmentation of agricultural patterns.
Calorimetry with Deep Learning: Particle Simulation and Reconstruction for Collider Physics
These networks can serve as fast and computationally light methods for particle shower simulation and reconstruction for current and future experiments at particle colliders.
TAB-VCR: Tags and Attributes based VCR Baselines
Despite impressive recent progress that has been reported on tasks that necessitate reasoning, such as visual question answering and visual dialog, models often exploit biases in datasets.
Graph Structured Prediction Energy Networks
For joint inference over multiple variables, a variety of structured prediction techniques have been developed to model correlations among variables and thereby improve predictions.
Towards Principled Objectives for Contrastive Disentanglement
%In fact, contrastive disentanglement and unsupervised recovery are often combined in that we seek additional variations that exhibit salient factors/properties.
Unsupervised Discovery of Dynamic Neural Circuits
What can we learn about the functional organization of cortical microcircuits from large-scale recordings of neural activity?
ViCo: Word Embeddings from Visual Co-occurrences
Through unsupervised clustering, supervised partitioning, and a zero-shot-like generalization analysis we show that our word embeddings complement text-only embeddings like GloVe by better representing similarities and differences between visual concepts that are difficult to obtain from text corpora alone.
Sequential Latent Spaces for Modeling the Intention During Diverse Image Captioning
We encourage this temporal latent space to capture the 'intention' about how to complete the sentence by mimicking a representation which summarizes the future.
Factor Graph Attention
We address this issue and develop a general attention mechanism for visual dialog which operates on any number of data utilities.
Ranked #1 on
Visual Dialog
on VisDial v0.9 val
Max-Sliced Wasserstein Distance and its use for GANs
Generative adversarial nets (GANs) and variational auto-encoders have significantly improved our distribution modeling capabilities, showing promise for dataset augmentation, image-to-image translation and feature learning.
Two Body Problem: Collaborative Visual Task Completion
Collaboration is a necessary skill to perform tasks that are beyond one agent's capabilities.
No-Frills Human-Object Interaction Detection: Factorization, Layout Encodings, and Training Techniques
We show that for human-object interaction detection a relatively simple factorized model with appearance and layout encodings constructed from pre-trained object detectors outperforms more sophisticated approaches.
GradiVeQ: Vector Quantization for Bandwidth-Efficient Gradient Aggregation in Distributed CNN Training
Data parallelism can boost the training speed of convolutional neural networks (CNN), but could suffer from significant communication costs caused by gradient aggregation.
Pipe-SGD: A Decentralized Pipelined SGD Framework for Distributed Deep Net Training
Distributed training of deep nets is an important technique to address some of the present day computing challenges like memory consumption and computational demands.
Deep Structured Prediction with Nonlinear Output Transformations
Deep structured models are widely used for tasks like semantic segmentation, where explicit correlations between variables provide important prior information which generally helps to reduce the data needs of deep nets.
Fast, Diverse and Accurate Image Captioning Guided By Part-of-Speech
We achieve the trifecta: (1) High accuracy for the diverse captions as evaluated by standard captioning metrics and user studies; (2) Faster computation of diverse captions compared to beam search and diverse beam search; and (3) High diversity as evaluated by counting novel sentences, distinct n-grams and mutual overlap (i. e., mBleu-4) scores.
Generative Modeling using the Sliced Wasserstein Distance
While this is particularly true for early GAN formulations, there has been significant empirically motivated and theoretically founded progress to improve stability, for instance, by using the Wasserstein distance rather than the Jenson-Shannon divergence.
Two can play this Game: Visual Dialog with Discriminative Question Generation and Answering
In addition, for the first time on the visual dialog dataset, we assess the performance of a system asking questions, and demonstrate how visual dialog can be generated from discriminative question generation and question answering.
Ranked #7 on
Visual Dialog
on VisDial v0.9 val
Asynchronous Parallel Coordinate Minimization for MAP Inference
Finding the maximum a-posteriori (MAP) assignment is a central task in graphical models.
Convolutional Image Captioning
In recent years significant progress has been made in image captioning, using Recurrent Neural Networks powered by long-short-term-memory (LSTM) units.
Dualing GANs
We start from linear discriminators in which case conjugate duality provides a mechanism to reformulate the saddle point objective into a maximization problem, such that both the generator and the discriminator of this 'dualing GAN' act in concert.
Creativity: Generating Diverse Questions using Variational Autoencoders
Generating diverse questions for given images is an important task for computational education, entertainment and AI assistants.
Statistical Inference, Learning and Models in Big Data
The need for new methods to deal with big data is a common theme in most scientific fields, although its definition tends to vary with the context.
Blending Learning and Inference in Structured Prediction
In this paper we derive an efficient algorithm to learn the parameters of structured predictors in general graphical models.
