Search Results for author:
Found 25 papers, 10 papers with code
BRAVE: Broadening the visual encoding of vision-language models
Our results highlight the potential of incorporating different visual biases for a more broad and contextualized visual understanding of VLMs.
Controlled Training Data Generation with Diffusion Models
In this work, we present a method to control a text-to-image generative model to produce training data specifically "useful" for supervised learning.
Unraveling the Key Components of OOD Generalization via Diversification
We show that (1) diversification methods are highly sensitive to the distribution of the unlabeled data used for diversification and can underperform significantly when away from a method-specific sweet spot.
4M: Massively Multimodal Masked Modeling
Current machine learning models for vision are often highly specialized and limited to a single modality and task.
Rapid Network Adaptation: Learning to Adapt Neural Networks Using Test-Time Feedback
We propose a method for adapting neural networks to distribution shifts at test-time.
Modality-invariant Visual Odometry for Embodied Vision
Our model outperforms previous methods while training on only a fraction of the data.
An Information-Theoretic Approach to Transferability in Task Transfer Learning
Task transfer learning is a popular technique in image processing applications that uses pre-trained models to reduce the supervision cost of related tasks.
PALMER: Perception-Action Loop with Memory for Long-Horizon Planning
For training these perceptual representations, we combine Q-learning with contrastive representation learning to create a latent space where the distance between the embeddings of two states captures how easily an optimal policy can traverse between them.
Task Discovery: Finding the Tasks that Neural Networks Generalize on
An intriguing question would be: what if, instead of fixing the task and searching in the model space, we fix the model and search in the task space?
MultiMAE: Multi-modal Multi-task Masked Autoencoders
We show this pre-training strategy leads to a flexible, simple, and efficient framework with improved transfer results to downstream tasks.
Ranked #1 on
Semantic Segmentation
on Hypersim
3D Common Corruptions and Data Augmentation
We introduce a set of image transformations that can be used as corruptions to evaluate the robustness of models as well as data augmentation mechanisms for training neural networks.
CLIPasso: Semantically-Aware Object Sketching
Abstraction is at the heart of sketching due to the simple and minimal nature of line drawings.
Simple Control Baselines for Evaluating Transfer Learning
Transfer learning has witnessed remarkable progress in recent years, for example, with the introduction of augmentation-based contrastive self-supervised learning methods.
Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scans
This paper introduces a pipeline to parametrically sample and render multi-task vision datasets from comprehensive 3D scans from the real world.
Measuring the Effectiveness of Self-Supervised Learning using Calibrated Learning Curves
Self-supervised learning has witnessed remarkable progress in recent years, in particular with the introduction of augmentation-based contrastive methods.
Robustness via Cross-Domain Ensembles
We present a method for making neural network predictions robust to shifts from the training data distribution.
Robustness via Probabilistic Cross-Task Ensembles
We present a method for making predictions using neural networks that, at the test time, is robust against shifts from the training data distribution.
Robust Policies via Mid-Level Visual Representations: An Experimental Study in Manipulation and Navigation
Vision-based robotics often separates the control loop into one module for perception and a separate module for control.
Robust Learning Through Cross-Task Consistency
Visual perception entails solving a wide set of tasks, e. g., object detection, depth estimation, etc.
Side-Tuning: A Baseline for Network Adaptation via Additive Side Networks
When training a neural network for a desired task, one may prefer to adapt a pre-trained network rather than starting from randomly initialized weights.
Learning to Navigate Using Mid-Level Visual Priors
How much does having visual priors about the world (e. g. the fact that the world is 3D) assist in learning to perform downstream motor tasks (e. g. navigating a complex environment)?
Gibson Env: Real-World Perception for Embodied Agents
Developing visual perception models for active agents and sensorimotor control are cumbersome to be done in the physical world, as existing algorithms are too slow to efficiently learn in real-time and robots are fragile and costly.
Taskonomy: Disentangling Task Transfer Learning
The product is a computational taxonomic map for task transfer learning.
Semantic Cross-View Matching
In this paper, we therefore explore this idea and propose an automatic method for detecting and representing the semantic information of an RGB image with the goal of performing cross-view matching with a (non-RGB) geographic information system (GIS).
Unsupervised Semantic Parsing of Video Collections
The proposed method is capable of providing a semantic "storyline" of the video composed of its objective steps.
