Search Results for author:
Found 16 papers, 4 papers with code
Vid2Robot: End-to-end Video-conditioned Policy Learning with Cross-Attention Transformers
Given a video demonstration of a manipulation task and current visual observations, Vid2Robot directly produces robot actions.
PIVOT: Iterative Visual Prompting Elicits Actionable Knowledge for VLMs
In each iteration, the image is annotated with a visual representation of proposals that the VLM can refer to (e. g., candidate robot actions, localizations, or trajectories).
SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities
By training a VLM on such data, we significantly enhance its ability on both qualitative and quantitative spatial VQA.
RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control
Our goal is to enable a single end-to-end trained model to both learn to map robot observations to actions and enjoy the benefits of large-scale pretraining on language and vision-language data from the web.
Towards Generalist Biomedical AI
While considerable work is needed to validate these models in real-world use cases, our results represent a milestone towards the development of generalist biomedical AI systems.
Large Language Models as General Pattern Machines
We observe that pre-trained large language models (LLMs) are capable of autoregressively completing complex token sequences -- from arbitrary ones procedurally generated by probabilistic context-free grammars (PCFG), to more rich spatial patterns found in the Abstraction and Reasoning Corpus (ARC), a general AI benchmark, prompted in the style of ASCII art.
PaLM-E: An Embodied Multimodal Language Model
Large language models excel at a wide range of complex tasks.
Ranked #2 on
Visual Question Answering (VQA)
on OK-VQA
Grounded Decoding: Guiding Text Generation with Grounded Models for Embodied Agents
Recent progress in large language models (LLMs) has demonstrated the ability to learn and leverage Internet-scale knowledge through pre-training with autoregressive models.
Reinforcement Learning with Neural Radiance Fields
This paper demonstrates that learning state representations with supervision from Neural Radiance Fields (NeRFs) can improve the performance of RL compared to other learned representations or even low-dimensional, hand-engineered state information.
Learning Multi-Object Dynamics with Compositional Neural Radiance Fields
We present a method to learn compositional multi-object dynamics models from image observations based on implicit object encoders, Neural Radiance Fields (NeRFs), and graph neural networks.
Learning to Execute: Efficient Learning of Universal Plan-Conditioned Policies in Robotics
Applications of Reinforcement Learning (RL) in robotics are often limited by high data demand.
Learning Models as Functionals of Signed-Distance Fields for Manipulation Planning
We show that representing objects as signed-distance fields not only enables to learn and represent a variety of models with higher accuracy compared to point-cloud and occupancy measure representations, but also that SDF-based models are suitable for optimization-based planning.
Learning Neural Implicit Functions as Object Representations for Robotic Manipulation
Robotic manipulation planning is the problem of finding a sequence of robot configurations that involves interactions with objects in the scene, e. g., grasp, placement, tool-use, etc.
Deep 6-DoF Tracking of Unknown Objects for Reactive Grasping
We propose a robotic manipulation system, which is able to grasp a wide variety of formerly unseen objects and is robust against object perturbations and inferior grasping points.
Deep Visual Reasoning: Learning to Predict Action Sequences for Task and Motion Planning from an Initial Scene Image
This is possible by encoding the objects of the scene in images as input to the neural network, instead of a fixed feature vector.
Describing Physics For Physical Reasoning: Force-based Sequential Manipulation Planning
Physical reasoning is a core aspect of intelligence in animals and humans.
Robotics
