Search Results for author:
Found 39 papers, 25 papers with code
Hierarchical Diffusion Policy for Kinematics-Aware Multi-Task Robotic Manipulation
This paper introduces Hierarchical Diffusion Policy (HDP), a hierarchical agent for multi-task robotic manipulation.
Speed Co-Augmentation for Unsupervised Audio-Visual Pre-training
This work aims to improve unsupervised audio-visual pre-training.
Language Reward Modulation for Pretraining Reinforcement Learning
Using learned reward functions (LRFs) as a means to solve sparse-reward reinforcement learning (RL) tasks has yielded some steady progress in task-complexity through the years.
Multi-View Masked World Models for Visual Robotic Manipulation
In this paper, we investigate how to learn good representations with multi-view data and utilize them for visual robotic manipulation.
StereoPose: Category-Level 6D Transparent Object Pose Estimation from Stereo Images via Back-View NOCS
To further improve the performance of the stereo framework, StereoPose is equipped with a parallax attention module for stereo feature fusion and an epipolar loss for improving the stereo-view consistency of network predictions.
Sim-to-Real via Sim-to-Seg: End-to-end Off-road Autonomous Driving Without Real Data
In this paper, we address this challenge by presenting Sim2Seg, a re-imagining of RCAN that crosses the visual reality gap for off-road autonomous driving, without using any real-world data.
Real-World Robot Learning with Masked Visual Pre-training
Finally, we train a 307M parameter vision transformer on a massive collection of 4. 5M images from the Internet and egocentric videos, and demonstrate clearly the benefits of scaling visual pre-training for robot learning.
Temporally Consistent Transformers for Video Generation
To generate accurate videos, algorithms have to understand the spatial and temporal dependencies in the world.
HARP: Autoregressive Latent Video Prediction with High-Fidelity Image Generator
Video prediction is an important yet challenging problem; burdened with the tasks of generating future frames and learning environment dynamics.
Masked World Models for Visual Control
Yet the current approaches typically train a single model end-to-end for learning both visual representations and dynamics, making it difficult to accurately model the interaction between robots and small objects.
Model-based Reinforcement Learning
Reinforcement Learning (RL)
+1
Patch-based Object-centric Transformers for Efficient Video Generation
In this work, we present Patch-based Object-centric Video Transformer (POVT), a novel region-based video generation architecture that leverages object-centric information to efficiently model temporal dynamics in videos.
On the Effectiveness of Fine-tuning Versus Meta-reinforcement Learning
From these findings, we advocate for evaluating future meta-RL methods on more challenging tasks and including multi-task pretraining with fine-tuning as a simple, yet strong baseline.
Coarse-to-fine Q-attention with Tree Expansion
Coarse-to-fine Q-attention enables sample-efficient robot manipulation by discretizing the translation space in a coarse-to-fine manner, where the resolution gradually increases at each layer in the hierarchy.
Sim-to-Real 6D Object Pose Estimation via Iterative Self-training for Robotic Bin Picking
To continuously improve the quality of pseudo labels, we iterate the above steps by taking the trained student model as a new teacher and re-label real data using the refined teacher model.
Coarse-to-Fine Q-attention with Learned Path Ranking
We propose Learned Path Ranking (LPR), a method that accepts an end-effector goal pose, and learns to rank a set of goal-reaching paths generated from an array of path generating methods, including: path planning, Bezier curve sampling, and a learned policy.
Reinforcement Learning with Action-Free Pre-Training from Videos
Our framework consists of two phases: we pre-train an action-free latent video prediction model, and then utilize the pre-trained representations for efficiently learning action-conditional world models on unseen environments.
ReorientBot: Learning Object Reorientation for Specific-Posed Placement
Robots need the capability of placing objects in arbitrary, specific poses to rearrange the world and achieve various valuable tasks.
SafePicking: Learning Safe Object Extraction via Object-Level Mapping
We evaluate our methods using the YCB objects in both simulation and the real world, achieving safe object extraction from piles.
Bingham Policy Parameterization for 3D Rotations in Reinforcement Learning
We propose a new policy parameterization for representing 3D rotations during reinforcement learning.
Auto-Lambda: Disentangling Dynamic Task Relationships
Unlike previous methods where task relationships are assumed to be fixed, Auto-Lambda is a gradient-based meta learning framework which explores continuous, dynamic task relationships via task-specific weightings, and can optimise any choice of combination of tasks through the formulation of a meta-loss; where the validation loss automatically influences task weightings throughout training.
Ranked #3 on
Robot Manipulation
on RLBench
(Succ. Rate (10 tasks, 100 demos/task) metric)
Autoregressive Latent Video Prediction with High-Fidelity Image Generator
Video prediction is an important yet challenging problem; burdened with the tasks of generating future frames and learning environment dynamics.
Coarse-to-Fine Q-attention: Efficient Learning for Visual Robotic Manipulation via Discretisation
We present a coarse-to-fine discretisation method that enables the use of discrete reinforcement learning approaches in place of unstable and data-inefficient actor-critic methods in continuous robotics domains.
Ranked #7 on
Robot Manipulation
on RLBench
Q-attention: Enabling Efficient Learning for Vision-based Robotic Manipulation
Despite the success of reinforcement learning methods, they have yet to have their breakthrough moment when applied to a broad range of robotic manipulation tasks.
Ranked #1 on
Robot Task Planning
on RLBench
Waypoint Planning Networks
We propose waypoint planning networks (WPN), a hybrid algorithm based on LSTMs with a local kernel - a classic algorithm such as A*, and a global kernel using a learned algorithm.
SIMstack: A Generative Shape and Instance Model for Unordered Object Stacks
At test time, our model can generate 3D shape and instance segmentation from a single depth view, probabilistically sampling proposals for the occluded region from the learned latent space.
End-to-End Egospheric Spatial Memory
Spatial memory, or the ability to remember and recall specific locations and objects, is central to autonomous agents' ability to carry out tasks in real environments.
Ivy: Templated Deep Learning for Inter-Framework Portability
We introduce Ivy, a templated Deep Learning (DL) framework which abstracts existing DL frameworks.
Ego-Centric Spatial Memory Networks
With our broad demonstrations, we show that ESMN represents a useful and general computation graph for embodied spatial reasoning, and the module forms a bridge between real-time mapping systems and differentiable memory architectures.
Attention-driven Robotic Manipulation
Despite the success of reinforcement learning methods, they have yet to have their breakthrough moment when applied to a broad range of robotic manipulation tasks.
MoreFusion: Multi-object Reasoning for 6D Pose Estimation from Volumetric Fusion
Robots and other smart devices need efficient object-based scene representations from their on-board vision systems to reason about contact, physics and occlusion.
Learning One-Shot Imitation from Humans without Humans
But is there a way to remove the need for real world human demonstrations during training?
RLBench: The Robot Learning Benchmark & Learning Environment
We present a challenging new benchmark and learning-environment for robot learning: RLBench.
PyRep: Bringing V-REP to Deep Robot Learning
PyRep is a toolkit for robot learning research, built on top of the virtual robotics experimentation platform (V-REP).
Sim-to-Real via Sim-to-Sim: Data-efficient Robotic Grasping via Randomized-to-Canonical Adaptation Networks
Using domain adaptation methods to cross this "reality gap" requires a large amount of unlabelled real-world data, whilst domain randomization alone can waste modeling power.
Task-Embedded Control Networks for Few-Shot Imitation Learning
Despite this, most robot learning approaches have focused on learning a single task, from scratch, with a limited notion of generalisation, and no way of leveraging the knowledge to learn other tasks more efficiently.
Sim-to-Real Reinforcement Learning for Deformable Object Manipulation
Moreover, due to the large amount of data needed to learn these end-to-end solutions, an emerging trend is to learn control policies in simulation and then transfer them over to the real world.
Transferring End-to-End Visuomotor Control from Simulation to Real World for a Multi-Stage Task
End-to-end control for robot manipulation and grasping is emerging as an attractive alternative to traditional pipelined approaches.
3D Simulation for Robot Arm Control with Deep Q-Learning
Building upon the recent success of deep Q-networks, we present an approach which uses 3D simulations to train a 7-DOF robotic arm in a control task without any prior knowledge.
