Search Results for author:
Found 45 papers, 18 papers with code
CoLLEGe: Concept Embedding Generation for Large Language Models
Current language models are unable to quickly learn new concepts on the fly, often requiring a more involved finetuning process to learn robustly.
Reawakening knowledge: Anticipatory recovery from catastrophic interference via structured training
We explore the training dynamics of neural networks in a structured non-IID setting where documents are presented cyclically in a fixed, repeated sequence.
Self-supervised learning of video representations from a child's perspective
These results suggest that important temporal aspects of a child's internal model of the world may be learnable from their visual experience using highly generic learning algorithms and without strong inductive biases.
Learning and Forgetting Unsafe Examples in Large Language Models
As the number of large language models (LLMs) released to the public grows, there is a pressing need to understand the safety implications associated with these models learning from third-party custom finetuning data.
LifelongMemory: Leveraging LLMs for Answering Queries in Long-form Egocentric Videos
In this paper we introduce LifelongMemory, a new framework for accessing long-form egocentric videographic memory through natural language question answering and retrieval.
BIM: Block-Wise Self-Supervised Learning with Masked Image Modeling
This approach significantly reduces computational costs in comparison with training each DNN backbone individually.
Towards Unsupervised Object Detection From LiDAR Point Clouds
In this paper, we study the problem of unsupervised object detection from 3D point clouds in self-driving scenes.
Rethinking Closed-loop Training for Autonomous Driving
Recent advances in high-fidelity simulators have enabled closed-loop training of autonomous driving agents, potentially solving the distribution shift in training v. s.
Gaussian-Bernoulli RBMs Without Tears
We revisit the challenging problem of training Gaussian-Bernoulli restricted Boltzmann machines (GRBMs), introducing two innovations.
Scaling Forward Gradient With Local Losses
Forward gradient learning computes a noisy directional gradient and is a biologically plausible alternative to backprop for learning deep neural networks.
Learning to Reason With Relational Abstractions
Large language models have recently shown promising progress in mathematical reasoning when fine-tuned with human-generated sequences walking through a sequence of solution steps.
Online Unsupervised Learning of Visual Representations and Categories
Real world learning scenarios involve a nonstationary distribution of classes with sequential dependencies among the samples, in contrast to the standard machine learning formulation of drawing samples independently from a fixed, typically uniform distribution.
Just Label What You Need: Fine-Grained Active Selection for Perception and Prediction through Partially Labeled Scenes
As data collection is often significantly cheaper than labeling in this domain, the decision of which subset of examples to label can have a profound impact on model performance.
Cost-Efficient Online Hyperparameter Optimization
Recent work on hyperparameters optimization (HPO) has shown the possibility of training certain hyperparameters together with regular parameters.
Adversarial Attacks On Multi-Agent Communication
Growing at a fast pace, modern autonomous systems will soon be deployed at scale, opening up the possibility for cooperative multi-agent systems.
Exploring Adversarial Robustness of Multi-Sensor Perception Systems in Self Driving
Yet, there have been limited studies on the adversarial robustness of multi-modal models that fuse LiDAR features with image features.
AdvSim: Generating Safety-Critical Scenarios for Self-Driving Vehicles
Importantly, by simulating directly from sensor data, we obtain adversarial scenarios that are safety-critical for the full autonomy stack.
Self-Supervised Representation Learning from Flow Equivariance
Motivated by this ability, we present a new self-supervised learning representation framework that can be directly deployed on a video stream of complex scenes with many moving objects.
SceneGen: Learning to Generate Realistic Traffic Scenes
Existing methods typically insert actors into the scene according to a set of hand-crafted heuristics and are limited in their ability to model the true complexity and diversity of real traffic scenes, thus inducing a content gap between synthesized traffic scenes versus real ones.
Exploring representation learning for flexible few-shot tasks
In this work, we consider a realistic setting where the relationship between examples can change from episode to episode depending on the task context, which is not given to the learner.
Probing Few-Shot Generalization with Attributes
Despite impressive progress in deep learning, generalizing far beyond the training distribution is an important open challenge.
Learning to Communicate and Correct Pose Errors
Learned communication makes multi-agent systems more effective by aggregating distributed information.
Perceive, Attend, and Drive: Learning Spatial Attention for Safe Self-Driving
In this paper, we propose an end-to-end self-driving network featuring a sparse attention module that learns to automatically attend to important regions of the input.
Theoretical bounds on estimation error for meta-learning
Machine learning models have traditionally been developed under the assumption that the training and test distributions match exactly.
SketchEmbedNet: Learning Novel Concepts by Imitating Drawings
Sketch drawings capture the salient information of visual concepts.
End-to-end Contextual Perception and Prediction with Interaction Transformer
We show that our approach can outperform the state-of-the-art on both datasets.
Perceive, Predict, and Plan: Safe Motion Planning Through Interpretable Semantic Representations
In this paper we propose a novel end-to-end learnable network that performs joint perception, prediction and motion planning for self-driving vehicles and produces interpretable intermediate representations.
LoCo: Local Contrastive Representation Learning
Deep neural nets typically perform end-to-end backpropagation to learn the weights, a procedure that creates synchronization constraints in the weight update step across layers and is not biologically plausible.
Multi-Agent Routing Value Iteration Network
In this paper we tackle the problem of routing multiple agents in a coordinated manner.
Wandering Within a World: Online Contextualized Few-Shot Learning
We aim to bridge the gap between typical human and machine-learning environments by extending the standard framework of few-shot learning to an online, continual setting.
Physically Realizable Adversarial Examples for LiDAR Object Detection
Modern autonomous driving systems rely heavily on deep learning models to process point cloud sensory data; meanwhile, deep models have been shown to be susceptible to adversarial attacks with visually imperceptible perturbations.
Identifying Unknown Instances for Autonomous Driving
In the past few years, we have seen great progress in perception algorithms, particular through the use of deep learning.
Learning to Remember from a Multi-Task Teacher
Recent studies on catastrophic forgetting during sequential learning typically focus on fixing the accuracy of the predictions for a previously learned task.
Jointly Learnable Behavior and Trajectory Planning for Self-Driving Vehicles
The motion planners used in self-driving vehicles need to generate trajectories that are safe, comfortable, and obey the traffic rules.
Deformable Filter Convolution for Point Cloud Reasoning
Point clouds are the native output of many real-world 3D sensors.
Incremental Few-Shot Learning with Attention Attractor Networks
This paper addresses this problem, incremental few-shot learning, where a regular classification network has already been trained to recognize a set of base classes, and several extra novel classes are being considered, each with only a few labeled examples.
Graph HyperNetworks for Neural Architecture Search
Neural architecture search (NAS) automatically finds the best task-specific neural network topology, outperforming many manual architecture designs.
Learning to Reweight Examples for Robust Deep Learning
Deep neural networks have been shown to be very powerful modeling tools for many supervised learning tasks involving complex input patterns.
Understanding Short-Horizon Bias in Stochastic Meta-Optimization
Careful tuning of the learning rate, or even schedules thereof, can be crucial to effective neural net training.
Meta-Learning for Semi-Supervised Few-Shot Classification
To address this paradigm, we propose novel extensions of Prototypical Networks (Snell et al., 2017) that are augmented with the ability to use unlabeled examples when producing prototypes.
SBNet: Sparse Blocks Network for Fast Inference
Conventional deep convolutional neural networks (CNNs) apply convolution operators uniformly in space across all feature maps for hundreds of layers - this incurs a high computational cost for real-time applications.
The Reversible Residual Network: Backpropagation Without Storing Activations
Deep residual networks (ResNets) have significantly pushed forward the state-of-the-art on image classification, increasing in performance as networks grow both deeper and wider.
Normalizing the Normalizers: Comparing and Extending Network Normalization Schemes
On the other hand, layer normalization normalizes the activations across all activities within a layer.
End-to-End Instance Segmentation with Recurrent Attention
While convolutional neural networks have gained impressive success recently in solving structured prediction problems such as semantic segmentation, it remains a challenge to differentiate individual object instances in the scene.
Exploring Models and Data for Image Question Answering
A suite of baseline results on this new dataset are also presented.
Ranked #5 on
Video Question Answering
on SUTD-TrafficQA
