Search Results for author:
Found 119 papers, 66 papers with code
Event Cameras Meet SPADs for High-Speed, Low-Bandwidth Imaging
Traditional cameras face a trade-off between low-light performance and high-speed imaging: longer exposure times to capture sufficient light results in motion blur, whereas shorter exposures result in Poisson-corrupted noisy images.
3D scene generation from scene graphs and self-attention
Synthesizing realistic and diverse indoor 3D scene layouts in a controllable fashion opens up applications in simulated navigation and virtual reality.
Few-shot point cloud reconstruction and denoising via learned Guassian splats renderings and fine-tuned diffusion features
Existing deep learning methods for the reconstruction and denoising of point clouds rely on small datasets of 3D shapes.
An N-Point Linear Solver for Line and Motion Estimation with Event Cameras
To recover the full linear camera velocity we fuse observations from multiple lines with a novel velocity averaging scheme that relies on a geometrically-motivated residual, and thus solves the problem more efficiently than previous schemes which minimize an algebraic residual.
Mitigating Motion Blur in Neural Radiance Fields with Events and Frames
Neural Radiance Fields (NeRFs) have shown great potential in novel view synthesis.
Learning Quadruped Locomotion Using Differentiable Simulation
This work provides several important insights into using differentiable simulations for legged locomotion in the real world.
Bootstrapping Reinforcement Learning with Imitation for Vision-Based Agile Flight
Our experiments in both simulated and real-world environments demonstrate that our approach achieves superior performance and robustness than IL or RL alone in navigating a quadrotor through a racing course using only visual information without explicit state estimation.
State Space Models for Event Cameras
We address this challenge by introducing state-space models (SSMs) with learnable timescale parameters to event-based vision.
Reaching the Limit in Autonomous Racing: Optimal Control versus Reinforcement Learning
A central question in robotics is how to design a control system for an agile mobile robot.
A 5-Point Minimal Solver for Event Camera Relative Motion Estimation
Event-based cameras are ideal for line-based motion estimation, since they predominantly respond to edges in the scene.
Contrastive Learning for Enhancing Robust Scene Transfer in Vision-based Agile Flight
The utility of a robot greatly depends on its ability to perform a task in the real world, outside of a well-controlled lab environment.
End-to-end Learned Visual Odometry with Events and Frames
Visual Odometry (VO) is crucial for autonomous robotic navigation, especially in GPS-denied environments like planetary terrains.
Contrastive Initial State Buffer for Reinforcement Learning
In Reinforcement Learning, the trade-off between exploration and exploitation poses a complex challenge for achieving efficient learning from limited samples.
Seeing Behind Dynamic Occlusions with Event Cameras
We present the first large-scale dataset consisting of synchronized images and event sequences to evaluate our approach.
E-Calib: A Fast, Robust and Accurate Calibration Toolbox for Event Cameras
However, conventional image-based calibration techniques are not applicable due to the asynchronous, binary output of the sensor.
Revisiting Token Pruning for Object Detection and Instance Segmentation
In relation to the dense counterpart that utilizes all tokens, our method realizes an increase in inference speed, achieving up to 34% faster performance for the entire network and 46% for the backbone.
From Chaos Comes Order: Ordering Event Representations for Object Recognition and Detection
However, selecting the appropriate representation for the task traditionally requires training a neural network for each representation and selecting the best one based on the validation score, which is very time-consuming.
Ranked #1 on
Object Detection
on GEN1 Detection
Neuromorphic Optical Flow and Real-time Implementation with Event Cameras
Optical flow provides information on relative motion that is an important component in many computer vision pipelines.
A Hybrid ANN-SNN Architecture for Low-Power and Low-Latency Visual Perception
Our hybrid ANN-SNN model thus combines the best of both worlds: It does not suffer from long state transients and state decay thanks to the ANN, and can generate predictions with high temporal resolution, low latency, and low power thanks to the SNN.
COVERED, CollabOratiVE Robot Environment Dataset for 3D Semantic segmentation
Despite the importance of semantic understanding for such applications, 3D semantic segmentation of collaborative robot workspaces lacks sufficient research and dedicated datasets.
Improving safety in physical human-robot collaboration via deep metric learning
This work uses the Deep Metric Learning (DML) approach to distinguish between non-contact robot movement, intentional contact aimed at physical human-robot interaction, and collision situations.
Event-based Shape from Polarization
Our method uses the continuous event stream caused by the rotation to reconstruct relative intensities at multiple polarizer angles.
Recurrent Vision Transformers for Object Detection with Event Cameras
By revisiting the high-level design of recurrent vision backbones, we reduce inference time by a factor of 6 while retaining similar performance.
SLAM for Visually Impaired People: a Survey
In recent decades, several assistive technologies have been developed to improve the ability of blind and visually impaired individuals to navigate independently and safely.
Data-driven Feature Tracking for Event Cameras
Because of their high temporal resolution, increased resilience to motion blur, and very sparse output, event cameras have been shown to be ideal for low-latency and low-bandwidth feature tracking, even in challenging scenarios.
Pushing the Limits of Asynchronous Graph-based Object Detection with Event Cameras
A recent line of work tackles this issue by modeling events as spatiotemporally evolving graphs that can be efficiently and asynchronously processed using graph neural networks.
Cracking Double-Blind Review: Authorship Attribution with Deep Learning
In this work, we present a transformer-based, neural-network architecture that only uses the text content and the author names in the bibliography to attribute an anonymous manuscript to an author.
Learning Deep Sensorimotor Policies for Vision-based Autonomous Drone Racing
The resulting policy directly infers control commands with feature representations learned from raw images, forgoing the need for globally-consistent state estimation, trajectory planning, and handcrafted control design.
Learning Perception-Aware Agile Flight in Cluttered Environments
Recently, neural control policies have outperformed existing model-based planning-and-control methods for autonomously navigating quadrotors through cluttered environments in minimum time.
Training Efficient Controllers via Analytic Policy Gradient
Conversely, learning-based offline optimization approaches, such as Reinforcement Learning (RL), allow fast and efficient execution on the robot but hardly match the accuracy of MPC in trajectory tracking tasks.
Data-Efficient Collaborative Decentralized Thermal-Inertial Odometry
Our system back-end uses a covariance-intersection fusion strategy to neglect the cross-correlation between agents so as to lower memory usage and computational cost.
E-NeRF: Neural Radiance Fields from a Moving Event Camera
We also show that combining events and frames can overcome failure cases of NeRF estimation in scenarios where only a few input views are available without requiring additional regularization.
Event-aided Direct Sparse Odometry
This opens the door to low-power motion-tracking applications where frames are sparingly triggered "on demand" and our method tracks the motion in between.
Exploring Event Camera-based Odometry for Planetary Robots
Due to their resilience to motion blur and high robustness in low-light and high dynamic range conditions, event cameras are poised to become enabling sensors for vision-based exploration on future Mars helicopter missions.
AEGNN: Asynchronous Event-based Graph Neural Networks
For this reason, recent works have adopted Graph Neural Networks (GNNs), which process events as ``static" spatio-temporal graphs, which are inherently "sparse".
Time Lens++: Event-based Frame Interpolation with Parametric Non-linear Flow and Multi-scale Fusion
Recently, video frame interpolation using a combination of frame- and event-based cameras has surpassed traditional image-based methods both in terms of performance and memory efficiency.
Are High-Resolution Event Cameras Really Needed?
We provide both empirical and theoretical evidence for this claim, which indicates that high-resolution event cameras exhibit higher per-pixel event rates, leading to higher temporal noise in low-illumination conditions and at high speeds.
Dense Continuous-Time Optical Flow from Events and Frames
To the best of our knowledge, our model is the first method that can regress dense pixel trajectories from event data.
ESS: Learning Event-based Semantic Segmentation from Still Images
Nonetheless, semantic segmentation with event cameras is still in its infancy which is chiefly due to the lack of high-quality, labeled datasets.
Ranked #5 on
Event-based Object Segmentation
on MVSEC-SEG
Real-time Neural-MPC: Deep Learning Model Predictive Control for Quadrotors and Agile Robotic Platforms
Our experiments, performed in simulation and the real world onboard a highly agile quadrotor platform, demonstrate the capabilities of the described system to run learned models with, previously infeasible, large modeling capacity using gradient-based online optimization MPC.
Multi-Bracket High Dynamic Range Imaging with Event Cameras
Modern high dynamic range (HDR) imaging pipelines align and fuse multiple low dynamic range (LDR) images captured at different exposure times.
Continuous-Time vs. Discrete-Time Vision-based SLAM: A Comparative Study
In the context of this work, we developed, and open source, a modular and efficient software architecture containing state-of-the-art algorithms to solve the SLAM problem in discrete and continuous time.
Perception-Aware Perching on Powerlines with Multirotors
Multirotor aerial robots are becoming widely used for the inspection of powerlines.
Visual Attention Prediction Improves Performance of Autonomous Drone Racing Agents
This work investigates whether neural networks capable of imitating human eye gaze behavior and attention can improve neural network performance for the challenging task of vision-based autonomous drone racing.
Policy Search for Model Predictive Control with Application to Agile Drone Flight
In this work, we provide an answer by using policy search for automatically choosing high-level decision variables for MPC, which leads to a novel policy-search-for-model-predictive-control framework.
ESL: Event-based Structured Light
Event cameras are bio-inspired sensors providing significant advantages over standard cameras such as low latency, high temporal resolution, and high dynamic range.
Event Guided Depth Sensing
The depth estimation is achieved by an event-based structured light system consisting of a laser point projector coupled with a second event-based sensor tuned to detect the reflection of the laser from the scene.
Learning High-Speed Flight in the Wild
Indeed, the subtasks are executed sequentially, leading to increased processing latency and a compounding of errors through the pipeline.
Jointly Learning Identification and Control for Few-Shot Policy Adaptation
We evaluate our approach on several complex systems and tasks, and experimentally analyze the advantages over model-free and model-based methods in terms of performance and sample efficiency.
The Hilti SLAM Challenge Dataset
The results of the challenge, which are summarized in this paper, show that the proposed dataset is an important asset in the development of new SLAM algorithms that are ready to be deployed in the real-world.
Bridging the Gap between Events and Frames through Unsupervised Domain Adaptation
However, event-based vision has been held back by the shortage of labeled datasets due to the novelty of event cameras.
E-RAFT: Dense Optical Flow from Event Cameras
Modern frame-based optical flow methods heavily rely on matching costs computed from feature correlation.
Time-Optimal Planning for Quadrotor Waypoint Flight
However, this time allocation is a priori unknown and renders previous works incapable of producing truly time-optimal trajectories.
Event-driven Vision and Control for UAVs on a Neuromorphic Chip
Here, we explore how an event-based vision algorithm can be implemented as a spiking neuronal network on a neuromorphic chip and used in a drone controller.
Time Lens: Event-Based Video Frame Interpolation
However, while these approaches can capture non-linear motions they suffer from ghosting and perform poorly in low-texture regions with few events.
TimeLens: Event-based Video Frame Interpolation
State-of-the-art frame interpolation methods generate intermediate frames by inferring object motions in the image from consecutive key-frames.
How to Calibrate Your Event Camera
We propose a generic event camera calibration framework using image reconstruction.
Autonomous Overtaking in Gran Turismo Sport Using Curriculum Reinforcement Learning
Professional race-car drivers can execute extreme overtaking maneuvers.
AutoTune: Controller Tuning for High-Speed Flight
To answer the first question, we study the relationship between parameters and performance and find out that the faster the maneuver, the more sensitive a controller becomes to its parameters.
Autonomous Drone Racing with Deep Reinforcement Learning
In many robotic tasks, such as autonomous drone racing, the goal is to travel through a set of waypoints as fast as possible.
DSEC: A Stereo Event Camera Dataset for Driving Scenarios
To address these challenges, we propose, DSEC, a new dataset that contains such demanding illumination conditions and provides a rich set of sensory data.
Combining Events and Frames using Recurrent Asynchronous Multimodal Networks for Monocular Depth Prediction
However, events only measure the varying component of the visual signal, which limits their ability to encode scene context.
Data-Driven MPC for Quadrotors
Aerodynamic forces render accurate high-speed trajectory tracking with quadrotors extremely challenging.
Gaussian Processes
Robotics
Primal-Dual Mesh Convolutional Neural Networks
Recent works in geometric deep learning have introduced neural networks that allow performing inference tasks on three-dimensional geometric data by defining convolution, and sometimes pooling, operations on triangle meshes.
Learning Monocular Dense Depth from Events
Event cameras are novel sensors that output brightness changes in the form of a stream of asynchronous events instead of intensity frames.
Unsupervised Feature Learning for Event Data: Direct vs Inverse Problem Formulation
Due to the asynchronous nature, efficient learning of compact representation for event data is challenging.
Flightmare: A Flexible Quadrotor Simulator
State-of-the-art quadrotor simulators have a rigid and highly-specialized structure: either are they really fast, physically accurate, or photo-realistic.
Super-Human Performance in Gran Turismo Sport Using Deep Reinforcement Learning
Autonomous car racing is a major challenge in robotics.
Fisher Information Field: an Efficient and Differentiable Map for Perception-aware Planning
However, computing the Fisher information from a set of sparse landmarks (i. e., a point cloud), which is the most common map for visual localization, is inefficient.
Robotics
Learning High-Level Policies for Model Predictive Control
In this work, we leverage probabilistic decision-making approaches and the generalization capability of artificial neural networks to the powerful online optimization by learning a deep high-level policy for the MPC (High-MPC).
Deep Drone Acrobatics
In this paper, we propose to learn a sensorimotor policy that enables an autonomous quadrotor to fly extreme acrobatic maneuvers with only onboard sensing and computation.
Robotics
AlphaPilot: Autonomous Drone Racing
This paper presents a novel system for autonomous, vision-based drone racing combining learned data abstraction, nonlinear filtering, and time-optimal trajectory planning.
Reference Pose Generation for Long-term Visual Localization via Learned Features and View Synthesis
Visual Localization is one of the key enabling technologies for autonomous driving and augmented reality.
Faster than FAST: GPU-Accelerated Frontend for High-Speed VIO
While most steps of a VIO pipeline work on visual features, they rely on image data for detection and tracking, of which both steps are well suited for parallelization.
Event-based Asynchronous Sparse Convolutional Networks
However, these approaches discard the spatial and temporal sparsity inherent in event data at the cost of higher computational complexity and latency.
Reducing the Sim-to-Real Gap for Event Cameras
We present strategies for improving training data for event based CNNs that result in 20-40% boost in performance of existing state-of-the-art (SOTA) video reconstruction networks retrained with our method, and up to 15% for optic flow networks.
Ranked #3 on
Video Reconstruction
on MVSEC
AirSim Drone Racing Lab
Autonomous drone racing is a challenging research problem at the intersection of computer vision, planning, state estimation, and control.
Event-Based Angular Velocity Regression with Spiking Networks
Due to their spike-based computational model, SNNs can process output from event-based, asynchronous sensors without any pre-processing at extremely lower power unlike standard artificial neural networks.
Redesigning SLAM for Arbitrary Multi-Camera Systems
We adapt a state-of-the-art visual-inertial odometry with these modifications, and experimental results show that the modified pipeline can adapt to a wide range of camera setups (e. g., 2 to 6 cameras in one experiment) without the need of sensor-specific modifications or tuning.
Voxel Map for Visual SLAM
We propose a voxel-map representation to efficiently retrieve map points for visual SLAM.
Learning Depth With Very Sparse Supervision
Motivated by the astonishing capabilities of natural intelligent agents and inspired by theories from psychology, this paper explores the idea that perception gets coupled to 3D properties of the world via interaction with the environment.
Augmenting Visual Place Recognition with Structural Cues
In this paper, we propose to augment image-based place recognition with structural cues.
Registration made easy -- standalone orthopedic navigation with HoloLens
In surgical navigation, finding correspondence between preoperative plan and intraoperative anatomy, the so-called registration task, is imperative.
Video to Events: Recycling Video Datasets for Event Cameras
Event cameras are novel sensors that output brightness changes in the form of a stream of asynchronous "events" instead of intensity frames.
Towards Low-Latency High-Bandwidth Control of Quadrotors using Event Cameras
In particular, this paper addresses the problem of one-dimensional attitude tracking using a dualcopter platform equipped with an event camera.
Robotics Systems and Control Systems and Control
Global-Local Network for Learning Depth with Very Sparse Supervision
Natural intelligent agents learn to perceive the three dimensional structure of the world without training on large datasets and are unlikely to have the precise equations of projective geometry hard-wired in the brain.
A General Framework for Uncertainty Estimation in Deep Learning
Current approaches for uncertainty estimation of neural networks require changes to the network and optimization process, typically ignore prior knowledge about the data, and tend to make over-simplifying assumptions which underestimate uncertainty.
High Speed and High Dynamic Range Video with an Event Camera
In this work we propose to learn to reconstruct intensity images from event streams directly from data instead of relying on any hand-crafted priors.
Ranked #3 on
Event-based Object Segmentation
on MVSEC-SEG
EVDodgeNet: Deep Dynamic Obstacle Dodging with Event Cameras
To our knowledge, this is the first deep learning -- based solution to the problem of dynamic obstacle avoidance using event cameras on a quadrotor.
Visual-Inertial Odometry of Aerial Robots
Visual-Inertial odometry (VIO) is the process of estimating the state (pose and velocity) of an agent (e. g., an aerial robot) by using only the input of one or more cameras plus one or more Inertial Measurement Units (IMUs) attached to it.
Robotics
CED: Color Event Camera Dataset
Event cameras are novel, bio-inspired visual sensors, whose pixels output asynchronous and independent timestamped spikes at local intensity changes, called 'events'.
Event-based Vision: A Survey
Event cameras offer attractive properties compared to traditional cameras: high temporal resolution (in the order of microseconds), very high dynamic range (140 dB vs. 60 dB), low power consumption, and high pixel bandwidth (on the order of kHz) resulting in reduced motion blur.
End-to-End Learning of Representations for Asynchronous Event-Based Data
Event cameras are vision sensors that record asynchronous streams of per-pixel brightness changes, referred to as "events".
Ranked #15 on
Robust classification
on N-ImageNet
Events-to-Video: Bringing Modern Computer Vision to Event Cameras
Since the output of event cameras is fundamentally different from conventional cameras, it is commonly accepted that they require the development of specialized algorithms to accommodate the particular nature of events.
Focus Is All You Need: Loss Functions For Event-based Vision
The proposed loss functions allow bringing mature computer vision tools to the realm of event cameras.
Event-Based Motion Segmentation by Motion Compensation
In contrast to traditional cameras, whose pixels have a common exposure time, event-based cameras are novel bio-inspired sensors whose pixels work independently and asynchronously output intensity changes (called "events"), with microsecond resolution.
Unsupervised Moving Object Detection via Contextual Information Separation
We propose an adversarial contextual model for detecting moving objects in images.
Matching Features without Descriptors: Implicitly Matched Interest Points
The extraction and matching of interest points is a prerequisite for many geometric computer vision problems.
Asynchronous, Photometric Feature Tracking using Events and Frames
By contrast, standard cameras provide intensity measurements (frames) that do not depend on motion direction.
Semi-Dense 3D Reconstruction with a Stereo Event Camera
Event cameras are bio-inspired sensors that offer several advantages, such as low latency, high-speed and high dynamic range, to tackle challenging scenarios in computer vision.
A 64mW DNN-based Visual Navigation Engine for Autonomous Nano-Drones
As part of our general methodology we discuss the software mapping techniques that enable the state-of-the-art deep convolutional neural network presented in [1] to be fully executed on-board within a strict 6 fps real-time constraint with no compromise in terms of flight results, while all processing is done with only 64 mW on average.
SIPs: Succinct Interest Points from Unsupervised Inlierness Probability Learning
In certain cases, our detector is able to obtain an equivalent amount of inliers with as little as 60% of the amount of points of other detectors.
Event-based Vision meets Deep Learning on Steering Prediction for Self-driving Cars
Event cameras are bio-inspired vision sensors that naturally capture the dynamics of a scene, filtering out redundant information.
Ranked #12 on
Robust classification
on N-ImageNet
A Unifying Contrast Maximization Framework for Event Cameras, with Applications to Motion, Depth, and Optical Flow Estimation
We present a unifying framework to solve several computer vision problems with event cameras: motion, depth and optical flow estimation.
A Real-Time Game Theoretic Planner for Autonomous Two-Player Drone Racing
To be successful in multi-player drone racing, a player must not only follow the race track in an optimal way, but also compete with other drones through strategic blocking, faking, and opportunistic passing while avoiding collisions.
Robotics
Differential Flatness of Quadrotor Dynamics Subject to Rotor Drag for Accurate Tracking of High-Speed Trajectories
In this paper, we prove that the dynamical model of a quadrotor subject to linear rotor drag effects is differentially flat in its position and heading.
Robotics
Fast, Autonomous Flight in GPS-Denied and Cluttered Environments
One of the most challenging tasks for a flying robot is to autonomously navigate between target locations quickly and reliably while avoiding obstacles in its path, and with little to no a-priori knowledge of the operating environment.
Robotics
Data-Efficient Decentralized Visual SLAM
In this work, we integrate state-of-the-art decentralized SLAM components into a new, complete decentralized visual SLAM system.
Robotics
Ultimate SLAM? Combining Events, Images, and IMU for Robust Visual SLAM in HDR and High Speed Scenarios
Event cameras are bio-inspired vision sensors that output pixel-level brightness changes instead of standard intensity frames.
Learning-based Image Enhancement for Visual Odometry in Challenging HDR Environments
One of the main open challenges in visual odometry (VO) is the robustness to difficult illumination conditions or high dynamic range (HDR) environments.
Independent Motion Detection with Event-driven Cameras
In this paper, we propose a method for segmenting the motion of an independently moving object for event-driven cameras.
Efficient Decentralized Visual Place Recognition From Full-Image Descriptors
As we show, casting this to a key-value lookup problem can be achieved with k-means clustering, and results in a much simpler system than [1].
PL-SLAM: a Stereo SLAM System through the Combination of Points and Line Segments
This paper proposes PL-SLAM, a stereo visual SLAM system that combines both points and line segments to work robustly in a wider variety of scenarios, particularly in those where point features are scarce or not well-distributed in the image.
Continuous-Time Visual-Inertial Odometry for Event Cameras
Recent work has shown that a continuous-time representation of the event camera pose can deal with the high temporal resolution and asynchronous nature of this sensor in a principled way.
The Event-Camera Dataset and Simulator: Event-based Data for Pose Estimation, Visual Odometry, and SLAM
New vision sensors, such as the Dynamic and Active-pixel Vision sensor (DAVIS), incorporate a conventional global-shutter camera and an event-based sensor in the same pixel array.
Event-based, 6-DOF Camera Tracking from Photometric Depth Maps
Event cameras are bio-inspired vision sensors that output pixel-level brightness changes instead of standard intensity frames.
Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Age
Simultaneous Localization and Mapping (SLAM)consists in the concurrent construction of a model of the environment (the map), and the estimation of the state of the robot moving within it.
Robotics
On-Manifold Preintegration for Real-Time Visual-Inertial Odometry
However, real-time optimization quickly becomes infeasible as the trajectory grows over time, this problem is further emphasized by the fact that inertial measurements come at high rate, hence leading to fast growth of the number of variables in the optimization.
Robotics
Event-based Camera Pose Tracking using a Generative Event Model
Event-based vision sensors mimic the operation of biological retina and they represent a major paradigm shift from traditional cameras.
