no code implementations • ECCV 2020 • Frank Dellaert, David M. Rosen, Jing Wu, Robert Mahony, Luca Carlone
Shonan Rotation Averaging is a fast, simple, and elegant rotation averaging algorithm that is guaranteed to recover globally optimal solutions under mild assumptions on the measurement noise.
no code implementations • 16 Feb 2024 • David Jin, Sushrut Karmalkar, Harry Zhang, Luca Carlone
We investigate a variation of the 3D registration problem, named multi-model 3D registration.
no code implementations • 18 Dec 2023 • Jared Strader, Nathan Hughes, William Chen, Alberto Speranzon, Luca Carlone
This paper proposes an approach to build 3D scene graphs in arbitrary indoor and outdoor environments.
no code implementations • 16 Feb 2023 • Dominic Maggio, Courtney Mario, Brett Streetman, Ted Steiner, Luca Carlone
Additionally, we evaluate performance on data collected by two cameras inside the capsule of Blue Origin's New Shepard rocket on payload flight NS-23, traveling at speeds up to 880 km/hr, and demonstrate less than 55 meters of average position error.
no code implementations • 12 Feb 2023 • Jingnan Shi, Rajat Talak, Dominic Maggio, Luca Carlone
Real-world robotics applications demand object pose estimation methods that work reliably across a variety of scenarios.
1 code implementation • 24 Oct 2022 • Antoni Rosinol, John J. Leonard, Luca Carlone
We propose a novel geometric and photometric 3D mapping pipeline for accurate and real-time scene reconstruction from monocular images.
1 code implementation • 3 Oct 2022 • Antoni Rosinol, John J. Leonard, Luca Carlone
We present a novel method to reconstruct 3D scenes from images by leveraging deep dense monocular SLAM and fast uncertainty propagation.
1 code implementation • 12 Sep 2022 • William Chen, Siyi Hu, Rajat Talak, Luca Carlone
Abstract semantic 3D scene understanding is a problem of critical importance in robotics.
1 code implementation • 22 Aug 2022 • Luca Carlone
In particular, we adapt and extend recent results on robust linear regression (applicable to the low-outlier regime with << 50% outliers) and list-decodable regression (applicable to the high-outlier regime with >> 50% outliers) to the setup commonly found in robotics and vision, where (i) variables (e. g., rotations, poses) belong to a non-convex domain, (ii) measurements are vector-valued, and (iii) the number of outliers is not known a priori.
1 code implementation • 24 Jun 2022 • Jingnan Shi, Heng Yang, Luca Carlone
We consider an active shape model, where -- for an object category -- we are given a library of potential CAD models describing objects in that category, and we adopt a standard formulation where pose and shape are estimated from 2D or 3D keypoints via non-convex optimization.
2 code implementations • 22 Jun 2022 • Rajat Talak, Lisa Peng, Luca Carlone
Our third contribution is a novel self-supervised training approach that uses our certificate of observable correctness to provide the supervisory signal to C-3PO during training.
no code implementations • 9 Jun 2022 • William Chen, Siyi Hu, Rajat Talak, Luca Carlone
Semantic 3D scene understanding is a problem of critical importance in robotics.
1 code implementation • 22 May 2022 • Pasquale Antonante, Heath Nilsen, Luca Carlone
This paper investigates runtime monitoring of perception systems.
no code implementations • 25 Sep 2021 • Amanda Prorok, Matthew Malencia, Luca Carlone, Gaurav S. Sukhatme, Brian M. Sadler, Vijay Kumar
In this survey article, we analyze how resilience is achieved in networks of agents and multi-robot systems that are able to overcome adversity by leveraging system-wide complementarity, diversity, and redundancy - often involving a reconfiguration of robotic capabilities to provide some key ability that was not present in the system a priori.
2 code implementations • 7 Sep 2021 • Heng Yang, Luca Carlone
Our third contribution is to solve the SDP relaxations at an unprecedented scale and accuracy by presenting STRIDE, a solver that blends global descent on the convex SDP with fast local search on the nonconvex POP.
no code implementations • 6 Aug 2021 • Antoni Rosinol, Luca Carlone
In this paper, we leapfrog these intermediate representations and build a 3D mesh directly from a depth map and the sparse landmarks triangulated with visual odometry.
1 code implementation • 2 Aug 2021 • Zachary Ravichandran, Lisa Peng, Nathan Hughes, J. Daniel Griffith, Luca Carlone
In this work, we present a reinforcement learning framework that leverages high-level hierarchical representations to learn navigation policies.
1 code implementation • 28 Jun 2021 • Yulun Tian, Yun Chang, Fernando Herrera Arias, Carlos Nieto-Granda, Jonathan P. How, Luca Carlone
This paper presents Kimera-Multi, the first multi-robot system that (i) is robust and capable of identifying and rejecting incorrect inter and intra-robot loop closures resulting from perceptual aliasing, (ii) is fully distributed and only relies on local (peer-to-peer) communication to achieve distributed localization and mapping, and (iii) builds a globally consistent metric-semantic 3D mesh model of the environment in real-time, where faces of the mesh are annotated with semantic labels.
1 code implementation • 28 May 2021 • Heng Yang, Ling Liang, Luca Carlone, Kim-Chuan Toh
In particular, we first design a globally convergent inexact projected gradient method (iPGM) for solving the SDP that serves as the backbone of our framework.
no code implementations • NeurIPS 2021 • Rajat Talak, Siyi Hu, Lisa Peng, Luca Carlone
We also prove that the number of parameters needed to achieve an $\epsilon$-approximation of the distribution function is exponential in the treewidth of the input graph, but linear in its size.
1 code implementation • 16 Apr 2021 • Jingnan Shi, Heng Yang, Luca Carlone
Our first contribution is to provide the first certifiably optimal solver for pose and shape estimation.
no code implementations • 21 Mar 2021 • Ali Agha, Kyohei Otsu, Benjamin Morrell, David D. Fan, Rohan Thakker, Angel Santamaria-Navarro, Sung-Kyun Kim, Amanda Bouman, Xianmei Lei, Jeffrey Edlund, Muhammad Fadhil Ginting, Kamak Ebadi, Matthew Anderson, Torkom Pailevanian, Edward Terry, Michael Wolf, Andrea Tagliabue, Tiago Stegun Vaquero, Matteo Palieri, Scott Tepsuporn, Yun Chang, Arash Kalantari, Fernando Chavez, Brett Lopez, Nobuhiro Funabiki, Gregory Miles, Thomas Touma, Alessandro Buscicchio, Jesus Tordesillas, Nikhilesh Alatur, Jeremy Nash, William Walsh, Sunggoo Jung, Hanseob Lee, Christoforos Kanellakis, John Mayo, Scott Harper, Marcel Kaufmann, Anushri Dixit, Gustavo Correa, Carlyn Lee, Jay Gao, Gene Merewether, Jairo Maldonado-Contreras, Gautam Salhotra, Maira Saboia Da Silva, Benjamin Ramtoula, Yuki Kubo, Seyed Fakoorian, Alexander Hatteland, Taeyeon Kim, Tara Bartlett, Alex Stephens, Leon Kim, Chuck Bergh, Eric Heiden, Thomas Lew, Abhishek Cauligi, Tristan Heywood, Andrew Kramer, Henry A. Leopold, Chris Choi, Shreyansh Daftry, Olivier Toupet, Inhwan Wee, Abhishek Thakur, Micah Feras, Giovanni Beltrame, George Nikolakopoulos, David Shim, Luca Carlone, Joel Burdick
This paper presents and discusses algorithms, hardware, and software architecture developed by the TEAM CoSTAR (Collaborative SubTerranean Autonomous Robots), competing in the DARPA Subterranean Challenge.
no code implementations • 11 Mar 2021 • Joshua Fishman, Samuel Ubellacker, Nathan Hughes, Luca Carlone
This paper presents the first prototype of a soft drone -- a quadrotor where traditional (i. e., rigid) landing gears are replaced with a soft tendon-actuated gripper to enable aggressive grasping.
Robotics
2 code implementations • CVPR 2021 • Heng Yang, Wei Dong, Luca Carlone, Vladlen Koltun
We present self-supervised geometric perception (SGP), the first general framework to learn a feature descriptor for correspondence matching without any ground-truth geometric model labels (e. g., camera poses, rigid transformations).
2 code implementations • 18 Jan 2021 • Antoni Rosinol, Andrew Violette, Marcus Abate, Nathan Hughes, Yun Chang, Jingnan Shi, Arjun Gupta, Luca Carlone
This mental model captures geometric and semantic aspects of the scene, describes the environment at multiple levels of abstractions (e. g., objects, rooms, buildings), includes static and dynamic entities and their relations (e. g., a person is in a room at a given time).
no code implementations • 11 Nov 2020 • Pasquale Antonante, David I. Spivak, Luca Carlone
The resulting temporal diagnostic graphs (i) provide a framework to reason over the consistency of perception outputs -- across modules and over time -- thus enabling fault detection, (ii) allow us to establish formal guarantees on the maximum number of faults that can be uniquely identified in a given perception system, and (iii) enable the design of efficient algorithms for fault identification.
no code implementations • 8 Nov 2020 • Yun Chang, Yulun Tian, Jonathan P. How, Luca Carlone
Our system, dubbed Kimera-Multi, is implemented by a team of robots equipped with visual-inertial sensors, and builds a 3D mesh model of the environment in real-time, where each face of the mesh is annotated with a semantic label (e. g., building, road, objects).
1 code implementation • 7 Nov 2020 • Jingnan Shi, Heng Yang, Luca Carlone
We also show that in practice the maximum k-core of the compatibility graph provides an approximation of the maximum clique, while being faster to compute in large problems.
1 code implementation • NeurIPS 2020 • Francesco Milano, Antonio Loquercio, Antoni Rosinol, Davide Scaramuzza, Luca Carlone
Recent works in geometric deep learning have introduced neural networks that allow performing inference tasks on three-dimensional geometric data by defining convolution, and sometimes pooling, operations on triangle meshes.
1 code implementation • 6 Aug 2020 • Frank Dellaert, David M. Rosen, Jing Wu, Robert Mahony, Luca Carlone
Shonan Rotation Averaging is a fast, simple, and elegant rotation averaging algorithm that is guaranteed to recover globally optimal solutions under mild assumptions on the measurement noise.
no code implementations • 29 Jul 2020 • Pasquale Antonante, Vasileios Tzoumas, Heng Yang, Luca Carlone
We extend ADAPT and GNC to the case where the user does not have prior knowledge of the inlier-noise statistics (or the statistics may vary over time) and is unable to guess a reasonable threshold to separate inliers from outliers (as the one commonly used in RANSAC).
1 code implementation • NeurIPS 2020 • Heng Yang, Luca Carlone
We propose the first general and practical framework to design certifiable algorithms for robust geometric perception in the presence of a large amount of outliers.
no code implementations • 24 May 2020 • Pasquale Antonante, David I. Spivak, Luca Carlone
Towards this goal, we draw connections with the literature on self-diagnosability for multiprocessor systems, and generalize it to (i) account for modules with heterogeneous outputs, and (ii) add a temporal dimension to the problem, which is crucial to model realistic perception systems where modules interact over time.
no code implementations • 11 May 2020 • Arjun Gupta, Luca Carlone
We investigate the problem of online output monitoring for neural networks that estimate 3D human shapes and poses from images.
3 code implementations • 15 Feb 2020 • Antoni Rosinol, Arjun Gupta, Marcus Abate, Jingnan Shi, Luca Carlone
Our second contribution is to provide the first fully automatic Spatial PerceptIon eNgine(SPIN) to build a DSG from visual-inertial data.
6 code implementations • 21 Jan 2020 • Heng Yang, Jingnan Shi, Luca Carlone
We propose the first fast and certifiable algorithm for the registration of two sets of 3D points in the presence of large amounts of outlier correspondences.
no code implementations • CVPR 2020 • Heng Yang, Luca Carlone
We study the problem of 3D shape reconstruction from 2D landmarks extracted in a single image.
12 code implementations • 6 Oct 2019 • Antoni Rosinol, Marcus Abate, Yun Chang, Luca Carlone
We provide an open-source C++ library for real-time metric-semantic visual-inertial Simultaneous Localization And Mapping (SLAM).
1 code implementation • 26 Sep 2019 • Pierre-Yves Lajoie, Benjamin Ramtoula, Yun Chang, Luca Carlone, Giovanni Beltrame
This paper introduces DOOR-SLAM, a fully distributed SLAM system with an outlier rejection mechanism that can work with less conservative parameters.
Robotics
4 code implementations • 18 Sep 2019 • Heng Yang, Pasquale Antonante, Vasileios Tzoumas, Luca Carlone
In this paper, we enable the simultaneous use of non-minimal solvers and robust estimation by providing a general-purpose approach for robust global estimation, which can be applied to any problem where a non-minimal solver is available for the outlier-free case.
4 code implementations • ICCV 2019 • Heng Yang, Luca Carlone
Our first contribution is to formulate the Wahba problem using a Truncated Least Squares (TLS) cost that is insensitive to a large fraction of spurious correspondences.
1 code implementation • 27 Mar 2019 • Vasileios Tzoumas, Pasquale Antonante, Luca Carlone
First, we show that even a simple linear instance of outlier rejection is inapproximable: in the worst-case one cannot design a quasi-polynomial time algorithm that computes an approximate solution efficiently.
2 code implementations • 20 Mar 2019 • Heng Yang, Luca Carlone
We propose a robust approach for the registration of two sets of 3D points in the presence of a large amount of outliers.
5 code implementations • 4 Mar 2019 • Antoni Rosinol, Torsten Sattler, Marc Pollefeys, Luca Carlone
We propose instead to tightly couple mesh regularization and state estimation by detecting and enforcing structural regularities in a novel factor-graph formulation.
1 code implementation • 27 Oct 2018 • Siyi Hu, Luca Carlone
We show that this approach, named Fast Unconstrained SEmidefinite Solver (FUSES), can solve large problems in milliseconds.
1 code implementation • 27 Oct 2018 • Pierre-Yves Lajoie, Siyi Hu, Giovanni Beltrame, Luca Carlone
Perceptual aliasing is one of the main causes of failure for Simultaneous Localization and Mapping (SLAM) systems operating in the wild.
Robotics 65K05, 62F10, 68T40, 68W40, 68W25, I.2.9; G.1.6
no code implementations • 7 Jan 2018 • Luca Carlone, Giuseppe C. Calafiore
Pose Graph Optimization involves the estimation of a set of poses from pairwise measurements and provides a formalization for many problems arising in mobile robotics and geometric computer vision.
1 code implementation • 4 Mar 2017 • Fangchang Ma, Luca Carlone, Ulas Ayaz, Sertac Karaman
We address the following question: is it possible to reconstruct the geometry of an unknown environment using sparse and incomplete depth measurements?
1 code implementation • 11 Feb 2017 • Siddharth Choudhary, Luca Carlone, Carlos Nieto, John Rogers, Henrik I. Christensen, Frank Dellaert
Our field tests show that the combined use of our distributed algorithms and object-based models reduces the communication requirements by several orders of magnitude and enables distributed mapping with large teams of robots in real-world problems.
2 code implementations • 19 Jun 2016 • Cesar Cadena, Luca Carlone, Henry Carrillo, Yasir Latif, Davide Scaramuzza, Jose Neira, Ian Reid, John J. Leonard
Simultaneous Localization and Mapping (SLAM)consists in the concurrent construction of a model of the environment (the map), and the estimation of the state of the robot moving within it.
Robotics
2 code implementations • 8 Dec 2015 • Christian Forster, Luca Carlone, Frank Dellaert, Davide Scaramuzza
However, real-time optimization quickly becomes infeasible as the trajectory grows over time, this problem is further emphasized by the fact that inertial measurements come at high rate, hence leading to fast growth of the number of variables in the optimization.
Robotics
1 code implementation • 13 May 2015 • Giuseppe Calafiore, Luca Carlone, Frank Dellaert
Our analysis shows that the duality gap is connected to the number of eigenvalues of the penalized pose graph matrix, which arises from the solution of the dual.
Robotics 68W01, 68W40, 68W25, 49K30 I.2.9; G.1.6