no code implementations • ECCV 2020 • Boyang Deng, JP Lewis, Timothy Jeruzalski, Gerard Pons-Moll, Geoffrey Hinton, Mohammad Norouzi, Andrea Tagliasacchi
Efficient representation of articulated objects such as human bodies is an important problem in computer vision and graphics.
no code implementations • 19 Apr 2024 • Ahan Shabanov, Shrisudhan Govindarajan, Cody Reading, Lily Goli, Daniel Rebain, Kwang Moo Yi, Andrea Tagliasacchi
Largely due to their implicit nature, neural fields lack a direct mechanism for filtering, as Fourier analysis from discrete signal processing is not directly applicable to these representations.
no code implementations • 18 Apr 2024 • Yalda Foroutan, Daniel Rebain, Kwang Moo Yi, Andrea Tagliasacchi
3D Gaussian Splatting has recently been embraced as a versatile and effective method for scene reconstruction and novel view synthesis, owing to its high-quality results and compatibility with hardware rasterization.
no code implementations • 15 Apr 2024 • Shakiba Kheradmand, Daniel Rebain, Gopal Sharma, Weiwei Sun, Jeff Tseng, Hossam Isack, Abhishek Kar, Andrea Tagliasacchi, Kwang Moo Yi
While 3D Gaussian Splatting has recently become popular for neural rendering, current methods rely on carefully engineered cloning and splitting strategies for placing Gaussians, which does not always generalize and may lead to poor-quality renderings.
no code implementations • 26 Mar 2024 • Sherwin Bahmani, Xian Liu, Yifan Wang, Ivan Skorokhodov, Victor Rong, Ziwei Liu, Xihui Liu, Jeong Joon Park, Sergey Tulyakov, Gordon Wetzstein, Andrea Tagliasacchi, David B. Lindell
We learn local deformations that conform to the global trajectory using supervision from a text-to-video model.
1 code implementation • 19 Dec 2023 • David Charatan, Sizhe Li, Andrea Tagliasacchi, Vincent Sitzmann
We introduce pixelSplat, a feed-forward model that learns to reconstruct 3D radiance fields parameterized by 3D Gaussian primitives from pairs of images.
Ranked #2 on Generalizable Novel View Synthesis on ACID
no code implementations • 11 Dec 2023 • Shaobo Xia, Jun Yue, Kacper Kania, Leyuan Fang, Andrea Tagliasacchi, Kwang Moo Yi, Weiwei Sun
We propose a weakly supervised semantic segmentation method for point clouds that predicts "per-point" labels from just "whole-scene" annotations while achieving the performance of recent fully supervised approaches.
no code implementations • 4 Dec 2023 • Weiwei Sun, Eduard Trulls, Yang-Che Tseng, Sneha Sambandam, Gopal Sharma, Andrea Tagliasacchi, Kwang Moo Yi
We overcome these problems with a simple representation that aggregates point clouds at multiple scale levels with sparse voxel grids at different resolutions.
no code implementations • 2 Dec 2023 • Gopal Sharma, Daniel Rebain, Kwang Moo Yi, Andrea Tagliasacchi
We propose a novel Neural Radiance Field (NeRF) representation for non-opaque scenes that allows fast inference by utilizing textured polygons.
1 code implementation • 29 Nov 2023 • Eric Hedlin, Gopal Sharma, Shweta Mahajan, Xingzhe He, Hossam Isack, Abhishek Kar Helge Rhodin, Andrea Tagliasacchi, Kwang Moo Yi
Unsupervised learning of keypoints and landmarks has seen significant progress with the help of modern neural network architectures, but performance is yet to match the supervised counterpart, making their practicability questionable.
Ranked #1 on Unsupervised Human Pose Estimation on Tai-Chi-HD
no code implementations • 29 Nov 2023 • Shakiba Kheradmand, Daniel Rebain, Gopal Sharma, Hossam Isack, Abhishek Kar, Andrea Tagliasacchi, Kwang Moo Yi
We present an approach to accelerate Neural Field training by efficiently selecting sampling locations.
no code implementations • 29 Nov 2023 • Sherwin Bahmani, Ivan Skorokhodov, Victor Rong, Gordon Wetzstein, Leonidas Guibas, Peter Wonka, Sergey Tulyakov, Jeong Joon Park, Andrea Tagliasacchi, David B. Lindell
Recent breakthroughs in text-to-4D generation rely on pre-trained text-to-image and text-to-video models to generate dynamic 3D scenes.
1 code implementation • 6 Sep 2023 • Lily Goli, Cody Reading, Silvia Sellán, Alec Jacobson, Andrea Tagliasacchi
Neural Radiance Fields (NeRFs) have shown promise in applications like view synthesis and depth estimation, but learning from multiview images faces inherent uncertainties.
1 code implementation • NeurIPS 2023 • Eric Hedlin, Gopal Sharma, Shweta Mahajan, Hossam Isack, Abhishek Kar, Andrea Tagliasacchi, Kwang Moo Yi
Text-to-image diffusion models are now capable of generating images that are often indistinguishable from real images.
Ranked #1 on Semantic correspondence on PF-WILLOW
no code implementations • CVPR 2023 • Kacper Kania, Stephan J. Garbin, Andrea Tagliasacchi, Virginia Estellers, Kwang Moo Yi, Julien Valentin, Tomasz Trzciński, Marek Kowalski
Generating faithful visualizations of human faces requires capturing both coarse and fine-level details of the face geometry and appearance.
no code implementations • ICCV 2023 • Sherwin Bahmani, Jeong Joon Park, Despoina Paschalidou, Xingguang Yan, Gordon Wetzstein, Leonidas Guibas, Andrea Tagliasacchi
In this work, we introduce CC3D, a conditional generative model that synthesizes complex 3D scenes conditioned on 2D semantic scene layouts, trained using single-view images.
1 code implementation • CVPR 2023 • Sara Sabour, Suhani Vora, Daniel Duckworth, Ivan Krasin, David J. Fleet, Andrea Tagliasacchi
To cope with distractors, we advocate a form of robust estimation for NeRF training, modeling distractors in training data as outliers of an optimization problem.
no code implementations • CVPR 2023 • Samarth Sinha, Jason Y. Zhang, Andrea Tagliasacchi, Igor Gilitschenski, David B. Lindell
Camera pose estimation is a key step in standard 3D reconstruction pipelines that operate on a dense set of images of a single object or scene.
1 code implementation • CVPR 2023 • Songyou Peng, Kyle Genova, Chiyu "Max" Jiang, Andrea Tagliasacchi, Marc Pollefeys, Thomas Funkhouser
Traditional 3D scene understanding approaches rely on labeled 3D datasets to train a model for a single task with supervision.
Ranked #5 on 3D Open-Vocabulary Instance Segmentation on Replica
3D Open-Vocabulary Instance Segmentation 3D Semantic Segmentation +1
1 code implementation • CVPR 2023 • Shitao Tang, Sicong Tang, Andrea Tagliasacchi, Ping Tan, Yasutaka Furukawa
State-of-the-art feature matching methods require each scene to be stored as a 3D point cloud with per-point features, consuming several gigabytes of storage per scene.
no code implementations • 3 Nov 2022 • Lily Goli, Daniel Rebain, Sara Sabour, Animesh Garg, Andrea Tagliasacchi
We introduce a technique for pairwise registration of neural fields that extends classical optimization-based local registration (i. e. ICP) to operate on Neural Radiance Fields (NeRF) -- neural 3D scene representations trained from collections of calibrated images.
no code implementations • CVPR 2023 • Cristina Vasconcelos, Cengiz Oztireli, Mark Matthews, Milad Hashemi, Kevin Swersky, Andrea Tagliasacchi
Neural fields have rapidly been adopted for representing 3D signals, but their application to more classical 2D image-processing has been relatively limited.
no code implementations • 6 Oct 2022 • Daniel Watson, William Chan, Ricardo Martin-Brualla, Jonathan Ho, Andrea Tagliasacchi, Mohammad Norouzi
We demonstrate that stochastic conditioning significantly improves the 3D consistency of a naive sampler for an image-to-image diffusion model, which involves conditioning on a single fixed view.
no code implementations • 21 Sep 2022 • Daniel Rebain, Mark J. Matthews, Kwang Moo Yi, Gopal Sharma, Dmitry Lagun, Andrea Tagliasacchi
Neural fields model signals by mapping coordinate inputs to sampled values.
no code implementations • 29 Aug 2022 • Andrea Tagliasacchi, Ben Mildenhall
Neural Radiance Fields employ simple volume rendering as a way to overcome the challenges of differentiating through ray-triangle intersections by leveraging a probabilistic notion of visibility.
1 code implementation • CVPR 2023 • Zhiqin Chen, Thomas Funkhouser, Peter Hedman, Andrea Tagliasacchi
Neural Radiance Fields (NeRFs) have demonstrated amazing ability to synthesize images of 3D scenes from novel views.
Ranked #1 on Novel View Synthesis on Mip-NeRF 360
no code implementations • 20 Jul 2022 • Weiwei Sun, Daniel Rebain, Renjie Liao, Vladimir Tankovich, Soroosh Yazdani, Kwang Moo Yi, Andrea Tagliasacchi
We introduce a method for instance proposal generation for 3D point clouds.
no code implementations • 31 May 2022 • Tianhao Wu, Fangcheng Zhong, Andrea Tagliasacchi, Forrester Cole, Cengiz Oztireli
We introduce Decoupled Dynamic Neural Radiance Field (D$^2$NeRF), a self-supervised approach that takes a monocular video and learns a 3D scene representation which decouples moving objects, including their shadows, from the static background.
no code implementations • CVPR 2022 • Abhijit Kundu, Kyle Genova, Xiaoqi Yin, Alireza Fathi, Caroline Pantofaru, Leonidas Guibas, Andrea Tagliasacchi, Frank Dellaert, Thomas Funkhouser
Our model builds a panoptic radiance field representation of any scene from just color images.
1 code implementation • CVPR 2022 • Klaus Greff, Francois Belletti, Lucas Beyer, Carl Doersch, Yilun Du, Daniel Duckworth, David J. Fleet, Dan Gnanapragasam, Florian Golemo, Charles Herrmann, Thomas Kipf, Abhijit Kundu, Dmitry Lagun, Issam Laradji, Hsueh-Ti, Liu, Henning Meyer, Yishu Miao, Derek Nowrouzezahrai, Cengiz Oztireli, Etienne Pot, Noha Radwan, Daniel Rebain, Sara Sabour, Mehdi S. M. Sajjadi, Matan Sela, Vincent Sitzmann, Austin Stone, Deqing Sun, Suhani Vora, Ziyu Wang, Tianhao Wu, Kwang Moo Yi, Fangcheng Zhong, Andrea Tagliasacchi
Data is the driving force of machine learning, with the amount and quality of training data often being more important for the performance of a system than architecture and training details.
2 code implementations • 4 Feb 2022 • Zhiqin Chen, Andrea Tagliasacchi, Thomas Funkhouser, Hao Zhang
We introduce neural dual contouring (NDC), a new data-driven approach to mesh reconstruction based on dual contouring (DC).
1 code implementation • 9 Dec 2021 • Anthony Simeonov, Yilun Du, Andrea Tagliasacchi, Joshua B. Tenenbaum, Alberto Rodriguez, Pulkit Agrawal, Vincent Sitzmann
Our performance generalizes across both object instances and 6-DoF object poses, and significantly outperforms a recent baseline that relies on 2D descriptors.
1 code implementation • CVPR 2022 • Kacper Kania, Kwang Moo Yi, Marek Kowalski, Tomasz Trzciński, Andrea Tagliasacchi
We extend neural 3D representations to allow for intuitive and interpretable user control beyond novel view rendering (i. e. camera control).
no code implementations • CVPR 2022 • Konstantinos Rematas, Andrew Liu, Pratul P. Srinivasan, Jonathan T. Barron, Andrea Tagliasacchi, Thomas Funkhouser, Vittorio Ferrari
The goal of this work is to perform 3D reconstruction and novel view synthesis from data captured by scanning platforms commonly deployed for world mapping in urban outdoor environments (e. g., Street View).
1 code implementation • 25 Nov 2021 • Naruya Kondo, Yuya Ikeda, Andrea Tagliasacchi, Yutaka Matsuo, Yoichi Ochiai, Shixiang Shane Gu
We hope VaxNeRF -- a careful combination of a classic technique with a deep method (that arguably replaced it) -- can empower and accelerate new NeRF extensions and applications, with its simplicity, portability, and reliable performance gains.
1 code implementation • CVPR 2022 • Mehdi S. M. Sajjadi, Henning Meyer, Etienne Pot, Urs Bergmann, Klaus Greff, Noha Radwan, Suhani Vora, Mario Lucic, Daniel Duckworth, Alexey Dosovitskiy, Jakob Uszkoreit, Thomas Funkhouser, Andrea Tagliasacchi
In this work, we propose the Scene Representation Transformer (SRT), a method which processes posed or unposed RGB images of a new area, infers a "set-latent scene representation", and synthesises novel views, all in a single feed-forward pass.
no code implementations • 25 Nov 2021 • Suhani Vora, Noha Radwan, Klaus Greff, Henning Meyer, Kyle Genova, Mehdi S. M. Sajjadi, Etienne Pot, Andrea Tagliasacchi, Daniel Duckworth
We present NeSF, a method for producing 3D semantic fields from posed RGB images alone.
no code implementations • CVPR 2022 • Daniel Rebain, Mark Matthews, Kwang Moo Yi, Dmitry Lagun, Andrea Tagliasacchi
We present a method for learning a generative 3D model based on neural radiance fields, trained solely from data with only single views of each object.
1 code implementation • CVPR 2022 • Ben Usman, Andrea Tagliasacchi, Kate Saenko, Avneesh Sud
In the era of deep learning, human pose estimation from multiple cameras with unknown calibration has received little attention to date.
Ranked #1 on 3D Human Pose Estimation on SkiPose
1 code implementation • 27 Jun 2021 • Zhiqin Chen, Andrea Tagliasacchi, Hao Zhang
The network is trained to reconstruct a shape using a set of convexes obtained from a BSP-tree built over a set of planes, where the planes and convexes are both defined by learned network weights.
1 code implementation • CVPR 2021 • Baptiste Angles, Yuhe Jin, Simon Kornblith, Andrea Tagliasacchi, Kwang Moo Yi
We propose a deep network that can be trained to tackle image reconstruction and classification problems that involve detection of multiple object instances, without any supervision regarding their whereabouts.
no code implementations • 7 Jun 2021 • Daniel Rebain, Ke Li, Vincent Sitzmann, Soroosh Yazdani, Kwang Moo Yi, Andrea Tagliasacchi
Implicit representations of geometry, such as occupancy fields or signed distance fields (SDF), have recently re-gained popularity in encoding 3D solid shape in a functional form.
4 code implementations • ICCV 2021 • Congyue Deng, Or Litany, Yueqi Duan, Adrien Poulenard, Andrea Tagliasacchi, Leonidas Guibas
Invariance and equivariance to the rotation group have been widely discussed in the 3D deep learning community for pointclouds.
1 code implementation • ICCV 2021 • Wei Jiang, Eduard Trulls, Jan Hosang, Andrea Tagliasacchi, Kwang Moo Yi
We propose a novel framework for finding correspondences in images based on a deep neural network that, given two images and a query point in one of them, finds its correspondence in the other.
Ranked #1 on Dense Pixel Correspondence Estimation on KITTI 2012
Dense Pixel Correspondence Estimation Optical Flow Estimation
no code implementations • 18 Dec 2020 • Francis Williams, Or Litany, Avneesh Sud, Kevin Swersky, Andrea Tagliasacchi
We introduce a technique for 3D human keypoint estimation that directly models the notion of spatial uncertainty of a keypoint.
1 code implementation • NeurIPS 2021 • Weiwei Sun, Andrea Tagliasacchi, Boyang Deng, Sara Sabour, Soroosh Yazdani, Geoffrey Hinton, Kwang Moo Yi
We propose a self-supervised capsule architecture for 3D point clouds.
no code implementations • NeurIPS 2020 • Chiyu Jiang, Jingwei Huang, Andrea Tagliasacchi, Leonidas J. Guibas
Such a space naturally allows the disentanglement of geometric style (coming from the source) and structural pose (conforming to the target).
no code implementations • 27 Nov 2020 • Sara Sabour, Andrea Tagliasacchi, Soroosh Yazdani, Geoffrey E. Hinton, David J. Fleet
Capsule networks aim to parse images into a hierarchy of objects, parts and relations.
no code implementations • CVPR 2021 • Daniel Rebain, Wei Jiang, Soroosh Yazdani, Ke Li, Kwang Moo Yi, Andrea Tagliasacchi
Moreover, we show that a Voronoi spatial decomposition is preferable for this purpose, as it is provably compatible with the Painter's Algorithm for efficient and GPU-friendly rendering.
no code implementations • 21 Oct 2020 • Soroosh Yazdani, Andrea Tagliasacchi
In this technical report, we investigate extending convolutional neural networks to the setting where functions are not sampled in a grid pattern.
no code implementations • NeurIPS 2020 • Xiaogang Wang, Yuelang Xu, Kai Xu, Andrea Tagliasacchi, Bin Zhou, Ali Mahdavi-Amiri, Hao Zhang
We introduce an end-to-end learnable technique to robustly identify feature edges in 3D point cloud data.
1 code implementation • NeurIPS 2020 • Emre Aksan, Thomas Deselaers, Andrea Tagliasacchi, Otmar Hilliges
We demonstrate qualitatively and quantitatively that our proposed approach is able to model the appearance of individual strokes, as well as the compositional structure of larger diagram drawings.
1 code implementation • 14 Jun 2020 • Chiyu "Max" Jiang, Jingwei Huang, Andrea Tagliasacchi, Leonidas Guibas
We illustrate the effectiveness of this learned deformation space for various downstream applications, including shape generation via deformation, geometric style transfer, unsupervised learning of a consistent parameterization for entire classes of shapes, and shape interpolation.
no code implementations • CVPR 2020 • Danhang Tang, Saurabh Singh, Philip A. Chou, Christian Haene, Mingsong Dou, Sean Fanello, Jonathan Taylor, Philip Davidson, Onur G. Guleryuz, yinda zhang, Shahram Izadi, Andrea Tagliasacchi, Sofien Bouaziz, Cem Keskin
We describe a novel approach for compressing truncated signed distance fields (TSDF) stored in 3D voxel grids, and their corresponding textures.
no code implementations • 6 Apr 2020 • Timothy Jeruzalski, David I. W. Levin, Alec Jacobson, Paul Lalonde, Mohammad Norouzi, Andrea Tagliasacchi
In this technical report, we investigate efficient representations of articulated objects (e. g. human bodies), which is an important problem in computer vision and graphics.
no code implementations • 8 Dec 2019 • Francis Williams, Daniele Panozzo, Kwang Moo Yi, Andrea Tagliasacchi
Voronoi diagrams are highly compact representations that are used in various Graphics applications.
no code implementations • 6 Dec 2019 • Boyang Deng, JP Lewis, Timothy Jeruzalski, Gerard Pons-Moll, Geoffrey Hinton, Mohammad Norouzi, Andrea Tagliasacchi
Efficient representation of articulated objects such as human bodies is an important problem in computer vision and graphics.
3 code implementations • CVPR 2020 • Zhiqin Chen, Andrea Tagliasacchi, Hao Zhang
The network is trained to reconstruct a shape using a set of convexes obtained from a BSP-tree built on a set of planes.
no code implementations • 25 Sep 2019 • Baptiste Angles, Simon Kornblith, Shahram Izadi, Andrea Tagliasacchi, Kwang Moo Yi
We propose a deep network that can be trained to tackle image reconstruction and classification problems that involve detection of multiple object instances, without any supervision regarding their whereabouts.
no code implementations • CVPR 2020 • Boyang Deng, Kyle Genova, Soroosh Yazdani, Sofien Bouaziz, Geoffrey Hinton, Andrea Tagliasacchi
We introduce a network architecture to represent a low dimensional family of convexes.
1 code implementation • CVPR 2020 • Weiwei Sun, Wei Jiang, Eduard Trulls, Andrea Tagliasacchi, Kwang Moo Yi
Many problems in computer vision require dealing with sparse, unordered data in the form of point clouds.
no code implementations • CVPR 2019 • Rohit Pandey, Anastasia Tkach, Shuoran Yang, Pavel Pidlypenskyi, Jonathan Taylor, Ricardo Martin-Brualla, Andrea Tagliasacchi, George Papandreou, Philip Davidson, Cem Keskin, Shahram Izadi, Sean Fanello
The key insight is to leverage previously seen "calibration" images of a given user to extrapolate what should be rendered in a novel viewpoint from the data available in the sensor.
1 code implementation • ICCV 2019 • Wei Jiang, Weiwei Sun, Andrea Tagliasacchi, Eduard Trulls, Kwang Moo Yi
We propose a novel image sampling method for differentiable image transformation in deep neural networks.
1 code implementation • 26 Nov 2018 • Baptiste Angles, Yuhe Jin, Simon Kornblith, Andrea Tagliasacchi, Kwang Moo Yi
We propose a deep network that can be trained to tackle image reconstruction and classification problems that involve detection of multiple object instances, without any supervision regarding their whereabouts.
Anomaly Detection In Surveillance Videos Image Reconstruction
no code implementations • ICLR 2018 • Fabrizio Pedersoli, George Tzanetakis, Andrea Tagliasacchi
Binary Deep Neural Networks (BDNNs) have been shown to be an effective way of achieving this objective.
no code implementations • 16 Nov 2017 • Abhishake Kumar Bojja, Franziska Mueller, Sri Raghu Malireddi, Markus Oberweger, Vincent Lepetit, Christian Theobalt, Kwang Moo Yi, Andrea Tagliasacchi
We propose an automatic method for generating high-quality annotations for depth-based hand segmentation, and introduce a large-scale hand segmentation dataset.
1 code implementation • ICCV 2017 • Edoardo Remelli, Anastasia Tkach, Andrea Tagliasacchi, Mark Pauly
We present a robust algorithm for personalizing a sphere-mesh tracking model to a user from a collection of depth measurements.
1 code implementation • 19 May 2017 • Fabrizio Pedersoli, George Tzanetakis, Andrea Tagliasacchi
In this paper, we show how Convolutional Neural Networks (CNNs) can be implemented using binary representations.