Search Results for author:
Found 59 papers, 33 papers with code
DrivAerNet: A Parametric Car Dataset for Data-Driven Aerodynamic Design and Graph-Based Drag Prediction
This study introduces DrivAerNet, a large-scale high-fidelity CFD dataset of 3D industry-standard car shapes, and RegDGCNN, a dynamic graph convolutional neural network model, both aimed at aerodynamic car design through machine learning.
FaceTalk: Audio-Driven Motion Diffusion for Neural Parametric Head Models
We propose a new latent diffusion model for this task, operating in the expression space of neural parametric head models, to synthesize audio-driven realistic head sequences.
PaSCo: Urban 3D Panoptic Scene Completion with Uncertainty Awareness
We propose the task of Panoptic Scene Completion (PSC) which extends the recently popular Semantic Scene Completion (SSC) task with instance-level information to produce a richer understanding of the 3D scene.
DPHMs: Diffusion Parametric Head Models for Depth-based Tracking
We introduce Diffusion Parametric Head Models (DPHMs), a generative model that enables robust volumetric head reconstruction and tracking from monocular depth sequences.
DiffCAD: Weakly-Supervised Probabilistic CAD Model Retrieval and Alignment from an RGB Image
We formulate this as a conditional generative task, leveraging diffusion to learn implicit probabilistic models capturing the shape, pose, and scale of CAD objects in an image.
GenZI: Zero-Shot 3D Human-Scene Interaction Generation
Given a natural language description and a coarse point location of the desired interaction in a 3D scene, we first leverage VLMs to imagine plausible 2D human interactions inpainted into multiple rendered views of the scene.
MeshGPT: Generating Triangle Meshes with Decoder-Only Transformers
We introduce MeshGPT, a new approach for generating triangle meshes that reflects the compactness typical of artist-created meshes, in contrast to dense triangle meshes extracted by iso-surfacing methods from neural fields.
CG-HOI: Contact-Guided 3D Human-Object Interaction Generation
Using this guidance to bridge human and object motion enables generating more realistic and physically plausible interaction sequences, where the human body and corresponding object move in a coherent manner.
Human-Object Interaction Detection
Human-Object Interaction Generation
+1
ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenes
Each scene is captured with a high-end laser scanner at sub-millimeter resolution, along with registered 33-megapixel images from a DSLR camera, and RGB-D streams from an iPhone.
Mesh2Tex: Generating Mesh Textures from Image Queries
The learned texture manifold enables effective navigation to generate an object texture for a given 3D object geometry that matches to an input RGB image, which maintains robustness even under challenging real-world scenarios where the mesh geometry approximates an inexact match to the underlying geometry in the RGB image.
HyperDiffusion: Generating Implicit Neural Fields with Weight-Space Diffusion
HyperDiffusion operates directly on MLP weights and generates new neural implicit fields encoded by synthesized MLP parameters.
UnScene3D: Unsupervised 3D Instance Segmentation for Indoor Scenes
We propose UnScene3D, the first fully unsupervised 3D learning approach for class-agnostic 3D instance segmentation of indoor scans.
DiffuScene: Denoising Diffusion Models for Generative Indoor Scene Synthesis
We introduce a diffusion network to synthesize a collection of 3D indoor objects by denoising a set of unordered object attributes.
Mask3D: Pre-training 2D Vision Transformers by Learning Masked 3D Priors
Current popular backbones in computer vision, such as Vision Transformers (ViT) and ResNets are trained to perceive the world from 2D images.
Panoptic Lifting for 3D Scene Understanding with Neural Fields
We propose Panoptic Lifting, a novel approach for learning panoptic 3D volumetric representations from images of in-the-wild scenes.
ObjectMatch: Robust Registration using Canonical Object Correspondences
We present ObjectMatch, a semantic and object-centric camera pose estimator for RGB-D SLAM pipelines.
ClipFace: Text-guided Editing of Textured 3D Morphable Models
Controllable editing and manipulation are given by language prompts to adapt texture and expression of the 3D morphable model.
Learning 3D Scene Priors with 2D Supervision
Holistic 3D scene understanding entails estimation of both layout configuration and object geometry in a 3D environment.
Neural Poisson: Indicator Functions for Neural Fields
Implicit neural field generating signed distance field representations (SDFs) of 3D shapes have shown remarkable progress in 3D shape reconstruction and generation.
FutureHuman3D: Forecasting Complex Long-Term 3D Human Behavior from Video Observations
Thus, we design our method to only require 2D RGB data while being able to generate 3D human motion sequences.
PatchComplete: Learning Multi-Resolution Patch Priors for 3D Shape Completion on Unseen Categories
While 3D shape representations enable powerful reasoning in many visual and perception applications, learning 3D shape priors tends to be constrained to the specific categories trained on, leading to an inefficient learning process, particularly for general applications with unseen categories.
Language-Grounded Indoor 3D Semantic Segmentation in the Wild
Recent advances in 3D semantic segmentation with deep neural networks have shown remarkable success, with rapid performance increase on available datasets.
Ranked #7 on
3D Semantic Segmentation
on ScanNet200
Texturify: Generating Textures on 3D Shape Surfaces
Texture cues on 3D objects are key to compelling visual representations, with the possibility to create high visual fidelity with inherent spatial consistency across different views.
Weakly-Supervised End-to-End CAD Retrieval to Scan Objects
CAD model retrieval to real-world scene observations has shown strong promise as a basis for 3D perception of objects and a clean, lightweight mesh-based scene representation; however, current approaches to retrieve CAD models to a query scan rely on expensive manual annotations of 1:1 associations of CAD-scan objects, which typically contain strong lower-level geometric differences.
Neural Part Priors: Learning to Optimize Part-Based Object Completion in RGB-D Scans
3D object recognition has seen significant advances in recent years, showing impressive performance on real-world 3D scan benchmarks, but lacking in object part reasoning, which is fundamental to higher-level scene understanding such as inter-object similarities or object functionality.
SPAMs: Structured Implicit Parametric Models
We observe that deformable object motion is often semantically structured, and thus propose to learn Structured-implicit PArametric Models (SPAMs) as a deformable object representation that structurally decomposes non-rigid object motion into part-based disentangled representations of shape and pose, with each being represented by deep implicit functions.
4DContrast: Contrastive Learning with Dynamic Correspondences for 3D Scene Understanding
We present a new approach to instill 4D dynamic object priors into learned 3D representations by unsupervised pre-training.
Ranked #21 on
3D Instance Segmentation
on ScanNet(v2)
ROCA: Robust CAD Model Retrieval and Alignment from a Single Image
We present ROCA, a novel end-to-end approach that retrieves and aligns 3D CAD models from a shape database to a single input image.
3D Dense Shape Correspondence
3D Object Detection From Monocular Images
+2
Pose2Room: Understanding 3D Scenes from Human Activities
To this end, we propose P2R-Net to learn a probabilistic 3D model of the objects in a scene characterized by their class categories and oriented 3D bounding boxes, based on an input observed human trajectory in the environment.
Panoptic 3D Scene Reconstruction From a Single RGB Image
Inspired by 2D panoptic segmentation, we propose to unify the tasks of geometric reconstruction, 3D semantic segmentation, and 3D instance segmentation into the task of panoptic 3D scene reconstruction - from a single RGB image, predicting the complete geometric reconstruction of the scene in the camera frustum of the image, along with semantic and instance segmentations.
Patch2CAD: Patchwise Embedding Learning for In-the-Wild Shape Retrieval from a Single Image
3D perception of object shapes from RGB image input is fundamental towards semantic scene understanding, grounding image-based perception in our spatially 3-dimensional real-world environments.
TransformerFusion: Monocular RGB Scene Reconstruction using Transformers
We introduce TransformerFusion, a transformer-based 3D scene reconstruction approach.
Pri3D: Can 3D Priors Help 2D Representation Learning?
Inspired by these advances in geometric understanding, we aim to imbue image-based perception with representations learned under geometric constraints.
NPMs: Neural Parametric Models for 3D Deformable Shapes
Crucially, once learned, our neural parametric models of shape and pose enable optimization over the learned spaces to fit to new observations, similar to the fitting of a traditional parametric model, e. g., SMPL.
RetrievalFuse: Neural 3D Scene Reconstruction with a Database
3D reconstruction of large scenes is a challenging problem due to the high-complexity nature of the solution space, in particular for generative neural networks.
Seeing Behind Objects for 3D Multi-Object Tracking in RGB-D Sequences
From a sequence of RGB-D frames, we detect objects in each frame and learn to predict their complete object geometry as well as a dense correspondence mapping into a canonical space.
Towards Part-Based Understanding of RGB-D Scans
Recent advances in 3D semantic scene understanding have shown impressive progress in 3D instance segmentation, enabling object-level reasoning about 3D scenes; however, a finer-grained understanding is required to enable interactions with objects and their functional understanding.
Neural Deformation Graphs for Globally-consistent Non-rigid Reconstruction
We introduce Neural Deformation Graphs for globally-consistent deformation tracking and 3D reconstruction of non-rigid objects.
Forecasting Characteristic 3D Poses of Human Actions
To predict characteristic poses, we propose a probabilistic approach that models the possible multi-modality in the distribution of likely characteristic poses.
Mask2CAD: 3D Shape Prediction by Learning to Segment and Retrieve
We propose to leverage existing large-scale datasets of 3D models to understand the underlying 3D structure of objects seen in an image by constructing a CAD-based representation of the objects and their poses.
SPSG: Self-Supervised Photometric Scene Generation from RGB-D Scans
We present SPSG, a novel approach to generate high-quality, colored 3D models of scenes from RGB-D scan observations by learning to infer unobserved scene geometry and color in a self-supervised fashion.
Neural Non-Rigid Tracking
We introduce a novel, end-to-end learnable, differentiable non-rigid tracker that enables state-of-the-art non-rigid reconstruction by a learned robust optimization.
SceneCAD: Predicting Object Alignments and Layouts in RGB-D Scans
We present a novel approach to reconstructing lightweight, CAD-based representations of scanned 3D environments from commodity RGB-D sensors.
Adversarial Texture Optimization from RGB-D Scans
In this work, we present a novel approach for color texture generation using a conditional adversarial loss obtained from weakly-supervised views.
SG-NN: Sparse Generative Neural Networks for Self-Supervised Scene Completion of RGB-D Scans
We present a novel approach that converts partial and noisy RGB-D scans into high-quality 3D scene reconstructions by inferring unobserved scene geometry.
Joint Embedding of 3D Scan and CAD Objects
We propose a novel approach to learn a joint embedding space between scan and CAD geometry, where semantically similar objects from both domains lie close together.
End-to-End CAD Model Retrieval and 9DoF Alignment in 3D Scans
We present a novel, end-to-end approach to align CAD models to an 3D scan of a scene, enabling transformation of a noisy, incomplete 3D scan to a compact, CAD reconstruction with clean, complete object geometry.
RevealNet: Seeing Behind Objects in RGB-D Scans
Thus, we introduce the task of semantic instance completion: from an incomplete RGB-D scan of a scene, we aim to detect the individual object instances and infer their complete object geometry.
3D-SIS: 3D Semantic Instance Segmentation of RGB-D Scans
We introduce 3D-SIS, a novel neural network architecture for 3D semantic instance segmentation in commodity RGB-D scans.
Ranked #3 on
3D Semantic Instance Segmentation
on ScanNetV2
Scan2CAD: Learning CAD Model Alignment in RGB-D Scans
For a 3D reconstruction of an indoor scene, our method takes as input a set of CAD models, and predicts a 9DoF pose that aligns each model to the underlying scan geometry.
Ranked #1 on
3D Reconstruction
on Scan2CAD
Scan2Mesh: From Unstructured Range Scans to 3D Meshes
We introduce Scan2Mesh, a novel data-driven generative approach which transforms an unstructured and potentially incomplete range scan into a structured 3D mesh representation.
3DMV: Joint 3D-Multi-View Prediction for 3D Semantic Scene Segmentation
We present 3DMV, a novel method for 3D semantic scene segmentation of RGB-D scans in indoor environments using a joint 3D-multi-view prediction network.
Ranked #1 on
Scene Segmentation
on ScanNet
ScanComplete: Large-Scale Scene Completion and Semantic Segmentation for 3D Scans
We introduce ScanComplete, a novel data-driven approach for taking an incomplete 3D scan of a scene as input and predicting a complete 3D model along with per-voxel semantic labels.
Matterport3D: Learning from RGB-D Data in Indoor Environments
Access to large, diverse RGB-D datasets is critical for training RGB-D scene understanding algorithms.
ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes
A key requirement for leveraging supervised deep learning methods is the availability of large, labeled datasets.
Ranked #11 on
Semantic Segmentation
on ScanNetV2
Shape Completion using 3D-Encoder-Predictor CNNs and Shape Synthesis
Although our 3D-EPN outperforms state-of-the-art completion method, the main contribution in our work lies in the combination of a data-driven shape predictor and analytic 3D shape synthesis.
Volumetric and Multi-View CNNs for Object Classification on 3D Data
Empirical results from these two types of CNNs exhibit a large gap, indicating that existing volumetric CNN architectures and approaches are unable to fully exploit the power of 3D representations.
Ranked #3 on
3D Object Recognition
on ModelNet40
BundleFusion: Real-time Globally Consistent 3D Reconstruction using On-the-fly Surface Re-integration
Our approach estimates globally optimized (i. e., bundle adjusted) poses in real-time, supports robust tracking with recovery from gross tracking failures (i. e., relocalization), and re-estimates the 3D model in real-time to ensure global consistency; all within a single framework.
Learning to Navigate the Energy Landscape
We demonstrate the efficacy of our approach on the challenging problem of RGB Camera Relocalization.
