Search Results for author:
Found 73 papers, 45 papers with code
ActAnywhere: Subject-Aware Video Background Generation
Generating video background that tailors to foreground subject motion is an important problem for the movie industry and visual effects community.
Refining Pre-Trained Motion Models
In the first stage, we use the pre-trained model to estimate motion in a video, and then select the subset of motion estimates which we can verify with cycle-consistency.
Towards Learning Geometric Eigen-Lengths Crucial for Fitting Tasks
In this work, we therefore for the first time formulate and propose a novel learning problem on this question and set up a benchmark suite including tasks, data, and evaluation metrics for studying the problem.
PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Tracking
Our goal is to advance the state-of-the-art by placing emphasis on long videos with naturalistic motion.
Ranked #1 on
Point Tracking
on TAP-Vid
Learning a Diffusion Prior for NeRFs
Neural Radiance Fields (NeRFs) have emerged as a powerful neural 3D representation for objects and scenes derived from 2D data.
Physics-informed PointNet: On how many irregular geometries can it solve an inverse problem simultaneously? Application to linear elasticity
In this article, we demonstrate that PIPN predicts the solution of desired partial differential equations over a few hundred domains simultaneously, while it only uses sparse labeled data.
CurveCloudNet: Processing Point Clouds with 1D Structure
Modern depth sensors such as LiDAR operate by sweeping laser-beams across the scene, resulting in a point cloud with notable 1D curve-like structures.
SceneHGN: Hierarchical Graph Networks for 3D Indoor Scene Generation with Fine-Grained Geometry
We propose SCENEHGN, a hierarchical graph network for 3D indoor scenes that takes into account the full hierarchy from the room level to the object level, then finally to the object part level.
Equivalence Between SE(3) Equivariant Networks via Steerable Kernels and Group Convolution
Most approaches for equivariance under the Euclidean group $\mathrm{SE}(3)$ of rotations and translations fall within one of the two major categories.
COPILOT: Human-Environment Collision Prediction and Localization from Egocentric Videos
In this work, we introduce the challenging problem of predicting collisions in diverse environments from multi-view egocentric videos captured from body-mounted cameras.
6D Camera Relocalization in Visually Ambiguous Extreme Environments
To this end, we propose: (i) a hierarchical localization system, where we leverage temporal information and (ii) a novel environment-aware image enhancement method to boost the robustness and accuracy.
Object Scene Representation Transformer
A compositional understanding of the world in terms of objects and their geometry in 3D space is considered a cornerstone of human cognition.
Fixing Malfunctional Objects With Learned Physical Simulation and Functional Prediction
Specifically, FixNet consists of a perception module to extract the structured representation from the 3D point cloud, a physical dynamics prediction module to simulate the results of interactions on 3D objects, and a functionality prediction module to evaluate the functionality and choose the correct fix.
GIMO: Gaze-Informed Human Motion Prediction in Context
We perform an extensive study of the benefits of leveraging the eye gaze for ego-centric human motion prediction with various state-of-the-art architectures.
ACID: Action-Conditional Implicit Visual Dynamics for Deformable Object Manipulation
Manipulating volumetric deformable objects in the real world, like plush toys and pizza dough, bring substantial challenges due to infinite shape variations, non-rigid motions, and partial observability.
ConDor: Self-Supervised Canonicalization of 3D Pose for Partial Shapes
ConDor is a self-supervised method that learns to Canonicalize the 3D orientation and position for full and partial 3D point clouds.
ADeLA: Automatic Dense Labeling With Attention for Viewpoint Shift in Semantic Segmentation
We describe a method to deal with performance drop in semantic segmentation caused by viewpoint changes within multi-camera systems, where temporally paired images are readily available, but the annotations may only be abundant for a few typical views.
Generating Useful Accident-Prone Driving Scenarios via a Learned Traffic Prior
Scenario generation is formulated as an optimization in the latent space of this traffic model, perturbing an initial real-world scene to produce trajectories that collide with a given planner.
Multiway Non-rigid Point Cloud Registration via Learned Functional Map Synchronization
We present SyNoRiM, a novel way to jointly register multiple non-rigid shapes by synchronizing the maps relating learned functions defined on the point clouds.
Unsupervised Discovery of Object Radiance Fields
We study the problem of inferring an object-centric scene representation from a single image, aiming to derive a representation that explains the image formation process, captures the scene's 3D nature, and is learned without supervision.
A Functional Approach to Rotation Equivariant Non-Linearities for Tensor Field Networks.
A fundamental problem in equivariant deep learning is to design activation functions which are both informative and preserve equivariance.
HuMoR: 3D Human Motion Model for Robust Pose Estimation
We introduce HuMoR: a 3D Human Motion Model for Robust Estimation of temporal pose and shape.
CAPTRA: CAtegory-level Pose Tracking for Rigid and Articulated Objects from Point Clouds
For the first time, we propose a unified framework that can handle 9DoF pose tracking for novel rigid object instances as well as per-part pose tracking for articulated objects from known categories.
Weakly Supervised Learning of Rigid 3D Scene Flow
We propose a data-driven scene flow estimation algorithm exploiting the observation that many 3D scenes can be explained by a collection of agents moving as rigid bodies.
3DIoUMatch: Leveraging IoU Prediction for Semi-Supervised 3D Object Detection
On KITTI, we are the first to demonstrate semi-supervised 3D object detection and our method surpasses a fully supervised baseline from 1. 8% to 7. 6% under different label ratios and categories.
ShapeFlow: Learnable Deformation Flows Among 3D Shapes
Such a space naturally allows the disentanglement of geometric style (coming from the source) and structural pose (conforming to the target).
A Point-Cloud Deep Learning Framework for Prediction of Fluid Flow Fields on Irregular Geometries
Grid vertices in a computational fluid dynamics (CFD) domain are viewed as point clouds and used as inputs to a neural network based on the PointNet architecture, which learns an end-to-end mapping between spatial positions and CFD quantities.
IF-Defense: 3D Adversarial Point Cloud Defense via Implicit Function based Restoration
The former aims to recover the surface of point cloud through implicit function, while the latter encourages evenly-distributed points.
DeformSyncNet: Deformation Transfer via Synchronized Shape Deformation Spaces
Shape deformation is an important component in any geometry processing toolbox.
Graphics
Pix2Surf: Learning Parametric 3D Surface Models of Objects from Images
We investigate the problem of learning to generate 3D parametric surface representations for novel object instances, as seen from one or more views.
DSG-Net: Learning Disentangled Structure and Geometry for 3D Shape Generation
While significant progress has been made, especially with recent deep generative models, it remains a challenge to synthesize high-quality shapes with rich geometric details and complex structure, in a controllable manner.
CaSPR: Learning Canonical Spatiotemporal Point Cloud Representations
We propose CaSPR, a method to learn object-centric Canonical Spatiotemporal Point Cloud Representations of dynamically moving or evolving objects.
Contact and Human Dynamics from Monocular Video
Existing deep models predict 2D and 3D kinematic poses from video that are approximately accurate, but contain visible errors that violate physical constraints, such as feet penetrating the ground and bodies leaning at extreme angles.
PointContrast: Unsupervised Pre-training for 3D Point Cloud Understanding
To this end, we select a suite of diverse datasets and tasks to measure the effect of unsupervised pre-training on a large source set of 3D scenes.
Object-Centric Multi-View Aggregation
We present an approach for aggregating a sparse set of views of an object in order to compute a semi-implicit 3D representation in the form of a volumetric feature grid.
Rethinking Sampling in 3D Point Cloud Generative Adversarial Networks
We further study how different evaluation metrics weigh the sampling pattern against the geometry and propose several perceptual metrics forming a sampling spectrum of metrics.
PT2PC: Learning to Generate 3D Point Cloud Shapes from Part Tree Conditions
3D generative shape modeling is a fundamental research area in computer vision and interactive computer graphics, with many real-world applications.
Curriculum DeepSDF
When learning to sketch, beginners start with simple and flexible shapes, and then gradually strive for more complex and accurate ones in the subsequent training sessions.
SAPIEN: A SimulAted Part-based Interactive ENvironment
To achieve this task, a simulated environment with physically realistic simulation, sufficient articulated objects, and transferability to the real robot is indispensable.
ImVoteNet: Boosting 3D Object Detection in Point Clouds with Image Votes
Compared to prior work on multi-modal detection, we explicitly extract both geometric and semantic features from the 2D images.
Ranked #2 on
3D Object Detection
on SUN-RGBD
(using extra training data)
Predicting the Physical Dynamics of Unseen 3D Objects
Experiments show that we can accurately predict the changes in state for unseen object geometries and initial conditions.
Learning multiview 3D point cloud registration
We present a novel, end-to-end learnable, multiview 3D point cloud registration algorithm.
A Condition Number for Joint Optimization of Cycle-Consistent Networks
A recent trend in optimizing maps such as dense correspondences between objects or neural networks between pairs of domains is to optimize them jointly.
StructEdit: Learning Structural Shape Variations
Learning to encode differences in the geometry and (topological) structure of the shapes of ordinary objects is key to generating semantically plausible variations of a given shape, transferring edits from one shape to another, and many other applications in 3D content creation.
Situational Fusion of Visual Representation for Visual Navigation
A complex visual navigation task puts an agent in different situations which call for a diverse range of visual perception abilities.
StructureNet: Hierarchical Graph Networks for 3D Shape Generation
We introduce StructureNet, a hierarchical graph network which (i) can directly encode shapes represented as such n-ary graphs; (ii) can be robustly trained on large and complex shape families; and (iii) can be used to generate a great diversity of realistic structured shape geometries.
Multiview Aggregation for Learning Category-Specific Shape Reconstruction
We investigate the problem of learning category-specific 3D shape reconstruction from a variable number of RGB views of previously unobserved object instances.
A Topology Layer for Machine Learning
Topology applied to real world data using persistent homology has started to find applications within machine learning, including deep learning.
ShapeGlot: Learning Language for Shape Differentiation
We also find that these models are amenable to zero-shot transfer learning to novel object classes (e. g. transfer from training on chairs to testing on lamps), as well as to real-world images drawn from furniture catalogs.
An Information-Theoretic Metric of Transferability for Task Transfer Learning
An important question in task transfer learning is to determine task transferability, i. e. given a common input domain, estimating to what extent representations learned from a source task can help in learning a target task.
Deep Hough Voting for 3D Object Detection in Point Clouds
Current 3D object detection methods are heavily influenced by 2D detectors.
3D Object Detection
3D Object Detection From Monocular Images
+2
KPConv: Flexible and Deformable Convolution for Point Clouds
Furthermore, these locations are continuous in space and can be learned by the network.
Ranked #1 on
3D Semantic Segmentation
on DALES
DeepSpline: Data-Driven Reconstruction of Parametric Curves and Surfaces
Reconstruction of geometry based on different input modes, such as images or point clouds, has been instrumental in the development of computer aided design and computer graphics.
Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation
The goal of this paper is to estimate the 6D pose and dimensions of unseen object instances in an RGB-D image.
Ranked #2 on
6D Pose Estimation using RGBD
on CAMERA25
Learning Generalizable Physical Dynamics of 3D Rigid Objects
In this work, we focus on predicting the dynamics of 3D rigid objects, in particular an object's final resting position and total rotation when subjected to an impulsive force.
PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understanding
We present PartNet: a consistent, large-scale dataset of 3D objects annotated with fine-grained, instance-level, and hierarchical 3D part information.
Ranked #3 on
3D Instance Segmentation
on PartNet
Topology-Aware Surface Reconstruction for Point Clouds
We present an approach to inform the reconstruction of a surface from a point scan through topological priors.
Computational Geometry Graphics
Parsing Geometry Using Structure-Aware Shape Templates
Real-life man-made objects often exhibit strong and easily-identifiable structure, as a direct result of their design or their intended functionality.
Modular Latent Spaces for Shape Correspondences
To achieve this, we use intermediate nonlinear embedding spaces, computed individually on every shape; the embedding functions use ideas from diffusion geometry and capture how different descriptors on the same shape inter‐relate.
FlowNet3D: Learning Scene Flow in 3D Point Clouds
In this work, we propose a novel deep neural network named $FlowNet3D$ that learns scene flow from point clouds in an end-to-end fashion.
Geometry Guided Convolutional Neural Networks for Self-Supervised Video Representation Learning
In addition, we also find that a progressive training strategy can foster a better neural network for the video recognition task than blindly pooling the distinct sources of geometry cues together.
Frustum PointNets for 3D Object Detection from RGB-D Data
In this work, we study 3D object detection from RGB-D data in both indoor and outdoor scenes.
Ranked #1 on
Object Localization
on KITTI Pedestrians Easy
PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space
By exploiting metric space distances, our network is able to learn local features with increasing contextual scales.
Ranked #2 on
Semantic Segmentation
on Toronto-3D L002
PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
Point cloud is an important type of geometric data structure.
Ranked #1 on
Semantic Segmentation
on S3DIS
(Number of params metric)
3D Semantic Segmentation
Few-Shot 3D Point Cloud Classification
+4
Learning Shape Abstractions by Assembling Volumetric Primitives
We present a learning framework for abstracting complex shapes by learning to assemble objects using 3D volumetric primitives.
Understanding and Exploiting Object Interaction Landscapes
We introduce a new general representation for proximal interactions among physical objects that is agnostic to the type of objects or interaction involved.
FPNN: Field Probing Neural Networks for 3D Data
Each field probing filter is a set of probing points --- sensors that perceive the space.
Ranked #5 on
3D Object Recognition
on ModelNet40
Volumetric and Multi-View CNNs for Object Classification on 3D Data
Empirical results from these two types of CNNs exhibit a large gap, indicating that existing volumetric CNN architectures and approaches are unable to fully exploit the power of 3D representations.
Ranked #3 on
3D Object Recognition
on ModelNet40
3D-Assisted Feature Synthesis for Novel Views of an Object
Comparing two images from different views has been a long-standing challenging problem in computer vision, as visual features are not stable under large view point changes.
Unsupervised Multi-Class Joint Image Segmentation
Joint segmentation of image sets is a challenging problem, especially when there are multiple objects with variable appearance shared among the images in the collection and the set of objects present in each particular image is itself varying and unknown.
Near-Optimal Joint Object Matching via Convex Relaxation
Joint matching over a collection of objects aims at aggregating information from a large collection of similar instances (e. g. images, graphs, shapes) to improve maps between pairs of them.
Wavelets on Graphs via Deep Learning
An increasing number of applications require processing of signals defined on weighted graphs.
WaldHash: sequential similarity-preserving hashing
Similarity-sensitive hashing seeks compact representation of vector data as binary codes, so that the Hamming distance between code words approximates the original similarity.
