Search Results for author:
Found 56 papers, 15 papers with code
LightIt: Illumination Modeling and Control for Diffusion Models
Our method is the first that enables the generation of images with controllable, consistent lighting and performs on par with specialized relighting state-of-the-art methods.
PF-LRM: Pose-Free Large Reconstruction Model for Joint Pose and Shape Prediction
We propose a Pose-Free Large Reconstruction Model (PF-LRM) for reconstructing a 3D object from a few unposed images even with little visual overlap, while simultaneously estimating the relative camera poses in ~1. 3 seconds on a single A100 GPU.
DMV3D: Denoising Multi-View Diffusion using 3D Large Reconstruction Model
We propose \textbf{DMV3D}, a novel 3D generation approach that uses a transformer-based 3D large reconstruction model to denoise multi-view diffusion.
Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Model
Text-to-3D with diffusion models has achieved remarkable progress in recent years.
LRM: Large Reconstruction Model for Single Image to 3D
We propose the first Large Reconstruction Model (LRM) that predicts the 3D model of an object from a single input image within just 5 seconds.
Controllable Dynamic Appearance for Neural 3D Portraits
The surface normals prediction is guided using 3DMM normals that act as a coarse prior for the normals of the human head, where direct prediction of normals is hard due to rigid and non-rigid deformations induced by head-pose and facial expression changes.
NeuManifold: Neural Watertight Manifold Reconstruction with Efficient and High-Quality Rendering Support
We present a method for generating high-quality watertight manifold meshes from multi-view input images.
PhotoMat: A Material Generator Learned from Single Flash Photos
Instead, we train a generator for a neural material representation that is rendered with a learned relighting module to create arbitrarily lit RGB images; these are compared against real photos using a discriminator.
PaletteNeRF: Palette-based Appearance Editing of Neural Radiance Fields
Recent advances in neural radiance fields have enabled the high-fidelity 3D reconstruction of complex scenes for novel view synthesis.
A Deep Perceptual Measure for Lens and Camera Calibration
Image editing and compositing have become ubiquitous in entertainment, from digital art to AR and VR experiences.
PhotoScene: Photorealistic Material and Lighting Transfer for Indoor Scenes
Most indoor 3D scene reconstruction methods focus on recovering 3D geometry and scene layout.
RigNeRF: Fully Controllable Neural 3D Portraits
In this work, we propose RigNeRF, a system that goes beyond just novel view synthesis and enables full control of head pose and facial expressions learned from a single portrait video.
TileGen: Tileable, Controllable Material Generation and Capture
The resulting materials are tileable, can be larger than the target image, and are editable by varying the condition.
Differentiable Rendering of Neural SDFs through Reparameterization
Our method leverages the distance to surface encoded in an SDF and uses quadrature on sphere tracer points to compute this warping function.
Physically-Based Editing of Indoor Scene Lighting from a Single Image
We tackle this problem using two novel components: 1) a holistic scene reconstruction method that estimates scene reflectance and parametric 3D lighting, and 2) a neural rendering framework that re-renders the scene from our predictions.
NeRFusion: Fusing Radiance Fields for Large-Scale Scene Reconstruction
We demonstrate that NeRFusion achieves state-of-the-art quality on both large-scale indoor and small-scale object scenes, with substantially faster reconstruction than NeRF and other recent methods.
Interactive Portrait Harmonization
To enable flexible interaction between user and harmonization, we introduce interactive harmonization, a new setting where the harmonization is performed with respect to a selected \emph{region} in the reference image instead of the entire background.
Point-NeRF: Point-based Neural Radiance Fields
Point-NeRF combines the advantages of these two approaches by using neural 3D point clouds, with associated neural features, to model a radiance field.
SSH: A Self-Supervised Framework for Image Harmonization
Image harmonization aims to improve the quality of image compositing by matching the "appearance" (\eg, color tone, brightness and contrast) between foreground and background images.
OpenRooms: An Open Framework for Photorealistic Indoor Scene Datasets
Finally, we demonstrate that our framework may also be integrated with physics engines, to create virtual robotics environments with unique ground truth such as friction coefficients and correspondence to real scenes.
NeuTex: Neural Texture Mapping for Volumetric Neural Rendering
We achieve this by introducing a 3D-to-2D texture mapping (or surface parameterization) network into volumetric representations.
Deep Denoising of Flash and No-Flash Pairs for Photography in Low-Light Environments
We introduce a neural network-based method to denoise pairs of images taken in quick succession, with and without a flash, in low-light environments.
MaterialGAN: Reflectance Capture using a Generative SVBRDF Model
We address the problem of reconstructing spatially-varying BRDFs from a small set of image measurements.
Neural Reflectance Fields for Appearance Acquisition
We combine this representation with a physically-based differentiable ray marching framework that can render images from a neural reflectance field under any viewpoint and light.
Deep Multi Depth Panoramas for View Synthesis
We propose a learning-based approach for novel view synthesis for multi-camera 360$^{\circ}$ panorama capture rigs.
OpenRooms: An End-to-End Open Framework for Photorealistic Indoor Scene Datasets
Finally, we demonstrate that our framework may also be integrated with physics engines, to create virtual robotics environments with unique ground truth such as friction coefficients and correspondence to real scenes.
Deep Reflectance Volumes: Relightable Reconstructions from Multi-View Photometric Images
We also show that our learned reflectance volumes are editable, allowing for modifying the materials of the captured scenes.
Single View Metrology in the Wild
Most 3D reconstruction methods may only recover scene properties up to a global scale ambiguity.
State of the Art on Neural Rendering
Neural rendering is a new and rapidly emerging field that combines generative machine learning techniques with physical knowledge from computer graphics, e. g., by the integration of differentiable rendering into network training.
Deep 3D Capture: Geometry and Reflectance from Sparse Multi-View Images
We introduce a novel learning-based method to reconstruct the high-quality geometry and complex, spatially-varying BRDF of an arbitrary object from a sparse set of only six images captured by wide-baseline cameras under collocated point lighting.
Deep CG2Real: Synthetic-to-Real Translation via Image Disentanglement
We present a method to improve the visual realism of low-quality, synthetic images, e. g. OpenGL renderings.
Basis Prediction Networks for Effective Burst Denoising with Large Kernels
Bursts of images exhibit significant self-similarity across both time and space.
Deep Parametric Indoor Lighting Estimation
We present a method to estimate lighting from a single image of an indoor scene.
All-Weather Deep Outdoor Lighting Estimation
We use this network to label a large-scale dataset of LDR panoramas with lighting parameters and use them to train our single image outdoor lighting estimation network.
Fast Spatially-Varying Indoor Lighting Estimation
We propose a real-time method to estimate spatiallyvarying indoor lighting from a single RGB image.
Inverse Rendering for Complex Indoor Scenes: Shape, Spatially-Varying Lighting and SVBRDF from a Single Image
Our inverse rendering network incorporates physical insights -- including a spatially-varying spherical Gaussian lighting representation, a differentiable rendering layer to model scene appearance, a cascade structure to iteratively refine the predictions and a bilateral solver for refinement -- allowing us to jointly reason about shape, lighting, and reflectance.
Learning to Separate Multiple Illuminants in a Single Image
We present a method to separate a single image captured under two illuminants, with different spectra, into the two images corresponding to the appearance of the scene under each individual illuminant.
Compositing-aware Image Search
We present a new image search technique that, given a background image, returns compatible foreground objects for image compositing tasks.
MT-VAE: Learning Motion Transformations to Generate Multimodal Human Dynamics
Our model jointly learns a feature embedding for motion modes (that the motion sequence can be reconstructed from) and a feature transformation that represents the transition of one motion mode to the next motion mode.
Ranked #7 on
Human Pose Forecasting
on Human3.6M
(ADE metric)
Fast Video Object Segmentation by Reference-Guided Mask Propagation
We validate our method on four benchmark sets that cover single and multiple object segmentation.
Self-supervised Multi-view Person Association and Its Applications
Reliable markerless motion tracking of people participating in a complex group activity from multiple moving cameras is challenging due to frequent occlusions, strong viewpoint and appearance variations, and asynchronous video streams.
Materials for Masses: SVBRDF Acquisition with a Single Mobile Phone Image
We propose a material acquisition approach to recover the spatially-varying BRDF and normal map of a near-planar surface from a single image captured by a handheld mobile phone camera.
A Perceptual Measure for Deep Single Image Camera Calibration
This network is trained using automatically generated samples from a large-scale panorama dataset, and considerably outperforms other methods, including recent deep learning-based approaches, in terms of standard L2 error.
Scene Parsing with Global Context Embedding
We present a scene parsing method that utilizes global context information based on both the parametric and non- parametric models.
Reflectance Capture Using Univariate Sampling of BRDFs
A collocated setup provides only a 1-D "univariate" sampling of the 4-D BRDF.
Illuminant Spectra-based Source Separation Using Flash Photography
Real-world lighting often consists of multiple illuminants with different spectra.
Neural Face Editing with Intrinsic Image Disentangling
Traditional face editing methods often require a number of sophisticated and task specific algorithms to be applied one after the other --- a process that is tedious, fragile, and computationally intensive.
Learning to Predict Indoor Illumination from a Single Image
We propose an automatic method to infer high dynamic range illumination from a single, limited field-of-view, low dynamic range photograph of an indoor scene.
Removing Shadows from Images of Documents
In this work, we automatically detect and remove distracting shadows from photographs of documents and other text-based items.
A Visual Representation for Editing Face Images
First, it allows a user to directly manipulate various illumination.
Deep Outdoor Illumination Estimation
We present a CNN-based technique to estimate high-dynamic range outdoor illumination from a single low dynamic range image.
Ranked #1 on
Outdoor Light Source Estimation
on SUN360
Single-image RGB Photometric Stereo With Spatially-varying Albedo
We present a single-shot system to recover surface geometry of objects with spatially-varying albedos, from images captured under a calibrated RGB photometric stereo setup---with three light directions multiplexed across different color channels in the observed RGB image.
Appearance Harmonization for Single Image Shadow Removal
In this work we propose a fully automatic shadow region harmonization approach that improves the appearance compatibility of the de-shadowed region as typically produced by previous methods.
PatchMatch-Based Automatic Lattice Detection for Near-Regular Textures
In this work, we investigate the problem of automatically inferring the lattice structure of near-regular textures (NRT) in real-world images.
Automatic Content-Aware Color and Tone Stylization
We introduce a new technique that automatically generates diverse, visually compelling stylizations for a photograph in an unsupervised manner.
