Search Results for author:
Found 64 papers, 21 papers with code
Object-Centric Domain Randomization for 3D Shape Reconstruction in the Wild
One of the biggest challenges in single-view 3D shape reconstruction in the wild is the scarcity of <3D shape, 2D image>-paired data from real-world environments.
Revisiting Learning-based Video Motion Magnification for Real-time Processing
Video motion magnification is a technique to capture and amplify subtle motion in a video that is invisible to the naked eye.
Noise Map Guidance: Inversion with Spatial Context for Real Image Editing
Text-guided diffusion models have become a popular tool in image synthesis, known for producing high-quality and diverse images.
FPRF: Feed-Forward Photorealistic Style Transfer of Large-Scale 3D Neural Radiance Fields
FPRF efficiently stylizes large-scale 3D scenes by introducing a style-decomposed 3D neural radiance field, which inherits AdaIN's feed-forward stylization machinery, supporting arbitrary style reference images.
Paint-it: Text-to-Texture Synthesis via Deep Convolutional Texture Map Optimization and Physically-Based Rendering
We present Paint-it, a text-driven high-fidelity texture map synthesis method for 3D meshes via neural re-parameterized texture optimization.
Learning-based Axial Video Motion Magnification
In this work, we focus on improving legibility by proposing a new concept, axial motion magnification, which magnifies decomposed motions along the user-specified direction.
SMILE: Multimodal Dataset for Understanding Laughter in Video with Language Models
We introduce this new task to explain why people laugh in a particular video and a dataset for this task.
LaughTalk: Expressive 3D Talking Head Generation with Laughter
Laughter is a unique expression, essential to affirmative social interactions of humans.
A Large-Scale 3D Face Mesh Video Dataset via Neural Re-parameterized Optimization
We propose NeuFace, a 3D face mesh pseudo annotation method on videos via neural re-parameterized optimization.
Sound Source Localization is All about Cross-Modal Alignment
However, prior arts and existing benchmarks do not account for a more important aspect of the problem, cross-modal semantic understanding, which is essential for genuine sound source localization.
An Iterative Method for Unsupervised Robust Anomaly Detection Under Data Contamination
Therefore, it has been a common practice to learn normality under the assumption that anomalous data are absent in a training dataset, which we call normality assumption.
The Devil in the Details: Simple and Effective Optical Flow Synthetic Data Generation
Recent work on dense optical flow has shown significant progress, primarily in a supervised learning manner requiring a large amount of labeled data.
SYNAuG: Exploiting Synthetic Data for Data Imbalance Problems
We live in an era of data floods, and deep neural networks play a pivotal role in this moment.
TextManiA: Enriching Visual Feature by Text-driven Manifold Augmentation
Given the hypothesis, TextManiA transfers pre-trained text representation obtained from a well-established large language encoder to a target visual feature space being learned.
Automatic Tuning of Loss Trade-offs without Hyper-parameter Search in End-to-End Zero-Shot Speech Synthesis
With our framework, we show superior performance compared to baselines in zero-shot TTS and VC, achieving state-of-the-art performance.
Prefix tuning for automated audio captioning
Audio captioning aims to generate text descriptions from environmental sounds.
Sound to Visual Scene Generation by Audio-to-Visual Latent Alignment
The key idea is to enrich the audio features with visual information by learning to align audio to visual latent space.
Unsupervised Pre-Training For Data-Efficient Text-to-Speech On Low Resource Languages
We empirically demonstrate the effectiveness of our proposed method in low-resource language scenarios, achieving outstanding performance compared to competing methods.
ENInst: Enhancing Weakly-supervised Low-shot Instance Segmentation
We address a weakly-supervised low-shot instance segmentation, an annotation-efficient training method to deal with novel classes effectively.
DFlow: Learning to Synthesize Better Optical Flow Datasets via a Differentiable Pipeline
Comprehensive studies of synthetic optical flow datasets have attempted to reveal what properties lead to accuracy improvement in learning-based optical flow estimation.
Computational Discovery of Microstructured Composites with Optimal Stiffness-Toughness Trade-Offs
The conflict between stiffness and toughness is a fundamental problem in engineering materials design.
Semi-Supervised Image Captioning by Adversarially Propagating Labeled Data
We present a novel data-efficient semi-supervised framework to improve the generalization of image captioning models.
Scratching Visual Transformer's Back with Uniform Attention
We observe that the inclusion of CB reduces the degree of density in the original attention maps and increases both the capacity and generalizability of the ViT models.
HDR-Plenoxels: Self-Calibrating High Dynamic Range Radiance Fields
Our voxel-based volume rendering pipeline reconstructs HDR radiance fields with only multi-view LDR images taken from varying camera settings in an end-to-end manner and has a fast convergence speed.
Cross-Attention of Disentangled Modalities for 3D Human Mesh Recovery with Transformers
Transformer encoder architectures have recently achieved state-of-the-art results on monocular 3D human mesh reconstruction, but they require a substantial number of parameters and expensive computations.
Ranked #3 on
3D Hand Pose Estimation
on FreiHAND
CLIP-Actor: Text-Driven Recommendation and Stylization for Animating Human Meshes
Then, our novel zero-shot neural style optimization detailizes and texturizes the recommended mesh sequence to conform to the prompt in a temporally-consistent and pose-agnostic manner.
Audio-Visual Fusion Layers for Event Type Aware Video Recognition
Human brain is continuously inundated with the multisensory information and their complex interactions coming from the outside world at any given moment.
Unified 3D Mesh Recovery of Humans and Animals by Learning Animal Exercise
To make the unstable disjoint multi-task learning jointly trainable, we propose to exploit the morphological similarity between humans and animals, motivated by animal exercise where humans imitate animal poses.
FICGAN: Facial Identity Controllable GAN for De-identification
In this work, we present Facial Identity Controllable GAN (FICGAN) for not only generating high-quality de-identified face images with ensured privacy protection, but also detailed controllability on attribute preservation for enhanced data utility.
FoxInst: A Frustratingly Simple Baseline for Weakly Few-shot Instance Segmentation
We propose the first weakly-supervised few-shot instance segmentation task and a frustratingly simple but strong baseline model, FoxInst.
FedPara: Low-Rank Hadamard Product for Communication-Efficient Federated Learning
We show that pFedPara outperforms competing personalized FL methods with more than three times fewer parameters.
CDS: Cross-Domain Self-Supervised Pre-Training
We present a two-stage pre-training approach that improves the generalization ability of standard single-domain pre-training.
Distilling Global and Local Logits With Densely Connected Relations
In prevalent knowledge distillation, logits in most image recognition models are computed by global average pooling, then used to learn to encode the high-level and task-relevant knowledge.
Ranked #31 on
Knowledge Distillation
on ImageNet
Dense Relational Image Captioning via Multi-task Triple-Stream Networks
To this end, we propose the multi-task triple-stream network (MTTSNet) which consists of three recurrent units responsible for each POS which is trained by jointly predicting the correct captions and POS for each word.
Monocular Reconstruction of Neural Face Reflectance Fields
The reflectance field of a face describes the reflectance properties responsible for complex lighting effects including diffuse, specular, inter-reflection and self shadowing.
Cross-domain Self-supervised Learning for Domain Adaptation with Few Source Labels
We show that when labeled source examples are limited, existing methods often fail to learn discriminative features applicable for both source and target domains.
Listen to Look: Action Recognition by Previewing Audio
In the face of the video data deluge, today's expensive clip-level classifiers are increasingly impractical.
Ranked #8 on
Action Recognition
on ActivityNet
Learning to Localize Sound Sources in Visual Scenes: Analysis and Applications
Visual events are usually accompanied by sounds in our daily lives.
Visuomotor Understanding for Representation Learning of Driving Scenes
We postulate that success on this task requires the network to learn semantic and geometric knowledge in the ego-centric view.
Image Captioning with Very Scarce Supervised Data: Adversarial Semi-Supervised Learning Approach
To this end, our proposed semi-supervised learning method assigns pseudo-labels to unpaired samples via Generative Adversarial Networks to learn the joint distribution of image and caption.
Speech2Face: Learning the Face Behind a Voice
How much can we infer about a person's looks from the way they speak?
Variational Prototyping-Encoder: One-Shot Learning with Prototypical Images
We take an approach to learn a generalizable embedding space for novel tasks.
Dense Relational Captioning: Triple-Stream Networks for Relationship-Based Captioning
Our goal in this work is to train an image captioning model that generates more dense and informative captions.
Neural Inverse Knitting: From Images to Manufacturing Instructions
Motivated by the recent potential of mass customization brought by whole-garment knitting machines, we introduce the new problem of automatic machine instruction generation using a single image of the desired physical product, which we apply to machine knitting.
Noise-tolerant Audio-visual Online Person Verification using an Attention-based Neural Network Fusion
In this paper, we present a multi-modal online person verification system using both speech and visual signals.
Globally Optimal Inlier Set Maximization for Atlanta Frame Estimation
In this work, we describe man-made structures via an appropriate structure assumption, called Atlanta world, which contains a vertical direction (typically the gravity direction) and a set of horizontal directions orthogonal to the vertical direction.
On Learning Associations of Faces and Voices
We computationally model the overlapping information between faces and voices and show that the learned cross-modal representation contains enough information to identify matching faces and voices with performance similar to that of humans.
Learning-based Video Motion Magnification
We show that the learned filters achieve high-quality results on real videos, with less ringing artifacts and better noise characteristics than previous methods.
Learning to Localize Sound Source in Visual Scenes
We show that even with a few supervision, false conclusion is able to be corrected and the source of sound in a visual scene can be localized effectively.
Disjoint Multi-task Learning between Heterogeneous Human-centric Tasks
Human behavior understanding is arguably one of the most important mid-level components in artificial intelligence.
Co-domain Embedding using Deep Quadruplet Networks for Unseen Traffic Sign Recognition
Recent advances in visual recognition show overarching success by virtue of large amounts of supervised data.
Gradient-based Camera Exposure Control for Outdoor Mobile Platforms
We introduce a novel method to automatically adjust camera exposure for image processing and computer vision applications on mobile robot platforms.
Personalized Cinemagraphs using Semantic Understanding and Collaborative Learning
Cinemagraphs are a compelling way to convey dynamic aspects of a scene.
Weakly- and Self-Supervised Learning for Content-Aware Deep Image Retargeting
Our method implicitly learns an attention map, which leads to a content-aware shift map for image retargeting.
Contextually Customized Video Summaries via Natural Language
Despite the challenging baselines, our method still manages to show comparable or even exceeding performance.
A Pseudo-Bayesian Algorithm for Robust PCA
Commonly used in many applications, robust PCA represents an algorithmic attempt to reduce the sensitivity of classical PCA to outliers.
Globally Optimal Manhattan Frame Estimation in Real-Time
Given a set of surface normals, we pose a Manhattan Frame (MF) estimation problem as a consensus set maximization that maximizes the number of inliers over the rotation search space.
Video-Story Composition via Plot Analysis
Inspired by plot analysis of written stories, our method generates a sequence of video clips ordered in such a way that it reflects plot dynamics and content coherency.
Robust and Globally Optimal Manhattan Frame Estimation in Near Real Time
Most man-made environments, such as urban and indoor scenes, consist of a set of parallel and orthogonal planar structures.
Human Attention Estimation for Natural Images: An Automatic Gaze Refinement Approach
Photo collections and its applications today attempt to reflect user interactions in various forms.
Pseudo-Bayesian Robust PCA: Algorithms and Analyses
Commonly used in computer vision and other applications, robust PCA represents an algorithmic attempt to reduce the sensitivity of classical PCA to outliers.
Fast Randomized Singular Value Thresholding for Low-rank Optimization
The problems related to NNM, or WNNM, can be solved iteratively by applying a closed-form proximal operator, called Singular Value Thresholding (SVT), or Weighted SVT, but they suffer from high computational cost of Singular Value Decomposition (SVD) at each iteration.
Fast Randomized Singular Value Thresholding for Nuclear Norm Minimization
The problems related to NNM (or WNNM) can be solved iteratively by applying a closed-form proximal operator, called Singular Value Thresholding (SVT) (or Weighted SVT), but they suffer from high computational cost to compute a Singular Value Decomposition (SVD) at each iteration.
Partial Sum Minimization of Singular Values in Robust PCA: Algorithm and Applications
Robust Principal Component Analysis (RPCA) via rank minimization is a powerful tool for recovering underlying low-rank structure of clean data corrupted with sparse noise/outliers.
