no code implementations • 8 May 2024 • Yaqi Wu, Zhihao Fan, Xiaofeng Chu, Jimmy S. Ren, Xiaoming Li, Zongsheng Yue, Chongyi Li, Shangcheng Zhou, Ruicheng Feng, Yuekun Dai, Peiqing Yang, Chen Change Loy, Senyan Xu, Zhijing Sun, Jiaying Zhu, Yurui Zhu, Xueyang Fu, Zheng-Jun Zha, Jun Cao, Cheng Li, Shu Chen, Liang Ma, Shiyang Zhou, Haijin Zeng, Kai Feng, Yongyong Chen, Jingyong Su, Xianyu Guan, Hongyuan Yu, Cheng Wan, Jiamin Lin, Binnan Han, Yajun Zou, Zhuoyuan Wu, Yuan Huang, Yongsheng Yu, Daoan Zhang, Jizhe Li, Xuanwu Yin, Kunlong Zuo, Yunfan Lu, Yijie Xu, Wenzong Ma, Weiyu Guo, Hui Xiong, Wei Yu, Bingchun Luo, Sabari Nathan, Priya Kansal
The increasing demand for computational photography and imaging on mobile platforms has led to the widespread development and integration of advanced image sensors with novel algorithms in camera systems.
no code implementations • 5 May 2024 • Honghua Chen, Chen Change Loy, Xingang Pan
Despite the emergence of successful NeRF inpainting methods built upon explicit RGB and depth 2D inpainting supervisions, these methods are inherently constrained by the capabilities of their underlying 2D inpainters.
no code implementations • 30 Apr 2024 • Yuekun Dai, Dafeng Zhang, Xiaoming Li, Zongsheng Yue, Chongyi Li, Shangchen Zhou, Ruicheng Feng, Peiqing Yang, Zhezhu Jin, Guanqun Liu, Chen Change Loy, Lize Zhang, Shuai Liu, Chaoyu Feng, Luyang Wang, Shuan Chen, Guangqi Shao, Xiaotao Wang, Lei Lei, Qirui Yang, Qihua Cheng, Zhiqiang Xu, Yihao Liu, Huanjing Yue, Jingyu Yang, Florin-Alexandru Vasluianu, Zongwei Wu, George Ciubotariu, Radu Timofte, Zhao Zhang, Suiyi Zhao, Bo wang, Zhichao Zuo, Yanyan Wei, Kuppa Sai Sri Teja, Jayakar Reddy A, Girish Rongali, Kaushik Mitra, Zhihao Ma, Yongxu Liu, Wanying Zhang, Wei Shang, Yuhong He, Long Peng, Zhongxin Yu, Shaofei Luo, Jian Wang, Yuqi Miao, Baiang Li, Gang Wei, Rakshank Verma, Ritik Maheshwari, Rahul Tekchandani, Praful Hambarde, Satya Narayan Tazi, Santosh Kumar Vipparthi, Subrahmanyam Murala, Haopeng Zhang, Yingli Hou, Mingde Yao, Levin M S, Aniruth Sundararajan, Hari Kumar A
The increasing demand for computational photography and imaging on mobile platforms has led to the widespread development and integration of advanced image sensors with novel algorithms in camera systems.
1 code implementation • 18 Apr 2024 • Mengyuan Liu, Zhongbin Fang, Xia Li, Joachim M. Buhmann, Xiangtai Li, Chen Change Loy
With the emergence of large-scale models trained on diverse datasets, in-context learning has emerged as a promising paradigm for multitasking, notably in natural language processing and image processing.
no code implementations • 16 Apr 2024 • Kang Liao, Zongsheng Yue, Zhonghua Wu, Chen Change Loy
To our knowledge, this is the first work that solves multiple practical warping tasks in one single model.
1 code implementation • 27 Mar 2024 • Yuekun Dai, Shangchen Zhou, Qinyue Li, Chongyi Li, Chen Change Loy
In this work, we introduce a new learning-based inclusion matching pipeline, which directs the network to comprehend the inclusion relationships between segments rather than relying solely on direct visual correspondences.
no code implementations • 27 Mar 2024 • Li SiYao, Tianpei Gu, Zhitao Yang, Zhengyu Lin, Ziwei Liu, Henghui Ding, Lei Yang, Chen Change Loy
We introduce a novel task within the field of 3D dance generation, termed dance accompaniment, which necessitates the generation of responsive movements from a dance partner, the "follower", synchronized with the lead dancer's movements and the underlying musical rhythm.
2 code implementations • 19 Mar 2024 • Shuai Yang, Yifan Zhou, Ziwei Liu, Chen Change Loy
In this paper, we introduce FRESCO, intra-frame correspondence alongside inter-frame correspondence to establish a more robust spatial-temporal constraint.
no code implementations • 18 Mar 2024 • Yushi Lan, Fangzhou Hong, Shuai Yang, Shangchen Zhou, Xuyi Meng, Bo Dai, Xingang Pan, Chen Change Loy
The latent is decoded by a transformer-based decoder into a high-capacity 3D neural field.
no code implementations • 14 Mar 2024 • Chaoyang Wang, Xiangtai Li, Henghui Ding, Lu Qi, Jiangning Zhang, Yunhai Tong, Chen Change Loy, Shuicheng Yan
In-context segmentation has drawn more attention with the introduction of vision foundation models.
1 code implementation • 12 Mar 2024 • Zongsheng Yue, Jianyi Wang, Chen Change Loy
While diffusion-based image restoration (IR) methods have achieved remarkable success, they are still limited by the low inference speed attributed to the necessity of executing hundreds or even thousands of sampling steps.
1 code implementation • 3 Mar 2024 • Kun-Yu Lin, Henghui Ding, Jiaming Zhou, Yi-Xing Peng, Zhilin Zhao, Chen Change Loy, Wei-Shi Zheng
To answer this, we establish a CROSS-domain Open-Vocabulary Action recognition benchmark named XOV-Action, and conduct a comprehensive evaluation of five state-of-the-art CLIP-based video learners under various types of domain gaps.
no code implementations • 16 Feb 2024 • Zhexin Liang, Zhaochen Li, Shangchen Zhou, Chongyi Li, Chen Change Loy
We also introduce a novel module based on self-attention and a content-guided deformable autoencoder to address the long-standing issues of color overflow and inaccurate coloring.
1 code implementation • 18 Jan 2024 • Xiangtai Li, Haobo Yuan, Wei Li, Henghui Ding, Size Wu, Wenwei Zhang, Yining Li, Kai Chen, Chen Change Loy
In this work, we address various segmentation tasks, each traditionally tackled by distinct or partially unified models.
no code implementations • 18 Jan 2024 • Jianzong Wu, Xiangtai Li, Chenyang Si, Shangchen Zhou, Jingkang Yang, Jiangning Zhang, Yining Li, Kai Chen, Yunhai Tong, Ziwei Liu, Chen Change Loy
We introduce a new task -- language-driven video inpainting, which uses natural language instructions to guide the inpainting process.
1 code implementation • 5 Jan 2024 • Haobo Yuan, Xiangtai Li, Chong Zhou, Yining Li, Kai Chen, Chen Change Loy
The CLIP and Segment Anything Model (SAM) are remarkable vision foundation models (VFMs).
1 code implementation • 18 Dec 2023 • Size Wu, Wenwei Zhang, Lumin Xu, Sheng Jin, Wentao Liu, Chen Change Loy
Our experimental results demonstrate that CLIM improves different baseline open-vocabulary object detectors by a large margin on both OV-COCO and OV-LVIS benchmarks.
Ranked #6 on Open Vocabulary Object Detection on LVIS v1.0
no code implementations • 11 Dec 2023 • Shangchen Zhou, Peiqing Yang, Jianyi Wang, Yihang Luo, Chen Change Loy
Text-based diffusion models have exhibited remarkable success in generation and editing, showing great promise for enhancing visual content with their generative prior.
1 code implementation • 11 Dec 2023 • Chong Zhou, Xiangtai Li, Chen Change Loy, Bo Dai
It is also the first SAM variant that can run at over 30 FPS on an iPhone 14.
no code implementations • 7 Dec 2023 • Zhongang Cai, Jianping Jiang, Zhongfei Qing, Xinying Guo, Mingyuan Zhang, Zhengyu Lin, Haiyi Mei, Chen Wei, Ruisi Wang, Wanqi Yin, Xiangyu Fan, Han Du, Liang Pan, Peng Gao, Zhitao Yang, Yang Gao, Jiaqi Li, Tianxiang Ren, Yukun Wei, Xiaogang Wang, Chen Change Loy, Lei Yang, Ziwei Liu
In this work, we present Digital Life Project, a framework utilizing language as the universal medium to build autonomous 3D characters, who are capable of engaging in social interactions and expressing with articulated body motions, thereby simulating life in a digital environment.
Ranked #2 on Motion Synthesis on InterHuman
no code implementations • 5 Dec 2023 • Yushi Lan, Feitong Tan, Di Qiu, Qiangeng Xu, Kyle Genova, Zeng Huang, Sean Fanello, Rohit Pandey, Thomas Funkhouser, Chen Change Loy, yinda zhang
We present a novel framework for generating photorealistic 3D human head and subsequently manipulating and reposing them with remarkable flexibility.
no code implementations • 1 Dec 2023 • Yuming Jiang, Tianxing Wu, Shuai Yang, Chenyang Si, Dahua Lin, Yu Qiao, Chen Change Loy, Ziwei Liu
In this paper, we study the task of video generation with image prompts, which provide more accurate and direct content control beyond the text prompts.
1 code implementation • 29 Nov 2023 • Xiaoming Li, Xinyu Hou, Chen Change Loy
Text-to-image diffusion models have remarkably excelled in producing diverse, high-quality, and photo-realistic images.
3 code implementations • CVPR 2023 • Jingkang Yang, Wenxuan Peng, Xiangtai Li, Zujin Guo, Liangyu Chen, Bo Li, Zheng Ma, Kaiyang Zhou, Wayne Zhang, Chen Change Loy, Ziwei Liu
PVSG relates to the existing video scene graph generation (VidSGG) problem, which focuses on temporal interactions between humans and objects grounded with bounding boxes in videos.
1 code implementation • 25 Oct 2023 • Guangcong Wang, Peng Wang, Zhaoxi Chen, Wenping Wang, Chen Change Loy, Ziwei Liu
In this paper, we present PERF, a 360-degree novel view synthesis framework that trains a panoramic neural radiance field from a single panorama.
no code implementations • 14 Oct 2023 • Jianhui Yu, Hao Zhu, Liming Jiang, Chen Change Loy, Weidong Cai, Wayne Wu
We first propose a novel score function, Denoised Score Distillation (DSD), which directly modifies the SDS by introducing negative gradient components to iteratively correct the gradient direction and generate high-quality textures.
1 code implementation • 2 Oct 2023 • Shilin Xu, Xiangtai Li, Size Wu, Wenwei Zhang, Yunhai Tong, Chen Change Loy
We refer to this approach as the self-training strategy, which enhances recall and accuracy for novel classes without requiring extra annotations, datasets, and re-training.
1 code implementation • 2 Oct 2023 • Size Wu, Wenwei Zhang, Lumin Xu, Sheng Jin, Xiangtai Li, Wentao Liu, Chen Change Loy
However, when transferring the vision-language alignment of CLIP from global image representation to local region representation for the open-vocabulary dense prediction tasks, CLIP ViTs suffer from the domain shift from full images to local image regions.
1 code implementation • ICCV 2023 • Li SiYao, Tianpei Gu, Weiye Xiao, Henghui Ding, Ziwei Liu, Chen Change Loy
To preserve the precision and detail of the line drawings, we propose a new approach, AnimeInbet, which geometrizes raster line drawings into graphs of endpoints and reframes the inbetweening task as a graph fusion problem with vertex repositioning.
2 code implementations • 26 Sep 2023 • Yaohui Wang, Xinyuan Chen, Xin Ma, Shangchen Zhou, Ziqi Huang, Yi Wang, Ceyuan Yang, Yinan He, Jiashuo Yu, Peiqing Yang, Yuwei Guo, Tianxing Wu, Chenyang Si, Yuming Jiang, Cunjian Chen, Chen Change Loy, Bo Dai, Dahua Lin, Yu Qiao, Ziwei Liu
To this end, we propose LaVie, an integrated video generation framework that operates on cascaded video latent diffusion models, comprising a base T2V model, a temporal interpolation model, and a video super-resolution model.
Ranked #4 on Text-to-Video Generation on EvalCrafter Text-to-Video (ECTV) Dataset (using extra training data)
1 code implementation • 22 Sep 2023 • Jiahao Xie, Wei Li, Xiangtai Li, Ziwei Liu, Yew Soon Ong, Chen Change Loy
We present MosaicFusion, a simple yet effective diffusion-based data augmentation approach for large vocabulary instance segmentation.
no code implementations • 19 Sep 2023 • Chong Zhou, Chen Change Loy, Bo Dai
There has been a debate about the superiority between vision Transformers and ConvNets, serving as the backbone of computer vision models.
1 code implementation • NeurIPS 2023 • Peiqing Yang, Shangchen Zhou, Qingyi Tao, Chen Change Loy
When combined with a diffusion prior, this partial guidance can deliver appealing results across a range of restoration tasks.
1 code implementation • 8 Sep 2023 • Junzhe Zhang, Yushi Lan, Shuai Yang, Fangzhou Hong, Quan Wang, Chai Kiat Yeo, Ziwei Liu, Chen Change Loy
In this paper, we address the challenging problem of 3D toonification, which involves transferring the style of an artistic domain onto a target 3D face with stylized geometry and texture.
1 code implementation • ICCV 2023 • Shangchen Zhou, Chongyi Li, Kelvin C. K. Chan, Chen Change Loy
We also propose a mask-guided sparse video Transformer, which achieves high efficiency by discarding unnecessary and redundant tokens.
Ranked #1 on Video Inpainting on YouTube-VOS 2018
no code implementations • 31 Aug 2023 • Linsen Song, Wayne Wu, Chaoyou Fu, Chen Change Loy, Ran He
Existing automated dubbing methods are usually designed for Professionally Generated Content (PGC) production, which requires massive training data and training time to learn a person-specific audio-video mapping.
1 code implementation • ICCV 2023 • YuHan Wang, Liming Jiang, Chen Change Loy
In this paper, we introduce a novel motion generator design that uses a learning-based inversion network for GAN.
no code implementations • 28 Aug 2023 • Zhongang Cai, Liang Pan, Chen Wei, Wanqi Yin, Fangzhou Hong, Mingyuan Zhang, Chen Change Loy, Lei Yang, Ziwei Liu
To tackle these challenges, we propose a principled framework, PointHPS, for accurate 3D HPS from point clouds captured in real-world settings, which iteratively refines point features through a cascaded architecture.
1 code implementation • ICCV 2023 • Yuxin Jiang, Liming Jiang, Shuai Yang, Chen Change Loy
The challenges of this task lie in the complexity of the scenes, the unique features of anime style, and the lack of high-quality datasets to bridge the domain gap.
1 code implementation • ICCV 2023 • Henghui Ding, Chang Liu, Shuting He, Xudong Jiang, Chen Change Loy
To investigate the feasibility of using motion expressions to ground and segment objects in videos, we propose a large-scale dataset called MeViS, which contains numerous motion expressions to indicate target objects in complex environments.
Ranked #2 on Referring Video Object Segmentation on MeViS
1 code implementation • NeurIPS 2023 • Zongsheng Yue, Jianyi Wang, Chen Change Loy
Diffusion-based image super-resolution (SR) methods are mainly limited by the low inference speed due to the requirements of hundreds or even thousands of sampling steps.
1 code implementation • ICCV 2023 • Wei Cheng, Ruixiang Chen, Wanqi Yin, Siming Fan, Keyu Chen, Honglin He, Huiwen Luo, Zhongang Cai, Jingbo Wang, Yang Gao, Zhengming Yu, Zhengyu Lin, Daxuan Ren, Lei Yang, Ziwei Liu, Chen Change Loy, Chen Qian, Wayne Wu, Dahua Lin, Bo Dai, Kwan-Yee Lin
Realistic human-centric rendering plays a key role in both computer vision and computer graphics.
no code implementations • 25 Jun 2023 • Haoying Li, Jixin Zhao, Shangchen Zhou, Huajun Feng, Chongyi Li, Chen Change Loy
Existing image deblurring methods predominantly focus on global deblurring, inadvertently affecting the sharpness of backgrounds in locally blurred images and wasting unnecessary computation on sharp pixels, especially for high-resolution images.
2 code implementations • NeurIPS 2023 • Zhongbin Fang, Xiangtai Li, Xia Li, Joachim M. Buhmann, Chen Change Loy, Mengyuan Liu
With the rise of large-scale models trained on broad data, in-context learning has become a new learning paradigm that has demonstrated significant potential in natural language processing and computer vision tasks.
no code implementations • 13 Jun 2023 • Shuai Yang, Yifan Zhou, Ziwei Liu, Chen Change Loy
The framework includes two parts: key frame translation and full video translation.
1 code implementation • 7 Jun 2023 • Shuai Yang, Liming Jiang, Ziwei Liu, Chen Change Loy
In this paper, we introduce a novel versatile framework, Generative Prior-guided UNsupervised Image-to-image Translation (GP-UNIT), that improves the quality, applicability and controllability of the existing translation models.
1 code implementation • 7 Jun 2023 • Yuekun Dai, Chongyi Li, Shangchen Zhou, Ruicheng Feng, Yihang Luo, Chen Change Loy
To address this issue, we additionally provide the annotations of light sources in Flare7K++ and propose a new end-to-end pipeline to preserve the light source while removing lens flares.
1 code implementation • 29 May 2023 • Yuhang Zang, Wei Li, Jun Han, Kaiyang Zhou, Chen Change Loy
Moreover, we present ContextDET, a unified multimodal model that is capable of end-to-end differentiable modeling of visual-language contexts, so as to locate, identify, and associate visual objects with language inputs for human-AI interaction.
no code implementations • 24 May 2023 • Yuhang Zang, Kaiyang Zhou, Chen Huang, Chen Change Loy
This paper focuses on long-tailed object detection in the semi-supervised learning setting, which poses realistic challenges, but has rarely been studied in the literature.
no code implementations • 23 May 2023 • Yuekun Dai, Chongyi Li, Shangchen Zhou, Ruicheng Feng, Qingpeng Zhu, Qianhui Sun, Wenxiu Sun, Chen Change Loy, Jinwei Gu
In this paper, we summarize and review the Nighttime Flare Removal track on MIPI 2023.
1 code implementation • NeurIPS 2023 • Dongwei Pan, Long Zhuo, Jingtan Piao, Huiwen Luo, Wei Cheng, Yuxin Wang, Siming Fan, Shengqi Liu, Lei Yang, Bo Dai, Ziwei Liu, Chen Change Loy, Chen Qian, Wayne Wu, Dahua Lin, Kwan-Yee Lin
It is a large-scale digital library for head avatars with three key attributes: 1) High Fidelity: all subjects are captured by 60 synchronized, high-resolution 2K cameras in 360 degrees.
no code implementations • ICCV 2023 • Yidi Shao, Chen Change Loy, Bo Dai
In this paper, we propose a novel data-driven method, called LayersNet, to model garment-level animations as particle-wise interactions in a micro physics system.
3 code implementations • 11 May 2023 • Jianyi Wang, Zongsheng Yue, Shangchen Zhou, Kelvin C. K. Chan, Chen Change Loy
We present a novel approach to leverage prior knowledge encapsulated in pre-trained text-to-image diffusion models for blind super-resolution (SR).
no code implementations • 27 Apr 2023 • Qingpeng Zhu, Wenxiu Sun, Yuekun Dai, Chongyi Li, Shangchen Zhou, Ruicheng Feng, Qianhui Sun, Chen Change Loy, Jinwei Gu, Yi Yu, Yangke Huang, Kang Zhang, Meiya Chen, Yu Wang, Yongchao Li, Hao Jiang, Amrit Kumar Muduli, Vikash Kumar, Kunal Swami, Pankaj Kumar Bajpai, Yunchao Ma, Jiajun Xiao, Zhi Ling
To evaluate the performance of different depth completion methods, we organized an RGB+sparse ToF depth completion competition.
no code implementations • 20 Apr 2023 • Qianhui Sun, Qingyu Yang, Chongyi Li, Shangchen Zhou, Ruicheng Feng, Yuekun Dai, Wenxiu Sun, Qingpeng Zhu, Chen Change Loy, Jinwei Gu
Developing and integrating advanced image sensors with novel algorithms in camera systems are prevalent with the increasing demand for computational photography and imaging on mobile platforms.
no code implementations • 20 Apr 2023 • Qianhui Sun, Qingyu Yang, Chongyi Li, Shangchen Zhou, Ruicheng Feng, Yuekun Dai, Wenxiu Sun, Qingpeng Zhu, Chen Change Loy, Jinwei Gu
Developing and integrating advanced image sensors with novel algorithms in camera systems are prevalent with the increasing demand for computational photography and imaging on mobile platforms.
2 code implementations • 19 Apr 2023 • Xiangtai Li, Henghui Ding, Haobo Yuan, Wenwei Zhang, Jiangmiao Pang, Guangliang Cheng, Kai Chen, Ziwei Liu, Chen Change Loy
Recently, transformers, a type of neural network based on self-attention originally designed for natural language processing, have considerably surpassed previous convolutional or recurrent approaches in various vision processing tasks.
1 code implementation • ICCV 2023 • Yuming Jiang, Shuai Yang, Tong Liang Koh, Wayne Wu, Chen Change Loy, Ziwei Liu
In this work, we present Text2Performer to generate vivid human videos with articulated motions from texts.
1 code implementation • CVPR 2023 • Ruicheng Feng, Chongyi Li, Huaijin Chen, Shuai Li, Jinwei Gu, Chen Change Loy
Due to the difficulty in collecting large-scale and perfectly aligned paired training data for Under-Display Camera (UDC) image restoration, previous methods resort to monitor-based image systems or simulation-based methods, sacrificing the realness of the data and introducing domain gaps.
1 code implementation • CVPR 2023 • Zeren Chen, Gengshi Huang, Wei Li, Jianing Teng, Kun Wang, Jing Shao, Chen Change Loy, Lu Sheng
In this work, we present Siamese DETR, a Siamese self-supervised pretraining approach for the Transformer architecture in DETR.
no code implementations • ICCV 2023 • Zhexin Liang, Chongyi Li, Shangchen Zhou, Ruicheng Feng, Chen Change Loy
To solve this issue, we devise a prompt learning framework that first learns an initial prompt pair by constraining the text-image similarity between the prompt (negative/positive sample) and the corresponding image (backlit image/well-lit image) in the CLIP latent space.
no code implementations • ICCV 2023 • Guangcong Wang, Zhaoxi Chen, Chen Change Loy, Ziwei Liu
Since coarse depth maps are not strictly scaled to the ground-truth depth maps, we propose a simple yet effective constraint, a local depth ranking method, on NeRFs such that the expected depth ranking of the NeRF is consistent with that of the coarse depth maps in local patches.
1 code implementation • CVPR 2023 • Yuekun Dai, Yihang Luo, Shangchen Zhou, Chongyi Li, Chen Change Loy
With the dataset, neural networks can be trained to remove the reflective flares effectively.
1 code implementation • CVPR 2023 • Xiaoming Li, WangMeng Zuo, Chen Change Loy
To restrict the generative space of StyleGAN so that it obeys the structure of characters yet remains flexible in handling different font styles, we store the discrete features for each character in a codebook.
1 code implementation • CVPR 2023 • Jianhui Yu, Hao Zhu, Liming Jiang, Chen Change Loy, Weidong Cai, Wayne Wu
This paper presents CelebV-Text, a large-scale, diverse, and high-quality dataset of facial text-video pairs, to facilitate research on facial text-to-video generation tasks.
1 code implementation • 23 Mar 2023 • Zeqi Xiao, Wenwei Zhang, Tai Wang, Chen Change Loy, Dahua Lin, Jiangmiao Pang
DEtection TRansformer (DETR) started a trend that uses a group of learnable queries for unified visual perception.
Ranked #1 on Panoptic Segmentation on SemanticKITTI
no code implementations • 23 Mar 2023 • Quanzhou Li, Jingbo Wang, Chen Change Loy, Bo Dai
Generating task-oriented human-object interaction motions in simulation is challenging.
2 code implementations • ICCV 2023 • Xiangtai Li, Haobo Yuan, Wenwei Zhang, Guangliang Cheng, Jiangmiao Pang, Chen Change Loy
Our framework is a near-online approach that takes a short subclip as input and outputs the corresponding spatial-temporal tube masks.
Ranked #3 on Video Semantic Segmentation on VSPW
1 code implementation • CVPR 2023 • Wei Li, Jiahao Xie, Chen Change Loy
We introduce Correlational Image Modeling (CIM), a novel and surprisingly effective approach to self-supervised visual pre-training.
1 code implementation • ICCV 2023 • Shuai Yang, Liming Jiang, Ziwei Liu, Chen Change Loy
Recent advances in face manipulation using StyleGAN have produced impressive results.
1 code implementation • CVPR 2023 • Size Wu, Wenwei Zhang, Sheng Jin, Wentao Liu, Chen Change Loy
The embeddings of regions in a bag are treated as embeddings of words in a sentence, and they are sent to the text encoder of a VLM to obtain the bag-of-regions embedding, which is learned to be aligned to the corresponding features extracted by a frozen VLM.
Ranked #7 on Open Vocabulary Object Detection on MSCOCO (using extra training data)
1 code implementation • 23 Feb 2023 • Chongyi Li, Chun-Le Guo, Man Zhou, Zhexin Liang, Shangchen Zhou, Ruicheng Feng, Chen Change Loy
Our approach is motivated by a few unique characteristics in the Fourier domain: 1) most luminance information concentrates on amplitudes while noise is closely related to phases, and 2) a high-resolution image and its low-resolution version share similar amplitude patterns. Through embedding Fourier into our network, the amplitude and phase of a low-light image are separately processed to avoid amplifying noise when enhancing luminance.
2 code implementations • ICCV 2023 • Jianzong Wu, Xiangtai Li, Henghui Ding, Xia Li, Guangliang Cheng, Yunhai Tong, Chen Change Loy
Experiments on the COCO dataset with two settings: Open Vocabulary Instance Segmentation (OVIS) and Open Set Panoptic Segmentation (OSPS) demonstrate the superiority of the CGG.
no code implementations • ICCV 2023 • Junzhe Zhang, Yushi Lan, Shuai Yang, Fangzhou Hong, Quan Wang, Chai Kiat Yeo, Ziwei Liu, Chen Change Loy
In this paper, we address the challenging problem of 3D toonification, which involves transferring the style of an artistic domain onto a target 3D face with stylized geometry and texture.
1 code implementation • 19 Dec 2022 • Yuming Jiang, Kelvin C. K. Chan, Xintao Wang, Chen Change Loy, Ziwei Liu
To tackle these challenges, we propose C2-Matching in this work, which performs explicit robust matching crossing transformation and resolution.
no code implementations • 19 Dec 2022 • Yushi Lan, Chen Change Loy, Bo Dai
The neural radiance field (NeRF) has shown promising results in preserving the fine details of objects and scenes.
no code implementations • CVPR 2023 • Yushi Lan, Xuyi Meng, Shuai Yang, Chen Change Loy, Bo Dai
In this paper, we study the challenging problem of 3D GAN inversion where a latent code is predicted given a single face image to faithfully recover its 3D shapes and detailed textures.
2 code implementations • 13 Dec 2022 • Zongsheng Yue, Chen Change Loy
Moreover, the transition distribution can contract the error of the restoration backbone and thus makes our method more robust to unknown degradations.
Ranked #5 on Blind Face Restoration on CelebA-Test
no code implementations • 12 Dec 2022 • Qixin Yan, Chunle Guo, Jixin Zhao, Yuekun Dai, Chen Change Loy, Chongyi Li
The key insights of this study are modeling component-specific correspondence for local makeup transfer, capturing long-range dependencies for global makeup transfer, and enabling efficient makeup transfer via a single-path structure.
1 code implementation • 10 Nov 2022 • Li SiYao, Yuhang Li, Bo Li, Chao Dong, Ziwei Liu, Chen Change Loy
Existing correspondence datasets for two-dimensional (2D) cartoon suffer from simple frame composition and monotonic movements, making them insufficient to simulate real animations.
1 code implementation • 13 Oct 2022 • Yuhang Zang, Wei Li, Kaiyang Zhou, Chen Huang, Chen Change Loy
Prompt tuning, a parameter- and data-efficient transfer learning paradigm that tunes only a small number of parameters in a model's input space, has become a trend in the vision community since the emergence of large vision-language models like CLIP.
1 code implementation • 12 Oct 2022 • Yuekun Dai, Chongyi Li, Shangchen Zhou, Ruicheng Feng, Chen Change Loy
In this paper, we introduce, Flare7K, the first nighttime flare removal dataset, which is generated based on the observation and statistics of real-world nighttime lens flares.
Ranked #2 on Flare Removal on Flare7K
1 code implementation • 11 Oct 2022 • Man Zhou, Hu Yu, Jie Huang, Feng Zhao, Jinwei Gu, Chen Change Loy, Deyu Meng, Chongyi Li
Existing convolutional neural networks widely adopt spatial down-/up-sampling for multi-scale modeling.
1 code implementation • 22 Sep 2022 • Shuai Yang, Liming Jiang, Ziwei Liu, Chen Change Loy
Although a series of successful portrait image toonification models built upon the powerful StyleGAN have been proposed, these image-oriented methods have obvious limitations when applied to videos, such as the fixed frame size, the requirement of face alignment, missing non-facial details and temporal inconsistency.
1 code implementation • 15 Sep 2022 • Qingyu Yang, Guang Yang, Jun Jiang, Chongyi Li, Ruicheng Feng, Shangchen Zhou, Wenxiu Sun, Qingpeng Zhu, Chen Change Loy, Jinwei Gu
A detailed description of all models developed in this challenge is provided in this paper.
1 code implementation • 15 Sep 2022 • Qingyu Yang, Guang Yang, Jun Jiang, Chongyi Li, Ruicheng Feng, Shangchen Zhou, Wenxiu Sun, Qingpeng Zhu, Chen Change Loy, Jinwei Gu
A detailed description of all models developed in this challenge is provided in this paper.
2 code implementations • 15 Sep 2022 • Kaiyang Zhou, Yuanhan Zhang, Yuhang Zang, Jingkang Yang, Chen Change Loy, Ziwei Liu
Another interesting observation is that the teacher-student gap on out-of-distribution data is bigger than that on in-distribution data, which highlights the capacity mismatch issue as well as the shortcoming of KD.
1 code implementation • 15 Sep 2022 • Qingyu Yang, Guang Yang, Jun Jiang, Chongyi Li, Ruicheng Feng, Shangchen Zhou, Wenxiu Sun, Qingpeng Zhu, Chen Change Loy, Jinwei Gu
A detailed description of all models developed in this challenge is provided in this paper.
1 code implementation • 15 Sep 2022 • Ruicheng Feng, Chongyi Li, Shangchen Zhou, Wenxiu Sun, Qingpeng Zhu, Jun Jiang, Qingyu Yang, Chen Change Loy, Jinwei Gu
In this paper, we summarize and review the Under-Display Camera (UDC) Image Restoration track on MIPI 2022.
1 code implementation • 15 Sep 2022 • Wenxiu Sun, Qingpeng Zhu, Chongyi Li, Ruicheng Feng, Shangchen Zhou, Jun Jiang, Qingyu Yang, Chen Change Loy, Jinwei Gu
A detailed description of all models developed in this challenge is provided in this paper.
1 code implementation • 18 Aug 2022 • Guodong Xu, Yuenan Hou, Ziwei Liu, Chen Change Loy
To further enhance the semantic consistency between the teacher and student model, we present a latent-direction-based distillation loss that preserves the semantic relations in latent space.
1 code implementation • 29 Jul 2022 • Kelvin C. K. Chan, Xiangyu Xu, Xintao Wang, Jinwei Gu, Chen Change Loy
While most existing perceptual-oriented approaches attempt to generate realistic outputs through learning with adversarial loss, our method, Generative LatEnt bANk (GLEAN), goes beyond existing practices by directly leveraging rich and diverse priors encapsulated in a pre-trained GAN.
1 code implementation • 29 Jul 2022 • Guangcong Wang, Yinuo Yang, Chen Change Loy, Ziwei Liu
To tackle this problem, we propose a coupled dual-StyleGAN panorama synthesis network (StyleLight) that integrates LDR and HDR panorama synthesis into a unified framework.
no code implementations • 28 Jul 2022 • Chongyi Li, Chunle Guo, Ruicheng Feng, Shangchen Zhou, Chen Change Loy
Our method inherits the zero-reference learning and curve-based framework from an effective low-light image enhancement method, Zero-DCE, with further speed up in its inference speed, reduction in its model size, and extension to controllable exposure adjustment.
1 code implementation • 25 Jul 2022 • Hao Zhu, Wayne Wu, Wentao Zhu, Liming Jiang, Siwei Tang, Li Zhang, Ziwei Liu, Chen Change Loy
Large-scale datasets have played indispensable roles in the recent success of face generation/editing and significantly facilitated the advances of emerging research fields.
Ranked #1 on Unconditional Video Generation on CelebV-HQ
1 code implementation • 25 Jul 2022 • Jianyi Wang, Kelvin C. K. Chan, Chen Change Loy
Measuring the perception of visual content is a long-standing problem in computer vision.
Ranked #9 on Video Quality Assessment on MSU SR-QA Dataset
1 code implementation • 22 Jul 2022 • Yidi Shao, Chen Change Loy, Bo Dai
Consequently, in this paper we propose a novel Transformer-based method, dubbed as Transformer with Implicit Edges (TIE), to capture the rich semantics of particle interactions in an edge-free manner.
1 code implementation • 20 Jul 2022 • Junzhe Zhang, Daxuan Ren, Zhongang Cai, Chai Kiat Yeo, Bo Dai, Chen Change Loy
Reconstruction is achieved by searching for a latent space in the 3D GAN that best resembles the target mesh in accordance with the single view observation.
1 code implementation • 20 Jul 2022 • Davide Moltisanti, Jinyi Wu, Bo Dai, Chen Change Loy
Estimating human keypoints from these videos is difficult due to the complexity of the dance, as well as the multiple moving cameras recording setup.
1 code implementation • 22 Jun 2022 • Shangchen Zhou, Kelvin C. K. Chan, Chongyi Li, Chen Change Loy
In this paper, we demonstrate that a learned discrete codebook prior in a small proxy space largely reduces the uncertainty and ambiguity of restoration mapping by casting blind face restoration as a code prediction task, while providing rich visual atoms for generating high-quality faces.
Ranked #1 on Blind Face Restoration on CelebA-Test
3 code implementations • 15 Jun 2022 • Jiahao Xie, Wei Li, Xiaohang Zhan, Ziwei Liu, Yew Soon Ong, Chen Change Loy
We present Masked Frequency Modeling (MFM), a unified frequency-domain-based approach for self-supervised pre-training of visual models.
no code implementations • CVPR 2022 • Yuenan Hou, Xinge Zhu, Yuexin Ma, Chen Change Loy, Yikang Li
This article addresses the problem of distilling knowledge from a large teacher model to a slim student network for LiDAR semantic segmentation.
Ranked #8 on LIDAR Semantic Segmentation on nuScenes (val mIoU metric)
2 code implementations • 31 May 2022 • Yuming Jiang, Shuai Yang, Haonan Qiu, Wayne Wu, Chen Change Loy, Ziwei Liu
In this work, we present a text-driven controllable framework, Text2Human, for a high-quality and diverse human generation.
3 code implementations • 12 May 2022 • Kenny T. R. Voo, Liming Jiang, Chen Change Loy
This paper performs comprehensive analysis on datasets for occlusion-aware face segmentation, a task that is crucial for many downstream applications.
no code implementations • 28 Apr 2022 • Zhongang Cai, Daxuan Ren, Ailing Zeng, Zhengyu Lin, Tao Yu, Wenjia Wang, Xiangyu Fan, Yang Gao, Yifan Yu, Liang Pan, Fangzhou Hong, Mingyuan Zhang, Chen Change Loy, Lei Yang, Ziwei Liu
4D human sensing and modeling are fundamental tasks in vision and graphics with numerous applications.
4 code implementations • 25 Apr 2022 • Jianglin Fu, Shikai Li, Yuming Jiang, Kwan-Yee Lin, Chen Qian, Chen Change Loy, Wayne Wu, Ziwei Liu
In addition, a model zoo and human editing applications are demonstrated to facilitate future research in the community.
1 code implementation • 11 Apr 2022 • Kelvin C. K. Chan, Shangchen Zhou, Xiangyu Xu, Chen Change Loy
The exploitation of long-term information has been a long-standing problem in video restoration.
1 code implementation • CVPR 2022 • Xiangtai Li, Wenwei Zhang, Jiangmiao Pang, Kai Chen, Guangliang Cheng, Yunhai Tong, Chen Change Loy
We hope this simple, yet effective method can serve as a new, flexible baseline in unified video segmentation design.
Ranked #1 on Video Panoptic Segmentation on KITTI-STEP (using extra training data)
1 code implementation • CVPR 2022 • Shuai Yang, Liming Jiang, Ziwei Liu, Chen Change Loy
In this work, we present a novel framework, Generative Prior-guided UNsupervised Image-to-image Translation (GP-UNIT), to improve the overall quality and applicability of the translation algorithm.
1 code implementation • CVPR 2022 • Yanbo Xu, Yueqin Yin, Liming Jiang, Qianyi Wu, Chengyao Zheng, Chen Change Loy, Bo Dai, Wayne Wu
In this study, we highlight the importance of interaction in a dual-space GAN for more controllable editing.
1 code implementation • CVPR 2022 • Li SiYao, Weijiang Yu, Tianpei Gu, Chunze Lin, Quan Wang, Chen Qian, Chen Change Loy, Ziwei Liu
With the learned choreographic memory, dance generation is realized on the quantized units that meet high choreography standards, such that the generated dancing sequences are confined within the spatial constraints.
Ranked #1 on Motion Synthesis on AIST++
1 code implementation • CVPR 2022 • Shuai Yang, Liming Jiang, Ziwei Liu, Chen Change Loy
Recent studies on StyleGAN show high performance on artistic portrait generation by transfer learning with limited data.
1 code implementation • 22 Mar 2022 • Yuhang Zang, Wei Li, Kaiyang Zhou, Chen Huang, Chen Change Loy
To this end, we propose a novel open-vocabulary detector based on DETR -- hence the name OV-DETR -- which, once trained, can detect any object given its class name or an exemplar image.
Ranked #21 on Open Vocabulary Object Detection on MSCOCO
1 code implementation • 21 Mar 2022 • Wenwei Zhang, Jiangmiao Pang, Kai Chen, Chen Change Loy
It also extracts a batch of region embeddings that correspond to some sub-regions in the overlapped area to be contrasted for region consistency.
Ranked #2 on Unsupervised Semantic Segmentation on COCO-All (mIoU metric)
9 code implementations • CVPR 2022 • Kaiyang Zhou, Jingkang Yang, Chen Change Loy, Ziwei Liu
With the rise of powerful pre-trained vision-language models like CLIP, it becomes essential to investigate ways to adapt these models to downstream datasets.
Ranked #3 on Prompt Engineering on ImageNet V2
1 code implementation • 7 Feb 2022 • Shangchen Zhou, Chongyi Li, Chen Change Loy
With the pipeline, we present the first large-scale dataset for joint low-light enhancement and deblurring.
Ranked #2 on Low-Light Image Enhancement on Sony-Total-Dark
no code implementations • 19 Dec 2021 • Wentao Zhu, Zhuoqian Yang, Ziang Di, Wayne Wu, Yizhou Wang, Chen Change Loy
Trained with the canonicalization operations and the derived regularizations, our method learns to factorize a skeleton sequence into three independent semantic subspaces, i. e., motion, structure, and view angle.
1 code implementation • 2 Dec 2021 • Chong Zhou, Chen Change Loy, Bo Dai
Contrastive Language-Image Pre-training (CLIP) has made a remarkable breakthrough in open-vocabulary zero-shot image recognition.
1 code implementation • CVPR 2022 • Kelvin C. K. Chan, Shangchen Zhou, Xiangyu Xu, Chen Change Loy
The diversity and complexity of degradations in real-world video super-resolution (VSR) pose non-trivial challenges in inference and training.
2 code implementations • NeurIPS 2021 • Liming Jiang, Bo Dai, Wayne Wu, Chen Change Loy
Generative adversarial networks (GANs) typically require ample data for training in order to synthesize high-fidelity images.
no code implementations • 1 Nov 2021 • Yu Rong, Jingbo Wang, Ziwei Liu, Chen Change Loy
In this paper, we make the first attempt to reconstruct 3D interacting hands from monocular single RGB images.
1 code implementation • NeurIPS 2021 • Xingang Pan, Xudong Xu, Chen Change Loy, Christian Theobalt, Bo Dai
Motivated by the observation that a 3D object should look realistic from multiple viewpoints, these methods introduce a multi-view constraint as regularization to learn valid 3D radiance fields from 2D images.
no code implementations • 25 Oct 2021 • Rui Xu, Xiangyu Xu, Kai Chen, Bolei Zhou, Chen Change Loy
Transformer becomes prevalent in computer vision, especially for high-level vision tasks.
no code implementations • 18 Oct 2021 • Linus Ericsson, Henry Gouk, Chen Change Loy, Timothy M. Hospedales
Self-supervised representation learning methods aim to provide powerful deep feature learning without the requirement of large annotated datasets, thus alleviating the annotation bottleneck that is one of the main barriers to practical deployment of deep learning today.
no code implementations • 14 Oct 2021 • Zhongang Cai, Mingyuan Zhang, Jiawei Ren, Chen Wei, Daxuan Ren, Zhengyu Lin, Haiyu Zhao, Lei Yang, Chen Change Loy, Ziwei Liu
Specifically, we contribute GTA-Human, a large-scale 3D human dataset generated with the GTA-V game engine, featuring a highly diverse set of subjects, actions, and scenarios.
1 code implementation • 9 Oct 2021 • Yihao Liu, Hengyuan Zhao, Kelvin C. K. Chan, Xintao Wang, Chen Change Loy, Yu Qiao, Chao Dong
We address this problem from a new perspective, by jointly considering colorization and temporal consistency in a unified framework.
no code implementations • 29 Sep 2021 • Yidi Shao, Chen Change Loy, Bo Dai
However, they force particles to interact with all neighbors without selection, and they fall short in capturing material semantics for different particles, leading to unsatisfactory performance, especially in generalization.
no code implementations • 29 Sep 2021 • Junzhe Zhang, Daxuan Ren, Zhongang Cai, Chai Kiat Yeo, Bo Dai, Chen Change Loy
Reconstruction is achieved by searching for a latent space in the 3D GAN that best resembles the target mesh in accordance with the single view observation.
no code implementations • 29 Sep 2021 • Guodong Xu, Yuenan Hou, Ziwei Liu, Chen Change Loy
To further enhance the semantic consistency between the teacher and student model, we present another latent-direction-based distillation loss that preserves the semantic relations in latent space.
1 code implementation • ICCV 2021 • Ke Yu, Zexian Li, Yue Peng, Chen Change Loy, Jinwei Gu
Image Signal Processor (ISP) is a crucial component in digital cameras that transforms sensor signals into images for us to perceive and understand.
1 code implementation • ICCV 2021 • Yuming Jiang, Ziqi Huang, Xingang Pan, Chen Change Loy, Ziwei Liu
In this work, we propose Talk-to-Edit, an interactive facial editing framework that performs fine-grained attribute manipulation through dialog between the user and the system.
Ranked #1 on Fine-Grained Facial Editing on CelebA-Dialog
1 code implementation • ICCV 2021 • Xiangyu Xu, Chen Change Loy
We propose a Transformer-based framework for 3D human texture estimation from a single image.
13 code implementations • 2 Sep 2021 • Kaiyang Zhou, Jingkang Yang, Chen Change Loy, Ziwei Liu
Large pre-trained vision-language models like CLIP have shown great potential in learning representations that are transferable across a wide range of downstream tasks.
Ranked #2 on Few-shot Age Estimation on MORPH Album2
1 code implementation • NeurIPS 2021 • Wenwei Zhang, Jiangmiao Pang, Kai Chen, Chen Change Loy
The framework, named K-Net, segments both instances and semantic categories consistently by a group of learnable kernels, where each kernel is responsible for generating a mask for either a potential instance or a stuff class.
Ranked #7 on Panoptic Segmentation on COCO test-dev
1 code implementation • NeurIPS 2021 • Jiahao Xie, Xiaohang Zhan, Ziwei Liu, Yew Soon Ong, Chen Change Loy
Extensive experiments on COCO show that ORL significantly improves the performance of self-supervised learning on scene images, even surpassing supervised ImageNet pre-training on several downstream tasks.
no code implementations • CVPR 2021 • Linsen Song, Wayne Wu, Chaoyou Fu, Chen Qian, Chen Change Loy, Ran He
We present a new application direction named Pareidolia Face Reenactment, which is defined as animating a static illusory face to move in tandem with a human face in the video.
1 code implementation • CVPR 2021 • Yuming Jiang, Kelvin C. K. Chan, Xintao Wang, Chen Change Loy, Ziwei Liu
However, performing local transfer is difficult because of two gaps between input and reference images: the transformation gap (e. g. scale and rotation) and the resolution gap (e. g. HR and LR).
2 code implementations • 1 Jun 2021 • Kaiyang Zhou, Chen Change Loy, Ziwei Liu
We find that the DG methods, which by design are unable to handle unlabeled data, perform poorly with limited labels in SSDG; the SSL methods, especially FixMatch, obtain much better results but are still far away from the basic vanilla model trained using full labels.
no code implementations • CVPR 2021 • Junzhe Zhang, Xinyi Chen, Zhongang Cai, Liang Pan, Haiyu Zhao, Shuai Yi, Chai Kiat Yeo, Bo Dai, Chen Change Loy
In contrast to previous fully supervised approaches, in this paper we present ShapeInversion, which introduces Generative Adversarial Network (GAN) inversion to shape completion for the first time.
3 code implementations • CVPR 2022 • Kelvin C. K. Chan, Shangchen Zhou, Xiangyu Xu, Chen Change Loy
We show that by empowering the recurrent framework with the enhanced propagation and alignment, one can exploit spatiotemporal information across misaligned video frames more effectively.
Ranked #1 on Video Enhancement on MFQE v2
1 code implementation • CVPR 2021 • Hang Zhou, Yasheng Sun, Wayne Wu, Chen Change Loy, Xiaogang Wang, Ziwei Liu
While speech content information can be defined by learning the intrinsic synchronization between audio-visual modalities, we identify that a pose code will be complementarily learned in a modulated convolution-based reconstruction framework.
1 code implementation • 21 Apr 2021 • Ren Yang, Radu Timofte, Jing Liu, Yi Xu, Xinjian Zhang, Minyi Zhao, Shuigeng Zhou, Kelvin C. K. Chan, Shangchen Zhou, Xiangyu Xu, Chen Change Loy, Xin Li, Fanglong Liu, He Zheng, Lielin Jiang, Qi Zhang, Dongliang He, Fu Li, Qingqing Dang, Yibin Huang, Matteo Maggioni, Zhongqian Fu, Shuai Xiao, Cheng Li, Thomas Tanay, Fenglong Song, Wentao Chao, Qiang Guo, Yan Liu, Jiang Li, Xiaochao Qu, Dewang Hou, Jiayu Yang, Lyn Jiang, Di You, Zhenyu Zhang, Chong Mou, Iaroslav Koshelev, Pavel Ostyakov, Andrey Somov, Jia Hao, Xueyi Zou, Shijie Zhao, Xiaopeng Sun, Yiting Liao, Yuanzhi Zhang, Qing Wang, Gen Zhan, Mengxi Guo, Junlin Li, Ming Lu, Zhan Ma, Pablo Navarrete Michelini, Hai Wang, Yiyun Chen, Jingyu Guo, Liliang Zhang, Wenming Yang, Sijung Kim, Syehoon Oh, Yucong Wang, Minjie Cai, Wei Hao, Kangdi Shi, Liangyan Li, Jun Chen, Wei Gao, Wang Liu, XiaoYu Zhang, Linjie Zhou, Sixin Lin, Ru Wang
This paper reviews the first NTIRE challenge on quality enhancement of compressed video, with a focus on the proposed methods and results.
3 code implementations • 21 Apr 2021 • Chongyi Li, Chunle Guo, Linghao Han, Jun Jiang, Ming-Ming Cheng, Jinwei Gu, Chen Change Loy
Low-light image enhancement (LLIE) aims at improving the perception or interpretability of an image captured in an environment with poor illumination.
1 code implementation • CVPR 2021 • Ruicheng Feng, Chongyi Li, Huaijin Chen, Shuai Li, Chen Change Loy, Jinwei Gu
Recent development of Under-Display Camera (UDC) systems provides a true bezel-less and notch-free viewing experience on smartphones (and TV, laptops, tablets), while allowing images to be captured from the selfie camera embedded underneath.
1 code implementation • CVPR 2021 • Xinya Ji, Hang Zhou, Kaisiyuan Wang, Wayne Wu, Chen Change Loy, Xun Cao, Feng Xu
In this work, we present Emotional Video Portraits (EVP), a system for synthesizing high-quality video portraits with vivid emotional dynamics driven by audios.
1 code implementation • 7 Apr 2021 • Linsen Song, Wayne Wu, Chaoyou Fu, Chen Qian, Chen Change Loy, Ran He
We present a new application direction named Pareidolia Face Reenactment, which is defined as animating a static illusory face to move in tandem with a human face in the video.
1 code implementation • CVPR 2021 • Li SiYao, Shiyu Zhao, Weijiang Yu, Wenxiu Sun, Dimitris N. Metaxas, Chen Change Loy, Ziwei Liu
In the animation industry, cartoon videos are usually produced at low frame rate since hand drawing of such frames is costly and time-consuming.
2 code implementations • 3 Mar 2021 • Kaiyang Zhou, Ziwei Liu, Yu Qiao, Tao Xiang, Chen Change Loy
Generalization to out-of-distribution (OOD) data is a capability natural to humans yet challenging for machines to reproduce.
1 code implementation • 2 Mar 2021 • Yuenan Hou, Zheng Ma, Chunxiao Liu, Zhe Wang, Chen Change Loy
Channel pruning is broadly recognized as an effective approach to obtain a small compact model through eliminating unimportant channels from a large cumbersome network.
4 code implementations • 1 Mar 2021 • Chongyi Li, Chunle Guo, Chen Change Loy
This paper presents a novel method, Zero-Reference Deep Curve Estimation (Zero-DCE), which formulates light enhancement as a task of image-specific curve estimation with a deep network.
1 code implementation • ICCV 2021 • Yuhang Zang, Chen Huang, Chen Change Loy
We propose a simple yet effective method, Feature Augmentation and Sampling Adaptation (FASA), that addresses the data scarcity issue by augmenting the feature space especially for rare classes.
2 code implementations • 18 Feb 2021 • Liming Jiang, Zhengkui Guo, Wayne Wu, Zhaoyang Liu, Ziwei Liu, Chen Change Loy, Shuo Yang, Yuanjun Xiong, Wei Xia, Baoying Chen, Peiyu Zhuang, Sili Li, Shen Chen, Taiping Yao, Shouhong Ding, Jilin Li, Feiyue Huang, Liujuan Cao, Rongrong Ji, Changlei Lu, Ganchao Tan
This paper reports methods and results in the DeeperForensics Challenge 2020 on real-world face forgery detection.
no code implementations • 29 Dec 2020 • Yu Rong, Ziwei Liu, Chen Change Loy
The reason is that most of the current models perform regression based on a single human prototype, which is similar to common poses while far from the rare poses.
8 code implementations • 23 Dec 2020 • Wenwei Zhang, Zhe Wang, Chen Change Loy
Due to the fact that multi-modality data augmentation must maintain consistency between point cloud and images, recent methods in this field typically use relatively insufficient data augmentation.
1 code implementation • ICCV 2021 • Liming Jiang, Bo Dai, Wayne Wu, Chen Change Loy
In this study, we show that narrowing gaps in the frequency domain can ameliorate image reconstruction and synthesis quality further.
1 code implementation • 17 Dec 2020 • Guodong Xu, Ziwei Liu, Chen Change Loy
Our goal is to achieve a performance comparable to conventional knowledge distillation with a lower computation cost during training.
no code implementations • CVPR 2021 • Rui Xu, Xintao Wang, Kai Chen, Bolei Zhou, Chen Change Loy
In this work, taking SinGAN and StyleGAN2 as examples, we show that such capability, to a large extent, is brought by the implicit positional encoding when using zero padding in the generators.
no code implementations • 7 Dec 2020 • Jiaqi Wang, Kai Chen, Rui Xu, Ziwei Liu, Chen Change Loy, Dahua Lin
Feature reassembly, i. e. feature downsampling and upsampling, is a key operation in a number of modern convolutional network architectures, e. g., residual networks and feature pyramids.
6 code implementations • CVPR 2021 • Kelvin C. K. Chan, Xintao Wang, Ke Yu, Chao Dong, Chen Change Loy
Video super-resolution (VSR) approaches tend to have more components than the image counterparts as they need to exploit the additional temporal dimension.
no code implementations • CVPR 2021 • Kelvin C. K. Chan, Xintao Wang, Xiangyu Xu, Jinwei Gu, Chen Change Loy
We show that pre-trained Generative Adversarial Networks (GANs), e. g., StyleGAN, can be used as a latent bank to improve the restoration quality of large-factor image super-resolution (SR).
1 code implementation • ICLR 2021 • Xingang Pan, Bo Dai, Ziwei Liu, Chen Change Loy, Ping Luo
Through our investigation, we found that such a pre-trained GAN indeed contains rich 3D knowledge and thus can be used to recover 3D shape from a single 2D image in an unsupervised manner.
no code implementations • 26 Oct 2020 • Chongyi Li, Chunle Guo, Qiming Ai, Shangchen Zhou, Chen Change Loy
This paper presents a new method, called FlexiCurve, for photo enhancement.
1 code implementation • 28 Sep 2020 • Rui Xu, Minghao Guo, Jiaqi Wang, Xiaoxiao Li, Bolei Zhou, Chen Change Loy
By bringing together the best of both paradigms, we propose a new deep inpainting framework where texture generation is guided by a texture memory of patch samples extracted from unmasked regions.
no code implementations • 15 Sep 2020 • Kelvin C. K. Chan, Xintao Wang, Ke Yu, Chao Dong, Chen Change Loy
Aside from the contributions to deformable alignment, our formulation inspires a more flexible approach to introduce offset diversity to flow-based alignment, improving its performance.
2 code implementations • 26 Aug 2020 • Jiahao Xie, Xiaohang Zhan, Ziwei Liu, Yew Soon Ong, Chen Change Loy
In this work, we present a comprehensive empirical study to better understand the role of inter-image invariance learning from three main constituting components: pseudo-label maintenance, sampling strategy, and decision boundary design.
2 code implementations • CVPR 2021 • Jiaqi Wang, Wenwei Zhang, Yuhang Zang, Yuhang Cao, Jiangmiao Pang, Tao Gong, Kai Chen, Ziwei Liu, Chen Change Loy, Dahua Lin
Instances of head classes dominate a long-tailed dataset and they serve as negative samples of tail categories.
no code implementations • ECCV 2020 • Zhongang Cai, Junzhe Zhang, Daxuan Ren, Cunjun Yu, Haiyu Zhao, Shuai Yi, Chai Kiat Yeo, Chen Change Loy
We present an interesting and challenging dataset that features a large number of scenes with messy tables captured from multiple camera views.
1 code implementation • ECCV 2020 • Liming Jiang, Changxu Zhang, Mingyang Huang, Chunxiao Liu, Jianping Shi, Chen Change Loy
We introduce a simple and versatile framework for image-to-image translation.
1 code implementation • ECCV 2020 • Tak-Wai Hui, Chen Change Loy
The keys to success lie in the use of cost volume and coarse-to-fine flow inference.
Ranked #4 on Optical Flow Estimation on KITTI 2012
1 code implementation • ECCV 2020 • Chongyi Li, Runmin Cong, Yongri Piao, Qianqian Xu, Chen Change Loy
Second, we propose an adaptive feature selection (AFS) module to select saliency-related features and suppress the inferior ones.
Ranked #8 on RGB-D Salient Object Detection on NJU2K
1 code implementation • NeurIPS 2020 • Shangchen Zhou, Jiawei Zhang, WangMeng Zuo, Chen Change Loy
Specifically, we dynamically construct a cross-scale graph by searching k-nearest neighboring patches in the downsampled LR image for each query patch in the LR image.
1 code implementation • CVPR 2020 • Xiaohang Zhan, Jiahao Xie, Ziwei Liu, Yew Soon Ong, Chen Change Loy
In this way, labels and the network evolve shoulder-to-shoulder rather than alternatingly.
2 code implementations • ECCV 2020 • Guodong Xu, Ziwei Liu, Xiaoxiao Li, Chen Change Loy
Knowledge distillation, which involves extracting the "dark knowledge" from a teacher network to guide the learning of a student network, has emerged as an important technique for model compression and transfer learning.
Ranked #32 on Knowledge Distillation on ImageNet
1 code implementation • CVPR 2020 • Yuenan Hou, Zheng Ma, Chunxiao Liu, Tak-Wai Hui, Chen Change Loy
We study the problem of distilling knowledge from a large deep teacher network to a much smaller student network for the task of road marking segmentation.
Ranked #1 on Semantic Segmentation on ApolloScape
1 code implementation • 7 Apr 2020 • Kai Chen, Yuhang Cao, Chen Change Loy, Dahua Lin, Christoph Feichtenhofer
Feature pyramid networks have been widely adopted in the object detection literature to improve feature representations for better handling of variations in scale.
2 code implementations • CVPR 2020 • Xiaohang Zhan, Xingang Pan, Bo Dai, Ziwei Liu, Dahua Lin, Chen Change Loy
This is achieved via Partial Completion Network (PCNet)-mask (M) and -content (C), that learn to recover fractions of object masks and contents, respectively, in a self-supervised manner.
3 code implementations • CVPR 2020 • Lei Yang, Dapeng Chen, Xiaohang Zhan, Rui Zhao, Chen Change Loy, Dahua Lin
With the vertex confidence and edge connectivity, we can naturally organize more relevant vertices on the affinity graph and group them into clusters.
no code implementations • CVPR 2020 • Zhuoqian Yang, Wentao Zhu, Wayne Wu, Chen Qian, Qiang Zhou, Bolei Zhou, Chen Change Loy
We present a lightweight video motion retargeting approach TransMoMo that is capable of transferring motion of a person in a source video realistically to another video of a target person.
1 code implementation • ECCV 2020 • Xingang Pan, Xiaohang Zhan, Bo Dai, Dahua Lin, Chen Change Loy, Ping Luo
Learning a good image prior is a long-term goal for image restoration and manipulation.
2 code implementations • 17 Mar 2020 • Yu Liu, Guanglu Song, Yuhang Zang, Yan Gao, Enze Xie, Junjie Yan, Chen Change Loy, Xiaogang Wang
Given such good instance bounding box, we further design a simple instance-level semantic segmentation pipeline and achieve the 1st place on the segmentation challenge.
no code implementations • 21 Feb 2020 • Mengya Gao, Yujun Shen, Quanquan Li, Chen Change Loy
Knowledge distillation (KD) is one of the most potent ways for model compression.
2 code implementations • ICLR 2020 • Yuanbo Xiangli, Yubin Deng, Bo Dai, Chen Change Loy, Dahua Lin
While generative adversarial networks (GAN) have been widely adopted in various topics, in this paper we generalize the standard GAN to a new perspective by treating realness as a random variable that can be estimated from multiple angles.
9 code implementations • CVPR 2020 • Chunle Guo, Chongyi Li, Jichang Guo, Chen Change Loy, Junhui Hou, Sam Kwong, Runmin Cong
The paper presents a novel method, Zero-Reference Deep Curve Estimation (Zero-DCE), which formulates light enhancement as a task of image-specific curve estimation with a deep network.
Ranked #1 on Color Constancy on INTEL-TUT2
no code implementations • 15 Jan 2020 • Linsen Song, Wayne Wu, Chen Qian, Ran He, Chen Change Loy
The audio-translated expression parameters are then used to synthesize a photo-realistic human subject in each video frame, with the movement of the mouth regions precisely mapped to the source audio.
1 code implementation • CVPR 2020 • Liming Jiang, Ren Li, Wayne Wu, Chen Qian, Chen Change Loy
The quality of generated videos outperforms those in existing datasets, validated by user studies.
no code implementations • CVPR 2020 • Dongzhan Zhou, Xinchi Zhou, Wenwei Zhang, Chen Change Loy, Shuai Yi, Xuesen Zhang, Wanli Ouyang
While many methods have been proposed to improve the efficiency of NAS, the search progress is still laborious because training and evaluating plausible architectures over large search space is time-consuming.
3 code implementations • ECCV 2020 • Jiaqi Wang, Wenwei Zhang, Yuhang Cao, Kai Chen, Jiangmiao Pang, Tao Gong, Jianping Shi, Chen Change Loy, Dahua Lin
To tackle the difficulty of precise localization in the presence of displacements with large variance, we further propose a two-step localization scheme, which first predicts a range of movement through bucket prediction and then pinpoints the precise position within the predicted bucket.
no code implementations • 18 Nov 2019 • Wu Shi, Tak-Wai Hui, Ziwei Liu, Dahua Lin, Chen Change Loy
Another important observation is that fashion textures are multi-modal.
1 code implementation • ICCV 2019 • Wenwei Zhang, Hui Zhou, Shuyang Sun, Zhe Wang, Jianping Shi, Chen Change Loy
Multi-sensor perception is crucial to ensure the reliability and accuracy in autonomous driving system, while multi-object tracking (MOT) improves that by tracing sequential movement of dynamic objects.
Ranked #10 on Multiple Object Tracking on KITTI Tracking test
1 code implementation • ICCV 2019 • Yu Rong, Ziwei Liu, Cheng Li, Kaidi Cao, Chen Change Loy
Specifically, we focus on the challenging task of in-the-wild 3D human recovery from single images when paired 3D annotations are not fully available.
2 code implementations • 5 Aug 2019 • Yunxuan Zhang, Siwei Zhang, Yue He, Cheng Li, Chen Change Loy, Ziwei Liu
However, in real-world scenario end-users often only have one target face at hand, rendering existing methods inapplicable.
2 code implementations • ICCV 2019 • Yuenan Hou, Zheng Ma, Chunxiao Liu, Chen Change Loy
Training deep models for lane detection is challenging due to the very subtle and sparse supervisory signals inherent in lane annotations.
Ranked #5 on Lane Detection on BDD100K val
144 code implementations • 17 Jun 2019 • Kai Chen, Jiaqi Wang, Jiangmiao Pang, Yuhang Cao, Yu Xiong, Xiaoxiao Li, Shuyang Sun, Wansen Feng, Ziwei Liu, Jiarui Xu, Zheng Zhang, Dazhi Cheng, Chenchen Zhu, Tianheng Cheng, Qijie Zhao, Buyu Li, Xin Lu, Rui Zhu, Yue Wu, Jifeng Dai, Jingdong Wang, Jianping Shi, Wanli Ouyang, Chen Change Loy, Dahua Lin
In this paper, we introduce the various features of this toolbox.
1 code implementation • ICLR Workshop DeepGenStruct 2019 • Wayne Wu, Kaidi Cao, Cheng Li, Chen Qian, Chen Change Loy
It is challenging to disentangle an object into two orthogonal spaces of content and style since each can influence the visual observation differently and unpredictably.
2 code implementations • CVPR 2019 • Rui Xu, Xiaoxiao Li, Bolei Zhou, Chen Change Loy
Then the synthesized flow field is used to guide the propagation of pixels to fill up the missing regions in the video.
Ranked #8 on Video Inpainting on DAVIS
One-shot visual object segmentation Optical Flow Estimation +2
11 code implementations • 7 May 2019 • Xintao Wang, Kelvin C. K. Chan, Ke Yu, Chao Dong, Chen Change Loy
In this work, we propose a novel Video Restoration framework with Enhanced Deformable networks, termed EDVR, to address these challenges.
Ranked #2 on Deblurring on REDS
3 code implementations • ICCV 2019 • Jiaqi Wang, Kai Chen, Rui Xu, Ziwei Liu, Chen Change Loy, Dahua Lin
CARAFE introduces little computational overhead and can be readily integrated into modern network architectures.
Ranked #3 on Feature Upsampling on ImageNet
1 code implementation • 23 Apr 2019 • Ke Yu, Xintao Wang, Chao Dong, Xiaoou Tang, Chen Change Loy
To leverage this, we propose Path-Restore, a multi-path CNN with a pathfinder that can dynamically select an appropriate route for each image region.
no code implementations • CVPR 2019 • Wayne Wu, Kaidi Cao, Cheng Li, Chen Qian, Chen Change Loy
Extensive experiments demonstrate the superior performance of our method to other state-of-the-art approaches, especially in the challenging near-rigid and non-rigid objects translation tasks.
1 code implementation • CVPR 2020 • Yuhang Cao, Kai Chen, Chen Change Loy, Dahua Lin
Our experiments demonstrate that it is often more effective to focus on prime samples than hard samples when training a detector.