no code implementations • 19 Mar 2024 • Wenjing Wang, Huan Yang, Jianlong Fu, Jiaying Liu
This prior serves as the bridge between normal and low-light images.
no code implementations • 19 Jan 2024 • Bo Zhao, Huan Yang, Jianlong Fu
Face inpainting requires the model to have a precise global understanding of the facial position structure.
no code implementations • 21 Sep 2023 • Feng Li, Yuqi Chai, Huan Yang, Pengfei Hu, Lingjie Duan
How to incentivize strategic workers using limited budget is a very fundamental problem for crowdsensing systems; nevertheless, since the sensing abilities of the workers may not always be known as prior knowledge due to the diversities of their sensor devices and behaviors, it is difficult to properly select and pay the unknown workers.
no code implementations • 31 Jul 2023 • Junchen Zhu, Huan Yang, Wenjing Wang, Huiguo He, Zixi Tuo, Yongsheng Yu, Wen-Huang Cheng, Lianli Gao, Jingkuan Song, Jianlong Fu, Jiebo Luo
In the basic generation, we take advantage of the pretrained image diffusion model, and adapt it to a high-quality open-domain vertical video generator for mobile devices.
no code implementations • 20 Jun 2023 • Huiguo He, Tianfu Wang, Huan Yang, Jianlong Fu, Nicholas Jing Yuan, Jian Yin, Hongyang Chao, Qi Zhang
The proposed framework consists of a large language model (LLM), a diffusion-based image generator, and a series of visual rewards by design.
no code implementations • 12 Jun 2023 • Junchen Zhu, Huan Yang, Huiguo He, Wenjing Wang, Zixi Tuo, Wen-Huang Cheng, Lianli Gao, Jingkuan Song, Jianlong Fu
To generate videos, we extend the capabilities of a pretrained text-to-image diffusion model through a two-stage process.
no code implementations • 24 May 2023 • Yiyang Ma, Huan Yang, Wenhan Yang, Jianlong Fu, Jiaying Liu
Diffusion models, as a kind of powerful generative model, have given impressive results on image super-resolution (SR) tasks.
1 code implementation • 18 May 2023 • Wenjing Wang, Huan Yang, Zixi Tuo, Huiguo He, Junchen Zhu, Jianlong Fu, Jiaying Liu
Moreover, to fully unlock model capabilities for high-quality video generation and promote the development of the field, we curate a large-scale and open-source video dataset called HD-VG-130M.
Ranked #1 on Text-to-Video Generation on WebVid
no code implementations • 22 Mar 2023 • Shengming Yin, Chenfei Wu, Huan Yang, JianFeng Wang, Xiaodong Wang, Minheng Ni, Zhengyuan Yang, Linjie Li, Shuguang Liu, Fan Yang, Jianlong Fu, Gong Ming, Lijuan Wang, Zicheng Liu, Houqiang Li, Nan Duan
In this paper, we propose NUWA-XL, a novel Diffusion over Diffusion architecture for eXtremely Long video generation.
1 code implementation • ICCV 2023 • Zixi Tuo, Huan Yang, Jianlong Fu, Yujie Dun, Xueming Qian
Existing real-world video super-resolution (VSR) methods focus on designing a general degradation pipeline for open-domain videos while ignoring data intrinsic characteristics which strongly limit their performance when applying to some specific domains (eg., animation videos).
no code implementations • 16 Mar 2023 • Yiyang Ma, Huan Yang, Wenjing Wang, Jianlong Fu, Jiaying Liu
Language-guided image generation has achieved great success nowadays by using diffusion models.
no code implementations • 1 Mar 2023 • Guanghao Yin, Zefan Qu, Xinyang Jiang, Shan Jiang, Zhenhua Han, Ningxin Zheng, Xiaohong Liu, Huan Yang, Yuqing Yang, Dongsheng Li, Lili Qiu
To facilitate the research on this problem, a new benchmark dataset named LDV-WebRTC is constructed based on a real-world online streaming system.
1 code implementation • 27 Dec 2022 • Zhongwei Qiu, Huan Yang, Jianlong Fu, Daochang Liu, Chang Xu, Dongmei Fu
Video Super-Resolution (VSR) aims to restore high-resolution (HR) videos from low-resolution (LR) videos.
Ranked #2 on Video Super-Resolution on REDS4- 4x upscaling
1 code implementation • CVPR 2023 • Ludan Ruan, Yiyang Ma, Huan Yang, Huiguo He, Bei Liu, Jianlong Fu, Nicholas Jing Yuan, Qin Jin, Baining Guo
To generate joint audio-video pairs, we propose a novel Multi-Modal Diffusion model (i. e., MM-Diffusion), with two-coupled denoising autoencoders.
1 code implementation • 22 Nov 2022 • Jun Liang, Songsen Yu, Huan Yang
ResNets and its variants play an important role in various fields of image recognition.
1 code implementation • 12 Oct 2022 • Yuchong Sun, Hongwei Xue, Ruihua Song, Bei Liu, Huan Yang, Jianlong Fu
Large-scale video-language pre-training has shown significant improvement in video-language understanding tasks.
Ranked #2 on Video Retrieval on QuerYD (using extra training data)
1 code implementation • 11 Oct 2022 • Jianbo Wang, Huan Yang, Jianlong Fu, Toshihiko Yamasaki, Baining Guo
Such a design usually destroys the spatial information of the input images and fails to transfer fine-grained style patterns into style transfer results.
1 code implementation • 7 Sep 2022 • Yiyang Ma, Huan Yang, Bei Liu, Jianlong Fu, Jiaying Liu
To address this issue, we propose a Prompt-based Cross-Modal Generation Framework (PCM-Frame) to leverage two powerful pre-trained models, including CLIP and StyleGAN.
no code implementations • 5 Sep 2022 • Chengxu Liu, Huan Yang, Jianlong Fu, Xueming Qian
In particular, we first introduce a lightweight context encoder and a parameter encoder to learn a context map for the pixel-level category and a group of image-adaptive coefficients, respectively.
Ranked #7 on Image Enhancement on MIT-Adobe 5k (SSIM on proRGB metric)
1 code implementation • 11 Aug 2022 • Tiankai Hang, Huan Yang, Bei Liu, Jianlong Fu, Xin Geng, Baining Guo
Specifically, we propose a recurrent motion generator to extract a series of semantic and motion information from the language and feed it along with visual information to a pre-trained StyleGAN to generate high-quality frames.
1 code implementation • 5 Aug 2022 • Zhongwei Qiu, Huan Yang, Jianlong Fu, Dongmei Fu
First, we divide a video frame into patches, and transform each patch into DCT spectral maps in which each channel represents a frequency band.
Ranked #3 on Video Super-Resolution on REDS4- 4x upscaling
no code implementations • 4 Aug 2022 • Jun Xiao, Xinyang Jiang, Ningxin Zheng, Huan Yang, Yifan Yang, Yuqing Yang, Dongsheng Li, Kin-Man Lam
Then, our proposed CKBG method enhances this lightweight base model by bypassing the original network with ``kernel grafts'', which are extra convolutional kernels containing the prior knowledge of external pretrained image SR models.
no code implementations • 19 Jul 2022 • Chengxu Liu, Huan Yang, Jianlong Fu, Xueming Qian
In particular, we formulate the warped features with inconsistent motions as query tokens, and formulate relevant regions in a motion trajectory from two original consecutive frames into keys and values.
no code implementations • 3 Jul 2022 • Fuzhi Yang, Huan Yang, Yanhong Zeng, Jianlong Fu, Hongtao Lu
The extractor estimates the degradations in LR inputs and guides the meta-restoration modules to predict restoration parameters for different degradations on-the-fly.
1 code implementation • 19 Apr 2022 • Guojia Hou, YuXuan Li, Huan Yang, Kunqian Li, Zhenkuan Pan
Achieving subjective and objective quality assessment of underwater images is of high significance in underwater visual perception and image/video processing.
1 code implementation • CVPR 2022 • Chengxu Liu, Huan Yang, Jianlong Fu, Xueming Qian
Existing approaches usually align and aggregate video frames from limited adjacent frames (e. g., 5 or 7 frames), which prevents these approaches from satisfactory results.
Ranked #4 on Video Super-Resolution on UDM10 - 4x upscaling
no code implementations • 29 Jan 2022 • Feng Li, Xuyang Yuan, Lina Wang, Huan Yang, Dongxiao Yu, Weifeng Lv, Xiuzhen Cheng
The efficacy of our proposed three-staged collaborative learning algorithm is finally verified by extensive experiments on both synthetic and real datasets.
1 code implementation • CVPR 2022 • Hongwei Xue, Tiankai Hang, Yanhong Zeng, Yuchong Sun, Bei Liu, Huan Yang, Jianlong Fu, Baining Guo
To enable VL pre-training, we jointly optimize the HD-VILA model by a hybrid Transformer that learns rich spatiotemporal features, and a multimodal Transformer that enforces interactions of the learned video features with diversified texts.
Ranked #16 on Video Retrieval on MSR-VTT
no code implementations • NeurIPS 2021 • Yanhong Zeng, Huan Yang, Hongyang Chao, Jianbo Wang, Jianlong Fu
Given a sequence of style tokens, the TokenGAN is able to control the image synthesis by assigning the styles to the content tokens by attention mechanism with a Transformer.
no code implementations • 6 Sep 2021 • Hongwei Xue, Bei Liu, Huan Yang, Jianlong Fu, Houqiang Li, Jiebo Luo
To tackle this problem, we propose a model named FGLA to generate high-quality and realistic videos by learning Fine-Grained motion embedding for Landscape Animation.
1 code implementation • ICCV 2021 • Heliang Zheng, Huan Yang, Jianlong Fu, Zheng-Jun Zha, Jiebo Luo
And the reference space is optimized to capture deep image priors that are useful for quality assessment.
1 code implementation • ICCV 2021 • Kibeom Hong, Seogkyu Jeon, Huan Yang, Jianlong Fu, Hyeran Byun
To this end, we design a novel domainness indicator that captures the domainness value from the texture and structural features of reference images.
no code implementations • 24 Feb 2021 • Xiaoyu Chen, Wen Duan, Xinwei Fan, Wenshan Hong, Kailun Chen, Huan Yang, Shiliang Li, Huiqian Luo, Hai-Hu Wen
We report the observation of discrete vortex bound states with the energy levels deviating from the widely believed ratio of 1:3:5 in the vortices of an iron based superconductor KCa2Fe4As4F2 through scanning tunneling microcopy (STM).
Superconductivity Strongly Correlated Electrons
no code implementations • 17 Feb 2021 • Wen Duan, Kailun Chen, Wenshan Hong, Xiaoyu Chen, Huan Yang, Shiliang Li, Huiqian Luo, Hai-Hu Wen
On the second type of surface which is rarely obtained, the fully gapped feature can still be observed on the tunneling spectra, although multiple gaps are obtained either from a single spectrum or separate ones, and the gap values determined from coherence peaks locate mainly in the range from 4 to 8 meV.
Superconductivity
no code implementations • 25 Jan 2021 • Weida Hu, Junxian Wang, Leopoldo Infante, James E. Rhoads, Zhen-Ya Zheng, Huan Yang, Sangeeta Malhotra, L. Felipe Barrientos, Chunyan Jiang, Jorge González-López, Gonzalo Prieto, Lucia A. Perez, Pascale Hibon, Gaspar Galaz, Alicia Coughlin, Santosh Harish, Xu Kong, Wenyong Kang, Ali Ahmad Khostovan, John Pharo, Francisco Valdes, Isak Wold, Alistair R. Walker, XianZhong Zheng
Here we report the discovery of the protocluster LAGER-z7OD1 at a redshift of 6. 93, when the Universe was only 770 million years old and could be experiencing rapid evolution of the neutral hydrogen fraction in the intergalactic medium.
Astrophysics of Galaxies
no code implementations • 22 Jan 2021 • Zhen Pan, Huan Yang
In this work, we calculate the rate of EMRIs of an alternative formation channel: EMRI formation assisted by the accretion flow around accreting massive black holes.
High Energy Astrophysical Phenomena General Relativity and Quantum Cosmology
no code implementations • 25 Nov 2020 • Qi Liu, Hui Yuan, Raouf Hamzaoui, Honglei Su, Junhui Hou, Huan Yang
In rate-distortion optimization, the encoder settings are determined by maximizing a reconstruction quality measure subject to a constraint on the bit rate.
1 code implementation • 7 Aug 2020 • Chenglizhao Chen, Hongmeng Zhao, Huan Yang, Chong Peng, Teng Yu
The screen content images (SCIs) usually comprise various content types with sharp edges, in which the artifacts or distortions can be well sensed by the vanilla structure similarity measurement in a full reference manner.
no code implementations • 9 Jul 2020 • Ying Xiang, Qing Li, Yueying Li, Huan Yang, Yuefeng Nie, Hai-Hu Wen
The angle dependent resistivity at a fixed temperature and different magnetic fields cannot be scaled to one curve, which deviates from the prediction of the anisotropic Ginzburg-Landau theory.
Superconductivity Materials Science Strongly Correlated Electrons
1 code implementation • CVPR 2020 • Fuzhi Yang, Huan Yang, Jianlong Fu, Hongtao Lu, Baining Guo
In this paper, we propose a novel Texture Transformer Network for Image Super-Resolution (TTSR), in which the LR and Ref images are formulated as queries and keys in a transformer, respectively.
no code implementations • 3 May 2020 • Kai Zhang, Shuhang Gu, Radu Timofte, Taizhang Shang, Qiuju Dai, Shengchen Zhu, Tong Yang, Yandong Guo, Younghyun Jo, Sejong Yang, Seon Joo Kim, Lin Zha, Jiande Jiang, Xinbo Gao, Wen Lu, Jing Liu, Kwangjin Yoon, Taegyun Jeon, Kazutoshi Akita, Takeru Ooba, Norimichi Ukita, Zhipeng Luo, Yuehan Yao, Zhenyu Xu, Dongliang He, Wenhao Wu, Yukang Ding, Chao Li, Fu Li, Shilei Wen, Jianwei Li, Fuzhi Yang, Huan Yang, Jianlong Fu, Byung-Hoon Kim, JaeHyun Baek, Jong Chul Ye, Yuchen Fan, Thomas S. Huang, Junyeop Lee, Bokyeung Lee, Jungki Min, Gwantae Kim, Kanghyu Lee, Jaihyun Park, Mykola Mykhailych, Haoyu Zhong, Yukai Shi, Xiaojun Yang, Zhijing Yang, Liang Lin, Tongtong Zhao, Jinjia Peng, Huibing Wang, Zhi Jin, Jiahao Wu, Yifu Chen, Chenming Shang, Huanrong Zhang, Jeongki Min, Hrishikesh P. S, Densen Puthussery, Jiji C. V
This paper reviews the NTIRE 2020 challenge on perceptual extreme super-resolution with focus on proposed solutions and results.
no code implementations • 1 Apr 2020 • Rui Nie, Huan Yang, Hejuan Peng, Wenbin Luo, Weiya Fan, Jie Zhang, Jing Liao, Fang Huang, Yufeng Xiao
Small intestinal capsule endoscopy is the mainstream method for inspecting small intestinal lesions, but a single small intestinal capsule endoscopy will produce 60, 000 - 120, 000 images, the majority of which are similar and have no diagnostic value.
no code implementations • 6 Mar 2018 • Huan Yang, Baoyuan Wang, Noranart Vesdapunt, Minyi Guo, Sing Bing Kang
We propose a reinforcement learning approach for real-time exposure control of a mobile camera that is personalizable.
no code implementations • ICCV 2015 • Huan Yang, Baoyuan Wang, Stephen Lin, David Wipf, Minyi Guo, Baining Guo
With the growing popularity of short-form video sharing platforms such as \em{Instagram} and \em{Vine}, there has been an increasing need for techniques that automatically extract highlights from video.