Search Results for author: Jinwei Gu

Found 51 papers, 23 papers with code

HDRFlow: Real-Time HDR Video Reconstruction with Large Motions

no code implementations6 Mar 2024 Gangwei Xu, Yujin Wang, Jinwei Gu, Tianfan Xue, Xin Yang

HDRFlow has three novel designs: an HDR-domain alignment loss (HALoss), an efficient flow network with a multi-size large kernel (MLK), and a new HDR flow training scheme.

Optical Flow Estimation Video Reconstruction

Event-Based Motion Magnification

no code implementations19 Feb 2024 Yutian Chen, Shi Guo, Fangzheng Yu, Feng Zhang, Jinwei Gu, Tianfan Xue

Detecting and magnifying imperceptible high-frequency motions in real-world scenarios has substantial implications for industrial and medical applications.

Motion Detection Motion Magnification

Cached Transformers: Improving Transformers with Differentiable Memory Cache

1 code implementation20 Dec 2023 Zhaoyang Zhang, Wenqi Shao, Yixiao Ge, Xiaogang Wang, Jinwei Gu, Ping Luo

This work introduces a new Transformer model called Cached Transformer, which uses Gated Recurrent Cached (GRC) attention to extend the self-attention mechanism with a differentiable memory cache of tokens.

Image Classification Instance Segmentation +6

AutoDIR: Automatic All-in-One Image Restoration with Latent Diffusion

1 code implementation16 Oct 2023 Yitong Jiang, Zhaoyang Zhang, Tianfan Xue, Jinwei Gu

To this end, we propose an all-in-one image restoration framework with latent diffusion (AutoDIR), which can automatically detect and address multiple unknown degradations.

Blind Image Quality Assessment Image Restoration

Reconstruct-and-Generate Diffusion Model for Detail-Preserving Image Denoising

no code implementations19 Sep 2023 Yujin Wang, Lingen Li, Tianfan Xue, Jinwei Gu

To address the trade-off between visual appeal and fidelity of high-frequency details in denoising tasks, we propose a novel approach called the Reconstruct-and-Generate Diffusion Model (RnG).

Image Denoising Image Restoration

MIPI 2023 Challenge on RGBW Remosaic: Methods and Results

no code implementations20 Apr 2023 Qianhui Sun, Qingyu Yang, Chongyi Li, Shangchen Zhou, Ruicheng Feng, Yuekun Dai, Wenxiu Sun, Qingpeng Zhu, Chen Change Loy, Jinwei Gu

Developing and integrating advanced image sensors with novel algorithms in camera systems are prevalent with the increasing demand for computational photography and imaging on mobile platforms.

SSIM

MIPI 2023 Challenge on RGBW Fusion: Methods and Results

no code implementations20 Apr 2023 Qianhui Sun, Qingyu Yang, Chongyi Li, Shangchen Zhou, Ruicheng Feng, Yuekun Dai, Wenxiu Sun, Qingpeng Zhu, Chen Change Loy, Jinwei Gu

Developing and integrating advanced image sensors with novel algorithms in camera systems are prevalent with the increasing demand for computational photography and imaging on mobile platforms.

SSIM

Generating Aligned Pseudo-Supervision from Non-Aligned Data for Image Restoration in Under-Display Camera

1 code implementation CVPR 2023 Ruicheng Feng, Chongyi Li, Huaijin Chen, Shuai Li, Jinwei Gu, Chen Change Loy

Due to the difficulty in collecting large-scale and perfectly aligned paired training data for Under-Display Camera (UDC) image restoration, previous methods resort to monitor-based image systems or simulation-based methods, sacrificing the realness of the data and introducing domain gaps.

Image Restoration

Random Weights Networks Work as Loss Prior Constraint for Image Restoration

no code implementations29 Mar 2023 Man Zhou, Naishan Zheng, Jie Huang, Xiangyu Rui, Chunle Guo, Deyu Meng, Chongyi Li, Jinwei Gu

In this paper, orthogonal to the existing data and model studies, we instead resort our efforts to investigate the potential of loss function in a new perspective and present our belief ``Random Weights Networks can Be Acted as Loss Prior Constraint for Image Restoration''.

Image Restoration Image Super-Resolution +1

Real-time Controllable Denoising for Image and Video

1 code implementation CVPR 2023 Zhaoyang Zhang, Yitong Jiang, Wenqi Shao, Xiaogang Wang, Ping Luo, Kaimo Lin, Jinwei Gu

Controllable image denoising aims to generate clean samples with human perceptual priors and balance sharpness and smoothness.

Image Denoising Video Denoising

Overexposure Mask Fusion: Generalizable Reverse ISP Multi-Step Refinement

no code implementations20 Oct 2022 Jinha Kim, Jun Jiang, Jinwei Gu

With the advent of deep learning methods replacing the ISP in transforming sensor RAW readings into RGB images, numerous methodologies solidified into real-life applications.

Raw reconstruction

Deep Fourier Up-Sampling

1 code implementation11 Oct 2022 Man Zhou, Hu Yu, Jie Huang, Feng Zhao, Jinwei Gu, Chen Change Loy, Deyu Meng, Chongyi Li

Existing convolutional neural networks widely adopt spatial down-/up-sampling for multi-scale modeling.

Image Dehazing Image Segmentation +4

Neighbourhood Representative Sampling for Efficient End-to-end Video Quality Assessment

4 code implementations11 Oct 2022 HaoNing Wu, Chaofeng Chen, Liang Liao, Jingwen Hou, Wenxiu Sun, Qiong Yan, Jinwei Gu, Weisi Lin

On the other hand, existing practices, such as resizing and cropping, will change the quality of original videos due to the loss of details and contents, and are therefore harmful to quality assessment.

Ranked #2 on Video Quality Assessment on KoNViD-1k (using extra training data)

Video Quality Assessment Visual Question Answering (VQA)

GLEAN: Generative Latent Bank for Image Super-Resolution and Beyond

1 code implementation29 Jul 2022 Kelvin C. K. Chan, Xiangyu Xu, Xintao Wang, Jinwei Gu, Chen Change Loy

While most existing perceptual-oriented approaches attempt to generate realistic outputs through learning with adversarial loss, our method, Generative LatEnt bANk (GLEAN), goes beyond existing practices by directly leveraging rich and diverse priors encapsulated in a pre-trained GAN.

Colorization Image Colorization +2

ReconfigISP: Reconfigurable Camera Image Processing Pipeline

1 code implementation ICCV 2021 Ke Yu, Zexian Li, Yue Peng, Chen Change Loy, Jinwei Gu

Image Signal Processor (ISP) is a crucial component in digital cameras that transforms sensor signals into images for us to perceive and understand.

Image Restoration Neural Architecture Search +2

Deep Camera Obscura: An Image Restoration Pipeline for Lensless Pinhole Photography

no code implementations12 Aug 2021 Joshua D. Rego, Huaijin Chen, Shuai Li, Jinwei Gu, Suren Jayasuriya

The lensless pinhole camera is perhaps the earliest and simplest form of an imaging system using only a pinhole-sized aperture in place of a lens.

Image Restoration

Low-Light Image and Video Enhancement Using Deep Learning: A Survey

3 code implementations21 Apr 2021 Chongyi Li, Chunle Guo, Linghao Han, Jun Jiang, Ming-Ming Cheng, Jinwei Gu, Chen Change Loy

Low-light image enhancement (LLIE) aims at improving the perception or interpretability of an image captured in an environment with poor illumination.

Face Detection Low-Light Image Enhancement +1

Removing Diffraction Image Artifacts in Under-Display Camera via Dynamic Skip Connection Network

1 code implementation CVPR 2021 Ruicheng Feng, Chongyi Li, Huaijin Chen, Shuai Li, Chen Change Loy, Jinwei Gu

Recent development of Under-Display Camera (UDC) systems provides a true bezel-less and notch-free viewing experience on smartphones (and TV, laptops, tablets), while allowing images to be captured from the selfie camera embedded underneath.

Image Restoration

HDR Video Reconstruction with Tri-Exposure Quad-Bayer Sensors

no code implementations19 Mar 2021 Yitong Jiang, Inchang Choi, Jun Jiang, Jinwei Gu

We propose a novel high dynamic range (HDR) video reconstruction method with new tri-exposure quad-bayer sensors.

Video Reconstruction

iToF2dToF: A Robust and Flexible Representation for Data-Driven Time-of-Flight Imaging

no code implementations12 Mar 2021 Felipe Gutierrez-Barragan, Huaijin Chen, Mohit Gupta, Andreas Velten, Jinwei Gu

Recently, data-driven methods that jointly denoise and mitigate MPI have become state-of-the-art without using the intermediate transient representation.

Denoising

GLEAN: Generative Latent Bank for Large-Factor Image Super-Resolution

no code implementations CVPR 2021 Kelvin C. K. Chan, Xintao Wang, Xiangyu Xu, Jinwei Gu, Chen Change Loy

We show that pre-trained Generative Adversarial Networks (GANs), e. g., StyleGAN, can be used as a latent bank to improve the restoration quality of large-factor image super-resolution (SR).

Image Super-Resolution

JPD-SE: High-Level Semantics for Joint Perception-Distortion Enhancement in Image Compression

1 code implementation24 May 2020 Shiyu Duan, Huaijin Chen, Jinwei Gu

Moreover, they focus mostly on rate-distortion and tend to underperform in perception quality especially in low bitrate regime, and often disregard the performance of downstream computer vision algorithms, which is a fast-growing consumer group of compressed images in addition to human viewers.

Image Compression

Video Stitching for Linear Camera Arrays

no code implementations31 Jul 2019 Wei-Sheng Lai, Orazio Gallo, Jinwei Gu, Deqing Sun, Ming-Hsuan Yang, Jan Kautz

Despite the long history of image and video stitching research, existing academic and commercial solutions still produce strong artifacts.

Autonomous Driving Spatial Interpolation

NRMVS: Non-Rigid Multi-View Stereo

no code implementations12 Jan 2019 Matthias Innmann, Kihwan Kim, Jinwei Gu, Matthias Niessner, Charles Loop, Marc Stamminger, Jan Kautz

We show that creating a dense 4D structure from a few RGB images with non-rigid changes is possible, and demonstrate that our method can be used to interpolate novel deformed scenes from various combinations of these deformation estimates derived from the sparse views.

3D Reconstruction Depth Estimation

Very Power Efficient Neural Time-of-Flight

no code implementations19 Dec 2018 Yan Chen, Jimmy Ren, Xuanye Cheng, Keyuan Qian, Jinwei Gu

Therefore, the design of a power efficient ToF camera always creates a painful dilemma for the illumination and the performance trade-off.

PlaneRCNN: 3D Plane Detection and Reconstruction from a Single Image

2 code implementations CVPR 2019 Chen Liu, Kihwan Kim, Jinwei Gu, Yasutaka Furukawa, Jan Kautz

This paper proposes a deep neural architecture, PlaneRCNN, that detects and reconstructs piecewise planar surfaces from a single RGB image.

3D Plane Detection 3D Reconstruction +1

Context-Aware Synthesis and Placement of Object Instances

2 code implementations NeurIPS 2018 Donghoon Lee, Sifei Liu, Jinwei Gu, Ming-Yu Liu, Ming-Hsuan Yang, Jan Kautz

Learning to insert an object instance into an image in a semantically coherent manner is a challenging and interesting problem.

Object Scene Parsing

Switchable Temporal Propagation Network

1 code implementation ECCV 2018 Sifei Liu, Guangyu Zhong, Shalini De Mello, Jinwei Gu, Varun Jampani, Ming-Hsuan Yang, Jan Kautz

Our approach is based on a temporal propagation network (TPN), which models the transition-related affinity between a pair of frames in a purely data-driven manner.

Video Compression

Geometry-Aware Learning of Maps for Camera Localization

1 code implementation CVPR 2018 Samarth Brahmbhatt, Jinwei Gu, Kihwan Kim, James Hays, Jan Kautz

Maps are a key component in image-based camera localization and visual SLAM systems: they are used to establish geometric constraints between images, correct drift in relative pose estimation, and relocalize cameras after lost tracking.

Camera Localization Visual Localization

Separating Reflection and Transmission Images in the Wild

no code implementations ECCV 2018 Patrick Wieschollek, Orazio Gallo, Jinwei Gu, Jan Kautz

The reflections caused by common semi-reflectors, such as glass windows, can impact the performance of computer vision algorithms.

Synthetic Data Generation

Learning Affinity via Spatial Propagation Networks

no code implementations NeurIPS 2017 Sifei Liu, Shalini De Mello, Jinwei Gu, Guangyu Zhong, Ming-Hsuan Yang, Jan Kautz

Specifically, we develop a three-way connection for the linear propagation model, which (a) formulates a sparse transformation matrix, where all elements can be the output from a deep CNN, but (b) results in a dense affinity matrix that effectively models any task-specific pairwise similarity matrix.

Colorization Face Parsing +4

Learning Affinity via Spatial Propagation Network

no code implementations3 Oct 2017 Sifei Liu, Shalini De Mello, Jinwei Gu, Guangyu Zhong, Ming-Hsuan Yang, Jan Kautz

Specifically, we develop a three-way connection for the linear propagation model, which (a) formulates a sparse transformation matrix, where all elements can be the output from a deep CNN, but (b) results in a dense affinity matrix that effectively models any task-specific pairwise similarity matrix.

Colorization Face Parsing +4

Learning to Segment Instances in Videos with Spatial Propagation Network

no code implementations14 Sep 2017 Jingchun Cheng, Sifei Liu, Yi-Hsuan Tsai, Wei-Chih Hung, Shalini De Mello, Jinwei Gu, Jan Kautz, Shengjin Wang, Ming-Hsuan Yang

In addition, we apply a filter on the refined score map that aims to recognize the best connected region using spatial and temporal consistencies in the video.

Object Segmentation +1

Polarimetric Multi-View Stereo

no code implementations CVPR 2017 Zhaopeng Cui, Jinwei Gu, Boxin Shi, Ping Tan, Jan Kautz

Multi-view stereo relies on feature correspondences for 3D reconstruction, and thus is fundamentally flawed in dealing with featureless scenes.

3D Reconstruction

Dynamic Facial Analysis: From Bayesian Filtering to Recurrent Neural Network

no code implementations CVPR 2017 Jinwei Gu, Xiaodong Yang, Shalini De Mello, Jan Kautz

We are inspired by the fact that the computation performed in an RNN bears resemblance to Bayesian filters, which have been used for tracking in many previous methods for facial analysis from videos.

 Ranked #1 on Head Pose Estimation on BIWI (MAE (trained with BIWI data) metric, using extra training data)

Face Alignment Feature Engineering +3

A Lightweight Approach for On-the-Fly Reflectance Estimation

no code implementations ICCV 2017 Kihwan Kim, Jinwei Gu, Stephen Tyree, Pavlo Molchanov, Matthias Nießner, Jan Kautz

In addition, we have created a large synthetic dataset, SynBRDF, which comprises a total of $500$K RGBD images rendered with a physically-based ray tracer under a variety of natural illumination, covering $5000$ materials and $5000$ shapes.

Color Constancy

Learning Discriminative Illumination and Filters for Raw Material Classification with Optimal Projections of Bidirectional Texture Functions

no code implementations CVPR 2013 Chao Liu, Geifei Yang, Jinwei Gu

We built an LED-based multispectral dome, with which we have acquired a BTF database of a variety of materials and demonstrated the effectiveness of the proposed approach for material classification.

Classification General Classification +1

Cannot find the paper you are looking for? You can Submit a new open access paper.