Retinexformer: One-stage Retinex-based Transformer for Low-light Image Enhancement

When enhancing low-light images, many deep learning algorithms are based on the Retinex theory. However, the Retinex model does not consider the corruptions hidden in the dark or introduced by the light-up process. Besides, these methods usually require a tedious multi-stage training pipeline and rely on convolutional neural networks, showing limitations in capturing long-range dependencies. In this paper, we formulate a simple yet principled One-stage Retinex-based Framework (ORF). ORF first estimates the illumination information to light up the low-light image and then restores the corruption to produce the enhanced image. We design an Illumination-Guided Transformer (IGT) that utilizes illumination representations to direct the modeling of non-local interactions of regions with different lighting conditions. By plugging IGT into ORF, we obtain our algorithm, Retinexformer. Comprehensive quantitative and qualitative experiments demonstrate that our Retinexformer significantly outperforms state-of-the-art methods on thirteen benchmarks. The user study and application on low-light object detection also reveal the latent practical values of our method. Code, models, and results are available at https://github.com/caiyuanhao1998/Retinexformer

PDF Abstract ICCV 2023 PDF ICCV 2023 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Low-Light Image Enhancement DICM Retinexformer User Study Score 3.71 # 1
Low-Light Image Enhancement LIME Rextinexformer User Study Score 4.3 # 1
Low-Light Image Enhancement LOL Retinexformer Average PSNR 25.16 # 11
SSIM 0.845 # 14
FLOPS (G) 15.57 # 4
Params (M) 1.61 # 2
Low-Light Image Enhancement LOL Retinexformer_ Average PSNR 27.18 # 5
SSIM 0.850 # 13
FLOPS (G) 15.57 # 4
Params (M) 1.61 # 2
Low-light Image Deblurring and Enhancement LOL-Blur RetinexFormer SSIM 0.824 # 3
LPIPS 0.236 # 3
Average PSNR 22.904 # 3
Low-Light Image Enhancement LOLv2 Retinexformer Average PSNR 27.71 # 6
SSIM 0.856 # 7
Low-Light Image Enhancement LOL-v2 Retinexformer Average PSNR 22.80 # 3
SSIM 0.840 # 3
Low-Light Image Enhancement LOL-v2-synthetic Retinexformer PSNR 25.67 # 2
SSIM 0.939 # 2
Low-Light Image Enhancement LOLv2-synthetic Retinexformer Average PSNR 29.04 # 3
SSIM 0.939 # 3
Low-Light Image Enhancement MEF Retinexformer User Study Score 3.91 # 1
Image Enhancement MIT-Adobe 5k Retinexformer PSNR on proRGB 25.98 # 1
SSIM on proRGB 0.957 # 1
PSNR on sRGB 24.94 # 1
SSIM on sRGB 0.907 # 1
Photo Retouching MIT-Adobe 5k Retinexformer PSNR 24.94 # 3
SSIM 0.907 # 3
Low-Light Image Enhancement MIT-Adobe FiveK Retinexformer PSNR 24.94 # 1
SSIM 0.907 # 1
Low-Light Image Enhancement NPE Retinexformer User Study Score 4.17 # 1
Low-Light Image Enhancement SDSD-indoor Retinexformer PSNR 29.77 # 1
Low-Light Image Enhancement SDSD-outdoor Retinexformer PSNR 29.84 # 1
Low-Light Image Enhancement SID Retinexformer PSNR 24.44 # 1
SSIM 0.680 # 1
Low-Light Image Enhancement SMID Retinexformer PSNR 29.15 # 1
Low-Light Image Enhancement VV Rextinexformer User Study Score 3.61 # 1

Methods