Learning to Generate Realistic Noisy Images via Pixel-level Noise-aware Adversarial Training
Existing deep learning real denoising methods require a large amount of noisy-clean image pairs for supervision. Nonetheless, capturing a real noisy-clean dataset is an unacceptable expensive and cumbersome procedure. To alleviate this problem, this work investigates how to generate realistic noisy images. Firstly, we formulate a simple yet reasonable noise model that treats each real noisy pixel as a random variable. This model splits the noisy image generation problem into two sub-problems: image domain alignment and noise domain alignment. Subsequently, we propose a novel framework, namely Pixel-level Noise-aware Generative Adversarial Network (PNGAN). PNGAN employs a pre-trained real denoiser to map the fake and real noisy images into a nearly noise-free solution space to perform image domain alignment. Simultaneously, PNGAN establishes a pixel-level adversarial training to conduct noise domain alignment. Additionally, for better noise fitting, we present an efficient architecture Simple Multi-scale Network (SMNet) as the generator. Qualitative validation shows that noise generated by PNGAN is highly similar to real noise in terms of intensity and distribution. Quantitative experiments demonstrate that a series of denoisers trained with the generated noisy images achieve state-of-the-art (SOTA) results on four real denoising benchmarks. Part of codes, pre-trained models, and results are available at https://github.com/caiyuanhao1998/PNGAN for comparisons.
PDF Abstract NeurIPS 2021 PDF NeurIPS 2021 AbstractCode
Task | Dataset | Model | Metric Name | Metric Value | Global Rank | Uses Extra Training Data |
Benchmark |
---|---|---|---|---|---|---|---|
Image Denoising | Nam | PNGAN | PSNR | 40.78 | # 1 | ||
SSIM | 0.986 | # 1 | |||||
Image Denoising | PolyU | PNGAN | PSNR | 40.55 | # 1 | ||
SSIM | 0.983 | # 1 | |||||
Noise Estimation | SIDD | PNGAN | PSNR Gap | 0.84 | # 1 | ||
Average KL Divergence | 0.153 | # 1 | |||||
Image Denoising | SIDD | PNGAN | PSNR (sRGB) | 40.07 | # 5 | ||
SSIM (sRGB) | 0.960 | # 6 |