Sequential Hierarchical Learning with Distribution Transformation for Image Super-Resolution

19 Jul 2020  ·  Yuqing Liu, Xinfeng Zhang, Shanshe Wang, Siwei Ma, Wen Gao ·

Multi-scale design has been considered in recent image super-resolution (SR) works to explore the hierarchical feature information. Existing multi-scale networks aim to build elaborate blocks or progressive architecture for restoration. In general, larger scale features concentrate more on structural and high-level information, while smaller scale features contain plentiful details and textured information. In this point of view, information from larger scale features can be derived from smaller ones. Based on the observation, in this paper, we build a sequential hierarchical learning super-resolution network (SHSR) for effective image SR. Specially, we consider the inter-scale correlations of features, and devise a sequential multi-scale block (SMB) to progressively explore the hierarchical information. SMB is designed in a recursive way based on the linearity of convolution with restricted parameters. Besides the sequential hierarchical learning, we also investigate the correlations among the feature maps and devise a distribution transformation block (DTB). Different from attention-based methods, DTB regards the transformation in a normalization manner, and jointly considers the spatial and channel-wise correlations with scaling and bias factors. Experiment results show SHSR achieves superior quantitative performance and visual quality to state-of-the-art methods with near 34\% parameters and 50\% MACs off when scaling factor is $\times4$. To boost the performance without further training, the extension model SHSR$^+$ with self-ensemble achieves competitive performance than larger networks with near 92\% parameters and 42\% MACs off with scaling factor $\times4$.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Image Super-Resolution Manga109 - 2x upscaling PMRN+ PSNR 39.15 # 10
SSIM 0.9781 # 8
Image Super-Resolution Manga109 - 3x upscaling PMRN+ PSNR 34.1 # 9
SSIM 0.9480 # 7
Image Super-Resolution Manga109 - 4x upscaling PMRN+ PSNR 31.07 # 25
SSIM 0.9144 # 26
Image Super-Resolution Set14 - 3x upscaling PMRN+ PSNR 29.24 # 17
SSIM 0.8087 # 9
Image Super-Resolution Set14 - 4x upscaling PMRN+ PSNR 27.72 # 64
SSIM 0.7405 # 66
Image Super-Resolution Set5 - 2x upscaling PMRN+ PSNR 38.22 # 11
SSIM 0.9612 # 9
Image Super-Resolution Set5 - 3x upscaling PMRN+ PSNR 34.65 # 13
SSIM 0.9289 # 9
Image Super-Resolution Urban100 - 2x upscaling PMRN+ PSNR 32.78 # 13
SSIM 0.9342 # 10

Methods