Video Super-Resolution with Recurrent Structure-Detail Network

ECCV 2020  ·  Takashi Isobe, Xu Jia, Shuhang Gu, Songjiang Li, Shengjin Wang, Qi Tian ·

Most video super-resolution methods super-resolve a single reference frame with the help of neighboring frames in a temporal sliding window. They are less efficient compared to the recurrent-based methods. In this work, we propose a novel recurrent video super-resolution method which is both effective and efficient in exploiting previous frames to super-resolve the current frame. It divides the input into structure and detail components which are fed to a recurrent unit composed of several proposed two-stream structure-detail blocks. In addition, a hidden state adaptation module that allows the current frame to selectively use information from hidden state is introduced to enhance its robustness to appearance change and error accumulation. Extensive ablation study validate the effectiveness of the proposed modules. Experiments on several benchmark datasets demonstrate the superior performance of the proposed method compared to state-of-the-art methods on video super-resolution.

PDF Abstract ECCV 2020 PDF ECCV 2020 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Video Super-Resolution MSU Super-Resolution for Video Compression RSDN + uavs3e BSQ-rate over ERQA 18.327 # 73
Compare
BSQ-rate over VMAF 9.796 # 77
Compare
BSQ-rate over PSNR 15.144 # 73
Compare
BSQ-rate over MS-SSIM 11.643 # 82
Compare
BSQ-rate over LPIPS 13.844 # 78
Compare
Video Super-Resolution MSU Super-Resolution for Video Compression RSDN + x265 BSQ-rate over ERQA 13.416 # 59
Compare
BSQ-rate over VMAF 6.467 # 67
Compare
BSQ-rate over PSNR 13.403 # 69
Compare
BSQ-rate over MS-SSIM 5.682 # 64
Compare
BSQ-rate over LPIPS 13.232 # 74
Compare
Video Super-Resolution MSU Super-Resolution for Video Compression RSDN + x264 BSQ-rate over ERQA 6.58 # 29
Compare
BSQ-rate over VMAF 1.5 # 31
Compare
BSQ-rate over PSNR 13.348 # 68
Compare
BSQ-rate over MS-SSIM 1.023 # 21
Compare
BSQ-rate over LPIPS 10.775 # 54
Compare
Video Super-Resolution MSU Super-Resolution for Video Compression RSDN + aomenc BSQ-rate over ERQA 20.617 # 79
Compare
BSQ-rate over VMAF 10.67 # 81
Compare
BSQ-rate over PSNR 15.144 # 73
Compare
BSQ-rate over MS-SSIM 11.643 # 82
Compare
BSQ-rate over LPIPS 14.574 # 80
Compare
Video Super-Resolution MSU Super-Resolution for Video Compression RSDN + vvenc BSQ-rate over ERQA 14.95 # 66
Compare
BSQ-rate over VMAF 10.145 # 78
Compare
BSQ-rate over PSNR 14.061 # 70
Compare
BSQ-rate over MS-SSIM 9.138 # 79
Compare
BSQ-rate over LPIPS 4.866 # 43
Compare
Video Super-Resolution MSU Video Super Resolution Benchmark: Detail Restoration RSDN Subjective score 5.566 # 10
Compare
ERQAv1.0 0.667 # 15
Compare
QRCRv1.0 0.619 # 7
Compare
SSIM 0.826 # 20
Compare
PSNR 25.321 # 25
Compare
FPS 1.961 # 8
Compare
1 - LPIPS 0.819 # 26
Compare
Video Super-Resolution Vid4 - 4x upscaling - BD degradation RSDN PSNR 27.92 # 9
Compare
SSIM 0.8505 # 9
Compare

Methods
Add Remove

No methods listed for this paper. Add relevant methods here
Contact us on: hello@paperswithcode.com . Papers With Code is a free resource with all data licensed under CC-BY-SA.
Terms Data policy Cookies policy