FAST: A Framework to Accelerate Super-Resolution Processing on Compressed Videos



Abstract

State-of-the-art super-resolution (SR) algorithms require significant computational resources to achieve real-time throughput (e.g., 60Mpixels/s for HD video). This paper introduces FAST (Free Adaptive Super-resolution via Transfer), a framework to accelerate any SR algorithm applied to compressed videos. FAST exploits the temporal correlation between adjacent frames such that SR is only applied to a subset of frames; SR pixels are then transferred to the other frames. The transferring process has negligible computation cost as it uses information already embedded in the compressed video (e.g., motion vectors and residual). Adaptive processing is used to retain accuracy when the temporal correlation is not present (e.g., occlusions). FAST accelerates state-of-the-art SR algorithms by up to 15x with a visual quality loss of 0.2dB. FAST is an important step towards real-time SR algorithms for ultra-HD displays and energy constrained devices (e.g., phones and tablets).


Pipeline of FAST

Pipeline of FAST

Comparison of visual quality

Comparison of visual quality


Video Quality Evaluation

FAST accelerates SRCNN by 10✕ and maintains the high visual quality.

Uncropped and uncompressed results here: SRCNN, FAST with SRCNN, Bicubic



BibTeX


@inproceedings{cvprw_2017_zhang_fast,
    author      = {{Zhang, Zhengdong and Sze, Vivienne}},
    title       = {{FAST: A Framework to Accelerate Super-Resolution Processing on Compressed Videos}},
    booktitle   = {{CVPR Workshop on New Trends in Image Restoration and Enhancement}},
    year        = {{2017}}
}