A Hardware-Aware and Content-Adaptive Super Resolution System for Bandwidth-Limited IoT Video Delivery

The demand for video delivery in the Internet of Things (IoT) is increasing.Limited Internet bandwidth is under immense pressure and struggles to meet the requirements for low latency.This paper proposes a low-bandwidth IoT video delivery solution that transmits low-resolution videos over the Internet and uses superresolution (SR) technology at the edge to upscale the videos and restore their quality.Firstly, a hardware-aware super-resolution (SR) neural architecture search (NAS) framework is proposed, which considers the latency of the target hardware to design super-resolution algorithms automatically, based on the observation that latency does not grow continuously with the amount of computation in the algorithm.This framework yields two super-resolution algorithms with different upscaling capabilities and latency.Additionally, a content-adaptive task scheduling scheme is introduced, which selects one of the two super-resolution algorithms for upscaling each image block based on the block's upscaling difficulty.Lastly, a heterogenous-multithreading deployment scheme is presented, which leverages the heterogeneous computing resources in the targeted hardware to parallelize the execution of various algorithm modules in the super-resolution system.The proposed superresolution system can achieve up to a 1.8 speedup.