Prediction of Imaging Data Based on Signal Processing Boosted by Neural Networks with Continual Learning

Modern video codecs rely on data prediction techniques to attain good compression performance. Such prediction techniques work on a block-by-block basis and rely on traditional signal processing, namely interpolation and extrapolation techniques, to predict each frame. Neural networks (NN) trained off-line have recently been shown to improve these traditional prediction techniques. However, as with any NN that requires training before operation, the prediction performance is conditioned on the training data. Furthermore, the learned parameters must be included in the compressed bitstream to reverse the prediction process when decompressing the frames, thus increasing bitrates. This work proposes a novel strategy to improve the prediction obtained by traditional signal processing by using a boosting technique that is based on fully-connected NNs (FC-NNs) that continually learn as the frames are predicted block-by-block. The proposed strategy does not require storing the learned parameters to reverse the prediction process because these parameters are refined in an on-line manner using only the data being predicted. Our evaluations show that the proposed strategy can improve the traditional block-based prediction and other related NN-based strategies with prediction accuracy gains of up to 17.064 dB PSNR. Our evaluations also show that the increase in computational complexity is negligible, with an average increase of 1.1%.