STC-Colornet: Fully Automatic Video Colorization via Dual Decoders

Most contemporary fully automated video colorization solutions stem from techniques initially designed for colorizing individual frames. Frame colorization presents a significant challenge due to its inherently multi-modal uncertainty and high degree of ill-posedness. Directly training a deep neural network often results in inaccurate semantic color assignments and diminished color richness. Moreover, analysing video frames independently results in abrupt color changes, undermining temporal coherence throughout the entire sequence. This inconsistency can cause visual fatigue for viewers. To address these issues, we propose an end-to-end network employing a dual decoder architecture for fully automated video colorization, which accurately reinstates the intricate structures and color representations of each frame while mitigating inter-frame color inconsistencies. The pixel decoder reinstates the image's spatial resolution, while the color decoder utilizes rich visual features to enhance color accuracy. Employing a recursive neural network architecture ensures the preservation of temporal and contextual coherence across successive video frames. Experimental findings demonstrate that STC-ColorNet effectively balances both the temporal consistency of videos and the quality of colorization in individual frames, substantially enhancing the quality of black and white video colorization through the integration of recursive neural network architecture and dual decoders.