Low-Latency H.264/AVC Encoder Hardware Design with Wavefront Parallelism on FPGA Platform

With network transmission delays reduced, video encoding latency emerges as a critical challenge for live video streaming. Nowadays, H.264/AVC remains the most popular video encoding standard in production due to its relatively low complexity. Therefore, an H.264 encoder with lower encoding latency and cost still has significant application value, but the fact that H.264 standard is not optimized for parallel processing has become a limitation to further reducing encoding latency. In this paper, we propose a dual-core hardware implementation of an H.264-based video encoder that performs wavefront parallelism on FPGA platform and provides full compatibility with the H.264 Baseline Profile. The main challenge is the maintenance of data dependencies of adjacent macroblock (MB) lines, introduced by the dual-core parallelism. To handle this problem, we propose a core scheduling mechanism and use a pair of single-input single-output (SISO) on-chip data channels to maintain data dependencies between adjacent MB lines. Furthermore, we combine the output from two sub-encoders to produce a bitstream compatible with a standard H.264 decoder. The encoder achieves 125fps for 1080p video and supports both intra and inter coding, providing real-time performance and flexibility in various live streaming scenarios. Results show that our design can reduce up to 49.4% of overall video encoding latency compared to MB serial processing without the loss of H.264 standard compatibility and rate-distortion performance.