A DVS-CIS Sensor Data Receiver on FPGA with a 10 Gbps MIPI Controller

Fusing a dynamic vision sensor (DVS) and a CMOS image sensor (CIS) is promising in real-time vision applications. However, unlike common CIS, DVS typically come with a custom data format due to their naturally sparse data, which becomes a challenge to fuse DVS and CIS data streams on a general-purpose CPU. To address this problem, this work proposes a DVS-CIS sensor stream receiver on FPGA. The proposed receiver incorporates a cost-effective address decoder and an inline transpose to receive and store a DVS stream on DRAM effectively. At a system level, a host PC can stream the DVS-CIS stream from FPGA via PCIe and display streams on a monitor. Experimental results demonstrate that our architecture can decode up to 13,900fps of DVS frames without incurring any frame drops while concurrently streaming frames at 60fps from a CIS. The design only uses 135 BRAM, 38 DSPs, 69489 LUTs, and 86626 FFs on a Xilinx Zynq+ ZCU106 FPGA board and consumes a power of 6.977 W.