HAST: A Hardware-Efficient Spatio-Temporal Correlation Near-Sensor Noise Filter for Dynamic Vision Sensors

The Dynamic Vision Sensor (DVS) is a bio-inspired image sensor which has many advantages such as high dynamic range, high bandwidth, high temporal resolution and low power consumption for Internet of Video Things and Edge Computing applications. However, spuriously generated Background Activity (BA) noise events can significantly degrade the quality of DVS output and cause unnecessary computations throughout the image processing chain, reducing its energy efficiency. Near-sensor filters can mitigate this problem by preventing the BA noise events from reaching downstream stages. In this paper, we propose a novel, hardware-efficient, spatio-temporal correlation filter (HAST) for near-sensor BA noise filtering. It uses compact two-dimensional binary arrays along with simple, arithmetic-free hash-based functions for storage and retrieval operations. This approach eliminates the need to use timestamps for determining the chronological order of events. HAST uses much lower memory and energy compared to other hardware-friendly filters (BAF/STCF) while matching their performance in simulations with standard datasets; for a sensor of resolution <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$346\times 260$ </tex-math></inline-formula> pixels, it requires only 5–18% of their memory, and about 15% of their energy per event for correlation time <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\tau $ </tex-math></inline-formula> ranging from 1 to 50 ms. The memory and energy gains of the filter increase with sensor resolution. In FPGA implementation, HAST achieves about 29% higher throughput than BAF/STCF while utilizing only about 5% of their memory. The filter parameter values can be chosen by Design Space Exploration (DSE) for optimized performance-resource trade-offs based on application requirements.