A Real-time Sparse Semantic Segmentation Approach for Event-based Vision Sensors

Event-based vision sensors are biomimetic devices designed to capture data asynchronously, discretely, and sparsely, thereby providing high temporal resolution. Because we need cumulative time to transform event data into frame images to adapt image-based semantic segmentation methods, these methods waste the high temporal resolution properties of the event data and affect the real-time performance of the semantic segmentation task. This work explores a sparse semantic segmentation approach for event data as the only input to address these issues. Firstly, the event stream is denoised by the proposed outlier degree approach. Besides, we characterize the event stream as a four-dimensional event cloud and enhance the SA layer of the PointNet++ network to bolster the geometric feature extraction, attaining a novel 92.2% F1 Score on the DVS-OUTLAB dataset.