Efficient data processing pipeline for event-based vision datasets: techniques and insights

Abstract Event-based vision datasets have emerged as a critical asset in advancing the capabilities of real-time perception systems, particularly in fields such as robotics, autonomous vehicles, and human-computer interaction. These datasets enable low-latency processing by capturing asynchronous, pixel-level changes in the scene, providing a distinct advantage over traditional frame-based systems. However, the diverse formats and characteristics of event-based datasets pose significant challenges for efficient processing and analysis, hindering their broader adoption and integration. In this paper, we present a versatile and comprehensive data processing pipeline designed to address these challenges by supporting multiple event-data formats, including newer formats such as EVT2 and EVT3. This pipeline not only converts data into widely supported formats like AEDAT and NPZ, but also ensures that the unique characteristics of event-based data-such as temporal precision and sparse event representation-are preserved throughout the conversion process. By applying this pipeline to several open-source datasets, we establish a standardized, efficient methodology for dataset manipulation that enhances compatibility and reproducibility in event-based vision research. Additionally, we introduce a novel high-resolution event-based action dataset, converted into various formats using our pipeline, which opens new avenues for exploring event-based techniques in action recognition. This dataset and our pipeline serve as valuable resources for the research community, enabling advancements in real-time vision applications and fostering greater collaboration and standardization across studies.