UAV-StrawFire: A visible and infrared dataset for real-time straw-fire monitoring with deep learning and image fusion

Straw burning poses significant threats to local air quality and nearby public health by emitting harmful pollutants during specific seasons. Traditional satellite-based remote sensing techniques encounter difficulties in monitoring small-scale straw-burning events due to long revisit intervals and low spatial resolution. To address this challenge, unmanned aerial vehicles (UAVs) equipped with imaging sensors have emerged as a rapid and cost-effective solution for monitoring and detecting straw fires. This paper presents the UAV-StrawFire dataset, which comprises RGB images, thermal infrared images, and videos captured during controlled straw residue burning experiments in southern China using drones. The dataset is annotated and labeled to support the application of detection, segmentation, and tracking algorithms. This study addresses three key machine learning tasks using the dataset: (1) flame detection, achieved through a feature-based multi-modal image fusion model (FF-YOLOv5n) reaching a mAP50-95 of 0.5764; (2) flame segmentation, which delineates fire boundaries using the real-time lightweight BiSeNetV2 model, achieving a high mean Intersection over Union (mIoU) score exceeding 0.88; and (3) flame tracking, which monitors the real-time progression of straw burning with a precision of 0.9065 and a success rate of 0.6593 using the Aba-ViTrack algorithm, suitable for on-board processing on UAVs at 50 frames per second (FPS). These experiments provide efficient baseline models for UAV-based straw-burning monitoring with edge computing capabilities. The UAV-StrawFire dataset enables the detection and monitoring of flame regions with varying sizes, textures, and opacities, thereby supporting potential straw-burning control efforts. The dataset is publicly available on IEEE Dataport, offering a valuable resource for researchers in the remote sensing and machine learning communities to advance the development of effective straw-burning monitoring systems. • Introduce an RGB-TIR paired straw fire detection, segmentation and tracking dataset. • Propose RGB-TIR fusion baseline model FF-YOLOv5n for straw fire detection. • Delineate straw fire flame boundaries using lightweight segmentation models. • Achieve real-time straw fire tracking onboard UAV with 50 FPS.