SPECTRA: A Drone-based Multispectral Sensing Platform for Complex Environment Perception

In complex environments where visibility is severely compromised, such as smoke-filled areas or dense forests, traditional single-sensor systems often fail to provide accurate and reliable data for robots to autonomously navigate and avoid obstacles effectively, posing significant operational challenges and safety risks. Ground-based sensing platforms are further limited by their restricted mobility, hindering access to remote or hazardous areas. Multimodal sensing, which combines various sensor technologies, offers a robust solution to these challenges. Drones, with their high maneuverability and ability to reach inaccessible locations, can capture detailed data from varied altitudes and perspectives. In this demonstration, we introduce SPECTRA, a drone-based multispectral sensing platform that combines thermal cameras, LiDAR, mmWave, and RGB cameras to reliably perform tasks in challenging environments. SPECTRA is a software-hardware co-design platform that can enable and fuse different sensors locally based on the dynamic environment and interpret user tasks using Large Language Models (LLM). We demonstrate SPECTRA's ability to explore and execute assigned tasks in complex environments.