Multimodal near-space surveillance system

Near-space surveillance is emerging as a pivotal tool to address the challenges of climate observation, resource exploration, and disaster evaluation. Large-scale multi-modal data enables to improve analysis accuracy, which however faces the challenge of limited downlink bandwidth and storage resources in the near-space platform. In this work, we developed a near-space multi-modal surveillance system, which not only enables multi-modal video acquisition but also realizes efficient data storage control and low-latency, low-bandwidth data transmission. Specifically, a snapshot hyperspectral camera (with 2048×2048 spatial resolution, 5 nm spectral resolution, covering a wide spectral range from 400 to 1000 nm), an infrared camera (with 640×512 pixel resolution), and an RGB camera (with 2448×2048 pixel resolution) were equipped together, to enable synchronous wide-spectrum multi-modal data acquisition with a maximum frame rate of 24 fps. To handle the massive heterogeneous data sets generated by the multiple cameras, a B+Tree index was constructed with the data acquisition time as the primary key, which reduces the time complexity of data retrieval from linear level to logarithmic level. A UDP based image transmission protocol was designed to reduce communication latency by eliminating head-of-line blocking and handshake delays caused by TCP. Remote resource management, on-demand image transmission selection and flexible acquiring control of multi-camera were implemented to further reduce storage space and transmission bandwidth usage. Experiments validated the system’s capability to operate normally under conditions of -50 degrees Celsius temperature and 5 kPa pressure, concurrently affirming the enhanced stability, reliability, and efficiency brought by the aforementioned design.