Adaptive model switching of collaborative inference for multi-CNN streams in UAV swarm

Unmanned Aerial Vehicles (UAVs) coupled with deep learning such as Convolutional Neural Networks (CNNs) have been widely applied across numerous domains, including agriculture, smart city monitoring, and fire rescue operations, owing to their malleability and versatility. However, the computation-intensive and latency-sensitive natures of CNNs present a formidable obstacle to their deployment on resource-constrained UAVs. Some early studies have explored a hybrid approach that dynamically switches between lightweight and complex models to balance accuracy and latency. However, they often overlook scenarios involving multiple concurrent CNN streams, where competition for resources between streams can substantially impact latency and overall system performance. In this paper, we first investigate the deployment of both lightweight and complex models for multiple CNN streams in UAV swarm. Specifically, we formulate an optimization problem to minimize the total latency across multiple CNN streams, under the constraints on UAV memory and the accuracy requirement of each stream. To address this problem, we propose an algorithm called Adaptive Model Switching of collaborative inference for Multi-CNN streams (AMSM) to identify the inference strategy with a low latency. Simulation results demonstrate that the proposed AMSM algorithm consistently achieves the lowest latency while meeting the accuracy requirements compared to benchmark algorithms.