DuSiamIE: A Lightweight Multidimensional Infrared-Enhanced RGBT Tracking Algorithm for Edge Device Deployment

Advancements in deep learning and infrared sensors have facilitated the integration of RGB-thermal (RGBT) tracking technology in computer vision. However, contemporary RGBT tracking methods handle complex image data, resulting in inference procedures with a large number of floating-point operations and parameters, which limits their performance on general-purpose processors. We present a lightweight Siamese dual-stream infrared-enhanced RGBT tracking algorithm, called DuSiamIE.It is implemented on the low-power NVIDIA Jetson Nano to assess its practicality for edge-device applications in resource-limited settings. Our algorithm replaces the conventional backbone network with a modified MobileNetV3 and incorporates light-aware and infrared feature enhancement modules to extract and integrate multimodal information. Finally, NVIDIA TensorRT is used to improve the inference speed of the algorithm on edge devices. We validated our algorithm on two public RGBT tracking datasets. On the GTOT dataset, DuSiamIE achieved a precision (PR) of 83.4% and a success rate (SR) of 66.8%, with a tracking speed of 40.3 frames per second (FPS). On the RGBT234 dataset, the algorithm achieved a PR of 75.3% and an SR of 52.6%, with a tracking speed of 34.7 FPS. Compared with other algorithms, DuSiamIE exhibits a slight loss in accuracy but significantly outperforms them in speed on resource-constrained edge devices. It is the only algorithm among those tested that can perform real-time tracking on such devices.