An Obstacle Tracking Pipeline for Autonomous Surface Vehicles Based on Optical Camera, Infrared Camera and LiDAR

This work presents a complete obstacle tracking pipeline for ASVs (Autonomous Surface Vehicles), employing sensor fusion based on optical camera, infrared camera and LiDAR. The system is thoroughly described, implemented and validated on the public MIT Grant Dataset, which collects LiDAR, optical camera and thermal camera data in various sea scenarios. Two YOLO-based neural networks are developed for obstacle detection on the image plane, respectively for handling optical and infrared images. Key results of this paper are the proposal of a full pipeline for obstacle detection and tracking and the presence of several tests on real data.