Integrating a Stereo Vision System on the F1Tenth Platform for Enhanced Perception

During the development of vehicle control algorithms, effective real-world validation is crucial. Model vehicle platforms provide a cost-effective and accessible method for such testing. The open-source F1Tenth project is a popular choice, but its reliance on lidar sensors limits certain applications. To enable more universal environmental perception, integrating a stereo camera system could be advantageous, although existing software packages do not yet support this functionality. Therefore, our research focuses on developing a modular software architecture for the F1Tenth platform, incorporating real-time stereo vision-based environment perception, robust state representation, and clear actuator interfaces. The system simplifies the integration and testing of control algorithms, while minimizing the simulation-to-reality gap. The framework’s operation is demonstrated through a real-world control problem. Environmental sensing, representation, and the control method combine classical and deep learning techniques to ensure real-time performance and robust operation. Our platform facilitates real-world testing and is suitable for validating research projects.