Hybrid-Eyecon: An End-To-End Controller With a Multi-Task and Multi-Dataset Perception Network

This work proposes an end-to-end learning-based controller for the task of lane-following by an autonomous vehicle. The perception module is developed as a unified camera model that is able to perform multiple tasks namely-object detection, lane segmentation, free-space segmentation, lane-detection and depth-estimation from a single RGB camera image. Since, there is no single data-source that contains labels for training all the above mentioned tasks at once, we propose a novel training methodology that enables us to train a single network on many different task-specific datasets in an asynchronous manner. Based on empirical studies we demonstrate that the multi-dataset training outperforms standard method in combined task with a marginal trade-off in the object detection task. Subsequently, we combine the perception module with a path-following learning based controller and deploy it on real-vehicle and test the end-to-end controller on real roads.