Review of Smart Auto X: Autonomous Self Driving Car Using IoT and Deep Learning Technologies

This study introduces a comprehensive framework for autonomous vehicle control that integrates data-driven modelling, deep learning, and multi-sensor fusion to enhance robustness and adaptability within dynamic driving contexts. Initially, the nonlinear vehicle dynamics are modelled using the Deep Koopman Operator (DK) methodology, wherein deep neural networks extract basis functions to approximate the infinite-dimensional Koopman operator within a lifted linear space. An Extended State Observer (ESO) is utilized to estimate total disturbances in real time and to compensate for model uncertainties, resulting in an ESO-based Deep Koopman Model Predictive Control (ESO-DKMPC) scheme that improves trajectory-tracking precision. To further bolster policy robustness, imitation learning is incorporated via perturbation-based data augmentation, facilitating effective generalization to previously unseen driving scenarios. Concurrently, an end-to-end convolutional neural network (CNN) is employed to directly map raw camera inputs to steering commands, thereby jointly optimizing perception, planning, and control processes. Additionally, sensor fusion techniques that integrate LiDAR, GNSS, IMU, and wheel encoder data are applied to enhance localization accuracy and navigation resilience in complex environments. Simulation and co-simulation experiments conducted on the Car Sim/Simulink platform demonstrate that the proposed hybrid control framework outperforms traditional linear and nonlinear model predictive control approaches, as well as standalone learning-based methods, in terms of tracking performance and generalization capability.