Machine Learning-enhanced Direction-of-Arrival Estimation for Coherent and Non-Coherent Sources

Accurate Direction-Of-Arrival (DOA) estimation for both coherent and non-coherent sources remains a critical challenge in array signal processing, particularly under sparse sensor configurations. This study introduces a novel 3D coprime array method that enhances source separation and spatial resolution. By leveraging a unique joint diagonalization framework with a full-rank Toeplitz matrix, the proposed approach effectively decorrelates coherent sources while preserving the accuracy of uncorrelated signals. A machine learning model can be employed to further refine the DOA estimates, utilizing a regression model or neural network to predict DOA based on features extracted from the covariance matrix. A new cost function, independent of the number of sources, is proposed to increase robustness in complex environments. Extensive simulations demonstrate that the proposed technique significantly outperforms established algorithms, including 3D Unitary Root-MUSIC, modified Root-MUSIC, ECA-MURE, and FBSS. The results reveal substantial improvements in Root Mean Square Error (RMSE) across various Signal-to-Noise Ratios (SNRs), affirming the method's effectiveness. Additionally, the approach's adaptability to different scenarios makes it suitable for real-world applications. These advances pave the way for improved applications in Unmanned Aerial Vehicles (UAVs), radar systems, and next-generation communication networks.