Early fusion of laser and acoustic features for human orientation detection in non-line-of-sight environments

For search and rescue, security, and autonomous systems, determining the orientation of a human presence outside the line of sight is as crucial as detecting them. Knowing the human's orientation plays a crucial role in determining the intervention strategy. In this study, data were obtained from scenarios involving human orientation outside the line of sight in a set of experimental settings. The original dataset was created using laser and acoustic chirp signals for the scenarios created to detect human orientation (behind, front, left, right). Data obtained from multiple sensors were correlated at the feature level using early fusion. A dataset with 42 features was created by combining two sensor sources, each with 21 features. Human orientation classification yielded strong performance under the adopted controlled experimental conditions using machine learning methods, artificial intelligence methods, and the proposed LAO-Net (Laser-Acoustic Orientation) model. Furthermore, explainable artificial intelligence analysis revealed the features that influence human orientation detection. The results of the study indicate that the fused laser-acoustic representation achieves strong human orientation detection performance under the experimental conditions evaluated. This study contributes to the limited literature on detecting human orientation in environments outside the line of sight.