A sensor fusion approach to autonomous ultrasound imaging of the lumbar region

Ultrasound imaging has several advantages compared to other diagnostic imaging techniques, including no use of radiation, low cost, and ease of use. However, it is heavily reliant on the experience of the sonographer, and work-related disorders are common. Automated ultrasound examination using a robot has been proposed to address these challenges. Previously proposed systems rely mainly on single-source RGB or depth imaging. RGB imaging is sensitive to ambient lighting variations and requires extensive training data. Depth imaging systems face alignment challenges due to data noise and environmental complexity. This study introduces a novel approach for autonomous ultrasound imaging through the fusion of depth and thermal imaging modalities, providing a comprehensive solution that addresses these limitations. Thermal imaging effectively distinguishes a patient’s contour from the surroundings, while depth imaging provides topographical information for efficient probe manipulation. The integration of these modalities creates a comprehensive depth map enriched with thermal data. This study primarily targets the lumbar region, which is an anatomical region where ultrasound imaging is commonly used for diagnoses. The developed fully autonomous system initially localizes the lumbar region in the fused depth-thermal image. Thereafter, it manipulates the tip of the ultrasound probe to an optimal scanning starting point and optimizes probe orientation to capture ultrasound images. These operations are executed through novel algorithms, including calibration-based fusion, intensity-based thresholding, depth image-based template matching, and force–torque control algorithms. The system’s performance was evaluated through repeatability and reproducibility tests, as well as a volunteer experiment, demonstrating consistent and reliable image acquisition. • Proposed an autonomous ultrasound imaging system using fused depth and thermal data. • Introduced a novel method for the fusion of depth and thermal imaging. • Proposed novel lumbar region localization using template matching in fused data. • Utilized novel algorithms for probe positioning and orientation optimization.