Robotic ultrasound scanning platform with autonomous control, multimodal human–machine interface and real-time image analysis

PURPOSE: Autonomous robots can streamline repetitive and time-consuming surgical tasks like ultrasound scanning. AI can provide the necessary intelligence, but for clinical acceptance, systems must move predictably, offer intuitive interaction, and maintain low latency on medical-grade hardware. METHODS: We present an integral robotic platform for autonomous ultrasound scanning. The platform allows to perform scanning on either manual or autonomous mode. The autonomous control combines AI-driven ultrasound target segmentation with geometric motion algorithms designed to focus the probe on the segmented target. We optimized the image processing models for specialized hardware to ensure real-time performance with constrained resources. Users can control the platform via natural language voice commands processed by a Large Language Model (LLM) and visualize the procedure through a synchronized 3D Digital Twin and an augmented reality (AR) environment. RESULTS: We tested the platform in hepatic tumor localization in a synthetic phantom. Autonomous tumor localization showed a success rate of 84% with an average execution time of 5.75 s. CONCLUSIONS: Our system successfully automates ultrasound tasks, allowing the user to control the process through an intuitive, multimodal interface. By pairing optimized AI with specialized hardware, we achieved low-latency, real-time adaptability with minimal hardware resources. In the future we plan to adapt this platform for more complex neurosurgical and urological procedures.