Bionic Hand Control with Real-Time B-Mode Ultrasound Web AI Vision

Aim: This basic research study aimed to assess the ability of Web AI Vision to classify anatomical movement patterns in real-time B-mode ultrasound scans for controlling a virtual bionic limb. Methods: A MobileNetV2 model, implemented via the TensorFlow.js library, was used for transfer learning and feature extraction from 400 B-mode ultrasound images of the distal forearm of one individual participant, corresponding to four different hand positions: 100 images of a fist position, 100 images of thumb palmar abduction, 100 images of a fist with an extended forefinger, and 100 images of an open palm. Results: After 32 epochs of training with a learning rate of 0.001 and a batch size of 16, the model achieved 100% validation accuracy, 100% test accuracy, and a test loss (crossentropy) of 0.0067 in differentiating ultrasound patterns associated with specific hand positions. During manual testing with 40 ultrasound images excluded from training, validation, and testing, the AI was able to correctly predict the hand position in all 40 cases (100%), with a mean predicted probability of 98.9% (SD ± 0.6). When tested with B-mode cine loops and live ultrasound scanning, the model successfully performed real-time predictions with a 20 ms interval between predictions, achieving 50 predictions per second. Conclusion: This study demonstrated the ability of Web AI Vision to classify anatomical movement patterns in real-time B-mode ultrasound scans for controlling a virtual bionic limb. Such ultrasound- and Web AI-powered bionic limbs can be easily and automatically retrained and recalibrated in a privacy-safe manner on the client side, within a web environment, and without extensive computational costs. Using the same ultrasound scanner that controls the limb, patients can efficiently adjust the Web AI Vision model with new B-mode scans as needed, without relying on external services. The advantages of this combination warrant further research into AI-powered muscle movement analysis and the utilization of ultrasound-powered Web AI in rehabilitation medicine, neuromuscular disease management, and advanced prosthetic control for amputees.