Semi‐autonomous Navigation of Magnetic Microcarriers for Knee Cartilage Regeneration

Autonomous navigation of microcarriers in complex 3D environments poses a significant challenge for effective minimally invasive intervention applications. Achieving this task using only monocular camera feedback further increases the difficulty. In this work, a semi‐autonomous strategy is proposed that leverages the combination of magnetic actuation control of magnetic microcarriers and visual feedback from a monocular camera device. The main challenges of this strategy include tracking the 3D position of the microcarriers in real‐time for autonomous control and determining the optimal magnetic force to steer the microcarriers from the injection point to the target area. To address these challenges, first, a 3D radio‐frequency localization method is introduced to track the pose of the camera, enabling the transformation of particle positions into global coordinates. This is achieved using a single transmitting coil and a miniature receiving unit attached to the camera. Next, a monocular depth estimation method based on handcrafted depth cues is implemented for real‐time 3D position tracking of the carriers. The most suitable magnetic force for steering the microcarriers to the target area with minimal effort is precomputed using a model‐based algorithm. The proposed autonomous targeting strategy is successfully demonstrated in a human‐sized knee phantom setup.