Toothbrushing Monitoring Using an RGB-D Camera and Smart Toothbrush-Integrated IMU

Accurate tracking of toothbrush position and identifying brushing regions are critical challenges in developing effective oral hygiene monitoring systems. This paper presents a novel approach that integrates an inertial measurement unit (IMU) and an orange ball marker attached to a toothbrush with an external red-green-blue-depth (RGB-D) camera. The proposed method consists of two primary components: a toothbrush position estimation algorithm and a toothbrushing region classification method. By leveraging sensor fusion from the IMU and RGB-D camera, the orientation and position of the toothbrush are accurately estimated. The brushing direction vector is then computed from the relative positions of the toothbrush head and the user’s mouth. This direction vector, combined with acceleration and orientation data, is used as input for a transformer encoder-based classification model, which categorizes brushing activity into 16 distinct regions. Experimental results demonstrate that our classification method outperforms existing approaches, achieving an F1-score of 94.85% in the all-subjects evaluation and 81.56% in the leave-one-subject-out scenario. Additionally, the toothbrush position estimation algorithm achieves an average root mean square errors (RMSE) of 1.4 cm. These results highlight the effectiveness and reliability of the proposed method, positioning it as a promising solution for toothbrushing assessment and real-time feedback systems.