Behavior Recognition in Mice Using RGB-D Videos Captured from Below

Changes in mouse behavior are useful for both basic and applied research. However, visual inspection by humans is subjective and time-consuming. With the advancement of deep learning, systems have been developed that are capable of automatically and quantitatively classifying mouse behavior from videos. As camera angles are typically from above or the side, consistently capturing keypoints related to limb movement can be challenging. In this study, a mouse was placed on a transparent acrylic plate and its movements were recorded from below using an RGB-D camera, successfully capturing its limbs in 3D at all times. Furthermore, by using DeepLabCut, the 3D coordinates of the mouse's keypoints were obtained. By using deep learning with the time-series data of these obtained keypoints coordinates and corresponding behavioral labels, we created a model that classifies mouse behaviors from videos. This method achieved a total accuracy of 96.7% and a walking classification accuracy of 94.5%, demonstrating higher precision compared to previous studies.