Integrating CNNS and LSTMs for mouse behavior classification Presenter

Understanding animal behavior patterns is central for advancing research efforts in behavioral science. Accurately classifying behaviors allows for insights into how animals respond to different stimuli. Data frames annotated for key points on a mouse’s body were collected to identify specific mouse behaviors, such as when the mouse was shaking or licking. These labeled data points were subsequently used to train a deep learning model. The model uses both Convolutional Neural Networks (CNNs), which can obtain spatial features from individual frames of data, and Long Short- Term Memory (LSTM) networks, which are skilled at modeling long-term temporal dependencies within sequential data. This combination enabled the model to ingest both the visual data of each frame, i.e., the position of key annotated points, and the temporal patterns of behavior. The final model provides a tool for classifying mouse behaviors based on spatial and temporal information. The model was able to detect when the mouse was performing certain behaviors at the same time with precision but had difficulties identifying when only one of the behaviors was present.