Semi-Supervised Action Recognition From Newborn Resuscitation Videos

Newborn Resuscitation Algorithm Activities (NRAA) are complex and critical actions performed to save lives of newborns who are not breathing spontaneously at birth, and include stimulation, ventilation, and suction. The algorithm guidelines, i.e. the sequence, timing and proposed duration of NRAA is based on limited evidence. Videos from newborn resuscitation episodes and AI-based activity recognition can be helpful to generate timelines from birth to end of resuscitation, to evaluate compliance with guidelines and for quality improvement initiatives. Traditional supervised machine learning algorithms require a large amount of labeled data to achieve optimal performance. However, obtaining adequate number of videos and manual annotation of videos is often impractical both due to privacy concerns and time consumption. In this research work, we present a semi-supervised approach where a modified SVFormer model is trained on a dataset of recorded newborn resuscitation videos for recognition of different activities. Performance of used model is increased by employing data-specific pre-trained backbone and proper utilization of unlabeled data. Results show that the proposed pipeline provides comparable performance to supervised approach by just using $30 \%$ of labeled data.