A Deep Learning-Based Automatic Data Acquisition System for Medical Monitors

In the cardiac operating room, several operators are essential to assist the surgeon, including the physician managing and monitoring the artificial heart-lung machine. The custodian must interpret the patient's vital signs from equipment data and make decisions, such as blood transfusion. However, the equipment lacks automated data acquisition and recording capabilities, posing significant challenges for documenting surgical information. This paper introduces a system for screen segmentation and text recognition based on visual methods. This system allows the operating equipment doctor to wear a head-mounted camera to capture real-time video similar to the doctor's perspective, and pulls the video stream of the camera through RTSP (Real-Time Streaming Protocol) on the PC side. We proceed by processing the video stream captured by the camera, leveraging the YOLO (You Only Look Once) algorithm and OCR (Optical Character Recognition) technology as our primary tools to do screen segmentation and text recognition. These technologies enable us to extract the information displayed on the medical equipment screens. By initially employing YOLO for detecting and segmenting the screen of interest, the process is approximately 127% faster than direct OCR processing of the entire video frame. Additionally, the accuracy rate of OCR recognition for clear pictures can also reach more than 95% by our method.