Accessible Automation: Evaluating Object Segmentation Solutions for Parent-Child Interaction Research

Lab-based parent-child interaction (PCI) studies enable researchers to observe real-time behaviours in a controlled setting. With the rise of head-mounted eye-tracking and cameras, these studies now capture even richer data. However, extracting meaningful variables often requires time-consuming manual annotation. One key variable of interest to developmental scientists, due to its links to attention and learning, is the size of objects in the child’s view. Manually extracting object sizes from a single 5-minute recording (9,000 frames at 30 FPS) can take up to 225 hours. Advances in computer vision now offer automated solutions. In this study, we evaluated six automated object segmentation solutions for their ability to extract object size from PCI videos featuring distinctly coloured objects: Colour-based extraction, Segment and Track Anything (SAM-Track), Segment Anything Model 2 (SAM2), DeepLabv3, PyTorch U-Net, and You Only Look Once (YOLOv11). Some solutions require minimal setup (Colour-based extraction, SAM-Track, and SAM2), whilst others require custom training by providing manually annotated frames (DeepLabv3, PyTorch U-Net, and YOLOv11). Two of the out-of-the-box models (SAM-Track and SAM2) and two of the custom-trained models (PyTorch U-Net and YOLOv11) demonstrated very high object segmentation accuracy (median Dice scores = .92 – .96; median IoU scores = .85 – .92). Therefore, these tools offer a scalable and accessible way to automate object segmentation, reducing annotation time from months to hours, and thus enabling broader application of this approach in developmental science.