Segmenting Dynamic Human-Object Interactions via Boundary Supervision

Dynamic human-object interaction segmentation from videos focuses on the dense labeling of sequences involving multiple human-object interaction classes. Previous approaches often combined classification and segmentation in frame-level learning models, leading to significant computational strain and over-segmentation errors. In this paper, we introduce a boundary supervision branch designed to regularize temporal features by incorporating boundary labels. Specifically, we generate boundary labels for human-object interaction videos and apply them to refine the temporal features and alleviate over-segmentation errors. Additionally, we propose a novel spatial-temporal fusion module that effectively captures and fuses spatial and temporal dependencies. The experimental results on two human-object interaction datasets, CAD-120 and Bimanual Actions, validate the effectiveness of our approach and demonstrate state-of-the-art performance.