Multi-Scale Spatial and Channel Context Boosting Networks for Portrait Segmentation

Portrait segmentation is a highly pervasive but challenging task. Existing portrait segmentation models usually deploy auxiliary knowledge for high precision, e.g., face, pose, boundary, etc. However, it requires additional inputs or extra annotations. This paper proposes a solid context boosting model, namely CSSNet, that makes the most of contexts from convolutional neural networks (CNNs) to achieve state-of-the-art performance without the supervision of auxiliary knowledge. Firstly, the CSSNet yields a top-down pathway formed by the global prior and a succession of scale-specific fusing blocks. The global prior from pyramid pooling guides the general localization of portraits on top of the CNN backbone, while the fusing blocks progressively boost and incorporate CNN contexts from coarse to fine scales for fine-grained details. Secondly, the scale-specific fusing blocks thoroughly boost contexts with three modules, including the channel boosting module to enrich feature representations by simple linear operations, the spatial boosting module to enhance feature quality with various spatial statistics via spatial pyramid pooling, and the scale boosting module to integrate multi-scale CNN contexts. The CSSNet facilitates end-to-end segmentation and executes at 57.21 FPS on 300×400 images. Moreover, we summarize existing portrait segmentation datasets and propose to deploy two evaluation metrics for experimental analysis. Evaluation over four public datasets shows that the CSSNet achieves state-of-the-art performance without auxiliary supervision in model training.