CESS: A Cascade-Exit Semantic Segmentation Network for High Performance Inference

Semantic segmentation constitutes an essential component of various visual tasks and is widely utilized in numerous applications. However, its implementation in real-world settings is frequently challenged by limitations pertaining to resources and latency. The early exit mechanism addresses these challenges by attaching Internal Segmentation Heads (ISHs) to intermediate layers of the network, facilitating a gradual segmentation of the entire image from less complex to more intricate regions. This methodology mitigates the aggregate computational burden by obviating the necessity to execute the entire network across all portions of the image. However, in antecedent methodologies, ISHs function in isolation from each other. This segregation engenders complications in which the latter ISHs, designated to manage more intricate and detailed regions, are deprived of access to high-resolution information. Furthermore, the subsequent ISHs are unable to fully capitalize on the computations executed by preceding ISHs, thereby resulting in inefficient utilization of computational resources. To ameliorate this challenge, we introduce the Cascade-Exit Semantic Segmentation Network (CESS). This innovative architecture propagates high-resolution features into the decoders of subsequent ISHs and establishes connections between the predictors of antecedent ISHs and those of the subsequent ones. Comprehensive empirical evaluations performed on the Cityscapes and CamVid datasets substantiate that our proposed methodology exhibits a superior balance between accuracy and inference time when juxtaposed with recently early exit techniques.