A comparison of spherical neural networks for surround-view fisheye image semantic segmentation

The automotive industry has made significant strides in enhancing road safety and enabling automated driving features through advanced computer vision techniques. This is particularly true for short-range vehicle automation, where non-linear fisheye cameras are commonly used. However, these cameras are challenged by optical distortions, known as fisheye geometric distortions, which lead to object deformation within the image and significant pixel distortion, particularly at the image periphery. Based on the observation that fisheye and spherical images exhibit at least superficially similar geometric characteristics, we investigate the applicability of spherical models—including Spherical Convolutional Neural Networks (CNNs) and Spherical Vision Transformers (ViTs)—to fisheye images, even though fisheye images are not truly spherical. We perform our comparison using fisheye datasets—Woodscape, SynWoodscape, and SynCityscapes in autonomous driving scenarios, with a specific focus on the ability of spherical methods (Spherical CNNs and ViTs) to manage fisheye distortions and compared them against traditional non-spherical methods. Our findings indicate that spherical methods effectively address fisheye distortions without needing extra data augmentations. This results in better mean Intersection over Union (mIoU) scores, pixel accuracy, and better surround-view perception than other modern approaches for fisheye semantic segmentation. However, we also find that spherical methods have a greater tendency to overfit smaller datasets compared with non-spherical models. These advancements highlight how non-linear camera images can take advantage of spherical approximations through spherical models in autonomous driving.