SMILE: Segmentation-Based Centroid Matching for Image Rectification via Aligning Epipolar Lines

Stereo images, which consist of left and right image pairs, are often unaligned when initially captured, as they represent raw data. Stereo images are typically used in scenarios requiring disparity between the left and right views, such as depth estimation. In such cases, image calibration is performed to obtain the necessary parameters, and, based on these parameters, image rectification is applied to align the epipolar lines of the stereo images. This preprocessing step is crucial for effectively utilizing stereo images. The conventional method for performing image calibration usually involves using a reference object, such as a checkerboard, to obtain these parameters. In this paper, we propose a novel approach that does not require any special reference points like a checkerboard. Instead, we employ object detection to segment object pairs and calculate the centroids of the segmented objects. By aligning the y-coordinates of these centroids in the left and right image pairs, we induce the epipolar lines to be parallel, achieving an effect similar to image rectification.