Stereo Online Self-Calibration Through the Combination of Hybrid Cost Functions with Shared Characteristics Considering Cost Uncertainty

Stereo cameras and stereo matching algorithms are core components for stereo digital image correlation to obtain 3D data robustly in various environments. However, its accuracy heavily relies on extrinsic calibration. In this work, we propose a markerless method for obtaining stereo extrinsic calibration by employing nonlinear optimization on a manifold, which leverages the inherent observability property. To ensure the stability of the optimization and the robustness to outliers when using natural features, we minimize the error constraint between spatial per-frame sparse natural features by stably combining cost functions with similar properties, considering cost uncertainty. Both constraints work in the same direction to reduce the difference in the y-axis coordinates of corresponding points. As a result, the optimization process proceeds smoothly, and it helps reduce the likelihood of overfitting. To extend the problem to the spatiotemporal domain, Bayesian filtering is applied using the logit of zero-shot-based semantic segmentation. Using publicly available data, we conducted experiments where the optimization converged with minimal variation in the number of iterations, and stability was validated through a comparison with state-of-the-art methods.