An optimization of the Fourier self-calibration algorithm for improved noise suppression capability

Abstract Self-calibration emerges as a promising approach in ultra-precision field, overcoming the limitations imposed by instrument capabilities in conventional calibration. In the self-calibration research, the algorithm based on the Fourier transform firstly established a rigorous theoretical framework. However, despite advancements in alternative self-calibration methods that have improved noise suppression, the traditional Fourier algorithm still faces challenges, particularly in mitigating random measurement noise in the separation results. In this study, we introduce methods on enhancing the noise suppression capabilities of the Fourier self-calibration algorithm. We investigate the hypothesis in the traditional algorithm and find that the computation of weighted averages had minimal effects. Then, we present our method by augmenting measurement positions. Considering both the increase in the number of measurement positions and the enhancement of noise suppression capabilities, augmenting translation positions in the opposite direction is proved to be the most effective strategy. Our algorithm can effectively suppress measurement noise when one-sided point number of the artifact is below 21. While the additional data processing slightly increases runtime, it does not alter the algorithm’s computational complexity, yet it significantly improves noise suppression. The effectiveness of our method is validated through a Monte Carlo comparison of uncertainties with the original algorithm. Experiments on Fizeau interferometer further prove robustness and practical feasibility of our proposed algorithm, with the measurement repeatability being reduced by 26.34% and 25.15% in the errors separated from the stage and the artifact.