How to describe Fourier filtering wavefront sensor nonlinearity with interaction matrices

Fourier filtering wavefront sensors (WFSs) are a class of highly sensitive sensors that can significantly enhance adaptive optics (AO) performance, particularly in low-flux regimes. However, their nonlinear behavior limits their effectiveness for measuring high-amplitude phases. This paper presents a method for characterizing the nonlinearity of Fourier filtering WFSs within the widely used matrix formalism. The nonlinearity arises from an over-modulation effect depending on the phase being measured. Consequently, the matrix describing the sensor outside its linearity range must also depend on this phase. We first propose a theoretical framework, derived from light propagation equations, to construct a reconstructor capable of accounting for the WFS’s nonlinear responses. This analytical approach yields an exact expression for the reconstructor within the matrix formalism, referred to as the specific matrix, as it depends on the phase to be measured, which makes it impractical to use. Therefore, a portion of the paper is dedicated to deriving an approximation of the specific matrix. A method for approximating the specific matrix using data fusion with a focal plane camera is introduced. Simulation results demonstrate the efficacy of this approach when applied to a nonlinear WFS, such as the unmodulated pyramid WFS, under challenging seeing conditions (up to 1.3″).