From associative learning to aberration compensation, how can we build a smart microscope?

Aberration is a prominent issue in microscopy imaging where the wavefront is distorted due to optical deficiencies, misalignments and non-uniform sample structures. Such phase errors can result in a reduced resolution, low contrast and compromised scientific information. Aberrations can be corrected optically by adaptive devices, but corrections are not normally informed through direct measurements; the optimisation process is often time-consuming with a limited working range, making it less ideal for practical use. Previously we proposed a novel machine learning assisted adaptive optics methods and showed its advanced performances in challenging imaging tasks such as three-photon functional and superresolution imaging. We further developed this method to cope with different imaging tasks. This includes analysis of information content to better inform the controller design, compress more information in limited image frames by adopting a multi-foci arrangements, and incoporating Walsh modes for complex aberration computation encapsulating knowledge of lattice-like geometry.