Deep learning–based image super‐resolution in microscopy: Why more pixels do not imply higher resolving ability?

Deep learning-based image super-resolution is a popular topic in computer vision and artificial intelligence (AI)-based imaging. A survey of literature reveals that it is also increasingly being used in the domain of microscopy. From the fundamental perspective of sampling, image super-resolution is clearly understood and implemented as signal interpolation. However, the widespread use of deep learning-based image super-resolution in microscopy has led to some misinterpretations regarding the fundamental nature and purpose of image super-resolution. This is particularly problematic in the domain of microscopy because image super-resolution may be erroneously considered equivalent to improving resolving ability (i.e. better distinction between finer visual details) of the processed image. We, therefore, rely on first principles of imaging to provide theoretical analysis into why deep learning-based image super-resolution cannot, in general, reconstruct uncaptured visual details in the context of microscopy. In addition, we provide analysis of the case where deep learning-based image super-resolution may be useful for the purpose of emphasizing high-frequency details. The rigorous analysis and conclusions reported in the paper are both relevant and timely as they provide grounded insights towards correct interpretation of deep learning-based image super-resolution in the context of microscopy.