Color-Blind Image Sensors: Toward Digital Twin of Human Retina

The human retina contains a complex arrangement of photoreceptors that converts light into visual information. Conventional image sensors mimic the trichromacy of the retina by using periodic filter mosaics responsive to three primary colors. However, this is, at best, an approximation, as an actual retina exhibits a quasi-random spatial distribution of light-sensitive rod and cone photoreceptors, where the ratio of rods to cones and their concentrations varies across the retina. Hence, periodic mosaics are limited to accurately simulating the properties of the eye. Here, we present an image sensor with similar distribution, spacing, ratios, and spectral characteristics of an actual foveal mosaic for emulating eye-like sampling and mimicking color blindness. To perform image reconstruction, we use a fully convolutional U-Net neural network adopting the concept of receptive fields in retinal circuitry. Our research will enable the development of a digital twin of a retina to further understand color vision deficiencies.