Evaluating the Role of Edge-to-Surface Reconstruction as a Unifying Explanation for Simple to Complex Lightness Illusions

Lightness illusions demonstrate that how bright an object appears depends on an elaborate constructive process, to the point that the same surface can be perceived as either black or white depending on the context. Why does the biological visual system work this way? Traditionally, distinct mechanisms have been proposed to account for simple lightness illusions (e.g. the Craik-O’Brien-Cornsweet Illusion) and for more complex illusions (e.g., the moon illusion: discs in different hazy backgrounds, Anderson & Winawer 2005). The Craik-O’Brien-Cornsweet illusion seems to depend on local cues — a dark/light difference at a singular edge— whereas the moon illusion seems to require more than just local cues by parsing the input into distinct scene properties. Our work examines the degree to which an edge-to-surface reconstruction mechanism can provide a unified explanation for a range of lightness illusions. First, across two behavioral tasks, 59 participants matched the brightness of a disc presented in a dark or light hazy background to illustrate the effect of the context on lightness perception. Second, we trained a reconstructive U-Net model to output a filled-in image from edge-only inputs, a computational goal that is analogous to filling in surfaces from edge-selective neurons in the biological visual system. Surprisingly, we found that, when reconstructing the discs, the U-Net model made systematic errors consistent with lightness illusions measured in people, suggesting that an edge-to-surface reconstruction is a plausible mechanism underlying this complex illusion. Finally, we applied the reconstructive model to a suite of additional lightness illusions — Adelson Haze Illusion, Snake Illusion, Koffka Illusions, 3D Cornsweet Illusion (Purves et al. 1999) and Kanizsa Square Illusion — finding that an edge-to-surface reconstruction mechanism successfully recapitulated illusions when edge information is present, and the polarity of the edge is consistent around the boundary of the illusory surface.