Mechanisms of a Convolutional Neural Network that Learns to Covertly Attend

The cueing task is one of the most prominent paradigms to study covert attention. Observers are quicker and more accurate in detecting a target when it appears with a cue (valid trials) than when it appears opposite to the cue (invalid trials). How neuronal populations across the visual hierarchy progressively represent and integrate visual information across the target, cues, and locations to give rise to the behavioral cueing effect is not well-understood. To gain a theoretical understanding of the plausible system-wide neuronal computations and mechanisms mediating the cueing effect, we analyze the response properties of 180k neurons per network across layers of ten feedforward Convolutional Neural Networks (CNN). The CNNs are trained on noisy images without any explicit attention mechanism and show human-like benefits of cues on detection accuracy. Early layers show retinotopic neurons separately tuned to target or cue with excitatory or inhibitory responses. Later layers show neurons jointly tuned to both target and cue and integrate information across locations. Consistent with physiological findings, we find increasing influence of the cue on target responses in deeper layers in the network and computational stages similar to those of a Bayesian ideal observer, but with more gradual transitions. The cue influences the mean neuronal response to the target and distractor, and changes target sensitivity with two mechanisms: integration of information across locations at the dense layer, and interaction with the thresholding Rectified Linear Unit (ReLU) in the last convolution layer. We find novel neuronal properties not yet reported in physiological studies: cue-inhibitory neurons, inhibitory cue influences on target neurons, and location-opponent cells, which are target-inhibitory at one location and target-excitatory at the other. Together, our analyses illustrate a system-wide analysis of the neuronal computations that might give rise to behavioral cueing effects and provide a theoretical framework to inform physiological studies.