Image Classification by Inferotemporal Cell Activity

Neuroscience has been a driving force in improving artificial intelligence (AI). To understand how our brain recognizes object images, we focused on the underlying neuronal mechanisms of representation and identification in the inferotemporal cortex, which is repeatedly reported as a critical area for object recognition. In this study, we trained two types of classification models using the electrophysiological activity of cell populations in the inferotemporal cortex. The performance of the trained models was used to evaluate the feasibility of image identification by the neuronal activity of the inferotemporal cortex. Responses in individual trials were used in the present study, while responses averaged across trials were used in previous studies. We assessed the performance of models for distinguishing individual images as well as models for categorizing images into groups. The discrimination performance for each image showed consistently high hit rates between the two monkeys, but there was also a tendency for high false alarm rates. Interestingly, image classification by certain category types demonstrated high performance. For categories important to the monkeys’ daily lives, such as animals and fruits, the difference between hit rates and false alarm rates was as high as 50-60%. In particular, animals showed the highest average difference at 61.4%. These results suggest that information crucial for object recognition by category is represented by the activity patterns of numerous neurons in the inferotemporal cortex.