Image Classification Using Neural Network for Feature-Extractable CMOS Image Sensor

For the forthcoming Internet of Things (IoT) era, it will be important to reduce the data output from sensors as well as their energy efficiency. Since conventional image sensor output data for photography are often redundant in AI applications, we propose a CMOS image sensor that can generate both RGB color images for humans and feature data for deep learning (DL). Use of feature data allows reducing the energy efficiency of the image classification system and saving storage space for imaging data. We performed experiments to demonstrate that the simulated feature data are suitable for use in image classification tasks. A five-layer convolutional neural network (CNN) classifier was trained and tested using the aggressively quantized feature data generated from a person dataset, where image classification accuracy was also improved when applying contrast enhancement. According to the experimental results, an accuracy of 95.9% was achieved using 1-bit feature data, resulting in 93.75% reduction in the amount of data compared to RGB color images.