Multimodal object recognition and real-time anomaly detection by physical reservoir computing using an Ag₂S reservoir

Abstract In recent years, the deployment of artificial intelligence in society has progressed, particularly in the field of edge computing such as used in an AI robot, where reservoir computing attracts much attention because of its low power consumption and real-time performance. Challenges still remain in terms of achieving high accuracy in object recognition and ensuring sufficient real-time performance. In this study, we integrated tactile information processing using an Ag 2 S physical reservoir and visual information processing using a convolutional neural network for multimodal processing. The accuracy of object recognition has improved to up to 97.1%, by complementing the weakness of each method. We also developed a real-time anomaly detection system for grasping an object. By training an Ag 2 S reservoir with data from normal gripping operations alone, the system can detect anomalies by comparing predicted values and actual input, which enables real-time control of a robot arm.