Numerical simulation of deep reinforcement learning using self-referential holographic deep neural network

Deep reinforcement learning (DRL) is an artificial intelligence that is capable of autonomous decision-making in complex situations through a complementary combination of deep neural networks (DNNs) and reinforcement learning (RL). The autonomy is based on the cooperation between RL, which explores adaptations through trial and error in the environment; database (DB), which stores experience; and DNN, which learns adaptive behavior from training on experience data. The cooperation has been successfully applied to tasks that must operate directly in the face of a complex world, such as robotics, Go AI, and data center cooling. Regarding DNNs, the advancement of applications and algorithms requires the improvement of energy efficiency. There are two approaches to address this issue. The first is based on the current mainstream of electronic digital computing, such as application-specific integrated circuits (ASICs) and field-programmable gate arrays (FPGAs). The second explores new computing paradigms not dependent on von Neumann architectures, such as the search for extreme performance using optical computing. Although ASIC- and FPGA-based approaches have been studied for DNNs used in DRLs named q-networks [1,2], to the best of the authors' knowledge, there is no research on non-von Neumann computing. Here, we focus on self-referential holography (SRH) [3], an optoelectronic technology that implements two functions, as large-scale DB and DNNs exploiting the spatial parallelism of light, in a single system. In this study, we numerically investigate the feasibility of q-networks based on self-referential holographic DNN (SR-HDNN) [4].