DEEP NEURAL NETWORKS AND UNIVERSAL APPROXIMATORS II

There are many studies of approximations using deep neural networks. In this paper, the authors provide yet another proof that deep neural networks are universal approximators. In their earlier work, the authors showed that an arbitrary binary target function can be effectively rewritten in terms of a set of strings, or a set of subsets, and that a single hidden neuron can identify and only identify a single string or a single subset. Therefore, an arbitrary binary target function can be effectively rewritten in the form of a neural network with one hidden layer. In this study, the authors imposed locality on the deep neural network, and will show here that an arbitrary binary target function can be effectively rewritten in the form of a locally connected deep neural network that can have many hidden layers. Although it will increase the neural network size, as a neural network is localized, it will generally increase the speed of training for large networks.