Architectural optimisation in deep neural networks. Tests of a theoretically inspired method

We study how to optimise the architecture of a Deep Neural Network by rearranging the neurons within the hidden layers. In order to do so, we follow a theoretical suggestion that comes from the thermodynamic properties of some Restricted Boltzmann machines (RBMs). Namely, for a suitable definition of temperature, we move the neurons towards colder areas of the network. We find that such a procedure, in the analysed cases, improves the robustness of the network respectively by 4.8%, 6% and 2.8% with respect to the default choice of neurons equidistribution, without changing the accuracy. The result has a particular relevance in AI applications to satellites where resources are limited.