Universal approximation theorem for nonlinear resistive networks

Resistive networks that train themselves using local learning rules such as equilibrium propagation show promise as energy-efficient alternatives to neural networks. Their computational capabilities remain unclear, though, as they solve circuit equations rather than standard neural-network equations. This study demonstrates mathematically that a deep resistive network built from (ideal) ohmic resistors, diodes, voltage sources, and voltage amplifiers can approximate to arbitrary accuracy any neural network based on the rectified-linear-unit activation function. This insight is expected to inform the design of self-learning resistor networks capable of universal function approximation.