Two Hidden Layers Suffice for Exact ReLU Representation
We prove that every continuous piecewise-affine function with finitely many polyhedral pieces has an exact, global representation by a finite ReLU network with two hidden layers. Thus increasing depth beyond two hidden layers does not enlarge the unrestricted scalar-output function class, although it may reduce the resources needed for representation. The construction is rational whenever the affine constituents are rational. Its central identity expresses an incremental maximum as asigned sum of explicit shallow functions. Boolean cancellation gives these functions a common domain of absolutely convergent Laplace transforms, and a uniform rational identity reduces the transformed sum to Lagrange interpolation. All hidden parameters are integral, and output denominators divide (n−1)!. For integral piecewise-affine functions on Rd, the output denominator can instead be bounded by d!. An arithmetic exponent characterizes universality over rational parameter rings. The result settles unrestricted exact depth, not efficient computation,numerical stability, or learnability.
Authors
- Seyed Ali Rastegar
- Surya Tallavarjula (ORCID: https://orcid.org/0009-0007-7360-3514)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-10-14
- DOI
- https://doi.org/10.5281/zenodo.23377647
- Primary Topic
- Neural Networks and Applications
- Type
- preprint