Two Hidden Layers Suffice for Exact ReLU Representation

We prove that every continuous piecewise-affine function with finitely many polyhedral pieces has an exact, global representation by a finite ReLU network with two hidden layers. Thus increasing depth beyond two hidden layers does not enlarge the unrestricted scalar-output function class, although it may reduce the resources needed for representation. The construction is rational whenever the affine constituents are rational. Its central identity expresses an incremental maximum as asigned sum of explicit shallow functions. Boolean cancellation gives these functions a common domain of absolutely convergent Laplace transforms, and a uniform rational identity reduces the transformed sum to Lagrange interpolation. All hidden parameters are integral, and output denominators divide (n−1)!. For integral piecewise-affine functions on Rd, the output denominator can instead be bounded by d!. An arithmetic exponent characterizes universality over rational parameter rings. The result settles unrestricted exact depth, not efficient computation,numerical stability, or learnability.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-10-14
DOI
https://doi.org/10.5281/zenodo.23377647
Primary Topic
Neural Networks and Applications
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Two Hidden Layers Suffice for Exact ReLU Representation

Seyed Ali Rastegar, Surya Tallavarjula
Zenodo (CERN European Organization for Nuclear Research)
Neural Networks and Applications
preprint

Two Hidden Layers Suffice for Exact ReLU Representation

Seyed Ali Rastegar, Surya Tallavarjula
preprint en

Abstract

We prove that every continuous piecewise-affine function with finitely many polyhedral pieces has an exact, global representation by a finite ReLU network with two hidden layers. Thus increasing depth beyond two hidden layers does not enlarge the unrestricted scalar-output function class, although it may reduce the resources needed for representation. The construction is rational whenever the affine constituents are rational. Its central identity expresses an incremental maximum as asigned sum of explicit shallow functions. Boolean cancellation gives these functions a common domain of absolutely convergent Laplace transforms, and a uniform rational identity reduces the transformed sum to Lagrange interpolation. All hidden parameters are integral, and output denominators divide (n−1)!. For integral piecewise-affine functions on Rd, the output denominator can instead be bounded by d!. An arithmetic exponent characterizes universality over rational parameter rings. The result settles unrestricted exact depth, not efficient computation,numerical stability, or learnability.

Zenodo (CERN European Organization for Nuclear Research)
Neural Networks and Applications
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.