Optimal Piecewise Linear Approximations for Sigmoid, Tanh, Probability Density and Cumulative Distribution Functions: Tabular Form and R Package pwlapprox2d

Abstract We present information in tabular form about optimal piecewise linear (PWL) approximations of the sigmoid and tanh activation functions as used in neural networks, as well as the probability density and cumulative distribution functions of the Normal and log-Normal distributions. The presented approximations minimise the maximum absolute difference between the PWL function and the continuous function being modelled; we also provide information on optimal over- and underestimators. We provide information on the optimal breakpoint locations for different numbers of breakpoints in a series of tables, as well as the accuracy provided by these approximations. This allows practitioners to utilise the PWL approximations whenever needed; for example, to enable the use of mixed-integer linear programming techniques to solve problems where these functions may appear, or to approximate integrals. The provided optimal breakpoint locations lead to an improvement over a uniform breakpoint location of the maximum absolute difference of up to 95% and an average of 84% among the six functions. This information is available online at https://doi.org/10.5281/zenodo.16362215 and can be recreated using the R package pwlapprox2d [1].