Statistical guarantees for italic k k $\textit{k}$ -degree neural networks via scale regularization

Abstract Neural networks have been widely used in data analysis, but there is relatively little research on the statistical theory of neural networks. In this paper we generalize the scale regularization method to the k -degree neural networks, and obtain a statistical guarantee for estimators with a least-squares term and a regularizer. On this basis, we exemplify this guarantee with locally Lipschitz activation functions and l Subscript beta l β $l_{\\beta}$ -regularization for beta element of left bracket 1 comma 2 right bracket β ∈ [ 1 , 2 ] $\\beta\\in[1,2]$ in the case k greater than 1 k > 1 $k>1$ , showing that the corresponding prediction error decrease is of the order of n Superscript minus 1 divided by 2 n − 1 / 2 $n^{-{{1}/{2}}}$ in the sample size n .