Generalization Gap Bound for Deep Belief Networks via Rademacher Complexity

Deep Belief Network (DBN) has achieved remarkable success in diverse machine learning tasks, however, the theoretical understanding of their generalization performance remains limited. This paper establishes the upper bound of the generalization gap for DBN. First, we develop a precise quantitative characterization of their Rademacher complexity with explicit parameter dependence by analyzing the hierarchical architecture and parameter constraints inherent in DBN., building upon this foundation and incorporating covering number theory, we propose a compact upper bound for the generalization gap. It quantifies the relationship between architectural parameters and generalization performance, thereby advancing both theoretical comprehension and practical deployment of deep generative architectures.