Universal Scaling Laws in Quantum-Probabilistic Machine Learning by Tensor Network: Toward Interpreting Representation and Generalization Powers

Abstract The interpretation of representations and generalization powers has been a long-standing challenge in the fields of machine learning (ML) and artificial intelligence. This study contributes to understanding the emergence of universal scaling laws in quantum-probabilistic ML. We consider the generative tensor network (GTN) in the form of a matrix-product state as an example and show that with an untrained GTN (such as a random TN state), the negative logarithmic likelihood (NLL) L generally increases linearly with the number of features M , that is, L ≃ kM + const. This is a consequence of the so-called “catastrophe of orthogonality,” which states that quantum many-body states tend to become exponentially orthogonal to each other as M increases. This study reveals that, while gaining information through training, the linear-scaling law is suppressed by a negative quadratic correction, leading to L ≃ βM – αM 2 + const. The scaling coefficients exhibit logarithmic relationships with the number of training samples and quantum channels χ . The emergence of a quadratic correction term in the NLL for the testing (training) set can be regarded as evidence of the generalization (representation) power of the GTN. Over-parameterization can be identified by the deviation in the values of α between the training and testing sets while increasing χ . We further investigate how orthogonality in the quantum-feature map relates to the satisfaction of quantum-probabilistic interpretation and the representation and generalization powers of the GTN. Unveiling universal scaling laws in quantum-probabilistic ML would be a valuable step toward establishing a white-box ML scheme interpreted within the quantum-probabilistic framework.