Data-Informed Model Complexity Metric for Optimizing Symbolic Regression Models

Selecting models from a well-fit, evolved population that generalizes beyond training data is difficult. We introduce a pragmatic method using Hessian rank to align model complexity with dataset intrinsic dimensionality (ID) for post-processing selection in symbolic regression. Our method efficiently estimates model complexity by averaging Hessian rank across strategic points, aligns this with dataset ID estimated using conventional ID estimators, and identifies an ideal complexity window that balances expressiveness with accuracy. Using StackGP with the Penn Machine Learning Benchmark, we demonstrate that models within this targeted complexity window generalize better than those outside it, without the bias common in methods reliant on user-defined parameters.