Reply to “Comment on ‘Experimentally adjudicating between different causal accounts of Bell-inequality violations via statistical model selection' ”

Our article described an experiment that adjudicates between different causal accounts of Bell inequality violations by a comparison of their predictive power, finding that certain types of models that are structurally radical but parametrically conservative---of which a class of superdeterministic models are an example---overfit the data relative to models that are structurally conservative but parametrically radical in the sense of endorsing an intrinsically quantum generalization of the framework of causal modeling. In their comment [Phys. Rev. A 109, 026201 (2024)], Hance and Hossenfelder argue that we have misrepresented the purpose of superdeterministic models. We here dispute this claim by recalling the different classes of superdeterministic models we defined in our article and our conclusions regarding which of these are disfavored by our experimental results. Their confusion on this point seems to have arisen in part from the fact that we characterized superdeterministic models within a causal modeling framework and from the fact that we referred to this framework as ``classical'' in order to contrast it with an intrinsically quantum alternative. In this Reply, therefore, we take the opportunity to clarify these points. They also claim that if one is adjudicating between a pair of models, where one model can account for strictly more operational statistics than the other, the first model will tend to overfit the data relative to the second. Because this model inclusion relation can arise for pairs of models in a reductionist hierarchy, they conclude that overfitting should not be taken as evidence against the first model. We point out here that, contrary to this claim, one does not expect overfitting to arise generically in cases of model inclusion, so that it is indeed sometimes appropriate to consider overfitting as a criterion for adjudicating between such models.