Comparing the Empirical Content of Economic Theories

A fitted theory that predicts well may do so because of the structure it commits to or because it was fitted to the data its evaluated on. We propose measures that make it possible to separate the two situations by operationalising a theory as a combination of a functional form, a support restriction on behavior and a distribution over the parameters, with no parameter fitted to the evaluation sample. Probabilistic completeness is the share of the predictable variation that a rule drawn from the theory captures out of sample. Probabilistic restrictiveness is how much conceivable behavior such a rule excludes, and it separates into what the functional form can reach and how the parameter distribution concentrates on it. A parameter distribution in this context can be interpreted either as uncertainty about the behavior of a representative agent or as the distribution of the parameters across a population of heterogeneous agents. We demonstrate how our measures can be adapted to both readings through the choice of scoring rule, baseline and ceiling. We also introduce a scope layer that prices the domains a theory claims and makes it possible to directly compare theories across multiple domains. We apply the measures to expected utility, cumulative prospect theory and disappointment aversion on certainty equivalents for binary lotteries. We find that the functional forms exclude little of conceivable behavior, while the literature's parameter distributions exclude most of what the forms leave open. Further, a rule drawn from a literature-informed prior keeps two thirds of fitted prospect theory's completeness, most of expected utility's and none of disappointment aversion's. The flexibility–fit trade-off found for functional forms disappears once the theories are held with their literature priors. Read as populations, disappointment aversion and expected utility cannot produce the dispersion the subjects show, and the literature priors are less well placed for losses than for gains.