What Does Permutation Importance Measure? Exact Results for Correlated Features, and Evidence from Power-Plant and Building-Energy Data

Permutation importance, the rise in prediction error when one feature's values are shuffled, is the most widely used way to say which inputs matter to a machine-learning model. I prove two exact results for linear models. The expected rise in mean squared error from a uniformly random permutation of feature j is 2βⱼ² Var(xⱼ) + 2βⱼ Cov(e, xⱼ), which contains no term for how strongly xⱼ is correlated with the other features. The rise from dropping the feature and refitting is, in sample, exactly βⱼ² Var(xⱼ)(1−Rⱼ²), where Rⱼ² is the share of xⱼ explained by the other features. The two measures therefore differ by a factor of about 2/(1−Rⱼ²). A simulation confirms both. On 9,568 hourly records from a combined-cycle power plant, ambient temperature has permutation importance 433.1 against 36.2 for drop-and-refit in a linear model, a ratio of 11.96 against 11.79 predicted, and 431.0 against 8.07 in a random forest. On 768 simulated buildings, where surface area is an exact linear function of wall and roof area, all five geometry features have large permutation importance (5.0 to 27.3) but drop-and-refit importance of 0.003 or less, while dropping all five together raises the error by 115.2. Single-feature importances of either kind mislead when features are strongly dependent; grouped importance answers the question users usually ask.