Regression Model Selection under Explicit Inference Costs

Abstract Does selecting one regression expert per input justify its inference cost? On 35 previously unused benchmark tasks in 33 source families, our validation-selected one-query gate has a geometric MSE ratio of 1.055 to a calibrated global ensemble, at 0.410 times its measured warm latency. A retrained global boosting model has both lower MSE (1.038) and lower latency (0.264); AutoGluon improves MSE to 0.851 at substantially greater inference cost. The evaluated gate therefore adds no point to the aggregate accuracy–cost frontier, although it does on six individual source families. We also prove that bounded, pointwise dominated actions can consume every split of a depth-limited shared contrast greedy tree while preserving a positive action margin; a scalar stump avoids this failure. The construction extends to specified forests, a leaf constraint, target normalization, and outcome noise. Real-data library growth from 2 to 16 experts, however, does not show consistent degradation from weak additions. Shrinkage, clipping and projection controls further limit a scalar selection explanation. The combined evidence supports checking strong global alternatives before attributing practical value to regression routing, and distinguishes a provable learner failure from its observed empirical prevalence.