Learning to Correct Overlapping Forecasts: Accuracy, Queries, and Local Errors

When several models predict the same outcome from partly overlapping inputs, combining their outputs can count shared evidence repeatedly or lose useful information. Residual correction offers an alternative: each model predicts the error remaining after earlier predictions, using only its own inputs, and its output is added as a correction. Learning these corrections from finite data introduces errors that affect later prediction targets. We develop a theory of the accuracy attainable with imperfect corrections and the conditions under which learning them reduces rather than amplifies error. In an independent additive-signal model, we identify which local errors later experts can remove and which survive, and show that residual queries can outperform ordinary forecasts even with imperfect responses. When response error scales with the predictable residual, we determine the optimal worst-case risk for unequal expert precisions and every repeated-query budget. An optimal allocation and ordering rule attains it without knowing the information overlap. We then analyze corrections learned by local ridge regression on fresh residual labels. For iid symmetric features with finite variance, we give a necessary and sufficient stability condition for fixed covering sweeps with common training parameters. Gaussian least-squares correction requires more samples per fit than the total feature dimension plus one to converge, even with correlated features. Regularization can stabilize smaller batches. Matched-data and fixed-budget experiments test how regularization and sample allocation turn these stability conditions into useful correction.