Residual Analytical Learning for Auditable Regression Correction

A fitted regressor may leave systematic errors whose correction depends on a much smaller subset of variables than the original prediction task. This observation motivates a different way to improve an existing regressor by preserving the fitted model and learning an explicit correction only for the structure it leaves behind. We develop Residual Analytical Learning, a framework that learns from out-of-fold residuals, identifies low-dimensional residual structure, and fits analytical correction laws within supported error regimes. The resulting rule is not a post-hoc explanation of another model. It is the computation actually added to the base prediction, providing an explicit instance-level account of how and why that prediction was changed. Across eight heterogeneous regression benchmarks, the proposed method improves the frozen linear base in all 40 outer folds, with dataset-level MAE reductions ranging from 9.6% to 97.0% and a median reduction of 26.5%. Under a correction search capped at four variables, seven of eight benchmarks select a median residual dimension of at most three, including a stable two-variable residual core on a 32-dimensional benchmark. This analytical structure comes with an accuracy–auditability trade-off. Flexible residual learners remain more accurate on most datasets, while the proposed method recovers a median 55.0% of the improvement achieved by the best tested residual corrector while retaining the exact computation responsible for each correction. We further find that learning systematic residual structure and predicting the error that remains after correction are distinct problems.