TAILOR: Adaptive Residual Shape and CRPS Geometry for Heteroscedastic Regression

Accurate uncertainty quantification remains a central challenge in neural regression. Gaussian heteroscedastic models can absorb mean prediction errors into larger variance estimates, weakening the updates needed to correct the mean. Decoupling helps address this feedback, but leaves the model’s distributional assumptions unchanged: even a well-trained Gaussian cannot generally capture differences in residual shape by adjusting its width alone. We introduce TAILOR to address these limitations together, bringing adaptive residual modelling and orthogonal mean–scale learning into a single formulation. Rather than fixing the noise distribution in advance, TAILOR learns a shape parameter that moves smoothly between lighter-than-Gaussian, Gaussian, and heavier-tailed behaviour. Crucially, the objective used to fit this flexible family also supplies its learning geometry. Under the continuous ranked probability score (CRPS), location is orthogonal to scale and shape at the predictive level, and we use this structure to derive updates that account for the shared neural representation. This connection between flexibility and geometry extends beyond the learning rule. For small departures from Gaussianity within our family, we show that the best local Gaussian fit retains a quadratic approximation gap even after adjusting location and scale. The coefficient of this gap is precisely the geometric quantity that distinguishes shape changes from rescaling. Across regression benchmarks, TAILOR reduces RMSE by up to 74% and CRPS by up to 67% relative to tested Adam-based comparators, while achieving central prediction-interval calibration error as low as 0.6 percentage points. TAILOR brings adaptive residual modelling and orthogonal predictive geometry into one framework, addressing distributional flexibility and learning dynamics together rather than correcting them in isolation.