When Interpolation Is Loss-Blind: Quantile Regression at the Interpolation Boundary

Quantile regression turns a flexible predictor into prediction intervals by fitting several conditional quantiles, and the intervals are informative only when these fits differ across levels. Models that can fit every training response exactly are usually regularized by a ridge penalty, or weight decay, which makes the penalized coefficients unique. In this article, we show that uniqueness does not separate the levels on an explicit range of penalties. For fixed features whose training feature matrix has full row rank, we derive for each quantile level a closed-form penalty boundary, computed with one linear solve, at or below which the fit of that level equals the minimum-norm interpolant, which does not depend on the level. Theoretical results show that two distinct levels share a fit exactly when the common penalty is at or below both boundaries, with or without an unpenalized intercept, and that the same certificate applies to the linear quantile heads of a network at any local minimum of training at which the learned feature matrix has full row rank. With Gaussian features and noise and a feature count growing faster than the sample size, the boundary shrinks at a fixed response scale, and extinct fits have coverage converging to one half. For bounded responses and features fixed before sampling, validation above the boundary over a fixed penalty grid controls the excess pinball risk up to the approximation error of the features. Through simulations, we confirm the predicted coverage transition at the boundary. On four regression data sets and a simulated model, the weight decay $10^{-4}$ used to train a network lies below the boundaries computed from its learned features in 499 of 500 splits, and the five trained quantile heads have mean test coverage between $0.482$ and $0.538$ on every data set, at nominal levels from $0.1$ to $0.9$. Refitting the heads with a validated weight decay far above the boundary moves the pooled mean coverage at the levels $0.1$ and $0.9$ to $0.166$ and $0.825$.