Closing the Coverage Gap in Simultaneous Quantile Regression

Simultaneous quantile regression builds on quantile regression by minimizing the expected pinball loss to approximate all quantiles at once. This enables the production of a complete implicit cumulative distribution of a prediction interval, or any subset of this function, without assuming any prior distribution shape. Existing implementations of this technique show under-covered distribution tails compared to the observed distribution of the data. Such calibration errors cause the underestimation of risk or uncertainty of a model. Contrary to this observation, regular quantile regression calibrates well on the same dataset and model configurations. In this work, we investigate the cause of the gap in coverage between simultaneous quantile regression and regular quantile regression and identify a category of solutions to the problem. Our empirical evaluation shows that input quantiles need to be dynamically scaled to result in non-linear tail coverage. Our own scaling approaches work just as well as previous literature, while using fewer parameters. We further show that while tail coverage could also be recalibrated with conformalized quantile regression, the post-processing produces a better distributional fit with scaled quantile models.