acceptodds
Under review as a conference paper at ICLR 2027

Closing the Coverage Gap in Simultaneous Quantile Regression

Abstract

Simultaneous quantile regression builds on quantile regression by minimizing the expected pinball loss to approximate all quantiles at once. This enables the production of a complete implicit cumulative distribution of a prediction interval, or any subset of this function, without assuming any prior distribution shape. Existing implementations of this technique show under-covered distribution tails compared to the observed distribution of the data. Such calibration errors cause the underestimation of risk or uncertainty of a model. Contrary to this observation, regular quantile regression calibrates well on the same dataset and model configurations. In this work, we investigate the cause of the gap in coverage between simultaneous quantile regression and regular quantile regression and identify a category of solutions to the problem. Our empirical evaluation shows that input quantiles need to be dynamically scaled to result in non-linear tail coverage. Our own scaling approaches work just as well as previous literature, while using fewer parameters. We further show that while tail coverage could also be recalibrated with conformalized quantile regression, the post-processing produces a better distributional fit with scaled quantile models.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Quantile Regression, Uncertainty Quantification, Prediction Interval Coverage

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_CbieOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies the training of neural networks with SQR, an extension of ordinary QR that takes the quantile level as an additional model input. The authors attribute a particular prediction interval tail coverage collapse phenomenon in SQR to (i) affine dependence on the quantile level τ\tau at model initialization and (ii) gradient updates that are biased toward high-density regions of the data. To correct this behavior, they suggest that perturbing τ\tau closes the calibration gap between QR and SQR at extreme quantiles. The paper tests this explanation empirically by proposing several non-linear τ\tau-scaling methods and benchmarking them against existing IQN and AQR approaches. Experiments across seven datasets show that the proposed non-linear scalings improve both average calibration error and Winkler score relative to vanilla SQR and match QR.

Critical strengths

Clear empirical motivation and supporting ablations. The authors provide empirical examples showing that the quantile function learned by SQR remains approximately affine in τ\tau throughout training, which motivates their proposed non-linear correction. This finding establishes connections with prior approaches, notably IQN and AQR, and provides an explanation for why these models perform well in practice. The experiments also rule out several alternative explanations for the tail collapse by testing model capacity, optimization regimes, activation functions, and calibration losses.

Critical weaknesses
  1. Insufficient theoretical grounding. Much of the motivation behind the collapsing tails in Section 2 is informal. For example, the “neural network activations” paragraph only considers a single hidden-layer MLP and heuristically looks at the ratio of the variance contributions of wj⊤xw_j^\top x and vjτv_j \tau. The “loss derivative” and “impact of data-sample density on learning” paragraphs argue that the accumulated non-linearity is biased toward areas of high data density, but this claim is not rigorously supported. Instead, I would like to see stronger theoretical motivation behind the affine quantile function finding, through some combination of training dynamics analysis, gradient updates in relation to data density, or a small-scale theoretical case study that would better ground the empirical motivation.
  2. Incomplete or incremental novelty. The main novelty of the paper appears to be the explanation behind the tail coverage collapse phenomenon in SQR. While this is an interesting direction, the motivation is mostly informal (as noted previously) and lacks theoretical grounding. As a result, it feels somewhat underdeveloped. The remaining contributions, such as the proposed τ\tau-scaling methods, benchmarking the coverage of existing SQR variants against QR, and analysis of post-hoc CQR, are mostly incremental.
  3. Tension with AQR results. The authors argue that non-linear scaling of τ\tau closes the coverage gap between SQR and QR. However, using affine embeddings like in AQR performs comparably. I understand that the authors acknowledge this and it may be partly explained by the magnitude analysis of vjτv_j \tau. Nonetheless, this seems to weaken the core message and suggests that some combination of the magnitude and non-linearity of the effect of τ\tau may be the key factor.
  4. Evaluation scope. The evaluation benchmark uses seven datasets, but all the datasets are tabular and, as the authors note, do not have “highly non-normal distributions” except for two. I believe that the evaluation scope is thus somewhat narrow. In particular, my main concern is whether the proposed scalings are suitable for datasets where the conditional CDFs are not S-shaped or non-Gaussian, in which case the proposed non-linearities may not suit the data.
Questions for authors

n/a

3 comments

Thank you for all of your comments. We understand and share a number of your concerns and will strive to address them in an updated version of our manuscript.

Theoretical grounding: Indeed, our hypothesis could benefit from a more rigorous theoretical examination. The hypothesis on what causes the gap is, however, not disproven by our empirical analysis, suggesting the validity of the idea. We argue that the manuscript provides a substantial contribution nonetheless (see our explanation in Novelty). We do, however, agree that your suggestions on more analyses of training dynamics would add more value to the manuscript.

Novelty: We would like to disagree that the main novelty of this paper is the explanation behind the collapse and offer our perspective: Given our literature review, the coverage gap between SQR and QR has never received any formal attention. Further, to the best of our knowledge, there are no reports on the extent of the effect. According to their respective papers, the inventors of IQN or AQR were not aware of their methodology being more effective than simple SQR: neither were IQN and AQR empirically compared against SQR, nor did either paper include an empirical coverage analysis in their work.

As far as we understand, we thus 1.) are the first to observe the prevalence and significance of the effect and investigate it systematically, 2.) offer a hypothesis for why it happens, and 3.) identify direct solutions to it of our own but also from the literature, and 4.) document other relevant solutions that did not work. In addition, our solutions need fewer parameters than AQR or IQN. We hope this is a clearer explanation and will go through relevant sections in the paper again to evaluate if we can communicate this more clearly there as well.

AQR: We agree that AQR is not entirely explained by our hypothesis. With respect to AQR, this paper aims to offer a hypothesis on why there is a gap and how to fix it. While a non-linear transformation of tau is sufficient, it can appear not to be necessary. Thus, we assume that there may exist many more solutions to the problem. In the next version of the manuscript, we plan to include an ablation on AQR, as it can be split into shift, scale, and sampler components.

Evaluation scope: We agree that our current analysis can not conclude whether any of the analysed approaches can handle non-Gaussian or S-shaped data. Reviewer gsSj had a similar concern and suggested an additional analysis with synthetic data in heteroscedastic, skewed, and multi-modal conditions, which we will implement and report on. This will also include IQN and AQR and should provide further insights into how these approaches work.

We will post another comment explaining our changes and added experiments as soon as they are done.

While I think the manuscript has some potential, even with this rebuttal, I am not yet persuaded that the contribution is at the level I would expect for a publication in ICLR. This assessment seems consistent with the other reviews. I am maintaining an open mind for the reviewer discussion.

Thank you for your comments! The experiments are done, and changes to the paper have been finalized. In this post, we will describe changes related to your previous comments. We also compiled more specific scope- and communication-related changes in the comment that is not a reply to any reviewer (for us, that is at the very top of the comment list). 

We can add some more points to our initial response now:

  1. There was not enough time for a more thorough theoretical analysis. However, we did some more empirical evaluations that support our claims: We take a look at the properties of the curvature displayed by each approach, what differences there are in the pre-ReLU sign switching,  and how gradient updates are coupled in tail quantiles vs. quantiles in the middle of the distribution.  (These are at the end of the Results section in Figure 4 and Appendix J.1)
  2. Regarding novelty, we updated our phrasing in the Abstract and, especially, the Contributions section to communicate more precisely what novelty and contribution we offer to the community.
  3. We are more precise with our claims now. It is not only non-linearity in τ\tau that can produce a non-linear quantile coverage distribution. However, we also observed that AQR did not perform as well as IQN or the SQR$_e/s$ approaches in non-Gaussian distributions. AQR appears to be just nonlinear enough to cover well in the original datasets. AQR also has improved performance through simply embedding τ\tau with logit, like in SQR$_s$. Appendix J.1, K, and the Discussion in Appendix L).
  4. As promised, we added experiments with synthetic heterogeneous, skewed, heavy-tailed, and bimodal data. IQN, AQR, and the SQR$_e/s$ stay mostly close to QR performance. In our bimodal condition, all approaches were worse than QR, and AQR was worse than all others in the heavy-tailed condition. (These results were added in Appendix J.1) In conclusion, the new approaches seem to transfer to more data shapes apart from bimodal.
Reviewer_qWgnOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the undercoverage of simultaneous quantile regression (SQR) and proposes probit or logit transformations of the quantile input. On seven regression datasets, these changes largely improve interval coverage and achieve performance comparable to QR and existing quantile embedding methods.

Critical strengths
  1. The proposed fix is simple. The parameter-free logit transformation improves average 95% coverage from 86.6% to 94.2%, with comparable pinball scores to QR. This is easy to implement and adds no parameters.
  2. Experiments seven datasets, five runs, and several ablations in sampling, architecture, and optimization. The uniform-sampling ablations also show that the gains do not depend entirely on sampling more tail quantiles.
Critical weaknesses
  1. The methodological novelty is limited. IQN and AQR already have similar coverage, while the proposed methods mainly introduce standard input transformations. For the constant-scaling baseline, its implementation and sensitivity to the scaling factor are not clearly discussed. Would a well tuned constant-scaling baseline outperform the proposed transformation method? A simple recalibration of the queried quantile levels would also be a useful comparison alongside CQR.
  2. The evaluation shows improved marginal interval coverage, but does show that the models recover an accurate conditional quantile function. Marginal coverage may not show unequal errors in the two tails and errors that cancel across inputs.
Questions for authors

Could the authors address the weaknesses above, particularly by addressing the novelty concern, discussing how the constant-scaling baseline is tuned, and how well conditional quantile function is recovered?

3 comments

Thank you for your critique. We will strive to build upon your suggestions.

Your comment can be split into methodological concerns and novelty concerns. Let us address the methodology first:

Regarding the constant tau scaling baseline, we used the constant scaling parameter that we found in the code of Tagasovska’s and Lopez-Paz’s (2019) original paper. We did not find evidence for a parameter sweep in that work, and we agree that this is a limited view of its potential effects. Thus, we will conduct a parameter sweep of constant tau scaling on a range of different values. We can also provide a recalibration of queried quantile levels in addition to the CQR implementation.

It is true that we only show marginal interval coverage, which should be unbiased in approximately Gaussian data. This is no guarantee, though, so thank you for pointing this out. We agree and will report quantile coverage in the revised version of the manuscript.

Second, regarding your novelty concern, we argue that our work is not limited to the comparison with IQN and AQR. Given our literature review, the coverage gap between SQR and QR has never received any formal attention. Further, to the best of our knowledge, there are no reports on the extent of the effect. According to their respective papers, the inventors of IQN or AQR were not aware of their methodology being more effective than simple SQR: IQN and AQR were not empirically compared against SQR, nor did either paper include an empirical coverage analysis in their work.

As far as we understand, we thus 1.) are the first to observe the prevalence and significance of the effect and investigate it systematically, 2.) offer a hypothesis for why it happens, and 3.) identify direct solutions to it of our own but also from the literature, and 4.) document other relevant solutions that did not work. In addition, our solutions need fewer parameters than AQR or IQN.

We hope this is a clearer explanation and will go through relevant sections in the paper again to evaluate if we can communicate this more clearly there as well. We will post another comment explaining our changes and added experiments as soon as they are done.

Thank you for the response and clarification. Since most of the experimental results are still pending, I am keeping my original rating. I encourage the authors to include these experiments in a revised submission.

Thank you for your comments! The experiments are done, and changes to the paper have been finalized. In this post, we will describe changes related to your previous comments. We also compiled more specific scope- and communication-related changes in the comment that is not a reply to any reviewer (for us, that is at the very top of the comment list).

  1. Regarding your question of whether a well-tuned constant τ\tau scaler would outperform the other models tested: It achieves comparable results when hyperparameter-optimized per dataset. The other models, however, did not receive tuning. A direct comparison would be biased this way and would require major updates to all experiments in the paper. Similarly, another hyperparameter would limit the ease of use for SQR. We summarized the results to be read below. 

  2. Otherwise, we did add a recalibration comparison to the CQR evaluation in the paper. The summary is that a simple recalibration that remaps τ\tau to a τ’\tau’ that fits better to the target also increases coverage for all models, but not for standard SQR. (See Appendix B for implementation and Appendix I for results)

  3. Regarding the quantile coverage concerns: IQN, AQR, and the SQR$_e/s$ show symmetric coverage in the tails. We examined the main experiments on the 7 datasets and also added a number of synthetic datasets (per another reviewer’s request) and checked for all. (These results were added in Appendix J)

In the code of Tagasovska and Lopez-Paz, the authors used a constant scale on τ\tau in which the first layer input of their MLP received c(τ−0.5)c(\tau-0.5) instead of a raw τ\tau with c=12c=12. We use the same hyperparameter settings as in Appendix G. We train for 300 epochs and keep the best validation epoch, chosen with pinball. We hyperparameter optimize c∈{0.1,1,3,12,48,192}c \in \{0.1, 1, 3, 12, 48, 192\}, where c=1c=1 is equivalent τ\tau-scaling to regular SQR.
Optimal cc for each approach is very dependent on the dataset, so we tuned for each dataset separately.

We can see some of the average training results over the 7 original datasets in the table below. Performance can get close to existing approaches. Interesting is that the constant approaches are noticeably sharper and have a less nonlinear slope in τ\tau.

ACEQCE⁻QCE⁺SharpnessPICP₉₅Slope ratio
QR0.0210.0160.0122.2380.93411.2
SQR0.0660.0360.0361.5800.8662.0
SQR_e (logit)0.0150.0110.0102.3800.94313.0
Constant scale, tuned c0.0240.0190.0142.0480.9344.1
Reviewer_gsSjOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper investigates tail undercoverage in simultaneous quantile regression (SQR) relative to conventional multi-quantile regression (QR). The authors hypothesize that weak quantile-input signals at initialization and smoothing under shared-parameter optimization contribute to this behavior. They propose transforming the quantile input using the inverse standard normal CDF or the logit function, either directly or through additional embeddings. Experiments on seven tabular regression datasets demonstrate improved interval coverage and calibration comparable to QR, with little or no additional parameter overhead. The paper also evaluates the effects of conformalized quantile regression (CQR) post-processing.

Critical strengths
  1. The paper clearly demonstrates that models with similar average pinball losses can exhibit substantially different interval coverage, highlighting the importance of explicitly evaluating tail coverage in uncertainty quantification. This observation provides useful guidance for researchers and practitioners using quantile networks.
  2. The proposed methods are simple and parameter-efficient. Direct input scaling introduces no additional trainable parameters, while the embedding variants add relatively few, making both approaches straightforward to integrate into existing SQR implementations.
Critical weaknesses
  1. The proposed explanation for tail undercoverage remains insufficiently substantiated. The initialization variance analysis in Section 2 and the observations in Figures 2 and 6 provide suggestive evidence, but do not establish why the network should remain approximately affine in τ\tau during training, or why sparse tail observations affect SQR differently from QR. The experiments do not sufficiently disentangle initialization scale, parameter sharing, activation switching, and quantile-sampling noise. Moreover, AQR improves coverage without an explicit nonlinear input transformation, leaving the relationship between “dynamic scaling” and nonlinear expressiveness unclear. Controlled interventions that vary the initialization scale of the weights associated with τ\tau or sweep constant scaling factors, would strengthen the explanation. Otherwise, the proposed mechanism should remain explicitly framed as a hypothesis.

  2. The evaluation does not fully establish calibration at individual quantile levels or the validity of the learned inverse CDF. ACE in Appendix E measures the coverage of symmetric prediction intervals, allowing calibration errors at the two endpoints to cancel. Accurate interval coverage therefore does not necessarily imply well-calibrated individual quantiles. Furthermore, the models do not explicitly enforce monotonicity in$\tau$, and quantile-crossing rates are not reported. The Sharpness metric, computed from the maximum and minimum predictions, also cannot reveal internal quantile crossings. Evaluating one-sided quantile coverage, lower- and upper-tail miscoverage, and quantile-crossing rates on a denser quantile grid using the existing models would provide stronger support for the claims.

  3. The contribution and scope of the conclusions need clearer qualification. Since IQN already employs cosine embeddings and multiplicative interactions, the primary added value of this work lies in reliably identifying the coverage failure mechanism and demonstrating a simpler remedy. Experiments with MLPs on seven tabular datasets provide useful evidence, but do not yet justify the broad recommendation across modern quantile regression tasks. In particular, the$\Phi^{-1}$ and logit transformations may introduce inductive biases favoring certain distributional shapes. The current experiments do not distinguish general optimization benefits from a favorable match between these transformations and the conditional distributions. Moreover, the marginal skewness and kurtosis reported in Table 3 do not directly characterize Y∣X=xY\mid X=x. A small set of synthetic experiments with known conditional distributions, including heteroscedastic, skewed, and bimodal cases, would strengthen the conclusions. Alternatively, the scope of the claims in the abstract and conclusion should be narrowed.

Questions for authors
  1. Have you conducted controlled experiments varying the initialization scale of the weights associated with τ\tau or the constant scaling factor to isolate the role of weak initial τ\tau signals from other optimization effects? How do these interventions affect ReLU activation patterns and tail coverage?
  2. Could you report one-sided coverage at individual quantile levels, lower- and upper-tail miscoverage rates, and quantile-crossing rates using the existing test predictions? Do the calibration improvements occur in both tails?
  3. Do the Φ−1\Phi^{-1} and logit transformations remain effective when the conditional distribution is highly skewed or multimodal? Could you provide a small set of synthetic experiments with known conditional distributions to assess their applicability, or qualify the scope of the conclusions accordingly?
3 comments

Thank you again for all your suggestions. We will strive to implement them. Your numbered questions are answered below:

  1. We agree with your suggestion to add a control regarding the initialization scale and constant scale of tau. We are currently conducting a sweep across a range of initialization scales of tau weights and the constant scaling factors (and will update you about the results when they are done). Controls regarding different activations, parameter sharing, and quantile sampling already exist, enabling a direct comparison of which techniques enabled a (partial) closing of the gap, and which did not. We will further add an ablation of the AQR mechanics to get deeper insights into what drives its performance.

  2. While ACE should not be biased in approximately Gaussian data, we agree that this bias could exist and will add an average quantile coverage error and lower- and upper miscoverage to the results. We will also report quantile crossing rates with higher quantile resolution to see if some techniques produce less crossover than others. Monotonicity in tau is indeed needed for the production of a legitimate CDF using SQR. Techniques to achieve this fall outside of the scope of this work, as one may argue that this necessitates a comparison of those techniques as well.

  3. We also agree on the limitation regarding non-Gaussian data and are happy to implement your suggestion of synthetic data experiments with heteroscedastic, skewed, and bimodal cases.

We will post another comment explaining our changes and the added experiments as soon as they are done.

Reviewer_gsSjOpenReview ↗

Additional Review Comments

I have read the authors’ response and the other two reviews. I found the response reasonable overall, and my remaining concerns overlap with those raised by the other reviewers. As the authors acknowledge, their hypothesis only partly explains why AQR achieves comparable interval coverage with an affine embedding of the quantile level. I am also unsure how well the proposed transformations work when the conditional distribution is non-Gaussian, particularly when it is skewed or multimodal. The AQR ablation and synthetic-data experiments proposed in the rebuttal would help clarify these questions.

On balance, I stand by my original score, with reservations about the explanation for AQR’s performance and the scope of the empirical findings. My direct research experience with these quantile regression models is limited, and I found parts of the technical analysis difficult to assess independently. I ask the Area Chair and the other reviewers to take this limitation into account when weighing my assessment.

Thank you for your comments! The experiments are done, and changes to the paper have been finalized. In this post, we will describe changes related to your previous comments. We also compiled more specific scope- and communication-related changes in the comment that is not a reply to any reviewer (for us, that is at the very top of the comment list).

  1. We have now done a hyperparameter optimization with an initialization scale of weights associated with τ\tau and the constant scaling factor. When optimizing per dataset, their coverage performance comes close to the performance of non-hyperparameter-optimized IQN, AQR, and the SQR$_{e/s}$ versions. We will include details regarding this here (see end of the post), but did not integrate it with the paper, as the mismatch in initial conditions does not permit a fair comparison without redesigning all experiments (for which there simply was not enough time). Additionally, the requirement of another hyperparameter to tune is arguably not beneficial for the ease of use of SQR. We did, however, try to be more precise in our claims about which scaling would help close the gap.
  2. Regarding the question of calibration improvements in both tails: Yes, IQN, AQR, and the SQR$_{e/s}$ methods show symmetric improvements in both tails. Only IQN shows a high quantile cross-over. (These results were added in Appendix J and J.2)
  3. We tested for heterogeneous, skewed, multimodal, and heavy-tailed synthetic data and found that IQN, AQR, and the SQR$_{e/s}$ stay mostly close to QR performance. In our bimodal condition, all approaches were worse than QR, and AQR was worse than all others in the heavy-tailed condition. (These results were added in Appendix J.1)

Tuned τ\tau-scale baselines:

In the code of Tagasovska and Lopez-Paz, the authors used a constant scale on τ\tau in which the first layer input of their MLP received c(τ−0.5)c(\tau-0.5) instead of a raw τ\tau with c=12c=12.
To modify the initialization of weights that interact with \tau, we take the standard SQR network and multiply all first-layer weights on τ\tau by a constant c at initialization of the network.

For both approaches, we use the same hyperparameter settings as in Appendix G. We train for 300 epochs and keep the best validation epoch, chosen with pinball. We hyperparameter optimize c∈{0.1,1,3,12,48,192}c \in \{0.1, 1, 3, 12, 48, 192\}, where c=1c=1 is equivalent τ\tau-scaling in regular SQR.
The optimal c for each approach is very dependent on the dataset, so we tuned for each dataset separately.

We can see some of the average training results over the 7 original datasets in the table below. Performance can get close to existing approaches. Interesting is that the constant approaches are noticeably sharper and have a less nonlinear slope in τ\tau.

ACEQCE⁻QCE⁺SharpnessPICP₉₅Slope ratio
QR0.0210.0160.0122.2380.93411.2
SQR0.0660.0360.0361.5800.8662.0
SQR_e (logit)0.0150.0110.0102.3800.94313.0
Constant scale, tuned c0.0240.0190.0142.0480.9344.1
Initial scale, tuned c0.0230.0150.0171.9660.9293.7

Comments on the paper

2 comments

Dear Reviewing Committee, please accept our utmost thanks for the many insightful suggestions and stimulating comments in your reviews of our work.

Many of your comments require follow-up experiments, which we are currently in the process of executing. In the meantime, we want to communicate to you what experiments we are running and how that will affect the paper. Please find a response tailored to your concerns as a comment on your review.

We plan to use this reply as a partial first response to your comments and then post a full response with updates in the PDF of the paper (including explanations of what has been updated). We are also happy about any further suggestions or discussion points you may want to add in the meantime.

Thank you all for waiting for the update. We have concluded our experiments and have updated the paper.

As for relevant PDF updates:
Generally, we changed phrasing in the Abstract, Contribution, and Conclusion to more effectively communicate the contributions, effects, and observations of the paper.
We added several new sections in the Appendix for additional experiments, and we also updated the Experiment section in the main body to include the requested synthetic data experiments.

As a general conclusion, the added experiments point towards the robustness of the novel methods under different distributional shapes. We also present some more training observations that reinforce the hypothesis regarding the coverage gap between QR and SQR: SQR always stays approximately linear in tau, while successful other approaches do not. Specifically, they show more bending into the correct direction, where SQR's non-linearities after ReLU tend to cancel. This may be due to SQR's gradient update in terms of a single tau shifting the output of other taus more than in the other approaches.

Now a list of specific scope- and communication-related changes in phrasing:

Abstract

"Input quantiles need to be dynamically scaled to result in non-linear tail coverage" was changed to "a non-linear transformation or an embedding of the input quantile suffices for SQR to match the tail coverage of QR except for bimodal data."

1 Introduction

We added "Further we investigate carry over to non-Gaussian setting on a number of synthetic data distributions."

1.2 Related Work

"However, those works do not investigate these observations with a specific focus on quantile collapse, nor show coverage comparisons with the goal of evaluating extensions on SQR" was changed to "narrow distributions or extreme value miscalibration in SQR have been observed before, but only as side observations. These observations were attributed also to different part of their respective methods. There is no work specifically investigating and showing the coverage gap between SQR and QR."

1.3 Contribution

"A thorough study of the prediction interval tail collapse in SQR" was changed to "a thorough empirical examination of the previously unexamined, but systematic, prediction interval tail collapse in SQR and how to prevent it."

"Large enough perturbations of τ\tau will enable the function of τ\tau to be non-linear in τ\tau" was changed to "non-linear transformation or embedding of τ\tau will enable the function of τ\tau to be non-linear in τ\tau."

"Our benchmark across seven independent tabular datasets reveals that approaches like IQN, AQR, and other τ\tau perturbations reach QR-level calibration" was changed to "Our benchmark across seven independent tabular datasets reveals that approaches like IQN, AQR, and other τ\tau scalers reach QR-level calibration. On five synthetic distribution shapes they also stay close to QR's performance, except for the bimodal case."

"Answer the questions of why SQR shows a too-sharp distribution" was changed to "examine the too-sharp distribution of SQR with respect to τ\tau and discover ways in which this distribution can be calibrated to match or surpass QR."

We added the bullet point "We identify and show the coverage gap between SQR and QR."

We removed the bullet point "We arrive at a final recommendation regarding optimal SQR construction for prediction interval coverage."

3 Solution Approaches

"Still produces coverage comparable to QR in our testing. So, it seems not a necessary condition for τ\tau to be scaled by a non-linear function to cause high enough expressiveness in f" was changed to "still produces coverage comparable to QR in many conditions of our testing. This however also showed AQR to have better performance when a non-linear transformation of τ\tau is applied."

5 Conclusion

"Scale or additionally embed τ\tau to increase the expressiveness of f in τ\tau" was changed to "scale or additionally embed τ\tau such that the tails can move independently."

"Lift SQR up to the coverage level of QR and recommend their usage in any modern quantile regression task" was changed to "lift SQR up to, or close to, the coverage level of QR except to multimodal distributions and recommend their usage in any applicable quantile regression task."

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.