acceptodds
Under review as a conference paper at ICLR 2027

Parameter-Free and Group Conditional Online Conformal Prediction

13 views

Abstract

Quantifying the uncertainty in predictions made by machine learning models is challenging in online settings where the characteristics of the data and labels change over time. Online conformal prediction (OCP) methods tackle this problem by producing intervals for each observation and guaranteeing that a proportion of them contain their labels, a property known as marginal coverage. OCP methods, however, may systematically miscover observations belonging to groups defined by attributes like age or race, raising fairness concerns. Group-conditional OCP (G-OCP) methods address this by producing intervals that provide coverage for multiple groups. Existing approaches rely on learning rates, which control how they react to changes in the data stream, but can be hard to tune in online settings. Leveraging advancements in learning-rate-free methods, we develop a learning-rate-free G-OCP algorithm with the strongest group-coverage guarantees among existing methods. Empirical results show that our method provides group coverage, produces interval lengths competitive with those of standard methods, and reacts more effectively to changes in the data than fixed-learning-rate methods.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
conformal prediction, parameter-free online learning, uncertainty quantification

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_oTtoOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Regular online conformal prediction targets marginal coverage and may fail on several subgroups. Recent work on group online conformal prediction addresses this issue; performance depends on the learning rate. To address this, this paper borrows the idea from Liu et al. (2026), i.e., online conformal prediction without learning-rate tuning, and extends it to group online conformal prediction. This paper provides a coverage rate theorem, conducts convergence analysis and comparisons, and runs experiments on both synthetic and real datasets.

Critical strengths
  • The narrative is logically clear and friendly. The method is clean and easy to understand.
  • The paper’s detailed analysis and comparisons in the theorem sections are strong.
Critical weaknesses
  • Although the parameter-free online group conformal is new, the contribution is not big. This paper is a modest extension of the existing work Liu et al. (2026), i.e., from marginal to group-wise coverage.
  • The abstract’s bold claim “the strongest group-coverage guarantees among existing methods” actually cannot be fully supported by the theoretical bound (the author acknowledges that in line 267).
  • The empirical results, especially in Figure 2, show that POGO has a larger gap to the target coverage, contradicting the claim in the conclusion: “experiments on synthetic and real-world data show that POGO consistently attains the target coverage level for every group”
Questions for authors
  • Line 215: What do you mean by this augmentation? Will the θt\theta_t be augmented accordingly? Could you articulate this?
  • Line 428: This is a gap compared to what the authors did for ff with time-indexed. Here, the authors even train the base model first and then compare the method on the test dataset. Can you train f at the same time?
  • Can you provide the trajectory of the online sequence instead of the terminal step's (TT) results? The worst group at TT is not necessarily the same as earlier. The full sequence can provide more information.
Reviewer_gWmaOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes Portfolios for Online Group Conformal (POGO), a learning-rate-free method for online conformal prediction with potentially overlapping groups. It parameterizes the interval radius as a linear function of group membership and maintains a Universal Portfolio wealth process for each coordinate, with all coordinates updated using the coverage error of the same prediction interval. Under polynomially growing nonconformity scores, the paper establishes a finite-time bound on each group's time-averaged coverage error. Experiments compare coverage, interval radii, and consecutive miscoverage events on synthetic streams, stock prices, and MIMIC-IV.

Critical strengths
  1. The paper is well written and easy to follow. It addresses the important problem of parameter-free online coverage for overlapping groups and establishes finite-time bound for long-run group-conditional coverage.

  2. The experiments are comprehensive, covering varying numbers of groups, distribution shifts, increasing score scales, and real-world applications, with evaluations of coverage and interval efficiency.

Critical weaknesses
  1. POGO combines the group-linear parameterization of Ramalingam et al. (2025) with the Universal Portfolio framework of Liu et al. (2026). Theorem 4.1 closely follows Theorem 4.4 of Liu et al. (2026), extending it to coordinate-wise wealth processes and soft memberships. The methodological and theoretical novelty therefore appears limited.

  2. POGO removes learning-rate tuning but fixes the portfolio prior to Jeffreys prior. The prior μ(λ)\mu(\lambda) remains an algorithmic design choice, and its influence on coverage, interval efficiency, and adaptation is not explored. The scope of the parameter-free claim and whether this prior is optimal for the conformal prediction objective deserve further discussion.

  3. The polynomial-growth experiment uses A(t/T)qA(t/T)^q with fixed AA in Eq.(13). Although scores grow within each run, this component remains bounded by AA across horizons, leaving robustness to score magnitudes growing without a horizon-independent bound insufficiently validated.

  4. Cumulative coverage and maximum consecutive miscovers provide limited information about reactivity to distribution shifts. The authors should additionally report empirical group-conditional coverage over moving windows, which would better reveal local coverage deviations and how quickly the method recovers after a shift.

References

[1] Liu, T., Dobriban, E., & Orabona, F. (2026). Online conformal prediction via universal portfolio algorithms. arXiv preprint arXiv:2602.03168.

[2] Ramalingam, R., Kiyani, S., & Roth, A. (2025). The relationship between no-regret learning and online conformal prediction. arXiv preprint arXiv:2502.10947.

Questions for authors
  1. What are the new technical ingredients beyond Liu et al. (2026)?

  2. Should the choice of μ(λ)\mu(\lambda) be regarded as a hyperparameter? How would alternative priors affect performance, and is Jeffreys prior optimal under a relevant criterion, or could another prior yield better results?

  3. Can the authors increase TT while fixing the polynomial-growth coefficient, allowing the score scale to grow?

  4. Can the authors further substantiate the advantages of POGO over existing methods, particularly under distribution shifts?

  5. Could the authors explain why POGO frequently undercovers in the experiments, especially on real-world data? Its apparent advantage in radius length seems to arise from this undercoverage.

[1] Liu, T., Dobriban, E., & Orabona, F. (2026). Online conformal prediction via universal portfolio algorithms. arXiv preprint arXiv:2602.03168.

Reviewer_99mrOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes POGO, a parameter-free algorithm for group-conditional online conformal prediction under potentially adversarial distribution shifts. By parameterizing prediction interval radii as linear functions of group memberships and employing multiple universal portfolio processes, POGO achieves simultaneous long-run coverage guarantees for potentially overlapping groups without requiring learning-rate tuning. The authors establish finite-time group-conditional coverage bounds and evaluate the proposed method on synthetic and real-world datasets, demonstrating competitive trade-offs between coverage, interval width, and adaptivity.

Critical strengths
  1. The paper addresses a meaningful limitation of existing online conformal prediction methods: marginal coverage does not necessarily ensure reliable uncertainty quantification across heterogeneous subpopulations.
  2. Compared with previous work, the resulting guarantee exhibits favorable dependence on the number of groups and does not require selecting a learning rate.
Critical weaknesses
  1. The real-world results do not fully support the claimed coverage reliability. In Table 2, POGO achieves a lowest group coverage of only 84.4% on DAL and 87.0% on MIMIC-IV, whereas GCACI achieves 88.9% and 89.8%, respectively.
  2. Algorithm 1 maintains a universal portfolio process for every group and evaluates integrals involving products of historical returns. The computational and memory costs of these operations are not analyzed in detail. In particular, it is unclear how the integrals are approximated, how numerical stability is maintained over long horizons, and whether numerical approximation introduces implementation hyperparameters.
  3. POGO controls the cumulative empirical coverage error over the entire observation horizon. Such a guarantee does not necessarily prevent substantial undercoverage within particular temporal segments, especially following abrupt distribution shifts. The longest consecutive miscoverage streak is informative but does not fully characterize local calibration. Reporting group-wise coverage over sliding windows and immediately after changepoints would strengthen the claims concerning adaptivity.
Questions for authors
  1. What explains the relatively low minimum group coverage achieved by POGO in Table 2?
  2. Since Equations (13) and (16) can potentially generate negative values, were the nonconformity scores clipped from below at zero in all experiments? If not, how are these synthetic scores reconciled with the nonnegativity assumption used in the theoretical development?
  3. How does POGO compare with distribution-informed approaches such as SPCI [1] and COP [2]? Could these methods capture group-conditional coverage through group-aware score distribution estimation?

[1]: Xu C, Xie Y. Sequential predictive conformal inference for time series[C]//ICML 2023.

[2]: Hu D, Wu J, Xia S T, et al. Distribution-informed online conformal prediction[C]//ICLR 2026.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.