acceptodds
Under review as a conference paper at ICLR 2027

Dual-Calibrated Online Conformal Inference with Hybrid Quantile Loss

Abstract

Conformal inference provides rigorous coverage guarantees for uncertainty quantification, yet its dynamic coverage accuracy is highly susceptible to distribution shifts in non-stationary environments. Existing approaches construct prediction sets via dynamically estimated conditional quantiles but suffer from two key limitations. First, most algorithms passively correct for coverage deviations, which often results in excessively conservative prediction intervals. Second, prevailing update strategies rely on the standard quantile loss, whose inherent drawbacks undermine coverage adaptability. To address these issues, we introduce an -insensitive Hybrid Quantile (-HQ) loss with a bounded quadratic component, formally establish its Lipschitz continuity and generalized weak pseudo-convexity (GWPC), and design a gain-balancing mechanism to match gain magnitudes between the two components. Building upon this loss, we propose a Dual-Calibrated Online Conformal Inference framework, -HQ‑DCOCI. The first calibration refines gradient signals via adaptive residual mapping and insensitive-zone tuning, enabling continuous feedback correction. The second calibration accelerates threshold updates under persistent drift via momentum-based smoothing of historical subgradient directions. Theoretically, we prove that -HQ‑DCOCI achieves sublinear regret bounds and approximately valid coverage in non-stationary settings. Experiments on real-world datasets, including financial time series and medical image classification, demonstrate that our framework consistently yields tighter and more efficient prediction sets compared to state-of-the-art baselines.

Primary area:
optimization
Keywords:
Online conformal inference, Uncertainty quantification, Lipschitz continuity, Generalized weak pseudo-convexity, Dynamic regret, Non-stationary time series

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_MybKOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies online conformal inference under distribution shifts. The authors propose an ε\varepsilon-insensitive hybrid quantile loss (ε\varepsilon-HQ) that combines the standard quantile loss with a bounded quadratic component to provide more informative gradient feedback. Building on this loss, they develop a dual-calibration framework that adaptively rescales residuals, adjusts the insensitive zone, and uses historical gradient directions for an additional threshold correction. The paper presents dynamic regret and coverage-error bounds, along with experiments on financial time series and medical image classification datasets.

Critical strengths
  1. The problem is relevant and well motivated. Improving the coverage–efficiency trade-off under non-stationarity is an important problem in online conformal inference. The limitations of binary quantile-loss feedback provide a reasonable motivation for exploring alternative update rules.

  2. The hybrid loss and modular design are interesting. Combining a bounded quadratic component with the standard quantile loss offers a way to incorporate residual magnitude without losing non-vanishing gradients for large deviations. The paper also makes a substantial effort to analyze the proposed loss and connect the resulting online updates to regret and coverage guarantees.

  3. The experimental evaluation is reasonably comprehensive. The authors compare against several recent baselines across different datasets and prediction models. The ablation and sensitivity analyses are also useful for understanding how the individual components affect performance.

Critical weaknesses
  1. The asymptotic coverage argument appears to have a gap. Theorem 5.2 establishes a dynamic regret bound of order O(T(1+VT))O(\sqrt{T(1+V_T)}), but Corollary 5.4 assumes only VT=O(T)V_T=O(T) when concluding that the normalized regret vanishes. This would generally require VT=o(T)V_T=o(T). Moreover, VTV_T is defined through instantaneous loss minimizers, which may fluctuate even under stationary score distributions. It is therefore unclear when the regret bound leads to a meaningful asymptotic coverage guarantee.

  2. Some parts of the theoretical analysis need clarification. The expression for λ(τ)\lambda(\tau) in Eq. (5) appears inconsistent with the preceding equal-peak condition, which also affects the constants in Eq. (6). In addition, Appendix B.1 and the coverage proof in Appendix B.4 treat the residual mapping as a single affine transformation, whereas Eq. (7) defines different mappings for positive and negative residuals. The arguments relying on a common slope do not seem to follow directly when the residuals have opposite signs.

  3. The interpretation of historical gradient directions as a drift signal is not fully convincing. The momentum correction uses an average of normalized gradient directions. However, these directions are inherently asymmetric for quantile estimation. For example, with a continuous score distribution, ε=0\varepsilon=0, and a correctly calibrated threshold, their expected average is 1−2α1-2\alpha, which equals 0.80.8 when α=0.1\alpha=0.1, even without distribution shift. Thus, a persistent directional signal does not necessarily indicate drift, raising questions about how the additional correction affects calibration.

  4. The empirical advantage of the full framework remains somewhat limited. In Table 1, with LightGBM, the simpler ε=0\varepsilon=0-HQ-OGD achieves better MIS than ε=0\varepsilon=0-HQ-DCOGD on three of the four financial datasets, suggesting that the momentum correction does not consistently improve performance. The full method also falls below the nominal 90% coverage on DERM12345. More importantly, the experiments mainly report aggregate metrics, making it difficult to assess whether the method actually adapts faster to distribution shifts. The class-ordered medical image streams provide only a limited test of this claim.

Questions for authors
  1. Could the authors clarify the conditions under which the normalized dynamic regret vanishes in Corollary 5.4? In particular, how should VTV_T behave under stationary or slowly varying score distributions, given that it is defined using instantaneous loss minimizers?

  2. Could the authors verify Eq. (5) and clarify how the proofs in Appendices B.1 and B.4 account for the sign-dependent residual mapping? Do the stated regret and coverage bounds apply directly to the implemented algorithm?

  3. How does the momentum mechanism distinguish genuine distribution drift from the inherent asymmetry of quantile-loss gradient directions? Could the authors provide rolling coverage and recovery-time comparisons with ε\varepsilon-HQ-OGD to demonstrate when the additional calibration is beneficial?

5 comments

We thank the reviewer for the positive evaluation of our research problem, loss function design, and theoretical/empirical analyses. Addressing your concerns regarding theoretical rigorousness, mapping consistency, and empirical evidence, our detailed responses follow.

Response 1: Theoretical conditions for the asymptotic coverage corollary (Weakness 1). We acknowledge that deducing the vanishing of normalized dynamic regret solely from VT=O(T)V_T=O(T) is invalid. Since the instantaneous comparator relies on the realized score, its path length may grow linearly even under a stationary distribution. To address this, we explicitly select the common instantaneous minimizer qtopt=stq_t^{opt}=s_t for both the original and scaled losses, and define VT=∑t=1T−1∣st+1−st∣V_T=\sum_{t=1}^{T-1}|s_{t+1}-s_t|. Under the specified regularity conditions, Theorem 5.2 provides the dynamic regret upper bound:

R~Tdyn≤CjointμT(4R2+3RVT)=O(T(1+VT)).\tilde{R}_ T^{\mathrm{dyn}} \le \frac{C_{\mathrm{joint}}}{\mu}\sqrt{T(4R^2+3R V_T)} = \mathcal{O}\bigl(\sqrt{T(1+V_T)}\bigr).

Consequently, the normalized regret is bounded by O((1+VT)/T)\mathcal{O}(\sqrt{(1+V_T)/T}). Only when VT=o(T)V_T=o(T) does this bound guarantee convergence to zero; for VT=Θ(T)V_T=\Theta(T), this cannot be deduced. Accordingly, the revised Corollary 5.4 retains the long-term regret term:

lim sup⁡T→∞∣C^T−(1−α)∣≤CμR~∞+2Camin⁡ε∞+CμΔ∞,\limsup_{T\to\infty}|\widehat C_T-(1-\alpha)| \le \frac C\mu\widetilde R_\infty + \frac{2C}{a_{\min}}\varepsilon_\infty + \frac C\mu\Delta_\infty,

where R~∞=lim sup⁡T→∞R~TdynT\widetilde R_\infty = \limsup_{T\to\infty} \frac{\widetilde R_T^{dyn}}{T}, ε∞=lim sup⁡T→∞1T∑t=1Tεt\varepsilon_\infty = \limsup_{T\to\infty} \frac{1}{T}\sum_{t=1}^T\varepsilon_t, and Δ∞=CMZabs22amin⁡2+1−τamin⁡+MZabs2amin⁡\Delta_\infty = \frac{CMZ_{\mathrm{abs}}^2}{2a_{\min}^2} + \frac{1-\tau}{a_{\min}} + \frac{MZ_{\mathrm{abs}}}{2a_{\min}}. This bound decomposes the asymptotic coverage error into a potentially non-vanishing normalized regret, the average insensitive-zone width, and the residual loss bias, while conservatively incorporating the clipping-induced boundary probability mass. The corollary is now rigorously formulated as an asymptotic coverage error upper bound under the specified conditions (see Section 5.1–5.2, Theorems 5.1–5.3, Corollary 5.4; Appendix B.2–B.5).

Response 2: Gain-balancing and sign-dependent residual mapping (Weakness 2). We acknowledge that Eq. (5) was inconsistent with the equal-peak condition, and that the common-slope arguments in Appendices B.1 and B.4 were insufficient. We address this from three aspects:

(1) Gain-balancing formula. Solving the equal-peak constraint τA=(1−τ)33λ8γB\tau A=(1-\tau)\frac{3\sqrt{3\lambda}}{8\gamma}B under γ=1\gamma=1 yields λ(τ)=13(8A3Bτ1−τ)2\lambda(\tau)=\frac13\left(\frac{8A}{3B}\frac{\tau}{1-\tau}\right)^2. The theoretical constants in Eq. (6) are synchronously verified as M=2τAM=2\tau A, μ=τmin⁡(α,1−α)\mu=\tau\min(\alpha,1-\alpha), and K=τA(1+433)K=\tau A(1+\frac{4}{3\sqrt3}) (see Section 4.1, Eqs. (5)–(6); Appendix A.1–A.2).

(2) Piecewise mapping and gradient consistency. We distinguish mapping branches. In the positive-slope piecewise-affine branch ϕt(z)=ats(z)z\phi_t(z)=a_t^{s(z)}z, we define a scaled loss ℓ~t(q)=1ats(z)Lε-HQ(ϕt(st−q))\tilde\ell_t(q)=\frac{1}{a_t^{s(z)}}L_{\varepsilon\text{-HQ}}\left(\phi_t(s_t-q)\right) to mitigate slope-factor interference. Within this branch, the chain rule yields −Lε-HQ′(ϕt(st−q))=gt-L'_{\varepsilon\text{-HQ}}\left(\phi_t(s_t-q)\right)=g_t, aligning with the actual update signal in Algorithm 1 (see Section 4.2–4.3, Eqs. (7), (11)–(15); Appendix B.1).

(3) Branch handling in proofs. B.1 selects the common minimizer qtopt=stq_t^{opt} = s_t and leverages Lε−HQ(u)≤K(∣u∣−εt)+L_{\varepsilon\mathrm{-HQ}}(u) \le K(|u|-\varepsilon_t)_+ for opposite-sign residuals. In B.4, threshold deviations are established using the triangle inequality ∣st−qt∣+∣st−qtqt∣≥∣qt−qtqt∣|s_t-q_t| + |s_t-q_t^{qt}| \ge |q_t-q_t^{qt}| with independent slopes at,1,at,2∈[amin⁡,amax⁡]a _ {t,1}, a _ {t,2} \in [a _ {\min}, a _ {\max}]. B.4 also independently estimates the clipping boundary mass $E [|u| \mid \mathcal F _{t-1}] \le \frac{C Z _{\mathrm{abs}} ^ 2}{a _ {\min}} + Z _{\mathrm{abs}} $ (see Section 5.2, Theorem 5.3; Appendix B.1, Eq. (40); Appendix B.4).

Response 3: Gradient direction bias and the EMA-centered mechanism (Weakness 3). We thank the reviewer for highlighting the systematic bias in momentum direction statistics. For correctly calibrated continuous scores, the standard quantile gradient direction satisfies E[sign⁡(gt)]=1−2α\mathbb E[\operatorname{sign}(g_t)] = 1-2\alpha (0.80.8 for α=0.1\alpha=0.1). Using the raw mean conflated this inherent asymmetry with actual drift.

To resolve this, we introduce an EMA-centered mechanism. With window mean mt=1W∑i=t−W+1tsign⁡(gi)m_t = \frac1W\sum_{i=t-W+1}^{t}\operatorname{sign}(g_i), the EMA baseline updates as b^t+1=(1−σ)b^t+σmt\hat b_{t+1} = (1-\sigma)\hat b_t+\sigma m_t (initialized at b^1=1−2α\hat b_1=1-2\alpha), and the centered momentum is νt+1=12(mt−b^t+1)\nu_{t+1} = \frac12(m_t-\hat b_{t+1}). This measures recent variations relative to the historical baseline, avoiding static bias. From the EMA update, νt+1=1−σ2σ(b^t+1−b^t)\nu_{t+1} = \frac{1-\sigma}{2\sigma}(\hat b_{t+1}-\hat b_t). Appendix B.3 utilizes this difference structure and Abel summation to establish ∣∑t=1Tνt+1(qt−qtopt)∣≤cσ[2R+T(ηM+η′)+VT]\left|\sum_{t=1}^{T}\nu_{t+1}(q_t-q_t^{opt})\right| \le c_\sigma[2R+T(\eta M+\eta')+V_T], strictly preserving the O(T(1+VT))\mathcal{O}(\sqrt{T(1+V_T)}) regret order.

Furthermore, the centered signal concurrently drives both calibrations: its sign dictates the momentum correction η′νt+1\eta'\nu_{t+1}, while its magnitude ∣νt∣|\nu_t| dynamically adjusts the insensitive-zone width εt=max⁡{dt(1−∣νt∣),εmin⁡}\varepsilon_t = \max\{d_t(1-|\nu_t|), \varepsilon_{\min}\}. As directional deviation increases, the insensitive zone shrinks to enhance drift sensitivity; as the baseline adapts, both corrections diminish (see Section 4.2-4.3, Eqs. (9)-(10), (13)-(15); Theorem 5.2; Appendix B.3; Appendix C.1, Algorithm 1).

Response 4: Incremental benefits and dynamic adaptability evaluation (Weakness 4). We acknowledge that the original aggregate metrics obscure transient recovery; conventional datasets are insufficient to evaluate adaptability to abrupt shifts. We add four synthetic drift scenarios and introduce Recovery Time (RT) and rolling coverage analyses.

(1) MIS increments. ε\varepsilon-HQ-DCOGD achieves lower MIS than ε\varepsilon-HQ-OGD in Variance Changepoint (1.356 vs. 1.362), Changepoint (1.203 vs. 1.236), and Extreme Drift (1.208 vs. 1.214), and comparable values under Smooth Drift (1.073). Incremental benefits primarily occur in structural shifts (see Section 6.3, Table 3; Appendix D.11, Table 19).

(2) Transient recovery. In Changepoint, the full model shortens average recovery times by 9.0 and 4.8 steps (68.4±23.223→59.4±17.77168.4\pm23.223 \to 59.4\pm17.771; 39.6±13.334→34.8±6.01739.6\pm13.334 \to 34.8\pm6.017), with lower RT standard deviation. MIS decreases from 1.236 to 1.203, and interval width from 0.920 to 0.915. Figure 2 illustrates the rolling coverage trajectory. Notably, ECI achieves a lower MIS (1.196) but at only 86.880% coverage vs. our 88.608%, reflecting a distinct coverage-efficiency trade-off (see Section 6.3; Appendix D.11, Tables 19-20, Figure 2).

(3) Medical trade-off. On DERM12345 and ResNet-50, ε\varepsilon-HQ-DCOGD yields Coverage 87.551%, PI-Size 1.981, MDSC 0.673, vs. ε\varepsilon-HQ-OGD's 89.238%, 2.089, and 0.671. We report this sub-nominal coverage, distinguishing empirical coverage, set efficiency, and composite scores without equating smaller sets to coverage improvement (see Section 6.2, Table 2; Appendix D.6-D.7).

Validated across 4 financial, 2 medical, and 4 synthetic datasets, the evidence substantiates the transient adaptation value in abrupt settings, while delineating that incremental benefits are not universally established (see Section 6.1-6.3; Appendices D.5-D.7, D.11).

Responses to Specific Questions

Question 1: Since VTV_T relies on realized scores, random fluctuations may cause VT=Θ(T)V_T=\Theta(T) even under stationarity. The O(T(1+VT))\mathcal{O}(\sqrt{T(1+V_T)}) bound implies VT=o(T)V_T=o(T) is sufficient for vanishing normalized regret; Corollary 5.4 explicitly retains R~∞\widetilde R_\infty (see Response 1).

Question 2: Eq. (5) is corrected for the 1/31/3 factor and Eq. (6) verified. B.1 defines the scaled loss on the positive-slope branch to align gradients and handles opposite-sign residuals via the loss upper bound. B.4 uses independent branch slopes and the triangle inequality, supplemented by conservative clipping mass control (see Response 2).

Question 3: The EMA baseline b^t\hat b_t isolates long-term bias via recent directional variations. This signal concurrently drives both the momentum term and the dynamic insensitive zone. Changepoint experiments show the full model shortens average recovery times by 9.0 and 4.8 steps vs. ε\varepsilon-HQ-OGD (see Response 3 and Response 4).

On the Rating and Confidence. We understand your concerns regarding mathematical rigorousness, inherent direction bias, and dynamic adaptation evidence. We hope these clarifications—spanning sign-dependent mapping proofs, EMA-centered momentum, condition-dependent regret bounds, and changepoint experiments—strengthen your confidence in our work.

AuthorsOpenReview ↗

Supplementary Update on the Clipping/Fallback Issue (W2)

We would like to provide a brief supplementary update regarding the clipping/fallback issue raised in W2.

In our previous response, we acknowledged that the affine-branch analysis did not establish global gradient consistency for the hard clipping/fallback branch. We have since performed a further theoretical revision to address this limitation.

For the integral surrogate loss defined in Appendix B.1, we have

∂qℓ~t(q)=−Lε-HQ′(ϕt(st−q))=gt.\partial_q \tilde{\ell}_ t(q) = -L'_{\varepsilon\text{-HQ}}\bigl(\phi_t(s_t-q)\bigr) = g_t.

Importantly, this identity does not involve ϕt′\phi'_t, and therefore remains consistent with the implemented gradient on both the affine and clipping/fallback branches.

We have also revised Assumption 5.1 to distinguish the positive-slope affine condition from the clipping-branch endpoint condition, introducing a unified radial lower bound:

∣ϕt(u)∣≥min⁡(amin⁡∣u∣,Z∗),Z∗>εt.|\phi_t(u)| \ge \min\bigl(a_{\min}|u|, Z_*\bigr), \qquad Z_* > \varepsilon_t.

Using this condition together with the sign-preserving property, Appendix B.4 re-establishes the residual-to-loss lower bound underlying Eqs.(63)–(64), including clipping/fallback cases. The finite-time contribution arising from the warm-up phase is also explicitly accounted for.

Consequently, under the stated regularity and endpoint conditions, the revised regret and coverage analyses apply to the implemented Algorithm 1, rather than being restricted to the non-saturated affine branch.

We appreciate the reviewer's earlier observation, which motivated us to further strengthen the consistency between the implemented algorithm and its theoretical guarantees. We hope this fully addresses the raised concern.

Revisions made in: Sections 4.2–4.3, Assumption 5.1, Theorem 5.3, Corollary 5.4, Appendices B.1, B.4 and B.5.

Reviewer_LuswOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes ε-HQ-DCOCI, a dual-calibrated online conformal inference framework designed to improve prediction-set efficiency and coverage adaptation under distribution shifts. The method introduces an ε-insensitive Hybrid Quantile (ε-HQ) loss combining linear and bounded quadratic components, together with adaptive residual mapping, dynamic insensitive-zone tuning, and momentum-based threshold calibration. The authors provide theoretical analyses of the loss properties, dynamic regret, and coverage validity. Experiments on four financial time series and two medical image classification datasets demonstrate improvements over several existing online conformal prediction methods.

Critical strengths
  1. The proposed ε-HQ loss combines the persistent gradient signal of a linear quantile loss with a bounded quadratic component that provides magnitude-dependent feedback.
  2. The proposed framework separates gradient refinement from momentum-based calibration, allowing the two components to be incorporated into existing online gradient-based conformal inference algorithms. This modularity potentially improves the applicability of the approach beyond a single threshold-update strategy.
Critical weaknesses
  1. The sublinear dynamic regret and asymptotic coverage claims require stronger justification. Theorems 5.1 and 5.2 establish dynamic regret bounds of order O((T(1+VT)))O(\sqrt{(T(1+V_T))}). However, Corollary 5.4 assumes VT=O(T)V_T = O(T) and concludes that the average regret vanishes. This conclusion does not follow from the stated bound: when VT=Θ(T)V_T = Θ(T), the upper bound is O(T)O(T), and dividing by T does not yield convergence to zero.
  2. The empirical improvements are relatively modest and do not consistently support the benefit of dual calibration. In Table 1, ε-HQ-OGD achieves better MIS than ε-HQ-DCOGD on Bitcoin, IXIC, and XAU, suggesting that the momentum calibration does not consistently improve the performance of the hybrid-loss update. Several improvements over standard OGD are also numerically small. Furthermore, the best MDSC result on DERM12345 in Table 2 is obtained by ε-HQ-DCOGD with coverage of only 87.271%, below the nominal 90% target. This raises concerns about whether the reported efficiency improvements consistently preserve the desired coverage level.
Questions for authors
  1. In Algorithm 1, why is the subgradient computed using the residual st−qts_t − q_t while the base update is performed on q^t\hat{q}_t?
  2. In Section 4.1, Eq. (5) appears inconsistent with the equal-peak alignment constraint stated immediately before it. Specifically, using the stated quadratic-component peak gradient, the constraint implies λ=[8Aτ/(33B(1−τ))]2λ = [8Aτ/(3\sqrt{3}B(1−τ))]^2 when γ=1,γ = 1, whereas Eq. (5) gives λ=[8Aτ/(3B(1−τ))]2.λ = [8Aτ/(3B(1−τ))]^2.
  3. Could the authors more clearly quantify the incremental contribution of momentum calibration over ε-HQ-OGD? Given that ε-HQ-OGD outperforms the full method in several financial experiments, under what distribution-shift regimes does the second calibration reliably help?
5 comments

We thank the reviewer for the positive evaluation of our paper's clarity, research value, and methodology, and for recognizing our ε\varepsilon-HQ loss (integrating linear quantile gradient feedback with bounded quadratic magnitude information) and the modular framework decoupling gradient refinement from momentum calibration. Addressing your concerns regarding theoretical rigorousness and empirical benefits, our detailed responses follow.

Response 1: Conditions for dynamic regret and asymptotic coverage guarantees (Weakness 1). We agree that deducing the vanishing of normalized dynamic regret solely from VT=O(T)V_T=O(T) is invalid. We re-clarify the relationship between the comparator path length, dynamic regret order, and coverage error.

(1) Dynamic regret bounds and their applicable conditions. The revision adopts the common instantaneous minimizer qtopt=stq_t^{opt}=s_t and defines VT=∑t=1T−1∣qt+1opt−qtopt∣V_T=\sum_{t=1}^{T-1}|q_{t+1}^{opt}-q_t^{opt}|. Theorem 5.2 establishes a bound of O(T(1+VT))\mathcal{O}(\sqrt{T(1+V_T)}) for the full model, guaranteeing vanishing normalized regret only when VT=o(T)V_T=o(T); for VT=Θ(T)V_T=\Theta(T), the bound degrades to O(T)\mathcal{O}(T), and vanishing average regret cannot be deduced. Since the comparator relies on realized scores, random fluctuations may yield a linear path length even under stationarity. Hence, we no longer equate stationarity with a sublinear path length (see Section 5.1, Theorems 5.1–5.2, Eqs. (16)–(17); Appendix B.2–B.3).

(2) Correspondence between the joint update and the regret proof. Aligning Theorem 5.2 with the algorithm, the full compound update is adopted: qt+1=ΠQ[ΠQ(qt−ηgt)−η′νt+1]q_{t+1}=\Pi_Q[\Pi_Q(q_t-\eta g_t)-\eta'\nu_{t+1}]. Leveraging the EMA-centered momentum νt+1=cσ(b^t+1−b^t)\nu_{t+1}=c_\sigma(\hat b_{t+1}-\hat b_t) (where cσ=1−σ2σc_\sigma=\frac{1-\sigma}{2\sigma}), Appendix B.3 derives a one-step regret inequality. Abel summation strictly bounds the cross term by cσ[2R+T(ηM+η′)+VT]c_\sigma[2R+T(\eta M+\eta')+V_T], preserving the base calibration's regret order without treating the intermediate threshold as an independent OGD sequence (see Section 4.3, Eq. (15); Section 5.1, Theorem 5.2; Appendix B.3; Appendix C.1, Algorithm 1).

(3) Corollary on asymptotic coverage error. Since normalized regret may not vanish, the revised Corollary 5.4 explicitly retains the long-term regret term:

lim sup⁡T→∞∣C^T−(1−α)∣≤CμR~∞+2Camin⁡ε∞+CμΔ∞,\limsup_{T\to\infty}|\widehat C_T-(1-\alpha)|\le \frac C\mu\widetilde R_\infty+\frac{2C}{a_{\min}}\varepsilon_\infty+\frac C\mu\Delta_\infty,

where R~∞=lim sup⁡T→∞R~TdynT\widetilde R_\infty=\limsup_{T\to\infty}\frac{\widetilde R_T^{dyn}}T, ε∞=lim sup⁡T→∞1T∑t=1Tεt\varepsilon_\infty=\limsup_{T\to\infty}\frac1T\sum_{t=1}^T\varepsilon_t, and Δ∞=CMZabs22amin⁡2+1−τamin⁡+MZabs2amin⁡\Delta_\infty=\frac{CMZ_{\mathrm{abs}}^2}{2a_{\min}^2}+\frac{1-\tau}{a_{\min}}+\frac{MZ_{\mathrm{abs}}}{2a_{\min}}. Here, the three terms represent the optimization regret, average insensitive-zone width, and residual bias induced by loss and mapping. The last term conservatively bounds the newly added clipping boundary probability mass. The corollary is rigorously formulated as an asymptotic coverage error decomposition under the specified conditions (see Section 5.1–5.2, Theorem 5.3, Corollary 5.4, Eqs. (18)–(19); Appendix B.4–B.5).

Response 2: Incremental benefits of dual calibration and the coverage-efficiency trade-off (Weakness 2). We agree that the original results were insufficient to demonstrate universal superiority. We have re-verified the experiments and incorporated evaluations of dynamic shifts.

(1) Incremental benefits in financial experiments. Across 12 financial combinations, ε\varepsilon-HQ-DCOGD achieves a lower MIS than ε\varepsilon-HQ-OGD in 7 combinations (LightGBM 1/4, Theta 3/4, Prophet 3/4). Notably, ε\varepsilon-HQ-OGD still outperforms on Bitcoin, IXIC, and XAU under LightGBM, indicating that momentum gains are not universally guaranteed. We thus characterize the second calibration's contribution as task- and shift-dependent (see Section 6.1, Table 1; Appendix D.5, Tables 7–8).

(2) Incremental contributions in synthetic drift experiments. To analyze which distribution changes benefit from momentum, we introduce four synthetic drift settings. ε\varepsilon-HQ-DCOGD achieves lower MIS in Changepoint (1.203 vs 1.236) and Extreme Drift (1.208 vs 1.214), and comparable MIS under Smooth Drift (1.073). Furthermore, under Changepoint, the full model shortens average recovery times after the two changepoints by 9.0 and 4.8 steps compared to ε\varepsilon-HQ-OGD (59.459.4 vs. 68.468.4; 34.834.8 vs. 39.639.6), substantiating its transient adaptation advantage in abrupt environments (see Section 6.3, Table 3; Appendix D.11, Tables 19–20, Figure 2).

(3) Empirical coverage and efficiency trade-off in medical tasks. On DERM12345 and ResNet-50, ε\varepsilon-HQ-DCOGD yields Coverage 87.551%, PI-Size 1.981, and MDSC 0.673, vs. ε\varepsilon-HQ-OGD's 89.238%, 2.089, and 0.671. The full model attains smaller prediction sets and slightly higher MDSC at the cost of lower coverage. We interpret this as an empirical trade-off among coverage, set size, and composite score, rather than claiming inherent coverage improvement, thereby allowing objective comparison across methods (see Section 6.2, Table 2; Appendix D.6, Tables 9–10).

Responses to Specific Questions

Question 1: The revision distinguishes the actual threshold from the intermediate variable. The gradient is computed at qtq_t, specifically gt=−Lε-HQ′(ϕt(st−qt))g_t = -L'_ {\varepsilon\text{-HQ}}(\phi_t(s_t-q_t)); the intermediate update q^t+1=ΠQ(qt−ηgt)\hat{q}_{t+1} = \Pi_Q(q_t - \eta g_t) is merely a base step variable. Appendix B.3 analyzes the full recursion in Eq. (15), eliminating prior inconsistencies between variable definitions and proof targets (see Response 1(2)).

Question 2: We confirm the omission of the 1/31/3 factor. Under γ=1\gamma=1, the recalculated analytical solution from the constraint τA=(1−τ)33λ8γB\tau A=(1-\tau)\frac{3\sqrt{3\lambda}}{8\gamma}B is λ(τ)=13(8Aτ3B(1−τ))2\lambda(\tau)=\frac13\left(\frac{8A\tau}{3B(1-\tau)}\right)^2. Eq. (5) is updated, and the Lipschitz, GWPC, and gradient lower bound constants in Eq. (6) are re-verified (see Section 4.1, Eqs. (5)–(6); Appendix A.1–A.2).

Question 3: Synthetic experiments demonstrate significant incremental benefits in abrupt drift environments. In Changepoint, ε\varepsilon-HQ-DCOGD achieves a lower MIS (1.203 vs. 1.236) and shortens average recovery times by 9.0 and 4.8 steps; under Smooth Drift, MIS values are comparable (1.073). This supports the transient adaptation value in specific abrupt environments, rather than universal superiority (see Response 2(2)).

On the Rating and Confidence. We understand your concerns regarding theoretical rigorousness and the consistency of momentum's incremental benefits. We hope the clarifications above—including relaxed asymptotic coverage assumptions, explicit regret bounds, expanded financial validation, and quantified transient stability analysis—strengthen your confidence in the effectiveness of our method.

The revised analysis substantially improves the consistency of the joint update and the regret proof. However, I remain concerned about the clipping/fallback branch. In saturated regions, the derivative of the clipped composite loss with respect to the threshold can be zero, whereas Algorithm 1 uses the nonzero derivative of the ε-HQ loss evaluated at the clipping boundary. Moreover, the identity used in Eq. (63)–(64) does not generally hold under clipping. Could the authors clarify how Assumption 5.1 and the coverage-error bound remain valid for the exact implemented algorithm, including all fallback cases?

Reviewer_JRm8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates online conformal inference under distribution shift, with the aim of keeping empirical coverage close to the target while producing efficient prediction sets. It introduces an ε-insensitive hybrid quantile loss that combines pinball loss and a bounded asymmetric least-squares component. The resulting ε-HQ-DCOCI method uses adaptive residual mapping, insensitive-zone tuning, and momentum-based calibration for threshold updates. The authors analyze Lipschitz continuity and generalized weak pseudo-convexity, claim dynamic regret and coverage guarantees, and present experiments on financial time series and medical image classification.

Critical strengths
  1. The paper considers modifying the loss used for online conformal threshold updates, rather than only adjusting the learning rate or adding correction terms. This is a reasonable direction to explore and addresses a relevant limitation of standard quantile-loss feedback.
  2. The evaluation covers both financial time series and medical image classification, with multiple base predictors, comparisons with several online conformal baselines, and additional ablation studies. This gives a useful empirical basis for examining the proposed components.
Critical weaknesses
  1. The main theoretical guarantees are not sufficiently supported by the stated analysis. Theorem 5.2 gives a dynamic regret bound of O(T(1+VT))O(\sqrt{T(1+V_T)}), but Corollary 5.4 assumes only VT=O(T)V_T=O(T) and then treats the normalized regret as vanishing; that conclusion does not follow from this bound. The comparator is also defined using the realized score and may have a linearly growing path length even in a stationary setting. In addition, the gain-balancing calculation in Eq. (5) appears to omit a factor of 1/3, affecting the constants in Eq. (6). Algorithm 1 computes its residual using qtq_t, whereas the regret proof relies on a subgradient at q^t\hat{q}_t. These issues affect the connection between the proposed algorithm and its claimed regret and coverage guarantees.
  2. The empirical results do not consistently establish the benefit of the full dual calibration. Across the 12 financial dataset-model combinations in Tables 1, 6, and 7, ε-HQ-DCOGD improves MIS over ε-HQ-OGD in only four cases. On DERM12345 with ResNet-50, ε-HQ-DCOGD achieves an MDSC of 0.673 but only 87.271% coverage, below the nominal 90% level. Since the hybrid loss combines existing components and the momentum step averages past gradient directions, the practical advantage of the full method needs stronger justification.
Questions for authors
  1. Could the authors clarify the derivation of Eq. (5) and the constants in Eq. (6)? In particular, is a factor of 1/3 missing from the expression for λ(τ)\lambda(\tau)?
  2. What additional assumptions ensure that R~Tdyn/T→0\widetilde{R}_T^{\mathrm{dyn}}/T \to 0 in Corollary 5.4? How is the path length VTV_T controlled when the comparator is based on realized scores?
  3. Is the gradient in Algorithm 1 evaluated at qtq_t or q^t\hat{q}_t? Can the authors reconcile the pseudocode with the assumptions in the regret and coverage proofs?
  4. Can the authors explain when the momentum step improves over the loss-only variant, perhaps with a clearer comparison of rolling coverage and prediction-set size?
6 comments

We thank the reviewer for the positive evaluation. In response to your concerns regarding theoretical guarantees, algorithm-theory consistency, and the practical benefits of dual calibration, we have re-derived the relevant formulas, revised the regret/coverage analyses, and incorporated distribution-drift experiments. Our detailed responses are provided below.

Response 1: Theoretical correctness, comparator path length, and algorithm consistency (Weakness 1). We acknowledge the three core issues: (i) deducing vanishing normalized regret from VT=O(T)V_T=O(T) in Corollary 5.4 is invalid, as the realized-score comparator may yield a linear path length even under stationarity; (ii) Eq. (5) omits the 1/31/3 factor; (iii) Algorithm 1 computes the residual at qtq_t, whereas the proof relied on q^t\hat q_t. We address these below:

(1) Comparator path length and asymptotic coverage. Because the instantaneous comparator qtopt=stq_t^{\mathrm{opt}} = s_t fluctuates with realized scores, the path length may remain Θ(T)\Theta(T) even under stationary distributions. Therefore, the original argument is invalid. We explicitly select the common instantaneous minimizer qtopt=stq_t^{\mathrm{opt}}=s_t and retain the potentially non-vanishing long-term normalized regret term in Corollary 5.4:

lim sup⁡T→∞∣C^T−(1−α)∣≤CμR~∞+2Camin⁡ε∞+CμΔ∞,\limsup_{T\to\infty}|\widehat C_T-(1-\alpha)|\le \frac C\mu\widetilde R_\infty+\frac{2C}{a_{\min}}\varepsilon_\infty+\frac C\mu\Delta_\infty,

where R~∞=lim sup⁡T→∞R~TdynT\widetilde R_\infty=\limsup_{T\to\infty}\frac{\widetilde R_T^{dyn}}T is the irreducible regret residual, ε∞=lim sup⁡T→∞1T∑t=1Tεt\varepsilon_\infty=\limsup_{T\to\infty}\frac1T\sum_{t=1}^T\varepsilon_t is the insensitive-zone bias, and the residual bias bound is

Δ∞=CMZabs22amin⁡2+1−τamin⁡+MZabs2amin⁡,Zabs=max⁡(∣Zmin⁡∣,Zmax⁡,amax⁡R).\Delta_\infty=\frac{CMZ_{\mathrm{abs}}^2}{2a_{\min}^2}+\frac{1-\tau}{a_{\min}}+\frac{MZ_{\mathrm{abs}}}{2a_{\min}},\qquad Z_{\mathrm{abs}}=\max(|Z_{\min}|,Z_{\max},a_{\max}R).

The newly added final term conservatively controls the clipping boundary probability mass. Only when VT=o(T)V_T=o(T) does the upper bound guarantee the normalized regret vanishes; for VT=Θ(T)V_T=\Theta(T), this cannot be deduced. The corollary is now strictly defined as an asymptotic coverage error upper bound under the stated regularity conditions (see Section 5.2, Theorem 5.3, Corollary 5.4, Eqs. (18)–(19); Appendix B.4–B.5).

(2) Gain-balancing formula and theoretical constants. The original Eq. (5) omitted the 1/31/3 factor required by the equal-peak condition. Based on the maximum gradient magnitudes Glinear=τAG_{\mathrm{linear}}=\tau A and Gquadratic=(1−τ)33λ8γBG_{\mathrm{quadratic}}=(1-\tau)\frac{3\sqrt{3\lambda}}{8\gamma}B (with A=max⁡(α,1−α)A=\max(\alpha,1-\alpha), B=max⁡(α,1−α)B=\max(\sqrt\alpha,\sqrt{1-\alpha})), solving the constraint τA=(1−τ)33λ8γB\tau A=(1-\tau)\frac{3\sqrt{3\lambda}}{8\gamma}B under γ=1\gamma=1 yields:

λ(τ)=13(8A3Bτ1−τ)2.\lambda(\tau)=\frac13\left(\frac{8A}{3B}\frac{\tau}{1-\tau}\right)^2.

Accordingly, Eq. (6) is re-verified: M=2τAM=2\tau A, μ=τmin⁡(α,1−α)\mu=\tau\min(\alpha,1-\alpha), and K=τA(1+433)K=\tau A\left(1+\frac{4}{3\sqrt3}\right) (see Section 4.1, Eqs. (5)–(6); Appendix A.1–A.2).

(3) Gradient evaluation point and algorithm-theory consistency. We resolve the notational inconsistencies between qtq_t and q^t\hat q_t by explicitly computing the current gradient from the mapped residual at the actual threshold qtq_t:

zt′=ϕt(st−qt),gt=−Lε-HQ′(zt′).z'_ t = \phi_t(s_t - q_t), \quad g_t= -L'_ {\varepsilon\text{-HQ}}(z'_ t).

(3a) Unified compound update. The base calibration yields q^t+1=ΠQ(qt−ηgt)\hat q_{t+1}=\Pi_Q(q_t-\eta g_t), and the momentum calibration yields qt+1=ΠQ(q^t+1−η′νt+1)q_{t+1}=\Pi_Q(\hat q_{t+1}-\eta'\nu_{t+1}), combining into Eq. (15):

qt+1=ΠQ(ΠQ(qt−ηgt)−η′νt+1).q_{t+1}=\Pi_Q\bigl(\Pi_Q(q_t-\eta g_t)-\eta'\nu_{t+1}\bigr).

Here, q^t+1\hat q_{t+1} is an intermediate variable; the next prediction uses qt+1q_{t+1}. To bridge the practical update and the theoretical analysis, Appendix B.1 defines the scaled loss ℓ~t(q)=1atsLε-HQ(ϕt(st−q))\tilde\ell_t(q)=\frac{1}{a_t^s}L_{\varepsilon\text{-HQ}}(\phi_t(s_t-q)). Under the positive-slope affine mapping condition, −Lε-HQ′(zt′)=gt-L'_{\varepsilon\text{-HQ}}(z'_t)=g_t, aligning the practical update with the scaled loss gradient; boundary impacts of the fallback mapping are conservatively handled in B.4 (see Section 4.3, Eq. (15); Appendix B.1; Appendix C.1, Algorithm 1).

(3b) Reconstruction of the joint regret proof. We fully rewrite Appendix B.3 to resolve the inadequate handling of base-momentum interdependence. Rather than treating the intermediate threshold as an independent OGD sequence, we directly establish a one-step projection inequality from Eq. (15). Utilizing the EMA-centered momentum νt+1=cσ(b^t+1−b^t)\nu_{t+1}=c_\sigma(\hat b_{t+1}-\hat b_t) (with cσ=1−σ2σc_\sigma=\frac{1-\sigma}{2\sigma}) and Abel summation, the cross term is bounded by:

∣∑t=1Tνt+1(qt−qtopt)∣≤cσ[2R+T(ηM+η′)+VT].\left|\sum_{t=1}^{T}\nu_{t+1}(q_t-q_t^{opt})\right|\le c_\sigma\bigl[2R+T(\eta M+\eta')+V_T\bigr].

Combining GWPC, the gradient lower bound, and projection non-expansiveness, Theorem 5.2 yields a bound of O(T(1+VT))\mathcal{O}(\sqrt{T(1+V_T)}), where r=η′/ηr=\eta'/\eta, Ar=(M+r)2+2cσr(M+r)A_r=(M+r)^2+2c_\sigma r(M+r), and Cjoint=K2(M+ArM+2cσr)C_{\mathrm{joint}}=\frac K2\left(M+\frac{A_r}{M}+2c_\sigma r\right). This proves that EMA centering effectively controls the cumulative impact of the second calibration (see Section 5.1, Theorem 5.2; Appendix B.3).

Response 2: Incremental value of the full dual calibration (Weakness 2). We agree that the second momentum calibration does not stably outperform the momentum-free version universally. To quantify its contribution accurately, we update the experiments and introduce dynamic evaluations on distribution drift.

(1) Financial time-series experiments. Across 12 combinations, ε\varepsilon-HQ-DCOGD achieves a lower MIS than ε\varepsilon-HQ-OGD in 7 combinations (LightGBM 1/4, Theta 3/4, Prophet 3/4). This improves upon the original 4/12, though benefits remain data- and predictor-dependent (Section 6.1, Table 1; Appendix D.5, Tables 7–8).

(2) Distribution drift experiments. In the newly introduced scenarios, ε\varepsilon-HQ-DCOGD outperforms ε\varepsilon-HQ-OGD in MIS under Changepoint (1.203 vs 1.236) and Extreme Drift (1.208 vs 1.214), while remaining comparable under Smooth Drift (both 1.073) (Section 6.3, Table 3; Appendix D.11, Table 19).

(3) Transient recovery experiments. Under Changepoint, the average recovery times for ε\varepsilon-HQ-DCOGD versus ε\varepsilon-HQ-OGD are t1:59.4t_1: 59.4 vs 68.468.4, and t2:34.8t_2: 34.8 vs 39.639.6—reductions of 9.0 and 4.8 steps, respectively. Figure 2 illustrates the rolling coverage error, and Table 19 provides interval widths and MIS (see Appendix D.11, Tables 19–20, Figure 2).

(4) Medical image experiments. On DERM12345 and ResNet-50, ε\varepsilon-HQ-DCOGD achieves a Coverage of 87.551%, PI-Size of 1.981, and MDSC of 0.673, compared to 89.238%, 2.089, and 0.671 for ε\varepsilon-HQ-OGD. While achieving smaller prediction sets and slightly higher MDSC, the coverage falls below the nominal level, clarifying a differentiated empirical trade-off between coverage accuracy and prediction efficiency (Section 6.2, Table 2; Appendix D.6, Tables 9–10).

Validated across 4 financial, 2 medical, and 4 synthetic datasets, the empirical evidence supports the transient adaptation value of the second calibration in abrupt-shift settings, while demonstrating that its incremental benefits are not universally established.

Responses to Specific Questions

Question 1: We confirm the omission of the 1/31/3 factor. The analytical solution has been recalculated, and the constants M,μ,KM, \mu, K re-verified. See Response 1(2).

Question 2: Guaranteeing vanishing normalized dynamic regret requires VT=o(T)V_T=o(T), which stationarity alone cannot ensure. Corollary 5.4 now retains the long-term regret term R~∞\widetilde R_\infty to delineate applicable conditions. See Response 1(1), Eq. (19), and Appendix B.5.

Question 3: The gradient in Algorithm 1 is computed at qtq_t; q^t+1\hat q_{t+1} is merely an intermediate base-step variable. The two steps form a unified compound recursion in Eq. (15), with B.2 and B.3 fully aligned. See Response 1(3).

Question 4: Momentum calibration demonstrates a pronounced advantage in abrupt-shift settings. Under Changepoint, ε\varepsilon-HQ-DCOGD achieves a lower MIS (1.203 vs 1.236) and shortens average recovery times by 9.0 and 4.8 steps. See Figure 2 and Table 19, and Response 2.

On the Rating and Confidence. We understand your concerns regarding theoretical guarantees and empirical benefits. We hope these clarifications—spanning corrected constants, joint regret reconstruction, coverage condition delineation, and transient recovery experiments—strengthen your confidence in our work.

Thank you for your detailed response and revisions. Several of my concerns have been addressed. However, I remain concerned about whether the theoretical guarantees fully apply to the clipping/fallback cases of the implemented algorithm. The empirical benefits of momentum calibration also remain inconsistent across settings. Therefore, I will maintain my original rating.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.