acceptodds
Under review as a conference paper at ICLR 2027

Decision-Focused Calibration with Adaptive Conformal Predictive Distributions

Abstract

Decision-focused learning can improve downstream decisions, but a predictive distribution trained for a particular task may misrepresent the outcomes it is meant to predict. Conversely, a distribution trained solely for predictive accuracy may miss features that matter most to the decision. We study how to adapt predictive distributions to decisions subject to a conformal calibration constraint. Our key observation is that a nonconformity score separates the remaining modeling problem into two parts: how probability varies across score levels with the context, and which outcomes occur within each level. We propose AdaCF, an adaptive conformal flow that learns both parts, optionally incorporates a downstream decision loss, and uses a rank-based correction to ensure finite-sample marginal validity under exchangeability. We decompose predictive error into the errors of the two learned components and bound regret for bounded decision losses in terms of that error. On synthetic problems and electricity price data, AdaCF improves predictive fidelity and decision quality over standard conformal flows while preserving validity.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Conformal prediction, uncertainty quantification, stochastic optimization, generative modeling, decision-focused learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_FdoaOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes AdaCF, which learns a context-dependent distribution over conformal score levels and a level-dependent distribution within each boundary. A randomized rank correction ensures finite-sample marginal calibration of generated samples relative to conformal prediction sets under exchangeability. The population analysis decomposes predictive KL error, characterizes the approximation cost of standard conformal flow restrictions through mutual information, and bounds decision regret for bounded losses. Synthetic benchmarks and electricity-price and solar-power applications evaluate predictive fidelity, calibration, and downstream decisions

Critical strengths
  1. An informative characterization of structural limitations.

The decomposition into across-level and within-level errors, together with the mutual-information characterization of restricted models, explains when each adaptation is necessary. This provides a principled rationale for the architecture and useful insight into the limitations of standard conformal flows.

  1. Substantial empirical investigation

Controlled ablations separately vary the two dependencies identified by the theory, while experiments across multiple decisions and two real-data applications examine practical usefulness.

Critical weaknesses
  1. The connection between the training objective and the decision guarantees remains incomplete.

The population KL analysis does not directly apply to the empirical sampler supported on finitely many score boundaries. Consequently, the theory does not establish that the implemented learning and calibration procedure controls the predictive error appearing in the regret bound.

  1. The practical advantage over stronger generators is insufficiently established.

The main experiments use no decision-focused loss, and its addition offers no significant improvement on the main Surge benchmark. These results leave the circumstances under which AdaCF provides a compelling advantage over well-fitted predictive generators insufficiently clear.

Questions for authors
  1. What are the corresponding calibration errors for the flexible-generator comparisons in Table 6?
Reviewer_aVyPOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper introduces AdaCF, an extension of conformal flows that learns both the context-dependent distribution of conformal levels and the distribution of outcomes within each level. A rank-based correction ensures marginal validity of generated samples relative to the conformal sets. The authors derive a KL error decomposition and a decision-regret bound, and evaluate the method on synthetic and electricity-related decision problems.

Critical strengths
  • The idea is incremental with respect to Harris (2026), but clean and well motivated: learn the context-dependent law of the conformal levels, then recalibrate it by rank so that finite-sample marginal validity is preserved.
  • The results are encouraging when the base generator is misspecified: with under-fitting or over-fitted bases, AdaCF substantially reduces the regret of Gen-SAA built on the same base.
  • The idea is well grounded theoretically by Corollary 3, which characterizes the irreducible cost of a uniform level law and of a shared base as I(A;X)I(A;X) and I(U;A∣X)I(U;A\mid X), and the controlled ablation confirms this prediction closely.
Critical weaknesses
  • Beyond Corollary 3, the theoretical results rely largely on standard KL and regret identities and are not connected to the performance of the trained, empirically calibrated generator.
  • It is unclear whether the increment over Harris (2026) is substantial enough. Empirically, much of the gain comes from the learned level law: CF-Across performs comparably to AdaCF on the regime-tail DGP and the solar case study, although the level-dependent base provides meaningful improvements on the surge DGP and in controlled ablations. Moreover, the decision-focused term is disabled in all main experiments (γ=0\gamma=0). The learned level law is closely related to estimating the conditional CDF of the nonconformity score, as in the pivotalization approaches of PIT-CP (Laplante, 2026) and Rectified CP (Plassier et al., 2025), neither of which is cited. The methodological distinction from these approaches deserves clarification.
  • (Minor) I found the paper hard to read; the notation and the organization of the subsections do not help. In particular, the roles of AA, A~\tilde A, A′A' and aia_i (population level, generated level, reference level, empirical level), and of μψ\mu_\psi, νψ\nu_\psi and QψQ_\psi, have to be reconstructed from several places.
Questions for authors
  • The empirical gains appear to come almost entirely from the learned level law: CF-Across matches AdaCF on the regime-tail DGP, on the solar case study (Table 4) and nearly on AEMO. Could the authors state precisely which results require the level-dependent base or the decision term?
  • The level A=1−F(S(X,Y)A=1-F(S(X,Y) is the complementary probability integral transform of the nonconformity score, so learning πϕ(⋅∣x)\pi_\phi(\cdot\mid x) is closely related to estimating the conditional CDF of the score. This resembles the pivotalization and rectification approaches of Laplante (2026) and Plassier et al. (2025), which are not cited. Could the authors clarify the relationship between these methods and AdaCF, particularly the rank correction in Eq. (6)? Can AdaCF be interpreted as a conformal flow based on a pivotalized score, and if so, under what conditions would the approximate conditional validity guarantees of these approaches carry over?
Reviewer_5cnVOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how to learn predictive distributions for contextual stochastic optimization that remain consistent with split-conformal prediction sets. It builds on conformal flows, which draw a level from uniform distribution and project a base-generator sample onto the boundary of the conformal set. The key observation is that the nonconformity score splits P(Y|X) into a level law (which score level the outcome falls on, given x) and a within-level law (where on that level set it lies). Standard conformal flows fix the level law to be uniform at every context and share one base distribution across levels, although validity does not require this.

The proposed method, AdaCF, learns both components, optionally adds a downstream decision loss, and applies a randomized rank correction on calibration data so that generated samples have finite-sample marginal coverage of the conformal sets. The theory gives an exact KL decomposition into across- and within-level errors, lower bounds on mutual entropies on the error of standard conformal flows, and a regret bound for bounded losses.

Experiments on two synthetic DGPs and on electricity price-spike insurance (AEMO), plus solar ramps in the appendix, show better fidelity and regret than standard conformal flows and competitive results against generative and decision-focused baselines. The main experiments use no decision loss (γ=0\gamma= 0) for some reason.

Critical strengths

The paper frames decision-focused learning as fitting a predictive distribution under an explicit conformal calibration constraint. This is a natural, practically relevant alternative to the usual ad hoc weighted sum of predictive and decision losses. The proposed model is original: the authors observe that the nonconformity score splits the conditional distribution into a level law and a within-level law, and that standard conformal flows needlessly fix both. Making the level law context-dependent while keeping it uniform on average (via the rank correction) is a simple, clean idea that preserves finite-sample marginal validity.

Critical weaknesses
  • The paper is framed as decision-focused calibration, and eq. (4) minimizes regret subject to a calibration constraint. In practice, all main experiments use gamma set to 0. Both components are fitted by maximum likelihood, and the constraint is imposed afterwards by the rank correction, so the method being evaluated does not use the downstream loss at all. When the decision term is added (Appendix B.6), it brings no significant gain on the main surge DGP or the bimodal DGP, it increases regret at gamma = 10, and it degrades fidelity as γ grows. It seems that the decision-focused framing in the title and abstract is not supported by the main results.

  • With gamma set to 0, AdaCF is a conditional density estimator written in score coordinates (a 20-bin classifier for the level, plus a Gaussian base for the direction), followed by a post-hoc rank recalibration and a projection onto the conformal sets. The same validity proof applies to any conditional generator recalibrated this way. The paper does not compare against this simpler pipeline, for example a flexible generator with level recalibration.

Questions for authors
  1. Could you explain why γ\gamma is set to 00 in all main experiments? With γ=0\gamma = 0 the method doesn't use the downstream loss, so the decision-focused part of the framing rests only on Appendix B.6. What, in your view, does the main evaluation show about decision-focused learning specifically?

  2. Could you compare AdaCF with a flexible conditional density estimator or sampler Q(⋅∣x)Q(\cdot \mid x), such as a conditional normalizing flow, followed by the same post-hoc calibration: rank recalibration of its implied levels and projection onto the conformal boundaries ∂Cα(x)\partial C_\alpha(x)? The validity argument of Proposition 1 seems to apply equally to that pipeline. This comparison would show what the score-coordinate parameterization (πϕ,μψ)(\pi_\phi, \mu_\psi) adds beyond a good conditional density estimate.

  3. Why not frame the paper as a context-adaptive (conditional) extension of conformal flows? That seems to be the core of what you propose: a learned, context-dependent level law πϕ(⋅∣x)\pi_\phi(\cdot \mid x) and a level-dependent base μψ(⋅∣x,α)\mu_\psi(\cdot \mid x, \alpha), with marginal validity kept through the rank correction. The decision alignment fits awkwardly, given that it is unused in the main experiments.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.