acceptodds
Under review as a conference paper at ICLR 2027

Preference Alignment Helps Probabilistic Forecasts Only When the Decision-Maker Cannot Adapt

Abstract

Preference alignment has been proposed as a way to make forecasts serve downstream decisions, but it is not known when, or whether, distorting a forecast can actually improve a decision. We settle this question for probabilistic forecasting, and the answer is sharply two-sided. If the decision-maker is adaptive, choosing the action that minimizes expected cost under the reported distribution, then reporting the true conditional distribution is optimal and no aligned forecast can ever do better. For newsvendor-type decisions, which cover the standard motivating examples of safety stock and reserve sizing, this is exact: minimizing expected cost and querying the critical-ratio quantile are the same operation, so calibration already suffices. If instead the decision-maker is non-adaptive, applying a fixed readout it cannot re-tune to the cost structure (most commonly a system that simply consumes the point forecast), then the cost-minimizing report is provably miscalibrated, and alignment has real value. We instantiate the positive regime with Prob-TPO, which aligns full quantile functions with a distributional DPO objective, and confirm both predictions on four benchmarks (energy, retail, transformer temperature, electricity load; 5 seeds, out-of-sample). Under a fixed point-forecast readout, Prob-TPO reduces realized newsvendor cost by 6–43% relative to calibrated quantile regression, closing up to 92% of the gap to a utility-optimal oracle, while an adaptive consumer sees no benefit, exactly as the theory predicts. We further show the auxiliary and calibration weights control how much of this gain is captured, and that alignment silently breaks quantile-label semantics: an aligned “95th percentile” carries 99% coverage, so downstream consumers must re-map labels.

Primary area:
learning on time series and dynamical systems
Keywords:
probabilistic forecasting, quantile regression, preference alignment, direct preference optimization, decision-focused learning, newsvendor, calibration, time series

Reviews

Among:
Shape:
1234this paper, Bottom 9%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.50 over 2 reviews, above 9% of the others.
Reviewer_Qo6pOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies whether modifying probabilistic forecasts can improve downstream decisions. It distinguishes consumers that select actions using the reported distribution and known costs from consumers restricted to a fixed forecast readout. The proposed Prob-TPO method fine-tunes a pretrained quantile transformer using a sigmoid comparison of current and reference utilities, auxiliary pinball loss, and a calibration penalty. Experiments on four datasets report reductions in asymmetric ordering costs, particularly when consumers must use the reported median as their order quantity.

The main shortcoming is the motivation for Prob-TPO’s training objective. The method assumes known asymmetric costs, but its headline evaluation prevents the standard pinball baseline from using those costs to select the appropriate newsvendor quantile. Moreover, the proposed utility is algebraically equivalent to weighted pinball loss at cost-dependent quantile levels. The experiments therefore do not establish whether the gains require the reference-based sigmoid objective or simply result from changing the forecast target to reflect the known costs. The practical necessity of the fixed-readout restriction also needs stronger justification.

Critical strengths
  1. The paper highlights the importance of how forecasts are consumed. Distinguishing cost-aware decision rules from fixed forecast interfaces is useful for understanding when forecast adjustments can improve decisions. The discussion of altered quantile-label semantics is also practically relevant: after alignment, a nominal percentile may no longer have its stated coverage.

  2. The evaluation includes downstream decision costs. The paper acknowledges that reducing its composite training cost does not establish a benefit to a decision-maker and additionally evaluates realized newsvendor costs. Chronological holdouts, five-seed results, and reporting of both decision costs and distributional metrics provide useful evidence about the observed trade-offs.

Critical weaknesses
  1. The practical motivation for the fixed-readout setting is insufficiently established. Prob-TPO uses known asymmetric costs during training, but the headline newsvendor evaluation forces every consumer to order the reported median rather than the cost-optimal critical-ratio quantile. This prevents the standard pinball baseline from using the known costs to select its action and favors a model trained to shift the median toward a cost-sensitive target.

A fixed interface could justify this restriction. However, the paper assumes access to the consumer’s costs and permission to modify the forecasting model while ruling out a cost-aware decision rule. It should demonstrate a concrete deployment where this separation of control is necessary and explain why changing the meaning of the reported quantile is acceptable. Otherwise, the gains primarily demonstrate compensation for an imposed suboptimal decision rule.

  1. The experiments do not isolate the benefit of the reference-based sigmoid objective. The proposed utility is exactly a weighted sum of negative pinball losses at cost-dependent quantile levels. Direct pinball regression at the newsvendor critical ratio already minimizes empirical asymmetric ordering cost. Even if the consumer must use a designated output, that output can be trained directly at the appropriate level.

A convincing evaluation should include:

  • Cost-aware readout: Evaluate each forecast using the critical-ratio quantile, ρ=Cu/(Cu+Co)\rho=C_u/(C_u+C_o), separately from selecting each model’s best-performing label.

  • Direct cost-sensitive training: Train a pinball baseline directly at level ρ\rho and use its prediction as the order quantity, including through the fixed interface if necessary.

The learned utility-optimal oracle is relevant, but the paper does not present it as a matched comparison isolating Eq. (11). Likewise, the explicit-pair ablation compares two surrogate formulations, while removing auxiliary penalties retains Eq. (11). Without matched architecture, initialization, training budget, and auxiliary terms, the reported gains may reflect correcting the target quantiles rather than a benefit of the reference-based comparison.

Direct neural-network training with newsvendor costs is established prior work: Oroojlooyjadid, Snyder, and Takáč, "Applying deep learning to the newsvendor problem," IISE Transactions, 52(4), 444–463 (2020). This work should be discussed and motivates the missing direct-cost baseline, although it does not address the same multi-quantile calibration problem.

Questions for authors
  1. Can you provide a concrete deployment where asymmetric costs are known and the forecasting model can be modified, but the consumer cannot query the critical-ratio quantile? Why is changing the meaning of the reported median acceptable while changing the decision rule is infeasible?

  2. How do the methods compare when each consumer orders the reported critical-ratio quantile, ρ=Cu/(Cu+Co)\rho=C_u/(C_u+C_o)? Please distinguish this from selecting each model’s best-performing quantile label and specify which data are used for that selection.

  3. Does Prob-TPO outperform direct cost-sensitive pinball training under matched architecture, initialization, training budget, and auxiliary penalties? Please include a model trained directly at level ρ\rho, whose prediction can also be supplied through the fixed interface, and explain how this differs from the learned utility-optimal oracle.

  4. How does the proposed method advance direct neural-network training with newsvendor costs? What demonstrated benefit does Eq. (11) provide over directly minimizing the corresponding asymmetric costs?

Reviewer_Z3xpOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies when preference alignment can improve downstream decisions by distorting probabilistic forecasts. The authors show that, for adaptive consumers who choose actions to minimize expected cost under the reported distribution, reporting the true conditional distribution is optimal, and preference alignment cannot provide further improvement. For newsvendor-type decisions, minimizing expected cost is equivalent to querying the critical-ratio quantile. In contrast, for non-adaptive consumers who apply a fixed readout, the authors show that the cost-minimizing report is generally miscalibrated. They also establish a calibration–preference trade-off under asymmetric quantile costs.

The authors propose Prob-TPO (Probabilistic Temporal Preference Optimization), which aligns full quantile functions using a distributional DPO objective with auxiliary pinball loss and calibration regularization. The authors evaluate Prob-TPO on four forecasting benchmarks. They show that Prob-TPO reduces realized newsvendor decision cost by 6.4%–42.5% relative to calibrated quantile regression under a fixed point-forecast readout, while the advantage largely vanishes for adaptive consumers. The authors further show that the auxiliary and calibration weights control the calibration–cost trade-off and that preference alignment breaks quantile-label semantics.

Critical strengths

The authors establish a clear distinction between adaptive and non-adaptive consumers. Propositions 7 and 9 characterize when reporting the true conditional distribution is optimal and when a miscalibrated report can improve decision quality. These results provide useful guidance on when preference alignment should be applied. The authors propose Prob-TPO, which extends preference alignment from point forecasts to full quantile functions. The method combines a distributional DPO objective with non-crossing quantile heads, auxiliary pinball loss, and calibration regularization. This provides a practical framework for managing the calibration–preference trade-off.

Critical weaknesses

My first concern is that the main theoretical results of the paper largely follow from classical decision theory and the newsvendor critical-ratio characterization. In particular, Proposition 7 follows directly from the optimality of minimizing expected cost under the true conditional distribution, while Proposition 9 follows from the standard newsvendor solution. The authors should better clarify the theoretical contributions beyond these established results.

In addition, Theorem 2 explicitly characterizes the utility-optimal quantile forecasts using asymmetric quantile costs. This suggests a simpler approach based on directly training cost-optimal quantiles using pinball loss, without distributional DPO. Although the authors include a utility-optimal oracle, they do not adequately compare Prob-TPO against this simpler approach as a practical baseline. This makes it difficult to assess the necessity and advantages of Prob-TPO. Proposition 7 establishes the optimality of reporting the true conditional distribution, but this does not imply that calibration, as measured by the paper's expected calibration error, is sufficient for optimal decisions. Moreover, the empirical evaluation focuses on newsvendor-type decisions with assumed cost ratios. The adaptive-consumer experiments also show an 18% improvement on Electricity, which weakens the empirical support for the claim that the advantage of preference alignment vanishes for adaptive consumers.

There is a substantial literature studying the relationship between calibration and downstream decision-making that is largely missing from the paper. For example, Sahoo et al. (NeurIPS 2021, Reliable Decisions with Threshold Calibration), Zhao et al. (NeurIPS 2021, Calibrating Predictions to Decisions: A Novel Approach to Multi-Class Calibration), Gopalan et al. (ITCS 2022, Omnipredictors), Kleinberg et al. (COLT 2023, U-Calibration: Forecasting for an Unknown Agent), Roth and Shi (EC 2024, Forecasting for Swap Regret for All Downstream Agents), Qiao and Zhao (COLT 2025, Truthfulness of Decision-Theoretic Calibration Measures), etc. have studied how different notions of calibration and forecast informativeness relate to downstream decision quality. These works are closely related to the central questions of this paper, but the authors do not adequately discuss them or distinguish their contributions from existing results. In particular, Proposition 7 establishes the optimality of reporting the true conditional distribution, which is stronger than the calibration guarantee defined in Definition 4. The authors should clarify this distinction and better position their theoretical results within the existing literature.

Questions for authors

Please see above comments.

Comments on the paper

1 comment
~Reza_Barati2OpenReview ↗

Does forecast alignment remain beneficial under state-dependent fixed policies?

Thank you for this interesting distinction between adaptive and non-adaptive forecast consumers. I work on sequential decision-making and decision-focused evaluation, and I have two questions about how it extends to state-dependent policies.

First, Section 5 evaluates single-period newsvendor decisions, and the multi-horizon formulation in Definition 1 does not appear to account for action-dependent state transitions. Consider multi-period inventory control with carryover, where a fixed point-forecast readout (the τ0 = 0.5 case of Proposition 9) determines an order-up-to level, but the actual order depends on the current inventory position: forecast interpretation is fixed, yet decisions affect future states and cumulative costs. Could a forecast transformation that improves single-period decision loss worsen cumulative performance under this policy? Does Proposition 9 extend to such settings, and under what assumptions does Proposition 7 extend to an optimal sequential decision-maker?

Second, on the proposed quantile-label remapping: for consumers that use quantile labels, would applying a predetermined coverage correction count as adaptation under Proposition 7, or does adaptation require optimization against the downstream objective? How much of the alignment benefit survives such a baseline?

A paired multi-period experiment — aligned vs. unaligned forecasts under the same state-aware fixed policy, with the remapping baseline evaluated separately — could clarify the scope of these results.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.