- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies whether modifying probabilistic forecasts can improve downstream decisions. It distinguishes consumers that select actions using the reported distribution and known costs from consumers restricted to a fixed forecast readout. The proposed Prob-TPO method fine-tunes a pretrained quantile transformer using a sigmoid comparison of current and reference utilities, auxiliary pinball loss, and a calibration penalty. Experiments on four datasets report reductions in asymmetric ordering costs, particularly when consumers must use the reported median as their order quantity.
The main shortcoming is the motivation for Prob-TPO’s training objective. The method assumes known asymmetric costs, but its headline evaluation prevents the standard pinball baseline from using those costs to select the appropriate newsvendor quantile. Moreover, the proposed utility is algebraically equivalent to weighted pinball loss at cost-dependent quantile levels. The experiments therefore do not establish whether the gains require the reference-based sigmoid objective or simply result from changing the forecast target to reflect the known costs. The practical necessity of the fixed-readout restriction also needs stronger justification.
Critical strengths
-
The paper highlights the importance of how forecasts are consumed. Distinguishing cost-aware decision rules from fixed forecast interfaces is useful for understanding when forecast adjustments can improve decisions. The discussion of altered quantile-label semantics is also practically relevant: after alignment, a nominal percentile may no longer have its stated coverage.
-
The evaluation includes downstream decision costs. The paper acknowledges that reducing its composite training cost does not establish a benefit to a decision-maker and additionally evaluates realized newsvendor costs. Chronological holdouts, five-seed results, and reporting of both decision costs and distributional metrics provide useful evidence about the observed trade-offs.
Critical weaknesses
- The practical motivation for the fixed-readout setting is insufficiently established. Prob-TPO uses known asymmetric costs during training, but the headline newsvendor evaluation forces every consumer to order the reported median rather than the cost-optimal critical-ratio quantile. This prevents the standard pinball baseline from using the known costs to select its action and favors a model trained to shift the median toward a cost-sensitive target.
A fixed interface could justify this restriction. However, the paper assumes access to the consumer’s costs and permission to modify the forecasting model while ruling out a cost-aware decision rule. It should demonstrate a concrete deployment where this separation of control is necessary and explain why changing the meaning of the reported quantile is acceptable. Otherwise, the gains primarily demonstrate compensation for an imposed suboptimal decision rule.
- The experiments do not isolate the benefit of the reference-based sigmoid objective. The proposed utility is exactly a weighted sum of negative pinball losses at cost-dependent quantile levels. Direct pinball regression at the newsvendor critical ratio already minimizes empirical asymmetric ordering cost. Even if the consumer must use a designated output, that output can be trained directly at the appropriate level.
A convincing evaluation should include:
-
Cost-aware readout: Evaluate each forecast using the critical-ratio quantile, , separately from selecting each model’s best-performing label.
-
Direct cost-sensitive training: Train a pinball baseline directly at level and use its prediction as the order quantity, including through the fixed interface if necessary.
The learned utility-optimal oracle is relevant, but the paper does not present it as a matched comparison isolating Eq. (11). Likewise, the explicit-pair ablation compares two surrogate formulations, while removing auxiliary penalties retains Eq. (11). Without matched architecture, initialization, training budget, and auxiliary terms, the reported gains may reflect correcting the target quantiles rather than a benefit of the reference-based comparison.
Direct neural-network training with newsvendor costs is established prior work: Oroojlooyjadid, Snyder, and Takáč, "Applying deep learning to the newsvendor problem," IISE Transactions, 52(4), 444–463 (2020). This work should be discussed and motivates the missing direct-cost baseline, although it does not address the same multi-quantile calibration problem.
Questions for authors
-
Can you provide a concrete deployment where asymmetric costs are known and the forecasting model can be modified, but the consumer cannot query the critical-ratio quantile? Why is changing the meaning of the reported median acceptable while changing the decision rule is infeasible?
-
How do the methods compare when each consumer orders the reported critical-ratio quantile, ? Please distinguish this from selecting each model’s best-performing quantile label and specify which data are used for that selection.
-
Does Prob-TPO outperform direct cost-sensitive pinball training under matched architecture, initialization, training budget, and auxiliary penalties? Please include a model trained directly at level , whose prediction can also be supplied through the fixed interface, and explain how this differs from the learned utility-optimal oracle.
-
How does the proposed method advance direct neural-network training with newsvendor costs? What demonstrated benefit does Eq. (11) provide over directly minimizing the corresponding asymmetric costs?
Does forecast alignment remain beneficial under state-dependent fixed policies?
Thank you for this interesting distinction between adaptive and non-adaptive forecast consumers. I work on sequential decision-making and decision-focused evaluation, and I have two questions about how it extends to state-dependent policies.
First, Section 5 evaluates single-period newsvendor decisions, and the multi-horizon formulation in Definition 1 does not appear to account for action-dependent state transitions. Consider multi-period inventory control with carryover, where a fixed point-forecast readout (the τ0 = 0.5 case of Proposition 9) determines an order-up-to level, but the actual order depends on the current inventory position: forecast interpretation is fixed, yet decisions affect future states and cumulative costs. Could a forecast transformation that improves single-period decision loss worsen cumulative performance under this policy? Does Proposition 9 extend to such settings, and under what assumptions does Proposition 7 extend to an optimal sequential decision-maker?
Second, on the proposed quantile-label remapping: for consumers that use quantile labels, would applying a predetermined coverage correction count as adaptation under Proposition 7, or does adaptation require optimization against the downstream objective? How much of the alignment benefit survives such a baseline?
A paired multi-period experiment — aligned vs. unaligned forecasts under the same state-aware fixed policy, with the remapping baseline evaluated separately — could clarify the scope of these results.