- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper asks whether the gains of decision losses come from the loss itself or from the optimizer. It splits a training step into a displacement set by the loss and a transport set by the parameter update, and it proposes RIFT, which fits the regression targets with a damped Gauss-Newton step. On five tasks, the authors claim that RIFT is not significantly worse than any baseline while training much faster than the strongest decision loss.
Critical strengths
- Asking whether the performance of decision losses actually comes from the loss is a meaningful question.
- The statistical analysis and the release of results are careful.
- The proofs are short and correct.
Critical weaknesses
-
The argument in the introduction is not convincing. DFL papers usually compare losses with the same backbone, data, optimizer and training pipeline, tune the learning rate of each method, and select the best model by validation regret. The benchmark of Geng et al. also searches the learning rate on a grid and selects every method by the same validation regret criterion. Since these experiments control the other variables and change only the loss, attributing the difference in results to the loss is the appropriate reading. The paper itself states that, under the same optimizer, a decision loss keeps the transport of regression. What can actually be confounded are a few settings, such as the loss-specific batch sizes or Adam. These can be fixed by aligning the settings, and they do not show that the effect of the loss cannot be identified. The displacement and transport decomposition is also a restatement of the chain rule and is hard to view as a new finding.
-
It is unclear why the number of seeds differs across tasks while standard deviations are reported. The first three tasks use five seeds and matching and portfolio use three, with no explanation. With three seeds, even large differences rarely become significant, so 10 of 12 methods on matching and 6 of 11 on portfolio fall into the leading group. As a result, claims that combine the five tasks are effectively decided by three of them.
-
Most papers report mean results, but this paper argues that means cannot separate the methods and summarizes each task by its leading group instead. By mean regret, RIFT ranks 5th, 1st, 4th, tied 3rd and 4th, so it has no advantage in means. Calling RIFT the only method in the leading group of every task, based on "not significantly worse" under low statistical power, reads as choosing a summary that favors the proposed method.
-
It is unclear why the baselines of the main table and of the appendix are organized separately. The eight methods of Table 10 were run only on generated knapsack because it is the fastest task, and they appear only in the appendix. However, energy prices knapsack is also cheap, with a few hundred to a few thousand seconds per baseline run, and the paper runs 20 RIFT variants with five seeds each even on energy scheduling, the most expensive task. In particular, PEAR, the only method in Table 10 with no significant difference from RIFT, is missing from both the main table and the timing comparison. The main table is not run on all tasks either. PFY with kNN targets, the strongest baseline, is missing on matching and portfolio. The omission on portfolio is explained by the form of its objective, but no reason is given for matching. The number of compared methods therefore differs across tasks, with 22 on generated knapsack, 14 on energy prices, 12 on energy scheduling, 11 on matching and 10 on portfolio, and the claim that RIFT is not significantly worse than any of 22 methods holds only on generated knapsack. Seven of the 14 methods that RIFT significantly outperforms on generated knapsack also appear only in the appendix.
-
The claim that RIFT is fast is made without comparing to methods that target training efficiency. RIFT is fast because it calls no solver during training, yet the timing comparison includes only methods that call a solver at every step. A comparison with recent methods that claim time efficiency, such as CaVE, PEAR and LAVA, would be more convincing.
-
The regret and time trade-off does not clearly show that RIFT is more efficient. On energy scheduling, pairwise LTR trains faster and has a lower mean regret, and on portfolio, MSE trains three times faster without a significant difference in regret. In addition, the training times include validation, and RIFT validates at most 30 times while the baselines validate up to 300 times, so MSE, which calls no solver, is slower than RIFT on three tasks. The time difference from MSE in particular seems to come from the validation frequency rather than the method. The training time of the gradient transport of Section 5.4, which uses the same schedule as RIFT, is not reported either.
Questions for authors
-
Why do some tasks use only three seeds? Please use the same number of seeds and also report mean results and mean ranks.
-
What criterion determines which baselines appear in the main table and which appear in the appendix? Why were the methods of Table 10 not run on energy prices, and why was PFY with kNN targets not run on matching?
-
How does RIFT compare with other baseline methods, including those that target training efficiency, in both regret and training time?
-
What are the training times under a matched validation schedule, and what is the training time of the gradient transport of Section 5.4?
3 comments
Response to Reviewer wZC8 - 2
W3 and Q1. Means and ranks. We report mean regret and mean rank next to the paired tests and withdraw the statement that RIFT is the only method in every leading group (Section 5.2, Appendix C.1).
- The 10 methods that run on all five tasks give RIFT a mean rank of 2.6, against 3.0 for SPO+ and listwise LTR and 4.8 for MSE (Table 9), and among all methods RIFT has the 5th, 1st, 4th, 6th and 2nd lowest mean on the five tasks.
- PFY with kNN targets has the lowest mean on generated knapsack, energy scheduling and matching, -0.38, -0.14 and -2.02 points from RIFT, and PFY is third on matching (-1.44), with no difference significant (Table 10).
- After tuning, PFY with kNN targets is 0.48 points below RIFT on generated knapsack ([-1.09, +0.13], Table 23), and the revision states that under the settings of the benchmark no compared method has a significantly lower regret than RIFT on the five tasks, and that after tuning RIFT at its default damping lies significantly above NCE on matching only (2.60 points, Holm p 0.021), a difference that a tuned damping removes (0.49 points, Holm p = 1.00).
W4 and Q2. Organization of baselines. Table 1 now holds the benchmark's methods, PFY, the kNN targets, CaVE and PEAR on every task where they run, Table 11 the six further PyEPO losses, and Appendix B.2 marks each empty cell as undefined or not evaluated.
- The six further PyEPO losses now also run on energy prices, all with a higher mean regret than RIFT (7.60%), 5 of them significant in the uncorrected test and none after the Holm correction (Table 11):
| CMAP | PG | AI-MLE | I-MLE | DPO | NID |
|---|---|---|---|---|---|
| 9.84 | 15.42 | 28.56 | 29.24 | 29.62 | 30.44 |
- PFY and the kNN targets now run on matching, where none of them differs significantly from RIFT (Table 1).
- The comparison holds 23 methods on each knapsack task, 13 on energy scheduling, 15 on matching and 11 on portfolio.
W5 and Q3. Methods that target training efficiency. CaVE and PEAR now run on both knapsack tasks, where RIFT has a lower mean regret than both and a median time no longer than either (Table 1, Table 12):
| Method | Gen. knapsack, % | Energy prices, % | Gen. knapsack, s | Energy prices, s |
|---|---|---|---|---|
| RIFT | 5.98 | 7.60 | 20 | 30 |
| CaVE | 15.50 | 15.96 | 70 | 60 |
| PEAR | 6.12 | 9.09 | 55 (four seeds) | 30 |
- CaVE and PEAR make 8 training solves per run (Table 13), and their mean regret is higher than that of RIFT, significantly so in three of the four comparisons in the paired t test and in two after the Holm correction (Table 10).
- LAVA (Berden et al., 2025) is cited in Section 6 among the losses that need no solver during training, and we did not run it in the rebuttal period.
W6 and Q4. Time and validation. On the knapsack tasks RIFT's time advantage over MSE comes only from validating less often, and its advantage over the solver-based decision losses comes from an update that solves no training instance and remains when no method validates (Section 5.2, Appendix C.2). Median run time and training solves on one machine type, five seeds:
| Method | Gen. knapsack, s | Energy prices, s | Gen. knapsack, solves | Energy prices, solves |
|---|---|---|---|---|
| RIFT | 20 | 30 | 0 | 0 |
| MSE | 40 | 60 | 0 | 0 |
| SPO+ | 140 | 420 | 61,440 | 150,822 |
| PFY, kNN targets | 750 | 1,120 | 416,000 | 471,870 |
- The submitted paper gave RIFT 45, 139 and 4,866 s on the two knapsack tasks and energy scheduling, measured on our original machine with the diagnostic solves of Appendix B.3, and on the machine of the table RIFT takes 40, 70 and 3,010 s with these solves and 20, 30 and 320 s without them, at the same validation and test regret (Appendix C.2).
- On energy scheduling a RIFT run takes 320 s against estimates from ten-epoch runs of 3,091 s for MSE, 4,292 s for pairwise LTR, whose mean regret is 1.86% against 1.89% without a significant difference, and 43,899 s for PFY with kNN targets (Table 12).
- Without validation for any method, timed on a second machine type, a run on the two knapsack tasks takes 20 and 30 s for RIFT, 20 and 20 s for MSE, 480 and 760 s for SPO+ and 2,080 and 3,770 s for PFY, so RIFT is not faster than MSE there, while SPO+ and PFY take 24 to 126 times as long as RIFT (Table 14).
- On that machine and with validation and diagnostic solves, gradient steps with step size 0.01 and 10 passes per stage take 60 and 60 s on the knapsack tasks and 2,550 s on one energy-scheduling seed, against 60, 100 and 5,770 s for the Gauss-Newton step (Table 14), so a Gauss-Newton run is not faster, and what it saves is the search over the seven step settings that selects the gradient transport, 3 of which diverge on both energy tasks (Appendix C.5).
- Matching and portfolio are not timed on one machine type, and in the submitted runs most baselines train faster than RIFT on matching (BCE 139 s and SPO+ 301 s against 824 s).
The revision does not address my main concern and makes it clearer. When the objective is affine in the cost, the optimal decision depends only on the conditional mean, so it is already known that in easy settings, where the predictor can approximate the conditional mean well, training regression well is enough to improve regret. That RIFT is statistically indistinguishable from tuned regression on the benchmark tasks (Table 23) is simply consistent with this known result. The central problem of decision-focused learning (DFL) is to reduce regret in settings where the predictor cannot represent the nonlinearity of the cost well enough to predict it accurately, and DFL methods have been developed to address this problem. However, it is in exactly these settings that RIFT is significantly worse than decision losses in most cells (Section 5.6). The paper therefore shows only that a well-trained regression performs similarly to decision losses in easy settings, and it shows no advantage on the problem that DFL aims to solve. Had RIFT performed on par with decision losses at every degree, the claim of the paper would have been convincing.
The budget experiment in Table 2, which supports the claim that regression baselines are undertrained, is also not a fair comparison. Because each method has a different loss landscape, most work in this area tunes the learning rate and other settings under controlled conditions and compares each method at the setting where it performs best. The number of steps is likewise a hyperparameter whose best value differs across methods, and the same number of steps can be too large for other methods. Table 2, however, extends the number of steps only for MSE and SPO+, without early stopping, and does not extend it for the other decision losses. A fair comparison under the same rules would extend the budget for every method and select each one by validation.
The remaining advantage that RIFT claims is fast training, because its parameter update calls no solver. The relevant comparison for this claim is other solver-free methods, such as LAVA (Berden et al., 2025) and WISE (Wan & Liu, 2026), which the paper cites in Section 6 but does not evaluate. These are essential baselines and are still missing from the experiments.
It is unclear to me what the paper intends to conclude about DFL, and I find it very difficult to see this work being accepted at ICLR as a DFL paper.
I keep my scores.
Response to Reviewer wZC8 - 1
We thank the reviewer for the detailed review. In the revised PDF, new and changed text outside figures is in blue.
W1. What the benchmark aligns. The benchmark aligns the optimizer, the learning rate and the step budget, and at its learning rate of 0.01 regression still improves after 300 steps on both knapsack tasks, so with a longer budget under identical settings the advantage of SPO+ is no longer significant on either knapsack task (Section 5.4, Table 2). Regret of SPO+ minus that of MSE without early stopping, five seeds, in points:
W1. Tuning the learning rate. With the grid of Geng et al. (0.1 to 0.001, patience 50), no tuned decision loss has a significantly lower regret than tuned MSE on generated knapsack, energy prices and portfolio after the Holm correction (Appendix C.7, Tables 22 and 23).
W1. The split is the chain rule. The split of a step into displacement and transport is a first-order expansion whose kernel is the empirical neural tangent kernel (Section 6), and it serves to design the exchanges.
W2 and Q1. Seeds. Matching and portfolio had three seeds because we ran them as secondary tasks, and every method now has five seeds there except pairwise LTR on portfolio, whose fifth run reached the time limit, with no method significantly below RIFT on either task (Table 1, Appendix C.1).