acceptodds
Under review as a conference paper at ICLR 2027

The Loss Is Only Half of the Step: Rethinking Decision Losses in Decision-Focused Learning

Abstract

Decision-focused learning replaces the squared error of a cost predictor with a decision loss and treats the choice of this loss as the main design decision. We find that the squared error already moves each prediction in a safe direction. Moving a prediction toward its realized cost never increases the regret of that sample when the objective is affine in the cost, and every task in our comparison has such an objective. Regret can rise only through the parameter update, which moves all predictions together and can carry a prediction off its path to the realized cost. Accordingly, we split a training step into the displacement that the loss assigns to each prediction and the transport by which one parameter update moves all predictions. Decision losses change only the displacement, so they correct a direction that is already safe. We introduce RIFT (regression in function space with trust regions), which fixes the transport as a damped Gauss-Newton step and accepts any displacement. RIFT with the regression displacement is not significantly worse than any of 22 methods on five tasks of PredictiveCO-Benchmark, and it trains 7 to 27 times faster than the strongest decision loss. Replacing the regression targets by the rescaled direction of the SPO+ surrogate or of a perturbed Fenchel-Young loss raises regret by 1.11 to 3.04 points on two knapsack tasks. Replacing the transport by gradient steps with a validated step size changes regret by at most 0.07 points, and both transports reach a regret 1.49 points below the regression baseline of the benchmark on energy prices. On these tasks, regression gains from a better transport and not from a decision loss.

Primary area:
optimization
Keywords:
decision-focused learning, predictive combinational optimization, predict-and-optimize, surrogate losses, neural tangent kernel

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_wZC8OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper asks whether the gains of decision losses come from the loss itself or from the optimizer. It splits a training step into a displacement set by the loss and a transport set by the parameter update, and it proposes RIFT, which fits the regression targets with a damped Gauss-Newton step. On five tasks, the authors claim that RIFT is not significantly worse than any baseline while training much faster than the strongest decision loss.

Critical strengths
  1. Asking whether the performance of decision losses actually comes from the loss is a meaningful question.
  2. The statistical analysis and the release of results are careful.
  3. The proofs are short and correct.
Critical weaknesses
  1. The argument in the introduction is not convincing. DFL papers usually compare losses with the same backbone, data, optimizer and training pipeline, tune the learning rate of each method, and select the best model by validation regret. The benchmark of Geng et al. also searches the learning rate on a grid and selects every method by the same validation regret criterion. Since these experiments control the other variables and change only the loss, attributing the difference in results to the loss is the appropriate reading. The paper itself states that, under the same optimizer, a decision loss keeps the transport of regression. What can actually be confounded are a few settings, such as the loss-specific batch sizes or Adam. These can be fixed by aligning the settings, and they do not show that the effect of the loss cannot be identified. The displacement and transport decomposition is also a restatement of the chain rule and is hard to view as a new finding.

  2. It is unclear why the number of seeds differs across tasks while standard deviations are reported. The first three tasks use five seeds and matching and portfolio use three, with no explanation. With three seeds, even large differences rarely become significant, so 10 of 12 methods on matching and 6 of 11 on portfolio fall into the leading group. As a result, claims that combine the five tasks are effectively decided by three of them.

  3. Most papers report mean results, but this paper argues that means cannot separate the methods and summarizes each task by its leading group instead. By mean regret, RIFT ranks 5th, 1st, 4th, tied 3rd and 4th, so it has no advantage in means. Calling RIFT the only method in the leading group of every task, based on "not significantly worse" under low statistical power, reads as choosing a summary that favors the proposed method.

  4. It is unclear why the baselines of the main table and of the appendix are organized separately. The eight methods of Table 10 were run only on generated knapsack because it is the fastest task, and they appear only in the appendix. However, energy prices knapsack is also cheap, with a few hundred to a few thousand seconds per baseline run, and the paper runs 20 RIFT variants with five seeds each even on energy scheduling, the most expensive task. In particular, PEAR, the only method in Table 10 with no significant difference from RIFT, is missing from both the main table and the timing comparison. The main table is not run on all tasks either. PFY with kNN targets, the strongest baseline, is missing on matching and portfolio. The omission on portfolio is explained by the form of its objective, but no reason is given for matching. The number of compared methods therefore differs across tasks, with 22 on generated knapsack, 14 on energy prices, 12 on energy scheduling, 11 on matching and 10 on portfolio, and the claim that RIFT is not significantly worse than any of 22 methods holds only on generated knapsack. Seven of the 14 methods that RIFT significantly outperforms on generated knapsack also appear only in the appendix.

  5. The claim that RIFT is fast is made without comparing to methods that target training efficiency. RIFT is fast because it calls no solver during training, yet the timing comparison includes only methods that call a solver at every step. A comparison with recent methods that claim time efficiency, such as CaVE, PEAR and LAVA, would be more convincing.

  6. The regret and time trade-off does not clearly show that RIFT is more efficient. On energy scheduling, pairwise LTR trains faster and has a lower mean regret, and on portfolio, MSE trains three times faster without a significant difference in regret. In addition, the training times include validation, and RIFT validates at most 30 times while the baselines validate up to 300 times, so MSE, which calls no solver, is slower than RIFT on three tasks. The time difference from MSE in particular seems to come from the validation frequency rather than the method. The training time of the gradient transport of Section 5.4, which uses the same schedule as RIFT, is not reported either.

Questions for authors
  1. Why do some tasks use only three seeds? Please use the same number of seeds and also report mean results and mean ranks.

  2. What criterion determines which baselines appear in the main table and which appear in the appendix? Why were the methods of Table 10 not run on energy prices, and why was PFY with kNN targets not run on matching?

  3. How does RIFT compare with other baseline methods, including those that target training efficiency, in both regret and training time?

  4. What are the training times under a matched validation schedule, and what is the training time of the gradient transport of Section 5.4?

3 comments
AuthorsOpenReview ↗

Response to Reviewer wZC8 - 1

We thank the reviewer for the detailed review. In the revised PDF, new and changed text outside figures is in blue.

W1. What the benchmark aligns. The benchmark aligns the optimizer, the learning rate and the step budget, and at its learning rate of 0.01 regression still improves after 300 steps on both knapsack tasks, so with a longer budget under identical settings the advantage of SPO+ is no longer significant on either knapsack task (Section 5.4, Table 2). Regret of SPO+ minus that of MSE without early stopping, five seeds, in points:

Task, learning rate101003001,000 steps
Generated knapsack, 0.01-6.93-0.86-0.14+0.72
Generated knapsack, 0.05-8.83+0.39+1.29+1.59
Energy prices, 0.01-6.35-1.90-0.90-0.38
Energy prices, 0.05-7.88-0.71-0.33+0.26
  • In the benchmark loop every method takes one full-batch Adam step per epoch, and on energy prices MSE improves from 8.73% after 300 steps to 8.01% after 1,000 at learning rate 0.01 and reaches 7.67% at 0.05, against 7.60% for RIFT, so the longer budget closes most of the gap of 1.49 points between RIFT and MSE in Table 1 (Table 21), and tuned MSE with validation stays 0.85 points above RIFT (Holm p 0.078, Table 23).
  • On energy scheduling the difference shrinks from -7.63 and -4.86 points after 10 steps to -0.10 and -0.16 after 300, and SPO+ stays ahead at learning rate 0.05 in the paired t test (p = 0.015), while no budget separates the two losses on matching.
  • After the Holm correction over the ten combinations of task and learning rate, no combination differs significantly at the largest budget, and the smallest corrected p value, 0.09, belongs to generated knapsack at learning rate 0.05, where MSE is lower (Appendix C.6).
  • The introduction now states that a benchmark comparison trains every loss with the same optimizer and step budget, and Section 3.2 that a decision loss trained with Adam also changes the transport through Adam's preconditioner.

W1. Tuning the learning rate. With the grid of Geng et al. (0.1 to 0.001, patience 50), no tuned decision loss has a significantly lower regret than tuned MSE on generated knapsack, energy prices and portfolio after the Holm correction (Appendix C.7, Tables 22 and 23).

  • Tuned MSE minus each tuned decision loss lies between +0.08 and +0.46 points on generated knapsack and between +0.30 and +0.77 on energy prices for all losses but pairwise LTR, with every corrected p value at least 0.068, and without the correction PFY with kNN targets is significantly below tuned MSE on energy prices (+0.68 points, p = 0.017).
  • PFY with kNN targets is not tuned on energy scheduling, where one epoch takes 674 s, and its release setting has the lowest mean there (1.74% against 1.89% for RIFT, not significant).
  • Tuned SPO+ stays 0.15 points below tuned MSE on energy scheduling and tuned NCE 2.03 points below tuned BCE on matching (Holm p 0.027 and 0.015), and with a tuned damping RIFT has no significantly higher regret than any tuned method, NCE on matching included (0.49 points, Holm p = 1.00, Appendix C.7).
  • Table 22 gives RIFT at its tuned and default damping on all five tasks.

W1. The split is the chain rule. The split of a step into displacement and transport is a first-order expansion whose kernel is the empirical neural tangent kernel (Section 6), and it serves to design the exchanges.

  • The inequality behind Proposition 1 is the classical monotonicity lemma of penalty methods (Luenberger and Ye, 2016, Section 13.1), also used by Vlastelica Pogančić et al. (2020) and Sahoo et al. (2023) and now cited (Section 3.1, Appendix A.2), and the paper applies it to the segment from a prediction to its realized cost, where the regret is nonincreasing for every tie-breaking rule, so under regression a rise in regret can come only from the transport.
  • With the Gauss-Newton transport fixed, the SPO+ and PFY directions raise regret by 1.11 to 3.04 points on both knapsack tasks, all four significant, and change it by no significant amount on energy scheduling, matching and portfolio, with intervals of about ±84 and ±130 points on portfolio (Section 5.3, Figure 3a, Table 18), while validated gradient steps in place of the Gauss-Newton step change it significantly on none of the five (Appendix C.5).

W2 and Q1. Seeds. Matching and portfolio had three seeds because we ran them as secondary tasks, and every method now has five seeds there except pairwise LTR on portfolio, whose fifth run reached the time limit, with no method significantly below RIFT on either task (Table 1, Appendix C.1).

  • On portfolio RIFT has the second-lowest mean of 11 methods, 6.30 points above SPO+ without significance, and 5 of the other 10 methods have a significantly higher regret after Holm.
  • With three seeds the leading groups held 10 of 12 and 6 of 11 methods, and the revision no longer summarizes the tasks by leading groups.
AuthorsOpenReview ↗

Response to Reviewer wZC8 - 2

W3 and Q1. Means and ranks. We report mean regret and mean rank next to the paired tests and withdraw the statement that RIFT is the only method in every leading group (Section 5.2, Appendix C.1).

  • The 10 methods that run on all five tasks give RIFT a mean rank of 2.6, against 3.0 for SPO+ and listwise LTR and 4.8 for MSE (Table 9), and among all methods RIFT has the 5th, 1st, 4th, 6th and 2nd lowest mean on the five tasks.
  • PFY with kNN targets has the lowest mean on generated knapsack, energy scheduling and matching, -0.38, -0.14 and -2.02 points from RIFT, and PFY is third on matching (-1.44), with no difference significant (Table 10).
  • After tuning, PFY with kNN targets is 0.48 points below RIFT on generated knapsack ([-1.09, +0.13], Table 23), and the revision states that under the settings of the benchmark no compared method has a significantly lower regret than RIFT on the five tasks, and that after tuning RIFT at its default damping lies significantly above NCE on matching only (2.60 points, Holm p 0.021), a difference that a tuned damping removes (0.49 points, Holm p = 1.00).

W4 and Q2. Organization of baselines. Table 1 now holds the benchmark's methods, PFY, the kNN targets, CaVE and PEAR on every task where they run, Table 11 the six further PyEPO losses, and Appendix B.2 marks each empty cell as undefined or not evaluated.

  • The six further PyEPO losses now also run on energy prices, all with a higher mean regret than RIFT (7.60%), 5 of them significant in the uncorrected test and none after the Holm correction (Table 11):
CMAPPGAI-MLEI-MLEDPONID
9.8415.4228.5629.2429.6230.44
  • PFY and the kNN targets now run on matching, where none of them differs significantly from RIFT (Table 1).
  • The comparison holds 23 methods on each knapsack task, 13 on energy scheduling, 15 on matching and 11 on portfolio.

W5 and Q3. Methods that target training efficiency. CaVE and PEAR now run on both knapsack tasks, where RIFT has a lower mean regret than both and a median time no longer than either (Table 1, Table 12):

MethodGen. knapsack, %Energy prices, %Gen. knapsack, sEnergy prices, s
RIFT5.987.602030
CaVE15.5015.967060
PEAR6.129.0955 (four seeds)30
  • CaVE and PEAR make 8 training solves per run (Table 13), and their mean regret is higher than that of RIFT, significantly so in three of the four comparisons in the paired t test and in two after the Holm correction (Table 10).
  • LAVA (Berden et al., 2025) is cited in Section 6 among the losses that need no solver during training, and we did not run it in the rebuttal period.

W6 and Q4. Time and validation. On the knapsack tasks RIFT's time advantage over MSE comes only from validating less often, and its advantage over the solver-based decision losses comes from an update that solves no training instance and remains when no method validates (Section 5.2, Appendix C.2). Median run time and training solves on one machine type, five seeds:

MethodGen. knapsack, sEnergy prices, sGen. knapsack, solvesEnergy prices, solves
RIFT203000
MSE406000
SPO+14042061,440150,822
PFY, kNN targets7501,120416,000471,870
  • The submitted paper gave RIFT 45, 139 and 4,866 s on the two knapsack tasks and energy scheduling, measured on our original machine with the diagnostic solves of Appendix B.3, and on the machine of the table RIFT takes 40, 70 and 3,010 s with these solves and 20, 30 and 320 s without them, at the same validation and test regret (Appendix C.2).
  • On energy scheduling a RIFT run takes 320 s against estimates from ten-epoch runs of 3,091 s for MSE, 4,292 s for pairwise LTR, whose mean regret is 1.86% against 1.89% without a significant difference, and 43,899 s for PFY with kNN targets (Table 12).
  • Without validation for any method, timed on a second machine type, a run on the two knapsack tasks takes 20 and 30 s for RIFT, 20 and 20 s for MSE, 480 and 760 s for SPO+ and 2,080 and 3,770 s for PFY, so RIFT is not faster than MSE there, while SPO+ and PFY take 24 to 126 times as long as RIFT (Table 14).
  • On that machine and with validation and diagnostic solves, gradient steps with step size 0.01 and 10 passes per stage take 60 and 60 s on the knapsack tasks and 2,550 s on one energy-scheduling seed, against 60, 100 and 5,770 s for the Gauss-Newton step (Table 14), so a Gauss-Newton run is not faster, and what it saves is the search over the seven step settings that selects the gradient transport, 3 of which diverge on both energy tasks (Appendix C.5).
  • Matching and portfolio are not timed on one machine type, and in the submitted runs most baselines train faster than RIFT on matching (BCE 139 s and SPO+ 301 s against 824 s).

The revision does not address my main concern and makes it clearer. When the objective is affine in the cost, the optimal decision depends only on the conditional mean, so it is already known that in easy settings, where the predictor can approximate the conditional mean well, training regression well is enough to improve regret. That RIFT is statistically indistinguishable from tuned regression on the benchmark tasks (Table 23) is simply consistent with this known result. The central problem of decision-focused learning (DFL) is to reduce regret in settings where the predictor cannot represent the nonlinearity of the cost well enough to predict it accurately, and DFL methods have been developed to address this problem. However, it is in exactly these settings that RIFT is significantly worse than decision losses in most cells (Section 5.6). The paper therefore shows only that a well-trained regression performs similarly to decision losses in easy settings, and it shows no advantage on the problem that DFL aims to solve. Had RIFT performed on par with decision losses at every degree, the claim of the paper would have been convincing.

The budget experiment in Table 2, which supports the claim that regression baselines are undertrained, is also not a fair comparison. Because each method has a different loss landscape, most work in this area tunes the learning rate and other settings under controlled conditions and compares each method at the setting where it performs best. The number of steps is likewise a hyperparameter whose best value differs across methods, and the same number of steps can be too large for other methods. Table 2, however, extends the number of steps only for MSE and SPO+, without early stopping, and does not extend it for the other decision losses. A fair comparison under the same rules would extend the budget for every method and select each one by validation.

The remaining advantage that RIFT claims is fast training, because its parameter update calls no solver. The relevant comparison for this claim is other solver-free methods, such as LAVA (Berden et al., 2025) and WISE (Wan & Liu, 2026), which the paper cites in Section 6 but does not evaluate. These are essential baselines and are still missing from the experiments.

It is unclear to me what the paper intends to conclude about DFL, and I find it very difficult to see this work being accepted at ICLR as a DFL paper.

I keep my scores.

Reviewer_Mu5yOpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
2
Soundness
1
Read review
Paper summary

This paper target Decision Focused Learning (DFL) with a "classical" setting, i.e. where a predictor is used to estimate the coefficients of a linear cost function, which then needs to be minimized under constraints to obtain the corresponding decision and its actual cost. DFL is usually contrasted with Prediction Focused Learning, which targets the same setting (estimate, then optimize), but trains the predictor separately, usually using the Mean Squared Error as the loss.

It has been known that, assuming that accurate expectations for the cost coefficients can be learned, then PFL and DFL asymptotically converge to the same solution. The authors build over a strengthened form of this observation, positing that the MSE gradient is "safe" from the point of view of DFL, meaning that it is also a sufficient condition for regret reduction.

The key idea in the paper is then to exploit this fact by trying to ensure that all components of the MSE loss are reduced at each gradient descent step; this not trivial, since a gradient step that reduces the MSE for one example in the mini-bach may increase it for another example, if (e.g.) the step length is sufficiently large. The authors achieve this objective by relying on trust region optimization (specifically, via a dampened Gauss-Newton step).

The main appeal of the method is that it avoids most calls to the optimization solver at training time. Some call are still needed, because the authors select the optimal parameter vector based on the validation regret, rather than on the MSE value.

In the reported empirical evaluation the proposed method achieves results in line with other DFL approach, but with a much smaller reported computational cost.

Critical strengths

The main strengths of this work are in its two main insights:

  • First, the observation that the MSE reduction direction is "safe" from the point of view of regret is new to the best of my knowledge, and stronger than just claiming asymptotic convergence to the same predictions.
  • Second, the idea of regulating the gradient descent step to avoid "unsafe" updates is fresh and potentially interesting, since it could in theory save substantial computational costs

The specific technique employed here (dampener Gauss-Newton steps) is also a reasonable choice and soundly executed, to the best I could check.

Critical weaknesses

Despite providing fresh (though very specific) insights, the work is not devoid of weaknesses.

First, given how the work focuses on the relation between DFL and PFL, it's strange to see some recent work on the topic not mentioned, such as [1] or [2]. Expanding the picture to include such (and similar) related work, the authors' main claim remains interesting, but its practical applicability appears to be restricted by two main factors.

First, the paper does not seem to account for model misalignment: the structure of the ML model itself may constraint gradient descent, in the worst case to the point that no safe step exists (e.g. when the MSE gradient is perpendicular to the feasible prediction space induced by the model structure).

Second, a MSE minimizer in the prediction space match the expected value of the cost coefficient only if the prediction noise is Normally distributed and homoscedastic (same variance everywhere). When these assumptions do not hold, MSE minimization will lead to a biased estimate for the expectation, and therefore to worse regret. Most DFL methods do not need to make distributional assumptions, though they often alter and bias the regret landscape for other reasons (e.g. smoothing).

Both limitations are very relevant in practice, as misaligned models are common and true homoscedastic processes are rare. The issue might be less evident in the considered benchmarks, since they are simulated and often have centered and symmetric noise. It is possible (though I cannot confirm this) that the practice of selecting the model weights with the lowest validation regret is partially mitigating for this issues.

Another, minor, weakness is directly linked to the use of validation regret. Even if the proposed method does not rely on calling the optimization solver to obtain gradients, it still requires calls to compute the validation regret. The situation is inverted for other DFL methods: they need solver calls to obtain gradients, but can operating without a validation set.

This situation can affect how the proposed method compares with DFL in terms of solver calls. Dedicated experiments where no validation set access is assumed on both method would settle the issue (the chosen weight vector would be the one for the last epoch); without changing the current evaluation, even just reporting separately the number of solver calls on the training and validation set would provide some useful information.

[1] Estimate-Then-Optimize Versus Integrated-Estimation-Optimization Versus Sample Average Approximation: A Stochastic Dominance Perspective. AN Elmachtoub, H Lam, H Zhang, Y Zhao

[2] Gupta, Vishal, and Michael Huang. "Decision-focused learning with directional gradients." Advances in Neural Information Processing Systems 37 (2024): 79194-79220.

Questions for authors
  1. Have you investigated how model and distribution misalignment might affect the proposed approach?

  2. Have you tried avoiding the use of a validation set altogether, by just picking the weight vector obtained at the last training epoch? How does that change the results?

I will consider improving my score depending on how my concerns are addressed.

3 comments
AuthorsOpenReview ↗

Response to Reviewer Mu5y - 1

We thank you for singling out the per-sample safety of the regression direction. In the revised PDF, new and changed text outside figures is in blue. We number your four weaknesses W1 to W4 in the order of the review.

W3 and Q1. Noise distribution. For every objective in the paper, least squares estimates E[c|x], the quantity the decision needs, under any noise with finite variance, and with three further noise families at degree 6 the three decision losses keep their lead over tuned MSE (Section 5.6).

  • With F(c, w) = g(w) + c^T phi(w), the expected cost given x depends on the noise only through E[c|x] (Hu et al., 2022).
  • For any predictor f, E||c - f(x)||^2 = E||c - E[c|x]||^2 + E||E[c|x] - f(x)||^2, so in the population least squares over any model class minimizes the distance to E[c|x], whatever the shape or heteroscedasticity of the noise.
  • The benchmark noise is neither Gaussian nor homoscedastic, since the benchmark's generator multiplies each cost of generated knapsack by a uniform factor in [0.5, 1.5] and rounds it up, and both energy tasks use real electricity prices.
  • On synthetic shortest path and knapsack with four noise families of the same conditional mean, the best decision loss lies between -0.67 and +0.36 points of tuned MSE in every cell of degree 1 or 2 (three seeds), and at degree 6 the additive Gaussian, additive heteroscedastic and log-normal families keep SPO+, PFY and PFY with kNN targets below tuned MSE, and RIFT above it in mean (Appendix C.9).

W2 and Q1. Model misalignment. SPO+, PFY and PFY with kNN targets help when a linear predictor meets costs of degree 4 or 6, which the revision states as the boundary of our claim (Section 5.6, Figure 6, Tables 25 and 26).

  • The split of a step into displacement and transport addresses the constraint that the model places on the step, since Proposition 1 holds for each prediction while one parameter update moves all predictions together, so any rise of regret under regression comes from the transport (Section 3.2).
  • On the benchmark tasks, where 852 or 321 parameters move 6,400 to 26,400 predicted costs, the Gauss-Newton step reaches only the orthogonal projection of the displacements onto the range of the Jacobian even as its damping goes to zero (Section 3.3, Proposition 2).
  • In all 8 cells with a linear predictor and costs of degree 4 or 6, each of the three has a lower regret than tuned MSE on every one of five new test seeds, by 0.45 to 4.57 points, and with 32 hidden units the best decision loss leads in every cell of degree 4 to 8 on the three selection seeds, while on the test seeds its lead at degree 4 is significant only on shortest path and only without the Holm correction (0.66 points, p = 0.024).
  • RIFT regresses on the realized costs and shares this boundary, since after the Holm correction over the cells it lies significantly above the best decision loss in 7 of the 8 misspecified cells and with 32 hidden units at degree 4 on shortest path (+0.53 points), and differs from tuned MSE in none of the 14 cells, while without the correction it is above the best decision loss in all 8 and above tuned MSE in one (+1.44 points, p = 0.033).
  • In the synthetic knapsack cell with the degree and width of generated knapsack (degree-4 costs, 32 hidden units) no decision loss is significantly better than tuned MSE, and under the settings of the benchmark no compared method has a significantly lower regret than RIFT on any of the five tasks (Table 1), while after tuning RIFT at its default damping lies significantly above NCE on matching only (2.60 points, Holm p 0.021), a difference that a tuned damping removes (0.49 points, Holm p = 1.00).

W1. Related work. We now cite and discuss [1] and the classical sources of Proposition 1, and [2] is cited in the introduction and Section 6, with its PG losses run on both knapsack tasks and in the synthetic study.

  • Proposition 1 rests on the monotonicity lemma of penalty methods (Luenberger and Ye, 2016, Section 13.1), whose two inequalities Vlastelica Pogančić et al. (2020) and Sahoo et al. (2023) also use for a cost perturbed along the loss gradient, and the paper applies them to the segment toward the realized cost, which makes regression a safe direction for every tie-breaking rule (Section 3.1, Appendix A.2).
  • [1] (Elmachtoub et al., 2025) shows that estimating and then optimizing asymptotically dominates integrated estimation in regret when the model class contains the true model and can be beaten under misspecification, in line with the gains of SPO+, PFY and PFY with kNN targets that our synthetic study finds with a linear predictor and costs of degree 4 or 6.
  • PG reaches 15.37% on generated knapsack and 15.42% on energy prices, against 5.98% and 7.60% for RIFT (Holm p 0.071 and 0.92, Table 11).
AuthorsOpenReview ↗

Response to Reviewer Mu5y - 2

W1. PG of [2] in the synthetic study. We ran the PG loss of [2] in the 14 test cells of the synthetic study with PyEPO's default step (sigma = 0.1, not tuned) and the training and learning-rate selection of the other decision losses, and with these settings PG has a higher mean regret than tuned MSE in every one of them (Section 5.6, Appendix C.9, Table 27).

  • In the 8 misspecified cells PG lies 0.24 to 2.58 points above tuned MSE, a difference that is significant in no cell after the Holm correction over the cells (smallest corrected p 0.057) and in one without it, knapsack at degree 4 (+1.98 points, p = 0.045).
  • PG is the best decision loss in none of the cells, so the comparisons of SPO+, PFY and PFY with kNN targets with tuned MSE and with RIFT stand as stated under W2 and Q1.
  • Gupta and Huang treat the step as a parameter, and these results describe PG with PyEPO's default step on the two generators and 14 cells of this study.

W4. Solver calls in training and validation. RIFT's update calls no solver, and the revision counts training and validation solves separately for every method on the knapsack and scheduling tasks (Appendix C.2, Table 13).

  • The RIFT runs of Tables 12 and 13 leave out the diagnostic solves of Appendix B.3 and reach the same validation and test regret at every stage as the runs with them, on all five seeds of the three tasks.
  • Median solves per run on one machine type, five seeds:
MethodGen. knapsack, trainingGen. knapsack, validationEnergy prices, trainingEnergy prices, validation
RIFT01,84003,330
MSE014,320033,300
SPO+61,44023,040150,82256,943
PFY, kNN targets416,000123,840471,870141,192
  • RIFT validates once per stage, at most 30 times, and MSE once per epoch, up to 300 times.

W4 and Q2. Training without a validation set. Without validation RIFT makes no solver call before the test, and its regret changes by -0.24, +0.11 and -0.02 points on the three tasks (Appendix C.3, Table 15). Mean test regret in percent, five seeds, under the settings of Table 1 and at the last epoch or stage:

MethodGen. knapsack, Table 1Gen. knapsack, lastEnergy prices, Table 1Energy prices, last
RIFT5.985.747.607.71
MSE6.085.849.098.73
SPO+5.885.708.697.83
PFY6.046.888.577.38
PFY, kNN targets5.605.637.937.39
Listwise LTR5.916.058.008.12
Pairwise LTR7.587.588.6913.98
  • On energy scheduling MSE and SPO+ change by -0.21 and -0.07 points, and pairwise LTR loses 1.51 points, significant in the paired t test.
  • Several decision losses gain without early stopping, and on energy prices PFY and PFY with kNN targets reach 7.38% and 7.39%, below RIFT with and without validation (7.60% and 7.71%).
Reviewer_Mu5yOpenReview ↗

Thanks for the response, but my key concerns remains unaddressed

I thanks the authors for their detailed response, which goes a long way about addressing some of my concerns, in particular those concerning solver calls and the role of the validation data.

However, my key concern about model misalignment and the assumptions behind the MSE loss still stand:

  • First, it can be trivially seen that, even with no model misalignment, training a predictor for MSE loss when the true P(c | x) is (say) log-normal results in a biased estimate of E[c | x]. In this setting, MSE training is structurally suboptimal even for a single context
  • I suspect the confusion might stem from the distinction between optimality for a given context -- i.e. for P(c | x) and optimality for a given realization -- i.e. c ~ P(c | x). MSE training is definitely sufficient for the latter, but not for the former, which is unfortunately what matters in DFL
  • Model misalignment complicates the picture, though I think the proposed dampened steps may be a good idea to address how misalignment links together the predictions over multiple examples

Overall, I keep my score, though I encourage the authors to move forward this line of research, perhaps by making a more general assumption on the loss function

Reviewer_CbMAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Decision-focused learning treats the choice of a decision loss as a major design decision. The paper challenges whether these specialized losses are truly necessary, or whether previous research has simply used weak regression baselines. Under certain assumptions, the authors present theoretical insight that moving a predicted cost towards its true value cannot worsen the downstream decision. The paper proposes RIFT, a method using damped Gauss-Newton optimization to make predictions follow desired directions. Experiments (but please see weaknesses) show that well-trained ordinary regression performs comparably to decision-focused learning across five benchmarks.

Critical strengths
  1. Interesting paper that highlights a potential drawback of decision-focused learning. The authors propose an alternate method with regression displacement that performs similar to five DFL tasks despite not including decision loss.

  2. Experiments specifically highlight that when comparing training signals of MSE and SPO+, MSE produces better directions on both tested knapsack problems, suggesting that the extra decision information in the losses is not necessarily improving training.

Critical weaknesses

The paper makes a broad claim on DFL with limited benchmark diversity. The central claim that decision-focused losses provide limited benefit over regression is heavily dependent on two knapsack experiments. Although the broader benchmark has five tasks, those experiments cannot further isolate the impact of these losses. It is unclear therefore whether these results can be extended to a broader DFL literature, or only similar knapsack tasks.

Please also see questions

Questions for authors
  1. How do the authors expect their conclusions to hold in non-knapsack tasks? Particularly your comparison of MSE vs SPO+/PFY directions and conclusion that decision-aware directions can actually perform worse.

  2. DFL could be said to be more useful when there is more of a "non-alignment" between prediction accuracy and decision quality. Is this true of the five benchmarks provided?

  3. Is there a conceptual boundary where regression is enough to make good decisions vs where DFL becomes useful? It would be helpful to clarify the strengths of each method, even when both are optimized equally well.

1 comment

We thank you for your close reading of the exchange experiments. In the revised PDF, new and changed text outside figures is in blue.

Weakness. Scope of the claim. The exchange result holds on the two knapsack tasks, and a training-budget test shows that the advantage of SPO+ over regression under the benchmark's Adam training shrinks there with longer training, and after 1,000 steps SPO+ is significantly lower on neither knapsack task (Section 5.4, Table 2).

  • With the same full-batch Adam step for both losses, the regret of SPO+ minus that of MSE goes from -6.35 to -8.83 points after 10 steps to -0.38 to +1.59 after 1,000 steps on the knapsack tasks (five seeds, in points):
Task, learning rate10 steps1,000 steps
Generated knapsack, 0.01-6.93+0.72
Generated knapsack, 0.05-8.83+1.59
Energy prices, 0.01-6.35-0.38
Energy prices, 0.05-7.88+0.26
  • On energy scheduling the difference shrinks to -0.10 and -0.16 points after 300 steps, and SPO+ stays 0.16 points ahead at learning rate 0.05 (p = 0.015, not significant after the Holm correction over the ten cells, Appendix C.6), while no budget separates the two losses on matching (Table 2).
  • The revised abstract states each result for the tasks on which it was measured and ends with the statement that on the two knapsack tasks, how regression is trained decides whether it trails a decision loss.

Q1. Directions beyond knapsack. Under the same Gauss-Newton transport, the SPO+ and PFY directions raise regret on both knapsack tasks and lower it significantly on none of the five tasks (Section 5.3, Figure 3a, Table 18). Regret of each direction minus that of the regression targets, five seeds, in points:

TaskSPO+ directionPFY direction
Generated knapsack+1.11+3.04
Energy prices+1.81+2.57
Energy scheduling+0.01+0.09
Bipartite matching+0.67-0.55
Portfolio-0.04-1.92
  • All four knapsack increases are significant in the paired t test and none of the six other differences is, with 95% intervals on portfolio of about ±84 and ±130 points around a mean regret of 634%.
  • Trained as losses on synthetic shortest path with a linear predictor, SPO+ and PFY are no better than tuned MSE at costs of degree 2 (+0.08 and +0.17 points) and lower at degree 4 (-1.11 and -1.08 points, the first significant after the Holm correction, Table 26).
  • On bipartite matching no method of Table 1 differs significantly from RIFT, and after tuning RIFT at its default damping lies significantly above NCE only (2.60 points, Holm p 0.021), a difference that a tuned damping removes (0.49 points, Holm p = 1.00, Appendix C.7).

Q2. Accuracy and decision quality on the five tasks. On the knapsack and scheduling tasks the prediction error and the regret of regression fall together during training, and after the Holm correction a regression baseline with a tuned learning rate leaves no significant gain to a decision loss on generated knapsack, energy prices and portfolio (Appendix C.7, Table 23).

  • From 10 steps to the largest budget of Table 2 at learning rate 0.01, the training MSE of regression falls by a factor of 7 to 12 on these three tasks and its test regret from 28.69%, 22.17% and 9.74% to 5.47%, 8.01% and 1.87% (Table 21).
  • On energy scheduling and matching a tuned decision loss keeps a gain, with tuned SPO+ 0.15 points below tuned MSE and tuned NCE 2.03 points below tuned BCE (Holm p 0.027 and 0.015).
  • On energy prices PFY with kNN targets is 0.68 points below tuned MSE, significant without the correction (p = 0.017) and not after it (Holm p 0.068).

Q3. Where regression is enough. In a controlled synthetic study on shortest path and knapsack, no decision loss beats tuned regression significantly at costs of degree 2, and SPO+, PFY and PFY with kNN targets do better with a linear predictor and costs of degree 4 or 6 (Section 5.6, Appendix C.9, Figure 6).

  • In all 8 such cells, chosen on three seeds, each of the three has a lower regret than tuned MSE on every one of five new test seeds, by 0.45 to 4.57 points.
  • On the three selection seeds, the best decision loss lies between -0.67 and +0.36 points of tuned MSE in every cell of degree 1 or 2 and below it in all 48 cells of degree 4 to 8, consistent with the result of Elmachtoub et al. (2025) that integrated estimation can do better under misspecification.
  • RIFT regresses on the realized costs and does not move this boundary, since after the Holm correction over the cells it lies significantly above the best decision loss in 7 of the 8 misspecified cells, by 1.04 to 5.89 points, and with 32 hidden units at degree 4 on shortest path (+0.53), and differs from tuned MSE in none of the 14 cells (Table 26), while without the correction it is above the best decision loss in all 8 misspecified cells and above tuned MSE in one (+1.44 points, p = 0.033, Appendix C.9).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.