- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 3
Read review
Paper summary
The paper proposes Functional-DFL, a decision-focused learning method for problems where the predictive model outputs a probability distribution rather than a single value. The main idea is that making good decisions does not necessarily require predicting the entire distribution accurately, but only the aspects of it that matter for the downstream optimization problem. The authors show how to compute the training gradient using these relevant quantities, without sampling scenarios or differentiating through an optimization solver (as related work does), provided the necessary expectations can be computed analytically. They then propose an additional loss that encourages the model to learn how the expected cost changes across different decisions, rather than only at the currently optimal decision. Experiments on four stochastic optimization problems show competitive computational efficiency and better average decision quality than the compared baselines.
Critical strengths
Relevant and well-motivated problem. The paper adds to relevant existing literature on decision-focused learning where entire distributions are predicted, instead of point predictions. I find the underlying idea of the paper appealing. Rather than trying to learn the entire probability distribution accurately, we should focus only on the aspects of it that actually matter for the downstream optimization problem.
Computationally efficient approach. Another strength is that the authors show how to compute the training gradient without sampling scenarios or differentiating through an optimization solver, provided the relevant expectations can be computed analytically. This is potentially useful for stochastic optimization problems where existing approaches can be computationally expensive.
Strong experimental results. The method shows very strong experimental results. Across the four benchmarks, it always achieves the best average decision quality, and while it is sometimes only modestly better than the second best method (which is typically diffusion-DFL), it trains much faster than diffusion-DFL (always at least one order of magnitude faster, typically more).
Good experimental analysis. I like that the paper includes several ablation studies that help separate the contributions of the exact regret gradient and the additional field-matching loss. The authors also investigate the effect of important hyperparameters and compare their approach with a differentiable optimization layer using the same predictive model.
Critical weaknesses
Unnecessarily dense and opaque presentation. Despite having extensive background in decision-focused learning, I found this paper really difficult to follow. Even he introduction, and particularly the paragraph introducing the distributional optimality field, reads as though the reader should already be familiar with the concepts being introduced. Rather than developing the intuition behind the ideas, the paper relies heavily on dense mathematical descriptions without accompanying “human interpretation”. This continues throughout the methodology, where central concepts are introduced almost entirely through mathematical definitions and propositions, with little explanation of what they actually mean or why they matter. For example, the distributional optimality field, which is a central concept of the paper, is introduced directly through its mathematical definition in Section 3.2. A simple explanation beforehand would help considerably. For instance, this can be as short as: “For a given predicted distribution, we can look at how the expected cost changes as we vary the decision z. This produced a gradient for every possible decision. The resulting function is what we call the distributional optimality field.” This same concern applies to most, if not all, of the rest of the paper. I consider this a critical weakness of the paper. The lack of intuitive explanations makes the work unnecessarily hard to read and makes it difficult for me to properly assess the novelty and significance of the technical contributions.
The writing is also often hard to parse. For example, in the introduction, the authors write “When a predictive distribution is trained by differentiating realized cost through the stochastic decision, the sample gradient is exactly the residual between the predictive field and the realized cost gradient at the induced decision, projected through the same local solution sensitivity, and for point predictions with bilinear costs this reduces to the prediction-error form.” This is a lot of information packed into a single sentence, making it very difficult to understand what the authors are actually trying to convey.
I am quite torn about this paper. On the one hand, the method’s underlying idea is interesting, and the experimental results are very strong. There is clearly something valuable here. On the other hand, I find the exposition critically lacking. Despite having substantial background in decision-focused learning, I struggled to understand what the authors were trying to communicate in many parts of the paper. The mathematical results are presented with very little intuition or discussion of their significance, making it unnecessarily difficult to appreciate and assess the contributions. I consider this a serious enough weakness that I would not be opposed to rejecting the paper on this basis alone, despite its strong results. I recognize that explaining all these concepts properly within nine pages may be difficult, but perhaps that also raises the question of whether the conference paper format is appropriate for work that requires this much mathematical development to be understood. At this point, I am leaning towards a borderline assessment, without a strong preference for either acceptance or rejection. I am interested to see what points other reviewers raise and whether the subsequent discussion helps clarify the contributions or resolve some of my concerns. I remain open to changing my assessment in either direction.
Questions for authors
/
5 comments
Thank you for your response and the revisions. I find the exposition clearer now.
However, undertanding the paper better, I do have a question regarding the novelty of the approach. As acknowledged in the paper, the local-only regret gradient is mathematically equivalent to that of Donti et al. (2017). While their particular implementation relies on SQP solver, I see no reason why their implicit differentiation framework could not be used with your forward solver. Does your comparison with Donti et al. therefore not primarily demonstrate the superiority of your forward solver rather than that of your gradient computation? More generally, what distinguishes local-only from applying the approach of Donti et al. with a suitable forward solver and analytically computed expectations?
Of course, Functional-DFL additionally incorporates field matching at other decisions, which I consider an interesting idea. However, Table 3 shows that this term provides relatively modest improvements on two benchmarks and essentially no improvement on the other two. If this is indeed the main methodological novelty, I am somewhat less convinced of the significance of the contribution.
But perhaps I am misunderstanding some of the differences with Donti et al. Could you please clarify this matter?
Follow-up response to Reviewer Ppt8
Thank you for the follow-up question.
- The Local only gradient is mathematically equal to that of Donti et al. (2017), but the numerical implementation differs. Our contribution is to identify which distributional error enters the gradient and how solution sensitivity transforms it.
- Functional field matching follows from this characterization. This analysis shows what regret minimization learns at the induced decision and motivates supervision at other decisions. It also helps interpret the gains across tasks.
Relation to Donti et al.
As you note, a suitable forward solver can be used in their general framework. Replacing the forward solver and differentiating the KKT conditions at the solution gives the sensitivity computation in Eqs. (6)–(8) used by Local only. The failure in Section 6.4 originates in the forward solver. The comparison shows how this affects training through the final QP of SQP.
Donti et al. differentiate that final QP through the OptNet layer. Such layers can be ill-conditioned (Magoon et al., 2025), and Lee et al. (2026) report lower regret and shorter training time by computing the same gradient through the reduced active-set system. In our mixture experiment, SQP fails to converge on 99.9% of training instances, so its final QP can differ from the one at the optimum in active constraints and curvature. We compute the decision and its sensitivity using the chosen solver and KKT system, then hold both fixed and differentiate the local field-matching loss through the predictive field. This gives the same regret gradient without a differentiable optimization layer.
Main contribution
Prior DFL methods obtain learning signals in different ways. Donti et al. differentiate the argmin, Diffusion-DFL uses generated scenarios, and DF$^2$ learns the expected objective. These methods provide a training signal but do not identify which distributional error the regret gradient uses. Lee et al. (2026) identified the corresponding relationship for point predictions, where prediction error is passed through local solution sensitivity.
We identify this relationship for predictive distributions. A distribution enters downstream optimality through its optimality field (Section 3). Theorem 1 expresses the regret gradient through field error at the induced decision, solution sensitivity, and the field's parameter Jacobian. To our knowledge, this field-error characterization had not been stated for predictive distributions. It identifies the decision-relevant error used by the regret gradient at a single decision. It also gives a local field-matching loss whose parameter gradient equals the sample regret gradient. Training requires one solve and one linear system. This implementation trains mixture predictors stably in our experiments. The full Functional-DFL method achieves better decision quality in less training time than the compared DFL methods (Tables 1 and 2).
Local field matching and Functional-DFL
Local field matching trains using this distributional regret gradient and is a complete method on its own. The regret gradient uses field error at one decision, but the field is a function over the decision space. The same observation provides field targets at other decisions without an additional solve. Functional-DFL adds a loss to reduce these errors at the probes. Differentiating the argmin gives the gradient at the induced decision. The field-error representation also identifies what can be matched at other decisions, leading to the additional learning objective.
Table 3 compares adding NLL or field supervision to Local only. NLL fits the outcome distribution, while functional field matching fits the field at other decisions. The latter gives similar performance on Allocation and Power Scheduling and improves the portfolio tasks (24.51 to 23.61 and 88.1 to 82.7). NLL helps on Mean–CVaR but increases regret on Allocation (0.325 to 0.772) and Mean–Variance (88.1 to 90.1). Functional field matching is comparable to or better than Local only on all four tasks and needs no additional solve, so we propose Functional-DFL as the default. Local only already outperforms the strongest baseline on three tasks, and Functional-DFL does so on all four (Tables 1 and 3).
Solution sensitivity helps interpret these gains. permits only decision changes compatible with the active constraints (Section 4.1). Each active zero-weight constraint in a long-only portfolio removes a feasible tangent direction. Measured over the trained models’ output biases, has full row rank on Allocation and Power Scheduling but much lower rank than the decision dimension on the portfolio tasks. Parameter changes that leave the decision unchanged receive no first-order regret signal, but the probe loss can still respond to field changes elsewhere (Eq. 20). This explains what the probes can learn beyond the current regret update and helps interpret the portfolio improvements.
Response to Reviewer Ppt8
Thank you for the thoughtful review. We appreciate your positive view of the idea and the experimental results, and we take seriously your concern that the presentation made the contribution hard to assess. We agree that the original submission presented the mathematical development without sufficiently explaining the meaning and purpose of the central concepts. We reread the paper as a reader would and revised it to make the theory and its message easier to follow. All section, equation, and line numbers below refer to the revised paper. We say "original" when we refer to the original submission.
Explanations and flow of the revised paper
You noted that the central concepts were introduced through definitions and propositions with little explanation, and that the writing was hard to parse. Sections 3–5 now follow one line of questions. Each concept is introduced where its question arises, with its meaning and purpose given before the derivation.
We also split sentences that combined several results. The introduction now explains the optimality field first and then presents the result from the sentence you quoted in several shorter sentences (lines 46–59). We revised the rest of the text in the same way so that it reads clearly throughout. A step-by-step explanation of the method in plain words is given in our response to Reviewer RfTE (Part 1, W1).
Organization of the theoretical results
The main text of the original submission stated four formal results: Proposition 1, Theorem 2, Proposition 3, and Proposition 4. The revised main text states one, Theorem 1 (original Theorem 2). Original Proposition 1 is replaced by the derivation of the solution sensitivity in Section 4.1. The regret bound and the curvature analysis (original Propositions 3 and 4) are now in Appendices A.5 and A.4 with their proofs. We used the space this freed for the explanations above.
Research motivation and significance
The question we set out to answer was what a predictive distribution learns from the regret gradient in DFL. Minimizing regret provides a clear objective, but it is less clear which errors in the predictive distribution affect the decision and what the regret gradient teaches it to capture. Our challenge was to connect the idea you found appealing, focusing on distributional information that matters for downstream decisions, to a concrete learning method.
In point-prediction DFL, prediction error is connected to the regret gradient through local solution sensitivity (Lee et al., 2026). In stochastic optimization, however, decisions may also depend on covariance or tail information. A difference in means is therefore not enough to describe the relevant distributional error in general. We first needed to understand which information the downstream problem uses from the distribution.
Existing approaches to DFL for stochastic optimization handle uncertainty in different ways. Diffusion-DFL uses generated scenarios in the downstream optimization, while DF$^2$ learns the expected objective. We wanted to keep the predictive distribution and to identify explicitly the error through which it enters the distributional regret gradient. Starting from the gradient of the expected objective and applying solution sensitivity shows that the distributional regret gradient contains the field error at the induced decision (Theorem 1). This identifies the distributional error that matters for learning through the structure of the downstream problem.
The field in this relationship is a function over the decision space. The same observed outcome used for learning at the induced decision can therefore provide objective gradients at other decisions, with no additional solve, and these give further information about the field. Functional-DFL uses this property to retain the exact distributional regret gradient while providing additional supervision. The question of how to understand the distributional regret gradient thus led to a way of obtaining more learning signal from the same observation.
We hope the revised paper now makes the contribution easier to assess. If any part is still unclear, please tell us and we will explain it further.