acceptodds
Under review as a conference paper at ICLR 2027

What Decisions Need from a Distribution: Decision-Focused Learning via Optimality-Field Matching

Abstract

Decision-Focused Learning (DFL) trains predictive models for downstream decision quality rather than predictive accuracy alone. For point predictions, recent work relates the regret gradient to prediction error and local solution sensitivity. In stochastic optimization, however, the optimal decision may depend on distributional information that a point prediction cannot represent. We show that a stochastic objective uses a predictive distribution through its *distributional optimality field*, a function over the decision space whose values are distributional functionals. Differentiating realized cost through the stochastic decision yields a sample gradient with the same structure: the field residual replaces prediction error and is transformed through the local solution sensitivity. When the field is available analytically, this gradient requires one solve and no predictive scenario sampling or differentiable optimization layer. Since the field being learned also determines the induced decision, this gradient constrains the field only where the decision currently lies. We therefore introduce **Functional-DFL**, which keeps the exact regret gradient and uses the same observation to match the field at additional decisions, so that regret remains controlled as the decision moves. Across two synthetic and two real-world problems, Functional-DFL achieves the best average decision quality among the compared baselines at low computational cost. Code is available at https://anonymous.4open.science/r/distributional_dfl-6B0B.

Primary area:
optimization
Keywords:
Decision-Focused Learning, Stochastic Optimization

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_Ppt8OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

The paper proposes Functional-DFL, a decision-focused learning method for problems where the predictive model outputs a probability distribution rather than a single value. The main idea is that making good decisions does not necessarily require predicting the entire distribution accurately, but only the aspects of it that matter for the downstream optimization problem. The authors show how to compute the training gradient using these relevant quantities, without sampling scenarios or differentiating through an optimization solver (as related work does), provided the necessary expectations can be computed analytically. They then propose an additional loss that encourages the model to learn how the expected cost changes across different decisions, rather than only at the currently optimal decision. Experiments on four stochastic optimization problems show competitive computational efficiency and better average decision quality than the compared baselines.

Critical strengths

Relevant and well-motivated problem. The paper adds to relevant existing literature on decision-focused learning where entire distributions are predicted, instead of point predictions. I find the underlying idea of the paper appealing. Rather than trying to learn the entire probability distribution accurately, we should focus only on the aspects of it that actually matter for the downstream optimization problem.

Computationally efficient approach. Another strength is that the authors show how to compute the training gradient without sampling scenarios or differentiating through an optimization solver, provided the relevant expectations can be computed analytically. This is potentially useful for stochastic optimization problems where existing approaches can be computationally expensive.

Strong experimental results. The method shows very strong experimental results. Across the four benchmarks, it always achieves the best average decision quality, and while it is sometimes only modestly better than the second best method (which is typically diffusion-DFL), it trains much faster than diffusion-DFL (always at least one order of magnitude faster, typically more).

Good experimental analysis. I like that the paper includes several ablation studies that help separate the contributions of the exact regret gradient and the additional field-matching loss. The authors also investigate the effect of important hyperparameters and compare their approach with a differentiable optimization layer using the same predictive model.

Critical weaknesses

Unnecessarily dense and opaque presentation. Despite having extensive background in decision-focused learning, I found this paper really difficult to follow. Even he introduction, and particularly the paragraph introducing the distributional optimality field, reads as though the reader should already be familiar with the concepts being introduced. Rather than developing the intuition behind the ideas, the paper relies heavily on dense mathematical descriptions without accompanying “human interpretation”. This continues throughout the methodology, where central concepts are introduced almost entirely through mathematical definitions and propositions, with little explanation of what they actually mean or why they matter. For example, the distributional optimality field, which is a central concept of the paper, is introduced directly through its mathematical definition in Section 3.2. A simple explanation beforehand would help considerably. For instance, this can be as short as: “For a given predicted distribution, we can look at how the expected cost changes as we vary the decision z. This produced a gradient for every possible decision. The resulting function is what we call the distributional optimality field.” This same concern applies to most, if not all, of the rest of the paper. I consider this a critical weakness of the paper. The lack of intuitive explanations makes the work unnecessarily hard to read and makes it difficult for me to properly assess the novelty and significance of the technical contributions.

The writing is also often hard to parse. For example, in the introduction, the authors write “When a predictive distribution is trained by differentiating realized cost through the stochastic decision, the sample gradient is exactly the residual between the predictive field and the realized cost gradient at the induced decision, projected through the same local solution sensitivity, and for point predictions with bilinear costs this reduces to the prediction-error form.” This is a lot of information packed into a single sentence, making it very difficult to understand what the authors are actually trying to convey.

I am quite torn about this paper. On the one hand, the method’s underlying idea is interesting, and the experimental results are very strong. There is clearly something valuable here. On the other hand, I find the exposition critically lacking. Despite having substantial background in decision-focused learning, I struggled to understand what the authors were trying to communicate in many parts of the paper. The mathematical results are presented with very little intuition or discussion of their significance, making it unnecessarily difficult to appreciate and assess the contributions. I consider this a serious enough weakness that I would not be opposed to rejecting the paper on this basis alone, despite its strong results. I recognize that explaining all these concepts properly within nine pages may be difficult, but perhaps that also raises the question of whether the conference paper format is appropriate for work that requires this much mathematical development to be understood. At this point, I am leaning towards a borderline assessment, without a strong preference for either acceptance or rejection. I am interested to see what points other reviewers raise and whether the subsequent discussion helps clarify the contributions or resolve some of my concerns. I remain open to changing my assessment in either direction.

Questions for authors

/

5 comments
AuthorsOpenReview ↗

Response to Reviewer Ppt8

Thank you for the thoughtful review. We appreciate your positive view of the idea and the experimental results, and we take seriously your concern that the presentation made the contribution hard to assess. We agree that the original submission presented the mathematical development without sufficiently explaining the meaning and purpose of the central concepts. We reread the paper as a reader would and revised it to make the theory and its message easier to follow. All section, equation, and line numbers below refer to the revised paper. We say "original" when we refer to the original submission.


Explanations and flow of the revised paper

You noted that the central concepts were introduced through definitions and propositions with little explanation, and that the writing was hard to parse. Sections 3–5 now follow one line of questions. Each concept is introduced where its question arises, with its meaning and purpose given before the derivation.

  1. What does a decision need from a predictive distribution? The optimality field. As you suggested, its definition is now preceded by a plain explanation of what the gradient of the expected objective describes (Section 3, lines 127–156).
  2. When does an error in the field change the decision? Solution sensitivity, derived from the KKT conditions (Section 4.1, lines 164–209).
  3. How does that error affect regret, and how can we train on it when the true field is unknown? Theorem 1 and the training loss (Sections 4.2–4.3, lines 213–266).
  4. The distributional regret gradient uses the field at one decision. What about the others? Probes and Functional-DFL (Section 5.1, lines 280–328).

We also split sentences that combined several results. The introduction now explains the optimality field first and then presents the result from the sentence you quoted in several shorter sentences (lines 46–59). We revised the rest of the text in the same way so that it reads clearly throughout. A step-by-step explanation of the method in plain words is given in our response to Reviewer RfTE (Part 1, W1).


Organization of the theoretical results

The main text of the original submission stated four formal results: Proposition 1, Theorem 2, Proposition 3, and Proposition 4. The revised main text states one, Theorem 1 (original Theorem 2). Original Proposition 1 is replaced by the derivation of the solution sensitivity in Section 4.1. The regret bound and the curvature analysis (original Propositions 3 and 4) are now in Appendices A.5 and A.4 with their proofs. We used the space this freed for the explanations above.


Research motivation and significance

The question we set out to answer was what a predictive distribution learns from the regret gradient in DFL. Minimizing regret provides a clear objective, but it is less clear which errors in the predictive distribution affect the decision and what the regret gradient teaches it to capture. Our challenge was to connect the idea you found appealing, focusing on distributional information that matters for downstream decisions, to a concrete learning method.

In point-prediction DFL, prediction error is connected to the regret gradient through local solution sensitivity (Lee et al., 2026). In stochastic optimization, however, decisions may also depend on covariance or tail information. A difference in means is therefore not enough to describe the relevant distributional error in general. We first needed to understand which information the downstream problem uses from the distribution.

Existing approaches to DFL for stochastic optimization handle uncertainty in different ways. Diffusion-DFL uses generated scenarios in the downstream optimization, while DF$^2$ learns the expected objective. We wanted to keep the predictive distribution and to identify explicitly the error through which it enters the distributional regret gradient. Starting from the gradient of the expected objective and applying solution sensitivity shows that the distributional regret gradient contains the field error at the induced decision (Theorem 1). This identifies the distributional error that matters for learning through the structure of the downstream problem.

The field in this relationship is a function over the decision space. The same observed outcome used for learning at the induced decision can therefore provide objective gradients at other decisions, with no additional solve, and these give further information about the field. Functional-DFL uses this property to retain the exact distributional regret gradient while providing additional supervision. The question of how to understand the distributional regret gradient thus led to a way of obtaining more learning signal from the same observation.


We hope the revised paper now makes the contribution easier to assess. If any part is still unclear, please tell us and we will explain it further.

Thank you for your response and the revisions. I find the exposition clearer now.

However, undertanding the paper better, I do have a question regarding the novelty of the approach. As acknowledged in the paper, the local-only regret gradient is mathematically equivalent to that of Donti et al. (2017). While their particular implementation relies on SQP solver, I see no reason why their implicit differentiation framework could not be used with your forward solver. Does your comparison with Donti et al. therefore not primarily demonstrate the superiority of your forward solver rather than that of your gradient computation? More generally, what distinguishes local-only from applying the approach of Donti et al. with a suitable forward solver and analytically computed expectations?

Of course, Functional-DFL additionally incorporates field matching at other decisions, which I consider an interesting idea. However, Table 3 shows that this term provides relatively modest improvements on two benchmarks and essentially no improvement on the other two. If this is indeed the main methodological novelty, I am somewhat less convinced of the significance of the contribution.

But perhaps I am misunderstanding some of the differences with Donti et al. Could you please clarify this matter?

AuthorsOpenReview ↗

Follow-up response to Reviewer Ppt8

Thank you for the follow-up question.

  1. The Local only gradient is mathematically equal to that of Donti et al. (2017), but the numerical implementation differs. Our contribution is to identify which distributional error enters the gradient and how solution sensitivity transforms it.
  2. Functional field matching follows from this characterization. This analysis shows what regret minimization learns at the induced decision and motivates supervision at other decisions. It also helps interpret the gains across tasks.

Relation to Donti et al.

As you note, a suitable forward solver can be used in their general framework. Replacing the forward solver and differentiating the KKT conditions at the solution gives the sensitivity computation in Eqs. (6)–(8) used by Local only. The failure in Section 6.4 originates in the forward solver. The comparison shows how this affects training through the final QP of SQP.

Donti et al. differentiate that final QP through the OptNet layer. Such layers can be ill-conditioned (Magoon et al., 2025), and Lee et al. (2026) report lower regret and shorter training time by computing the same gradient through the reduced active-set system. In our mixture experiment, SQP fails to converge on 99.9% of training instances, so its final QP can differ from the one at the optimum in active constraints and curvature. We compute the decision and its sensitivity using the chosen solver and KKT system, then hold both fixed and differentiate the local field-matching loss through the predictive field. This gives the same regret gradient without a differentiable optimization layer.

Main contribution

Prior DFL methods obtain learning signals in different ways. Donti et al. differentiate the argmin, Diffusion-DFL uses generated scenarios, and DF$^2$ learns the expected objective. These methods provide a training signal but do not identify which distributional error the regret gradient uses. Lee et al. (2026) identified the corresponding relationship for point predictions, where prediction error is passed through local solution sensitivity.

We identify this relationship for predictive distributions. A distribution enters downstream optimality through its optimality field (Section 3). Theorem 1 expresses the regret gradient through field error at the induced decision, solution sensitivity, and the field's parameter Jacobian. To our knowledge, this field-error characterization had not been stated for predictive distributions. It identifies the decision-relevant error used by the regret gradient at a single decision. It also gives a local field-matching loss whose parameter gradient equals the sample regret gradient. Training requires one solve and one linear system. This implementation trains mixture predictors stably in our experiments. The full Functional-DFL method achieves better decision quality in less training time than the compared DFL methods (Tables 1 and 2).

Local field matching and Functional-DFL

Local field matching trains using this distributional regret gradient and is a complete method on its own. The regret gradient uses field error at one decision, but the field is a function over the decision space. The same observation provides field targets at other decisions without an additional solve. Functional-DFL adds a loss to reduce these errors at the probes. Differentiating the argmin gives the gradient at the induced decision. The field-error representation also identifies what can be matched at other decisions, leading to the additional learning objective.

Table 3 compares adding NLL or field supervision to Local only. NLL fits the outcome distribution, while functional field matching fits the field at other decisions. The latter gives similar performance on Allocation and Power Scheduling and improves the portfolio tasks (24.51 to 23.61 and 88.1 to 82.7). NLL helps on Mean–CVaR but increases regret on Allocation (0.325 to 0.772) and Mean–Variance (88.1 to 90.1). Functional field matching is comparable to or better than Local only on all four tasks and needs no additional solve, so we propose Functional-DFL as the default. Local only already outperforms the strongest baseline on three tasks, and Functional-DFL does so on all four (Tables 1 and 3).

Solution sensitivity helps interpret these gains. SθS_\theta permits only decision changes compatible with the active constraints (Section 4.1). Each active zero-weight constraint in a long-only portfolio removes a feasible tangent direction. Measured over the trained models’ output biases, SθJθS_\theta J_\theta has full row rank on Allocation and Power Scheduling but much lower rank than the decision dimension on the portfolio tasks. Parameter changes that leave the decision unchanged receive no first-order regret signal, but the probe loss can still respond to field changes elsewhere (Eq. 20). This explains what the probes can learn beyond the current regret update and helps interpret the portfolio improvements.

Reviewer_Z45ZOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies decision-focused learning for stochastic optimization. The authors introduce Distributional Optimality Field to characterize the distributional information relevant to downstream decisions, and derive an exact regret gradient based on field residuals and solution sensitivity. Based on this analysis, they propose Functional-DFL, which introduces additional field-matching supervision at multiple decision points. Experiments on four tasks show that Functional-DFL achieves the best decision quality among the compared methods, while significantly reducing training time compared with Diffusion-DFL.

Critical strengths

1: Theoretical contribution: The paper introduces Distributional Optimality Field to characterize how predictive distributions affect downstream decisions. The derived regret gradient extends existing point-based DFL analysis to stochastic optimization and provides a theoretical foundation for the proposed method.

2: Efficiency: Functional-DFL avoids scenario sampling and differentiable optimization layers when the optimality field is analytically available. It achieves the best quality across four stochastic optimization tasks and reduces training time compared with Diffusion-DFL.

Critical weaknesses

1: Limited theoretical guarantees: Proposition 3 requires sufficient probe coverage, which is difficult to achieve in high-dimensional decision spaces. The resulting bound is only qualitative for the 10-dimensional Allocation task. Proposition 4 also relies on exact field matching and a positive-definite probe-induced Gram matrix, whose validity in practical neural network settings remains unclear.

2: Limited computational applicability: The computational advantage of Functional-DFL relies on the availability of analytically tractable optimality fields. It remains unclear whether this advantage can be maintained when the fields require numerical approximation or sampling, as such settings are not evaluated in the paper.

Questions for authors

1: Proposition 3: For the 10-dimensional Allocation task, the authors acknowledge that 32 probes provide insufficient coverage and that the resulting regret bound is only qualitative. It would be helpful to see the numerical bound under this setting and how it compares with the actual test regret.

2: Proposition 4: The analysis assumes that the model can exactly match the true optimality field and that the probe-induced Gram matrix is positive definite. It is unclear whether these conditions hold for the neural networks used in the experiments. An empirical analysis of the field matching error and the Gram matrix spectrum would help support these theoretical claims.

3: Computational efficiency: The efficiency of Functional-DFL comes from analytically computing the optimality field and its parameter Jacobian. In problems where these quantities require numerical approximation or Monte Carlo estimation, the computational cost could be quite different. A comparison of training time under different sample sizes would help clarify whether the efficiency advantage remains.

4: Baseline comparison: I noticed that the two methods use different covariance formulations in the Mean-Variance Portfolio task: Diffusion-DFL combines historical and generated-return covariances, whereas Functional-DFL uses the expected-window covariance. The rationale for this difference should be clarified, ideally with an additional comparison under the same risk formulation.

6 comments
AuthorsOpenReview ↗

Response to Reviewer Z45Z (Part 1/2)

We thank the reviewer for the thoughtful feedback.

We revised the paper in response to the reviewers’ feedback. We would appreciate your taking a look at the revised paper. All section, equation, and table numbers below refer to the revised paper. We use “original” when referring to the original submission.


W1, Q1 (The regret bound with 32 probes)

For the Allocation task with 32 probes, the coverage bound computed using global estimates is 6.29×10126.29\times10^{12}, compared with a mean population regret of 0.04477 over five seeds. Following your suggestion, we have added this comparison to Appendix D.2.

The proof first bounds regret using the field error at the induced decision. To estimate this error from measurements at the probes, we also need to account for how much the field error can increase between the probes and that decision. Using conservative estimates that hold throughout the feasible set leads to a very large coverage bound.

Since the data-generating distribution is known in this synthetic task, we directly computed the field error at the learned decisions. Substituting this error into Eq. (24) in Appendix A.5 gives an average upper bound of 0.12342, much closer to the population regret of 0.04477, with the bound satisfied in all five runs. Thus, directly evaluating the field error at the induced decision gives a much tighter regret bound using the same analysis.

Functional-DFL achieves the best mean decision quality across all four tasks, supporting the effectiveness of learning through the optimality field.


W1, Q2 (Field matching and the Gram condition)

In Proposition 4 of the original submission, positive definiteness was an additional sufficient condition for a strict local minimum. The curvature decomposition and the additional curvature along individual directions hold at exact matching even when the Gram matrix is singular. We removed the positive definiteness condition and strict local minimum conclusion because they are unnecessary for explaining the curvature added by probes. Nevertheless, Proposition 2 in the revised paper (Appendix A.4) retains the curvature decomposition and its directional implication. The Gram matrix formerly denoted by Γκ\Gamma_\kappa is written explicitly as EX,T[JX∗(T)⊤JX∗(T)]\mathbb E_{X,T}[J_X^{\ast}(T)^\top J_X^{\ast}(T)] in Eq. (22).

We also examined the conditions in the trained models. At the 32 training probes, mean squared field errors over five seeds were 851.635 for the Allocation task and 7.621 for Mean–CVaR, using pooled predictions and conditional predictions at the test contexts, respectively. Relative to the mean squared true-field norms, the errors are 0.949 and 0.157, so exact matching was not attained. The Gram matrix also has a null direction because adding the same constant to all mixture logits leaves the predictive distribution unchanged. This establishes that its minimum eigenvalue is zero without computing the full spectrum. Such parameter redundancy does not prevent probes from adding curvature along other directions.

To verify that contribution directly, we constructed an exact matching point in a two-dimensional Gaussian problem using the same MLP backbone. Across all 1,084,465 parameters, the regret Hessian had rank 2, which increased to 5 with the probe term. The Hessian from the theoretical expression agreed with an independent calculation to numerical precision, confirming that probes added curvature along directions with zero regret curvature (Appendix D.2).

During training, field error also affects the Hessian. The probe loss can still provide learning signals along directions that leave the induced decision unchanged to first order (Section 5.2). Table 3 shows improved decision quality with field supervision, and Appendix C.6 shows lower held-out field error on all four tasks at the largest perturbation scale. These results support the additional supervision provided by the field.

AuthorsOpenReview ↗

Response to Reviewer Z45Z (Part 2/2)

W2, Q3 (Computational efficiency under Monte Carlo estimation)

Our main experiments use analytic optimality fields to compute regret gradients efficiently. We also tested Monte Carlo estimation on the Allocation task (Appendix D.3, Table 10).

With other settings fixed, we estimated the objective, field, and derivatives using 16, 64, or 256 samples. The table below summarizes the results for the Allocation task over five seeds (mean ± standard deviation).

\rule[-.4em]{0pt}{1.4em}Method  \rule[-.4em]{0pt}{1.4em}Test regret ↓  \rule[-.4em]{0pt}{1.4em}Total training time (s) ↓
\rule[-.4em]{0pt}{1.4em}Analytic Functional-DFL  $\rule[-.4em]{0pt}{1.4em}$0.325 ± 0.005  \rule[-.4em]{0pt}{1.4em}\5.7 ± 3.8
\rule[-.4em]{0pt}{1.4em}Monte Carlo, 16 samples  $\rule[-.4em]{0pt}{1.4em}$0.336 ± 0.022  \rule[-.4em]{0pt}{1.4em}\7.1 ± 2.1
\rule[-.4em]{0pt}{1.4em}Monte Carlo, 64 samples  $\rule[-.4em]{0pt}{1.4em}$0.333 ± 0.010  \rule[-.4em]{0pt}{1.4em}\6.4 ± 2.1
\rule[-.4em]{0pt}{1.4em}Monte Carlo, 256 samples  $\rule[-.4em]{0pt}{1.4em}$0.328 ± 0.003  \rule[-.4em]{0pt}{1.4em}\7.3 ± 2.2
\rule[-.4em]{0pt}{1.4em}Diffusion-DFL  $\rule[-.4em]{0pt}{1.4em}$0.372 ± 0.052  \rule[-.4em]{0pt}{1.4em}\519 ± 147

At 256 samples, mean regret is 0.328, close to 0.325 with analytic fields. Training takes 7.3 seconds, close to the analytic version's 5.7 seconds and substantially less than the 519 seconds of Diffusion-DFL.

These results suggest that Functional-DFL can also be applied when optimality fields need to be estimated by Monte Carlo sampling. We consider evaluation across a wider range of such tasks an important direction for future work and appreciate your suggestion to investigate it.


Q4 (MV portfolio evaluation under the same risk formulation)

Following the previous study (Lee et al., 2026), we use 63 days of historical returns to choose a portfolio for the next 21 days. We estimate covariance from the combined historical and future returns and use its expectation in the mean–variance objective. This requires predictive information about the daily returns over the future 21-day period.

To compare both methods under this formulation, we extended the original portfolio setup of Diffusion-DFL, which uses one-day-ahead return predictions, to generate daily return sequences over 21 days. For each of 50 generated paths, we computed covariance over the combined historical and future returns, then averaged these matrices for use in training and evaluation. This configuration yielded a normalized regret of 95.22 ± 0.68% over five seeds.

We therefore sought an adaptation that preserved the original prediction structure of Diffusion-DFL while still representing the same 21-day investment problem. We changed the prediction target to average returns over that period. For covariance estimation, we took a weighted average of the covariance of the past 63 daily returns and the covariance across the generated average-return vectors. The historical term retains information about daily co-movement between assets, and the generated term incorporates uncertainty in their predicted returns over the investment period. This adaptation improved normalized regret to 87.8 ± 6.3% and is the baseline reported in the paper. We have expanded the explanation in Appendix C.2.

Both comparisons use the same data split, historical observations, investment horizon, portfolio constraints, risk coefficient, and evaluation procedure. We thus evaluated Diffusion-DFL both under the common covariance formulation and with an adaptation suited to its prediction structure. With these adaptations and consistent evaluation conditions, Functional-DFL still achieved better decision quality.


We hope the additional analyses and experiments answer your questions. If anything remains open, we would be glad to continue the discussion.

Thank you for the additional experiments and clarifications. The Monte Carlo results and baseline comparison address some of my concerns. However, several issues remain:

(1) Proposition 3: The coverage-based regret bound remains extremely loose (6.29 × 10¹²). The tighter bound relies on field error evaluated directly at the induced decision, rather than estimated from the 32 probes. Thus, the concern about probe coverage remains unresolved.

(2) Proposition 4: The revised theoretical guarantee is more limited, and the curvature improvement is demonstrated only on a constructed 2D Gaussian problem. Its validity on the actual benchmark tasks remains unclear.

(3) Experimental validation: Table 5 evaluates the two methods at different query locations, making the field error comparison less conclusive. Moreover, the Monte Carlo experiment is limited to Allocation, leaving its effectiveness on other tasks unclear.

Overall, I maintain my original score.

Reviewer_RfTEOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper introduces Functional-DFL, a method to train a predictive distribution using a loss that encourages the predictive distribution to be decision-optimal, without necessarily matching the true conditional data distribution.

Critical strengths
  1. The paper identifies a sufficient criteria for decision-optimal predictive distributions and defines a loss function based on that sufficient criteria.

  2. The experiments show substantial improvements over existing methods.

Critical weaknesses
  1. The paper is very clearly AI-written. While AI-writing alone is not disqualifying, I find it challenging to understand the paper. See the Questions section for some of these issues. Sections 4 and 5 were generally intractable for me to parse. These challenges carry into the other weaknesses I identify below. Even if the broad ideas of the paper are good, if they can't be explained in human-interpretable language, I don't think that this paper deserves acceptance as-is. This paper needs substantial rewriting for clarity. I would be happy to raise my score to acceptance if the authors can actually explain how their method works in a human-interpretable manner.

  2. The paper does not adequately explain how its contributions differ from prior work. The paper clearly does novel things, but the distinction between which parts are using existing results vs. introducing new ideas is not especially clear. For example, it seems that Shapiro (1990) already studies what this paper calls “distributional sensitivity,” under some milder assumptions than what is assumed in this work (e.g., MFCQ instead of LICQ). Does this paper use Shapiro’s results wholesale, or are there any generalizations beyond Shapiro?

  3. The paper claims in the introduction that “the field is available analytically […] for Gaussian mixtures in our tasks” but this is never shown in the main text. I think this might be discussed in the appendix. If so, the authors need to at least mention in the main text that the reader should consult the appendix for the details.

  4. Too many implementation details are pushed into the appendix, reducing the clarity of the experimental contribution. For example, only in the appendix can I find out that the trained Gaussian mixture models are parameterized by neural networks, or how the number of mixture components were chosen. It is also not clear to me why the authors only test GMM models.

  5. Too many assumptions are given without explanation of where they come from, how restrictive they are, or what sufficient conditions ensure their satisfaction. For example, Line 139 defines ψ(y,z)\psi(y,z) “wherever ϕ(y,⋅)\phi(y,\cdot) is differentiable.” What happens when it is not differentiable? Likewise, Line 221 in Theorem 2 says, “If these gradients are integrable under P and differentiation and expectation commute …” Generally, theorem statements should give sufficient conditions for these assumptions.

Questions for authors
  1. Line 113: Why only consider polyhedral feasible sets? Is the method general enough to consider more general convex feasible sets?

  2. Line 129: what is the “reduced Hessian”?

  3. Line 129: why assume LICQ, when Shapiro (1990) assumes MFCQ?

  4. Line 132: does an orthonormal basis of ker⁡CQ\ker C_Q always exist?

  5. Line 132: does SQS_Q depend on the choice of orthonormal basis?

  6. Line 134: why is SQS_Q called the “local solution sensitivity operator”? What does that mean?

  7. Line 162: Does “Differentiating the KKT conditions” refer to the KKT conditions of equation (1)?

  8. Line 167: why would you restrict the system to ker⁡CQ\ker C_Q?

  9. There are many different notations used for derivatives in this paper, and it is confusing. See Line 171 and Line 206 for examples.

  10. Line 229: what does “The equality is sample-wise, and the equivalence with lreg holds for the parameter gradient rather than the loss value” mean?

  11. Line 337: what are the “directly evaluable fields”?

  12. Line 427: what does “at the same weight” refer to? What was the original weight?

3 comments
AuthorsOpenReview ↗

Response to Reviewer RfTE (Part 1/3): How the method works

Thank you for the careful review. It helped us improve the paper a great deal. To fit everything into the page limit, we wrote the original submission in a compressed, equation-heavy style, and we used generative AI tools to improve the readability of that text. After reading your review, we reread the paper as a first-time reader would, and we fully agree that the writing was very hard to follow and did not explain the method in a human-interpretable way. We do want to be clear, though, that this is not a fully AI-written paper. The research direction and the content of the paper were all decided and carried out by the authors.

We have reorganized the whole paper and rewritten Sections 3–5 in particular. All section, equation, and table numbers below refer to the revised paper. We say "original" when we refer to the original submission, and the line numbers you cite are from it.


W1. How the method works

The model predicts a distribution over the outcome, and we solve the downstream problem under that distribution to obtain a decision. We call this the induced decision. The goal is for it to be a good decision under the true distribution. Sections 3–5 of the revised paper explain it in this order.

  1. What the decision needs from the distribution (Section 3). The solution of the downstream problem is determined by the gradient of the expected objective and the constraints. The optimality field is this gradient, viewed as a function of the decision. For a linear cost, the field uses only the mean. In Power Scheduling, it uses the mean demand and the probability that demand exceeds the schedule.

  2. Field error and regret (Sections 4.1–4.2). The field error is the predictive field minus the true field. Theorem 1 gives the regret gradient as

    ∇θRP(Qθ)=Jθ⊤Sθ Δθ(zθ∗)\nabla_\theta R_P(Q_\theta)=J_\theta^\top S_\theta\,\Delta_\theta(z_\theta^*)

    Here Δθ\Delta_\theta is the field error at the induced decision, SθS_\theta describes how the optimal decision moves when the field changes, and JθJ_\theta describes how the model parameters change the field. This has the same structure as training a point predictor on its prediction error.

  3. Learning from a single observation: local field matching (Section 4.3). The true field is unknown, but the objective gradient at an observed outcome averages to the true field. Replacing the true field with this gradient gives the sample field error. With the sample field error in place of the field error, the formula of Theorem 1 gives exactly the parameter gradient of the objective at the observed outcome, taken through the induced decision (Eq. 13). Computing it takes one solve and one linear system, with no differentiable optimization layer. Because the field is matched at a single point, the induced decision, we call this local field matching.

  4. Extending to probes: functional field matching (Section 5). The regret gradient above uses the field error at a single decision, the induced decision, and this decision can change as the model trains. The optimality field, however, is a function over the decision space. Because of this functional structure, the same observation gives a sample field error at other decisions as well, with no additional solve. We evaluate the squared sample field error at a set of fixed decisions, which we call probes, and add it to the loss (Eq. 15). This term is the probe loss. In expectation, it equals the squared field discrepancy at the probes, that is, the squared difference between the predictive field and the true field, plus a constant (Eq. 18).

One training step: predict the distribution, solve the downstream problem once, compute SθS_\theta, evaluate the field at the induced decision and at the probes, and backpropagate Eq. (15) through the field only.


W3, Q11. "this is never shown in the main text", "what are the 'directly evaluable fields'?"

You are right, the original main text did not show this. The first paragraph of Section 6 in the revised paper now states that the field is available in closed form for Gaussian mixtures and points to Appendix B for the expressions. Section 6.1 lists the quantities the field uses in each task.

  • Allocation: the expectation of the exponential objective (the moment generating function of a Gaussian)
  • Mean–CVaR: the mean return, the tail probability, and the tail-weighted return
  • Power Scheduling: the mean and CDF of hourly demand
  • Mean–Variance: the mean and expected covariance of returns

"Directly evaluable fields" (original line 337) meant fields like these, which are computed from the mixture weights, means, and covariances without sampling. The phrase was unclear, so the revised paper no longer uses it.

AuthorsOpenReview ↗

Response to Reviewer RfTE (Part 2/3): Prior work and assumptions

W2, Q3. Existing results vs. new ideas, and LICQ vs. MFCQ

There is no generalization beyond Shapiro (1990). The distinction is as follows.

  • Existing results: The solution sensitivity in Section 4.1. It is the classical result obtained by differentiating the KKT conditions with respect to the model parameters (Fiacco, 1976), and it is not our contribution. Proposition 1 of the original submission ("distributional sensitivity") applied the same analysis to perturbations of the predictive distribution. Because it could be read as a new result, we removed it from the revised paper.

  • What is new:

    1. Theorem 1 expresses the regret gradient of a predictive distribution in terms of the field error.
    2. Eqs. (13)–(14) show the same relationship for a single observation and give a training loss that does not need a differentiable optimization layer.
    3. Functional-DFL uses the same observation to supervise the field at the probes.

    Items 1 and 2 extend the result of Lee et al. (2026) from point predictions to predictive distributions, with the field error taking the place of the prediction error.

We assume LICQ because training needs a gradient. Under LICQ, strict complementarity, and the second-order condition (positive definiteness of the reduced Hessian), the multipliers are unique and the solution is differentiable, and its derivative is obtained from the linear system in Eq. (6). Under MFCQ alone, the multipliers may not be unique and the solution is in general only directionally differentiable, so there is no single sensitivity matrix to use in backpropagation. These assumptions are common in differentiable optimization. OptNet (Amos & Kolter, 2017), dQP (Magoon et al., 2025), and PEAR (Lee et al., 2026), among others, assume the same conditions or their equivalents. LICQ holds for the box, simplex, and ramp constraints of our four tasks (Appendix B).


W5, Q1. Assumptions and polyhedral feasible sets

  • Where they come from: LICQ, strict complementarity, and the second-order condition are the standard assumptions of classical sensitivity analysis (Fiacco, 1976; Robinson, 1980). Section 4.1 now states the role of each condition.
  • How restrictive they are: OptNet showed that the solution of a QP is differentiable except on a measure-zero set of parameters. In our four tasks, LICQ and the reduced Hessian condition hold (Appendix B). Strict complementarity is an assumption. It fails only at degenerate solutions where a constraint is active and its multiplier is zero.
  • "What happens when it is not differentiable?" (original line 139): The question concerns the objective for a single outcome, ϕ(y,⋅)\phi(y,\cdot), which can have kinks, such as the hinge in CVaR or the shortfall penalty in Power Scheduling. At a fixed decision, however, the outcomes for which the decision sits on a kink have probability zero under a continuous distribution. The expected objective is therefore differentiable, and the field is well defined. The revised paper defines the field as the gradient of the expected objective (Eq. 3).
  • "theorem statements should give sufficient conditions" (original line 221): We agree. Theorem 2 of the original submission is Theorem 1 in the revised paper, and it now assumes only the regularity conditions and differentiability of the expected objective. The integrability and the interchange of differentiation and expectation in the original statement are needed only for the single-observation identity in Eq. (13). A sufficient condition for them is almost-sure differentiability together with an integrable local Lipschitz bound. We state it in Appendix A.2 and verify it for each task in Appendix B.
  • "Why only consider polyhedral feasible sets?" (original line 113): The restriction was only there to simplify the exposition, and the method applies to more general convex feasible sets. All four of our tasks have linear constraints, and with linear constraints the Hessian of the Lagrangian equals the Hessian of the objective, which makes the formulas simpler. The revised paper assumes a closed convex feasible set in Section 3.1 and smooth constraints near the solution in Section 4.1.

Q2. "what is the 'reduced Hessian'?" It is the Hessian of the Lagrangian restricted to the directions that keep the active constraints satisfied. Its positive definiteness means that the solution is a strict minimum along the feasible directions. The definition is now given in Section 4.1.

Q4. "does an orthonormal basis of ker⁡CQ\ker C_Q always exist?" Yes. The directions that keep the active constraints satisfied form a linear subspace, and every linear subspace has an orthonormal basis.

Q5. "does SQS_Q depend on the choice of orthonormal basis?" No. Any orthonormal basis gives the same matrix. The revised paper also gives a definition that does not use a basis: the primal block of the inverse KKT matrix (Eq. 7).

AuthorsOpenReview ↗

Response to Reviewer RfTE (Part 3/3): Experiments and remaining questions

W4. "Too many implementation details are pushed into the appendix"

  • "parameterized by neural networks": This is now stated in the first paragraph of Section 6.
  • "how the number of mixture components were chosen": We did not tune it. We used four components as the default in every task and increased it to eight only for Allocation, which is the most multimodal task because each of its ten coordinates is bimodal. The default is not always the best choice. In Power Scheduling, two components (3.02) do better than four (3.27). The conclusions of the paper do not depend on the number of components NN. Across the 16 combinations of four tasks and four values of NN (1, 2, 4, 8), Functional-DFL has better decision quality than every baseline in Table 1 in 15 (Table 9). The exception is Power Scheduling with one component (3.74, against 3.59 for Diffusion-DFL).
  • "why the authors only test GMM models": For Gaussian mixtures the field is available in closed form, so we can evaluate the method without sampling noise. Following Reviewer Z45Z's question (W2, Q3), we also ran Allocation with sampled fields estimated by Monte Carlo, and the method worked in that setting too. With 256 samples, the test regret was 0.328, close to 0.325 with the closed form (Table 10). We did not test other model families. Since the decision only needs an accurate field, the method does not need a model large enough to represent the whole distribution, and we see this as one of its advantages. A small Gaussian mixture outperformed Diffusion-DFL, which uses a diffusion model, on all four tasks (Table 1).

Q6. "why is SQS_Q called the 'local solution sensitivity operator'?" The term (original line 134) referred to the matrix that gives how the optimal solution moves when the objective gradient at the solution changes slightly. "Local" means that this holds only for changes small enough to keep the active set unchanged. The original submission introduced the name without deriving it. The revised paper first derives Eq. (7) from the KKT conditions in Section 4.1 and calls the matrix SθS_\theta there the sensitivity operator.

Q7. "Does 'Differentiating the KKT conditions' refer to the KKT conditions of equation (1)?" Yes, the KKT conditions of the downstream problem in Eq. (1), which is also Eq. (1) in the revised paper. The revised paper writes them out as Eq. (5), and Eq. (6) is their derivative with respect to the model parameters.

Q8. "why would you restrict the system to ker⁡CQ\ker C_Q?" Because the solution can move only in those directions. The second row of the KKT system requires any movement of the solution to keep the active constraints satisfied. Writing the movement in coordinates of those directions eliminates the multipliers and leaves a linear system in the reduced Hessian (Eq. 8). The revised paper presents the unrestricted block-inverse form first (Eq. 7).


Q9. "many different notations used for derivatives" The revised paper uses only three.

  • ∇z\nabla_z: the gradient with respect to the decision
  • ∇θ\nabla_\theta: the parameter gradient of a scalar such as the regret or a loss
  • ∂θ\partial_\theta: the Jacobian of a vector quantity such as the induced decision or the field

Q10. "The equality is sample-wise, and the equivalence with ℓreg\ell_{\mathrm{reg}} holds for the parameter gradient rather than the loss value" (original line 229). This sentence made two points at once.

  • The equality holds for each observed outcome, not only on average over outcomes.
  • The value of the loss differs from the regret, but at the current parameters its gradient equals the sample parameter gradient in Eq. (13).

Section 4.3 of the revised paper spells this out (Eq. 13), and Appendix A.2 states that the equality concerns gradients, not values.

Q12. "what does 'at the same weight' refer to? What was the original weight?" The phrase (original line 427) refers to the weight λ\lambda on the probe loss of Functional-DFL (the second term of Eq. 15). "+ NLL" replaces the probe loss with the negative log-likelihood and uses the same weight. The value for each task is given in Appendix C.4.

We did not tune λ\lambda either. We looked at the relative weight that the released code of Diffusion-DFL, one of our baselines, places on its auxiliary loss in each task, and chose λ\lambda from 0.1, 1, and 10 to match that order of magnitude. Mean–CVaR is not among the Diffusion-DFL tasks, so we used the same value as for Mean–Variance Portfolio. In Table 7, where λ\lambda takes the values 0.1, 0.5, 1, and 10, Functional-DFL is better than every baseline in Table 1 in 15 of the 16 combinations. The exception is Allocation with λ=0.1\lambda=0.1 (0.398, against 0.372 for Diffusion-DFL).


We hope this addresses your concerns. If you have further questions or would like to discuss any point, please let us know.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.