- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents a diffusion-based decision-focused learning framework equipped with a novel cost-weighted cross-entropy loss. In addition to the standard diffusion model loss, this new loss aligns the normalized expected cost profile with the learned distribution. While the method achieves substantial experimental gains, comparisons against several foundational DFL baselines are absent.
Critical strengths
- The proposed method is technically sound: it introduces an auxiliary loss, alongside the diffusion model loss, to enforce alignment for decision-making tasks.
- Empirical results demonstrate considerable performance improvements over existing diffusion-based DFL methods.
Critical weaknesses
- Notation requires clarification. For instance, in lines 48–49 of the introduction, symbols appear without prior definition, which may confuse readers. The meaning of is also left unspecified in line 395.
- Baseline comparisons are limited primarily to diffusion-based DFL approaches, omitting other relevant baselines. Although the paper’s core contribution targets improved alignment for diffusion-based solvers, comprehensive comparisons against classic DFL methods (e.g., [1,2]) remain essential.
- The related work discussion appears insufficient.
[1] S. Shah, K. Wang, B. Wilder, A. Perrault, and M. Tambe. Decision-focused learning without decision-making: Learning locally optimized decision losses. (NeurIPS), 2022.
[2] J. Mandi, P. J. Stuckey, T. Guns, et al. Smart predict-and-optimize for hard combinatorial optimization problems. (AAAI), 2020.
Questions for authors
- Scope of applicability. Section 2 formulates the optimization problem in the discrete setting. Can the proposed method extend to continuous settings? Are the challenges outlined in the paper exclusive to discrete problems? If yes, this distinction should be explicitly stated in the introduction (and also title).
3 comments
Q1. Scope of applicability. Section 2 formulates the optimization problem in the discrete setting. Can the proposed method extend to continuous settings? Are the challenges outlined in the paper exclusive to discrete problems? If yes, this distinction should be explicitly stated in the introduction (and also title).
AW1. We thank the reviewer for raising this important question. We clarify that CWCE is not inherently restricted to discrete decision spaces and can also be extended to continuous optimization problems. Although our current formulation and experiments focus on discrete optimization, the underlying principle of aligning decision-relevant expected-cost profiles is more broadly applicable.
Extension to continuous decision spaces. The main difference is that a continuous decision space generally contains infinitely many feasible decisions, making it impractical to compute CWCE over the entire decision space. However, a similar computational challenge arises in large-scale discrete combinatorial optimization problems, where enumerating all feasible decisions is also prohibitively expensive.
To address this challenge, we can adopt the same finite candidate-set approximation already introduced in our paper (Lines 287–294), following the candidate-based strategies used in Mandi et al. (2022) and Wang et al. (2026). Specifically, we construct a finite candidate set and evaluate CWCE over instead of the entire decision space.
We initialize the candidate set using the optimal decisions associated with observed training outcomes:
z^{\star}(y_i)\in\arg\min_{z\in\mathcal{Z}}f(y_i,z).$$ During training, we additionally draw $M$ samples $\\{y_i^{(m)}\\}_{m=1}^{M}\sim p_{\theta}(\cdot\mid x_i)$ and, with probability $p_{\mathrm{solve}}=0.1$, compute $$\widehat z(x_i)\in\arg\min_{z\in\mathcal{Z}} \frac{1}{M}\sum_{m=1}^{M}f(y_i^{(m)},z).$$ We then update $\mathcal{S}\leftarrow\mathcal{S}\cup\{\widehat z(x_i)\}$, allowing the candidate set to adapt to decisions that become relevant under the evolving learned distribution. This procedure avoids exhaustive enumeration of $\mathcal{Z}$ and is applicable to both discrete and continuous optimization problems, provided that the corresponding downstream optimization problems can be solved. Thus, the finite candidate-set formulation offers a practical way to extend CWCE to continuous decision spaces, although its approximation quality depends on the coverage of decision-relevant candidates. **Challenges in discrete and continuous settings.** The fundamental challenges motivating our work are not exclusive to discrete optimization. First, distributional fidelity does not necessarily guarantee downstream decision quality, as even small distributional errors can alter the expected-cost structure and induce suboptimal decisions. Second, for general nonlinear stochastic objectives, deterministic point predictions may fail to capture the distributional information required for optimal decision-making. Both challenges arise regardless of whether the decision space is discrete or continuous. However, discrete optimization presents an additional difficulty: the downstream argmin mapping is generally non-differentiable, making direct gradient propagation through the optimizer particularly challenging. Although differentiability issues may also arise in continuous optimization, continuous problems can admit implicit differentiation or other gradient-based approaches under suitable regularity conditions. CWCE avoids differentiating through the downstream optimizer by directly aligning the expected-cost profiles induced by the learned distribution. **Clarification of scope.** Therefore, our focus on discrete optimization is motivated by the particular difficulty of incorporating decision-aware supervision through non-differentiable downstream optimization, rather than by the assumption that distributional misalignment is exclusive to discrete problems. We have clarified this distinction in the Introduction. The current title, *Cost-Weighted Cross-Entropy for Diffusion-Based Decision-Focused Learning*, does not restrict the proposed framework to discrete optimization and therefore remains appropriate.Dear Reviewer,
Thank you again for your constructive feedback. We have addressed your concerns with additional clarifications and experimental results in our rebuttal. As the discussion period is approaching its end, we would greatly appreciate it if you could review our responses and let us know whether they adequately address your concerns.
Thank you for your time and consideration.
W1. Notation requires clarification. For instance, in lines 48–49 of the introduction, symbols appear without prior definition, which may confuse readers. The meaning of is also left unspecified in line 395.
AW1. We thank the reviewer for pointing out the unclear notation. We agree that introducing mathematical symbols without prior definitions in Introduction Section may affect readability. To address this concern, we have revised Lines 48–49 to explain the distinction between deterministic and distribution-aware decision-making in words, without introducing unnecessary mathematical notation. Specifically, we clarify that, for certain structured objectives, such as those that are linear in the decision variables, conditional expected costs depend only on specific conditional expectations, whereas more general nonlinear stochastic objectives may require additional distributional information. This revision improves readability while preserving the original motivation. We clarify that nt denotes the number of test samples, and Dt=(xi,yi)i=1nt denotes the test set.
W2. Baseline comparisons are limited primarily to diffusion-based DFL approaches, omitting other relevant baselines. Although the paper’s core contribution targets improved alignment for diffusion-based solvers, comprehensive comparisons against classic DFL methods (e.g., [1,2]) remain essential.
AW2. We thank the reviewer for emphasizing the importance of comparisons with classical DFL methods. We would first like to clarify that our original experiments already include several representative deterministic DFL baselines, namely Pointwise Ranking, Listwise Ranking, and the Quadratic surrogate proposed by Shah et al. [1], in addition to PtO and the generative DFL methods Diffusion-DFL and Gen-DFL. Therefore, our evaluation is not limited to diffusion-based DFL approaches.
To further strengthen the experimental comparison, we have additionally included SPO+, proposed by Mandi et al. [2], as suggested by the reviewer. The new results are reported below.
These additional results demonstrate that CWCE consistently outperforms both the Quadratic surrogate [1] and SPO+ [2] across all three tasks, achieving higher normalized decision quality (nDQ) and lower regret. Together with our existing comparisons, these results provide a more comprehensive evaluation against classical deterministic and recent generative DFL methods.
W3. The related work discussion appears insufficient.
AW3. We thank the reviewer for this valuable suggestion. We agree that a more systematic discussion of related work would help better contextualize our contributions. To address this concern, we have added a dedicated Related Work section, organized into two main categories: deterministic DFL and generative DFL.
For deterministic DFL, we provide a more comprehensive discussion of differentiable optimization approaches and decision-focused surrogate losses, including SPO+, locally optimized surrogates, and pointwise/listwise ranking-based methods. For generative DFL, we expand the discussion of recent approaches, particularly Gen-DFL and Diffusion-DFL, and clarify their methodological differences from CWCE in terms of decision-aware supervision and the treatment of downstream discrete optimization.
We have also streamlined the Introduction to avoid redundant discussions of individual methods. These revisions provide a more comprehensive overview of existing research and more clearly position CWCE within the DFL literature.