acceptodds
Under review as a conference paper at ICLR 2027

Probability Measure Gradient Flows for Diffusion-Based Amortized Optimization

Abstract

Nonconvex optimization problems are ubiquitous but can be computationally expensive to solve repeatedly. Recent work on amortized optimization accelerates repeated optimization by learning a mapping from problem parameters to their solutions. However, existing methods typically learn a deterministic solution mapping, limiting their ability to represent and explore multiple promising regions of a nonconvex solution landscape. In this work, we introduce DiFlowOpt, a self-supervised amortized optimization method that models solutions as a distribution parameterized by a diffusion model. We lift amortized optimization to the space of probability measures and study its optimization dynamics under Wasserstein and Fisher–Rao gradient flows. We show that these two geometries induce complementary gradient estimators for training diffusion-based amortized optimizers, leveraging first- and zeroth-order objective information, respectively. Motivated by this connection, DiFlowOpt updates the diffusion model along a Wasserstein–Fisher–Rao flow that combines both sources of information. To accommodate constraints, we further develop a primal-dual formulation in which instance-specific dual variables are learned by a neural network. Across three standard high-dimensional nonconvex optimization problems and two real-world applications, DiFlowOpt achieves better solution quality than classical solvers and existing amortized optimization methods, with substantially lower solve time than classical solvers.

Primary area:
optimization
Keywords:
Nonconvex Optimization, Diffusion Models, Amortized Optimization

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_rEGoOpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents a constrained-optimization solver built on diffusion models, driven by two gradient flows: the Wasserstein gradient flow and the Fisher-Rao gradient flow. Given a fixed diffusion model, the authors compute the update direction for the score field;. The paper also introduces a Dual Network to adaptively adjust penalty for constraints during training. Finally, this paper conduct experiments on several nonconvex problems and ablation studies on gradient flows.

Critical strengths
  1. The Wasserstein gradient flow and the Fisher-Rao gradient flow are applied to a diffusion-based constrained-optimization solver at the same time. This is a direction I had thought about and explored before, and the authors have completed it here.
  2. The writing is clear. The paper involves a lot of concepts, but they are strung together in a sensible order; the propositions come with proofs, and the implementation methods are efficient.
Critical weaknesses
  1. The experiments are not enough. As a constrained-optimization solver, the paper is missing some classical optimization problems. The Wasserstein gradient flow can also be disturbed by unbounded gradients. DiOpt [1] gives one example, concave quadratic optimization: its objective goes to negative infinity at infinity, and this may mislead methods that rely on reward gradients.
  2. The paper mentions a Self-Distillation step. Its role is to keep the velocity field canonical, or in other words, close to the correct score field. But the paper provides no experiment showing that this Self-Distillation is necessary. There are works in other fields that follow the same Fisher-Rao gradient flow paradigm, for example DiffusionNFT [2]; that paper has no Self-Distillation, and yet it still gets decent results in image post-training. (One can show that, when roughly one gradient descent step is used, the two-branch scheme in that paper is equivalent to first constructing a target direction from the gradient flow and then fitting it.)

[1] Ding, Shutong, et al. "Diffusion-Based Learning Framework for Constrained Nonconvex Optimization with Weighted Bootstrapped Refinement." Forty-Third International Conference on Machine Learning, 2026.

[2] Zheng, Kaiwen, et al. "DiffusionNFT: Online Diffusion Reinforcement with Forward Process." International Conference on Learning Representations, 2026.

Questions for authors
  1. If possible, could the authors run an experiment on concave quadratic optimization? If the equality constraints are indeed hard to satisfy, they could start with a problem that has only inequality constraints, such as box constraints, so that the original problem keeps a finite optimum.
  2. I would like the authors to add experimental evidence on whether Self-Distillation is necessary. For example, would training collapse without it?

If the authors are willing to address these two points, I would consider raising my score further.

4 comments

We sincerely thank the reviewer for the recognition of our contributions and for the constructive comments. We address each point below. We have updated the manuscript and marked the modified parts in blue.

Experiments on concave quadratic optimization

We thank the reviewer for this suggestion. We conducted additional experiments on the concave quadratic optimization problem

\min_{x\in \mathbb{R}^d} \quad & \frac{1}{2}x^\top Qx + p^\top x,\\\\ \text{s.t.} \quad & Ax-b \leq 0,\\\\ & -1 \leq x_i \leq 1,\quad i=1,\dots,d, \end{aligned}$$ where $Q$ is a diagonal matrix and satisfies $Q \preceq 0$. We use $d=100$ and the problem parameter is the vector $b$. The results are reported in the table below. | **Method** | **Obj.** ↓ | **Viol.** ↓ | |------------|:----------:|:-----------:| | IPOPT | -58.228 | 0.000 | | MS-IPOPT | -58.620 | 0.000 | | PDL\$^{*}\$ | N/A | N/A | | DiOpt | -50.469 | 0.000 | | **DiFlowOpt** | -51.163 | 0.000 | | **DiFlowOpt (Wasserstein flow only)**$^{\dagger}$ | -51.339 | 0.001 | $^{*}$PDL diverges during training. $^{\dagger}$Diverges in the late stage of training and test results are reported at the iteration with the best validation performance. PDL diverges during training, while both DiOpt and DiFlowOpt (WFR flow) converge to feasible solutions. The diffusion-based methods do not show advantage compared to the classical solver in this setting, since the objective is quadratic and does not have a rugged landscape, making the multimodal exploration from the diffusion model less necessary. We also evaluate DiFlowOpt with the Wasserstein flow only. At its best iteration it reaches an objective comparable to the WFR flow version, but training is unstable and eventually diverges. Making the Wasserstein flow version more stable would require extra tuning. From our empirical observations, whether training diverges on this problem depends on the specific experimental settings, such as the learning rate of the dual variables, i.e., whether the constraint term can grow fast enough to pull the solutions back before the gradient of the objective term becomes too large. PDL moves a single solution point based only on the gradient at that point, and is more likely to reach the region that leads to divergence quickly before the dual variables can catch up. We have added this experiment to Appendix D.8. > On the self-distillation We thank the reviewer for the insightful question and for drawing the connection with DiffusionNFT. We agree that DiffusionNFT can be viewed as following the Fisher-Rao flow, as also discussed in [1]. Empirically, we find that without the self-distillation stage, our method still achieves decent performance, but the objective is noticeably worse as the training becomes unstable after the first few iterations (Appendix D.1 and the table below). | Self-distillation | Obj. ↓ | Viol. ↓ | |-------------------|:---------------:|:-------------:| | w/ (default) | 10.533 ± 0.885 | 0.000 ± 0.000 | | w/o | 77.183 ± 25.512 | 0.000 ± 0.000 | One recent work [2] discusses that in the DiffusionNFT loss, the velocity field can also become noncanonical, which introduces a "forward-reverse consistency force". It also shows that the two-branch training cancels that term partially while the EMA-based off-policy training further stabilizes training with restoration. In contrast, our method is trained on-policy without EMA, so the self-distillation stage plays this stabilizing role and is necessary for the training. We believe that how to stabilize training in iterative reward-based diffusion training/fine-tuning is an important question, and we plan to conduct more ablation studies on both our method and DiffusionNFT to investigate it further. Thank you again for raising this point. We hope these additional results address the reviewer's concerns, and we are happy to provide further clarification if needed. [1] https://www.cs.utexas.edu/~lqiang/notes/taylor_diffusion_nft.pdf [2] Wan, Zhenglin, et al. "DiffusionART: Efficient, Stable and Scalable Reinforcement of Diffusion Models." arXiv preprint arXiv:2610.12572 (2026).

Thank you for your patient answers to my questions. However, there is still one remaining minor issue that I would like to discuss with you. I have conducted self-distillation experiments myself, and the results showed that discretization itself changes the output distribution (for example, few-step Euler discretization compresses the variance). In this case, repeated self-distillation may instead corrupt the original distribution and lead to instability. This seems to be a kind of inconsistency between theory and practice. I wonder whether you have encountered this problem, and how you addressed it. If you are willing to answer this question, I will raise my score.

Thank you for the insightful follow-up question. We also encountered this practical problem when implementing our algorithm.

In the literature, prior works also discussed that the iterative training on generated samples can lead to distribution corruption (e.g., variance shrinkage) and unstable training (due to the approximation error from learning with finite samples [1, 2] and the discretization error in diffusion sampling [3]).

Empirically, we find that this problem can be mitigated in a few ways. Under the same total self-distillation budget, it works better to run self-distillation as a separate periodic stage with enough steps in each stage and enough primal-dual training between stages, than to perform shorter self-distillation more frequently. Using enough steps makes the fitted distribution more accurate, while limiting the frequency allows the solution improvement to make sufficient progress between self-distillation stages. Another technique that can be helpful in some settings is adding Gaussian noise to the samples after diffusion sampling to compensate for the variance shrinkage. In our main experiments, we did not add this noise, since the variance shrinkage is not severe with the periodic self-distillation design and the method performs well without it.

We thank the reviewer again for the insightful and detailed question.

[1] Alemohammad, Sina, et al. "Self-consuming generative models go mad." International Conference on Learning Representations. Vol. 2024. 2024.

[2] Bertrand, Quentin, et al. "On the stability of iterative retraining of generative models on their own data." International Conference on Learning Representations. Vol. 2024. 2024.

[3] Sabour, Amirmojtaba, Sanja Fidler, and Karsten Kreis. "Align your steps: Optimizing sampling schedules in diffusion models." arXiv preprint arXiv:2404.14507 (2024).

I appreciate your patient reply, which helped clear up a question I had been struggling with for some time. On that basis, I personally consider this manuscript worthy of acceptance, and I have raised my score accordingly.

Reviewer_gyvROpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies self-supervised amortized optimization for parametric, nonconvex, constrained problems min⁡xfs(x)\min_x f_s(x) s.t. gs(x)≤0g_s(x) \le 0, hs(x)=0h_s(x) = 0. It lifts the problem to min⁡pEx∼p[fs(x)]\min_p \mathbb{E}_{x \sim p}[f_s(x)] over probability measures and represents pp with a conditional diffusion model. Training moves the model's distribution along a gradient flow in "optimization time" τ\tau. Prop. 1 writes the time derivative of the perturbed score, gτ,t=∂τ∇log⁡pτ,tg_{\tau,t} = \partial_\tau \nabla \log p_{\tau,t}, in terms of ∂τpτ\partial_\tau p_\tau. Props. 2-3 give estimators of gτ,tg_{\tau,t} under the Wasserstein flow (uses ∇f\nabla f) and the Fisher-Rao flow (uses only ff, through Cov(x0,f(x0)∣xt)\mathrm{Cov}(x_0, f(x_0) \mid x_t)). DiFlowOpt uses a linear combination of the two (Wasserstein-Fisher-Rao, WFR). The primal update regresses the network's score onto a stop-gradient target "current score +ρsg^+ \rho_s \hat{g}". A dual network λϕ(s)\lambda_\phi(s) predicts instance-wise multipliers for relaxed expected constraints and is trained by regression to a projected ascent target. A self-distillation step refits the model on its own samples.

Critical strengths
  1. A clean, simple way to train a diffusion model as an amortized optimizer, with a useful unifying view. Prop. 1 gives one recipe that turns a gradient flow on measures into a regression target for the score. The W and FR cases give a first-order and a zeroth-order estimator that are easy to compute from Tweedie quantities, and they combine linearly. Remark 4 links the FR case to KL-regularized fine-tuning. The Levy ablation (WFR 10.5 vs. FR 16.8 vs. W 29.3) supports the claim that combining both kinds of information helps. The method is easy to adopt and fully self-supervised (unlike DiOpt, which needs supervised pretraining to work well).

  2. Large gains over amortized baselines on rugged high-dimensional problems. Against PDL and DiOpt (same hidden-layer count; DiOpt also shares the denoiser and the 10 diffusion steps), DiFlowOpt is much better on Levy (10.5 vs. 516 / 4084) and Schwefel (150.6 vs. 395 / 418), with zero or small violation. The 2-D Levy example (Fig. 2) shows the expected exploration of several basins. If these results hold up under the clarifications below, this is a meaningful step for learned solvers of nonconvex problems.

Critical weaknesses
  1. The comparison with classical solvers is not convincing, and key experimental information is missing.

    • Classical solvers are capped at 200 iterations with uniform random starts, and MS-IPOPT uses only 10 starts. On Morse, SLSQP and IPOPT return objectives of about 2.4×1062.4 \times 10^6 and 1.0×1061.0 \times 10^6 (DE 2409, MS-IPOPT 72.4). This looks like non-converged runs from overlapping-atom starts, not the real ability of these solvers.
    • Standard global baselines (basin hopping, CMA-ES, simulated annealing at matched wall-clock) are missing. No best-known reference values are reported. For example, the constrained Levy optimum should be near 0, and long basin-hopping runs give references for Morse clusters. So we cannot tell how far any method is from optimal, and the claim "better solution quality than classical solvers" is weakly supported.
    • Training time, the number of training and test instances, and the training budget are not reported. Without these, the amortization trade-off cannot be judged. The 0.002 s inference (network on GPU) is compared with CPU solve time, but training cost is left out.
    • Feasibility is mixed. On trajectory optimization, DiFlowOpt has mean violation 0.805, while DE is feasible (0.000, objective 9.107), so "best combination" depends on how one trades objective against violation. Schwefel (0.008) and Morse (0.031) are also not feasible, while the solvers are near 0. The obstacle constraints are indicator functions with zero gradients, which handicaps SLSQP/IPOPT by construction. On Hartmann, MS-IPOPT is better (-1.903 vs. -1.877).
    • Hyperparameters are not given: WFR weights a,ba, b, ρs\rho_s, ρλ\rho_\lambda, step sizes, RR, NimpN_{\mathrm{imp}}, NsdN_{\mathrm{sd}}, batch/particle counts, and the δ\delta schedule. There is no sensitivity study. The std over 3 seeds appears only in the appendix and not for Table 2. This also hurts reproducibility.
  2. The theory is thin relative to the claims, and the main motivation is not tested.

    • Prop. 1 is differentiation under the integral. Prop. 3 is the known Tweedie covariance identity ∇xtE[f(x0)∣xt]=αtσt2Cov(x0,f(x0)∣xt)\nabla_{x_t} \mathbb{E}[f(x_0) \mid x_t] = \frac{\alpha_t}{\sigma_t^2} \mathrm{Cov}(x_0, f(x_0) \mid x_t). Prop. 5 is linearity. Prop. 2 is a useful but direct derivation.
    • "Unbiased" holds only with the exact posterior mean. In practice mτ,tm_{\tau,t} comes from the current, imperfect model, and self-distillation adds another approximation. There is no analysis of when the discretized WFR flow with a diffusion parametrization reaches a global optimum, or of how estimator variance grows with dimension.
    • Multimodality is the main motivation (abstract, Sec. 1), but the paper never measures whether the model keeps several modes during or after training, and inference uses a single sample.
    • Remark 4 says KL-regularized fine-tuning and mirror-descent training are FR discretizations, but no such method is compared (e.g., reward-weighted regression, DDPO-style policy gradient, adjoint matching, or the dual-trained constrained diffusion of Khalafi et al., 2024). DC3 (Donti et al., 2021) is also missing. These comparisons are needed to show that the gradient-flow view gives a better algorithm, not only a new description.
Questions for authors

Clarifications (could change my assessment):

  1. What is the training time per problem, how many training and test instances are used, and how many instances are needed to break even against MS-IPOPT?
  2. What are the values of a,b,ρs,ρλa, b, \rho_s, \rho_\lambda, the step sizes, RR, NimpN_{\mathrm{imp}}, NsdN_{\mathrm{sd}}, δ\delta, and the batch size? How were they chosen (validation set?), and how sensitive are results to a/ba/b?
  3. Why cap SLSQP/IPOPT at 200 iterations? What do they give with default convergence criteria, with a sensible Morse initialization (non-overlapping atoms), and with more starts at matched wall-clock?
  4. How is "Viol." defined (sum, max, or number of colliding points)? What is the objective after a simple feasibility repair, or under a fixed tolerance?
  5. In Eq. (22), how many x0x_0 samples per xtx_t are used to estimate g^\hat g, and how do you control the variance of the FR term in 100-D?

Experiments that would materially strengthen the paper:

  1. Best-known reference values (e.g., long basin-hopping or large-budget DE runs) and optimality gaps for all problems.
  2. Baselines: CMA-ES or basin hopping; KL-regularized fine-tuning / reward-weighted regression; DC3.
  3. Evidence of mode retention during training (diversity metrics, or best-of-KK vs. single sample).
5 comments

We thank the reviewer for the detailed comments. We address each concern below and have updated the manuscript with the changes marked in blue.

Comparison with classical solvers in more complete settings

We conducted additional experiments against the classical solvers under much larger computation budgets. We vary the maximum number of iterations from 10 to 5000 and the number of starts from 1 to 64 for IPOPT and DE (we focus on these two since they are the best-performing classical solvers in Tables 1 and 2). The time is measured on Intel Xeon Gold 5215 CPU, with each instance pinned to a single CPU core. For the multi-start versions, we set the maximum number of iterations per start to 200 for IPOPT and 3000 for DE, since the objective value decreases only negligibly with more iterations.

The results below confirm that the 200-iteration cap used in the main experiments does not disadvantage IPOPT. The objective obtained from IPOPT is nearly unchanged from 200 to 5000 iterations. We further observe that DiFlowOpt achieves the best Pareto performance. Even with 64 starts and over 8000 s of computation, DE remains well above the objective that DiFlowOpt reaches in 0.002 s.

MethodStartsMax iter.Obj.Viol.Time (s)
IPOPT1104855.3511.2550.76
1502135.060.0001.74
11002090.040.0003.25
12002089.560.0006.29
15002089.550.00013.14
110002089.550.00015.33
13000†2089.550.00015.91
150002089.550.00015.93
IPOPT12002089.560.0006.29
2200794.290.00011.80
4200590.380.00023.06
8200487.110.00045.60
16200415.220.00090.66
32200361.780.000181.09
64200278.820.000360.57
DE1105166.83153.5140.67
1504210.2043.3363.18
11003631.088.3736.31
12002460.520.03612.57
15001494.050.00431.35
11000†662.780.00262.81
13000245.250.000129.44
15000245.250.000131.96
DE13000245.250.000129.44
23000187.810.000253.93
43000129.600.000497.70
8300095.920.0001014.84
16300074.080.0002059.53
32300053.980.0004133.48
64300043.700.0008279.51
DiFlowOpt––10.530.0000.002

(† marks the solver's default termination setting.)

Details on the hyperparameters and experiment settings. How are the hyperparameters selected?

We use 5000 training, 5000 validation and 5000 test instances for each problem. The same hyperparameters are used on Levy, Schwefel, Hartmann and Morse: a=b=1a=b=1, radii ρs=2\rho_s=2 and ρλ=1\rho_\lambda=1, Adam step sizes ηimp=10−4\eta_{\mathrm{imp}}=10^{-4}, ηsd=10−3\eta_{\mathrm{sd}}=10^{-3} and ηϕ=10−3\eta_\phi=10^{-3}, R=5R=5, Nimp=100N_{\mathrm{imp}}=100, Nsd=1000N_{\mathrm{sd}}=1000, δ=0.01\delta=0.01, and batch size 512512. On TO, we use ρλ=10\rho_\lambda=10 and δ=0.003\delta=0.003 to place more emphasis on constraint handling. For standard hyperparameters such as the Adam step sizes and batch size, we use common settings in machine learning, and the remaining hyperparameters are selected based on the results on the validation set. These details have been added to Appendix C.5.

How sensitive are results to a/ba/b?

The performance is not sensitive to a/ba/b over a wide range. We have added a sensitivity analysis of the ratio a/ba/b between the Wasserstein and Fisher-Rao coefficients in Appendix D.1. In Figure 4, we show that the objective is nearly flat for a/ba/b in a wide range around 1 and increases only as the flow moves toward the pure Wasserstein or pure Fisher-Rao flow. The violation remains 0.0000.000 for all a/ba/b ratios tested.

How is "Viol." defined?

For every problem, Viol. of an instance is the sum of the constraint violations, ∑i[gi(x)]++∑j∣hj(x)∣\sum_i [g_i(\mathbf{x})]_+ + \sum_j |h_j(\mathbf{x})|, and the reported Viol. is averaged over the 5000 test instances.

What is the objective after a simple feasibility repair?

To evaluate how DiFlowOpt performs when zero violation is enforced with a simple feasibility repair stage, we apply up to 10 gradient steps on the constraints for the Schwefel problem. For the trajectory optimization (TO) problem, the constraint is defined through an indicator function and thus has zero gradient, so we instead warm start the DE solver with the solutions predicted by the diffusion model. The tables below show that the repair stage achieves zero violation on both problems. On Schwefel, the objective is almost unchanged. On TO, the objective increases from 5.9075.907 to 8.9848.984 but remains lower than that of DE (9.1079.107).

(a) Schwefel.

MethodObj. ↓Viol. ↓
DiFlowOpt w/o repair150.558 ± 14.7730.008 ± 0.004
DiFlowOpt w/ repair150.407 ± 14.8630.000 ± 0.000

(b) Trajectory optimization.

MethodObj. ↓Viol. ↓
DiFlowOpt w/o repair5.907 ± 0.4850.805 ± 0.074
DiFlowOpt w/ repair8.984 ± 0.1480.000 ± 0.000

How does estimator variance grow with dimension?

To examine how the problem dimension affects the estimator variance, we run DiFlowOpt on the Levy problem with dimension dd ranging from 2 to 200 and report the coefficient of variation σ/μ\sigma/\mu of the estimator. The results are averaged over diffusion timesteps and measured in the early stage of training. As shown in Appendix D.2, the coefficient of variation increases sublinearly with the problem dimension dd. The variance can be further reduced by using more particles. In this experiment we use 128 particles, as in the main experiments.

Analysis of multimodal exploration

To further demonstrate the multimodal exploration ability of DiFlowOpt, we track the number of basins covered by the sampled solutions during training and compare it with PDL on the same Levy2D problem as in Section 5.1. The number of basins is measured as the number of distinct basins that the sampled solutions for one instance fall into after refinement with a local solver (SLSQP). Figures 2 and 7 show that DiFlowOpt explores multiple basins simultaneously and converges to the optimal basin, whereas PDL explores only one basin at a time.

Remark 4 says KL-regularized fine-tuning and mirror-descent training are FR discretizations, but no such method is compared (e.g., reward-weighted regression)

We would like to point out that DiOpt, which we compare against in all the main experiments, is itself a reward-weighted regression method. Specifically, the training loss for DiOpt is L(θ):=Es,x,ϵ,t[ω(x;s)∥ϵ−ϵθ(xt,s,t)∥22],\mathcal{L}(\theta) := \mathbb{E}_{\boldsymbol{s},\boldsymbol{x},\boldsymbol{\epsilon},t}\left[\omega(\boldsymbol{x};\boldsymbol{s}) \left\Vert\boldsymbol{\epsilon}-\boldsymbol{\epsilon}_\theta\left(\boldsymbol{x}_t,\boldsymbol{s},t\right)\right\Vert_2^2\right], with ω(x;s)\omega(\boldsymbol{x};\boldsymbol{s}) the weighting function that depends on the objective function and the constraints (see Equations 10-11 in [1]). We have revised the paper to emphasize the connection between DiOpt and KL-regularized training methods.

Training time

We report the training time (in minutes) of DiFlowOpt and the learning-based baselines on a single NVIDIA L40S GPU in the table below. For DiFlowOpt, we report both the time for its default number of training iterations (as used in the main experiments) and the time it takes to first surpass the best result of the strongest learning-based baseline.

ProblemPDLDiOptDiFlowOpt (Full training)DiFlowOpt (To surpass baselines)
Levy6.9317.33770.8856.271
Schwefel3.6025.46231.6952.046
Hartmann2.4484.27519.5960.881

In Eq. (22), how many x0\mathbf{x}_0 samples per xt\mathbf{x}_t are used to estimate g^\hat{g}?

We use K=128K=128 samples in the main experiments. We have added an ablation on the Levy problem (table below), which shows that the algorithm is robust to this choice once KK is moderately large (e.g., K≥32K \geq 32), while too few samples increase the variance of the estimate.

KObj. ↓Viol. ↓
816.034 ± 7.3860.000 ± 0.000
3212.360 ± 0.6510.000 ± 0.000
128 (default)10.533 ± 0.8850.000 ± 0.000
51213.783 ± 3.9860.000 ± 0.000

[1] Ding, Shutong, et al. "Diffusion-based learning framework for Constrained Nonconvex Optimization with Weighted Bootstrapped Refinement." arXiv preprint arXiv:2502.10330 (2025).

Reviewer_BDiUOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2(+1)
Presentation
2
Soundness
2
Read review
Paper summary

DiFlowOpt learns a conditional diffusion model for solving nonconvex constrained problems. The paper studies Wasserstein and Fisher-Rao gradient flows and combines them to derive training objectives of diffusion model that use first-/zeroth-order objective information. The authors also develop a primal-dual formulation and use a network that predicts constraint multipliers.

Critical strengths
  • The theory translates Wasserstein and Fisher-Rao flows into explicit score update, connecting distribution-level optimization to training of diffusion model.

  • In the reported Levy experiments, the proposed method outperforms either flow used alone, supporting the use of both objective values and gradients.

Critical weaknesses
  • Because λih\lambda_i^h is unrestricted, equation (21) enforces Ep[∣hi(x)∣]=δ\mathbb E_p[|h_i(x)|]=\delta rather than the stated inequality. These constraints coincide at δ=0\delta=0, but for δ>0\delta>0 the formulation excludes even distributions with zero violation.

  • The PDL baseline uses a standard Lagrangian instead of its augmented Lagrangian objective. The comparison therefore does not establish superiority over the original PDL method.

  • DiOpt is trained without its supervised warm-start. This is a valid label-free comparison, but the baseline should be identified as a restricted variant of DiOpt.

  • In trajectory optimization, DiFlowOpt achieves a lower objective with nonzero constraint violation, whereas differential evolution returns feasible solutions. The results therefore do not directly imply better solution quality at comparable feasibility.

Questions for authors
  • Which values of δ\delta and which multiplier projections are used in the implementation? If this can be satisfied the aforementioned weakness, how can equations (21)--(24) be corrected for positive δ\delta?

  • In [Li and He, 2026], the authors show that xx-prediction, which directly predicts the data rather than the noise, can improve generative performance. Could this choice also improve amortized optimization or the proposed method?

  • How do posterior-mean error, the score-update stepsize, and self-distillation affect agreement with the intended flow? A controlled example with a known evolving distribution would help to understand this approximation.

  • In my opinion, reporting learning rates, training steps, test-instance counts, and total training time would be beneficial. Inference latency alone does not measure the total cost of amortization.

3 comments

We greatly appreciate your valuable feedback. Please find our responses to your comments below. We have updated the manuscript and marked the modified parts in blue.

Because λih\lambda_i^h is unrestricted, equation (21) enforces Ep[∣hi(x)∣]=δ\mathbb{E}_p[|h_i(x)|] = \delta rather than the stated inequality. These constraints coincide at δ=0\delta = 0, but for δ>0\delta > 0 the formulation excludes even distributions with zero violation.

Thank you for your careful reading of our formulation. This is indeed a typo in the paper, where λih\lambda_i^h should be constrained to be nonnegative, i.e., λih≥0\lambda_i^h \geq 0, so that Equation (21) enforces the inequality Ep[∣hi(x)∣]≤δ\mathbb{E}_p[|h_i(\mathbf{x})|] \leq \delta as intended. We have corrected this in the revised manuscript. Our implementation already enforces λih≥0\lambda_i^h \geq 0 and the experimental results are unaffected.

The PDL baseline uses a standard Lagrangian instead of its augmented Lagrangian objective. The comparison therefore does not establish superiority over the original PDL method.

Thank you for raising this point about the experimental setting. In our implementation, we use the standard Lagrangian for both PDL and DiFlowOpt to keep the constraint handling approach aligned and focus on the effect of diffusion-based learning versus point-based neural network learning.

To make the results more complete, we also evaluate both methods with the Augmented Lagrangian Method (ALM). The formulation of ALM is kept the same as the one used in PDL [1]. As shown in the table below, DiFlowOpt shows a clear advantage over PDL in both the standard and augmented Lagrangian settings. ALM does not change the performance of PDL much, since the bottleneck for PDL is not constraint handling, but rather the highly non-convex landscape of the objective. We further observe that ALM enables better constraint handling for our DiFlowOpt algorithm. On the Schwefel problem, ALM reduces the constraint violation of DiFlowOpt from 0.0080.008 to 0.0000.000 while increasing the objective value only slightly. We will complete the ALM experiments on the remaining problems and add the results to the paper.

MethodLevy Obj.Levy Viol.Schwefel Obj.Schwefel Viol.Hartmann Obj.Hartmann Viol.
PDL516.255 ± 14.3580.000 ± 0.000394.617 ± 0.2080.000 ± 0.000-1.213 ± 0.0000.000 ± 0.000
PDL (ALM)515.679 ± 15.1730.000 ± 0.000394.329 ± 1.7500.000 ± 0.000-1.213 ± 0.0000.000 ± 0.000
DiFlowOpt10.533 ± 0.8850.000 ± 0.000150.558 ± 14.7730.008 ± 0.004-1.877 ± 0.0000.000 ± 0.000
DiFlowOpt (ALM)11.275 ± 1.0160.000 ± 0.000168.320 ± 13.3420.000 ± 0.000-1.877 ± 0.0000.000 ± 0.000

DiOpt is trained without its supervised warm-start. This is a valid label-free comparison, but the baseline should be identified as a restricted variant of DiOpt.

Thank you for your careful observation regarding the baseline setting. We have revised the paper to clarify this setting in Table 1.

Since our paper focuses on self-supervised amortized optimization, we only keep the self-supervised learning stage of DiOpt for a clear comparison. To further investigate the performance of DiOpt on the high-dimensional highly nonconvex optimization problems we evaluate, we conduct additional experiments on Levy 100D, where DiOpt training is warm-started with solutions obtained from the IPOPT solver under the default criteria (maximum iterations = 3000), following the practice in DiOpt [2].

MethodObj.Viol.
DiOpt (w/o warm start)4084.261 ± 176.6420.949 ± 0.852
DiOpt (w/ warm start)216.105 ± 60.1880.000 ± 0.000
DiFlowOpt (ours)10.533 ± 0.8850.000 ± 0.000

We observe that the warm start can improve the performance for DiOpt, but DiOpt still underperforms DiFlowOpt (trained from scratch), likely because the solutions from classical solvers are not optimal on these highly nonconvex problems. Getting near-optimal solutions from the classical solvers would require impractically high computational cost. For fair comparison, we did not use warm start for either method in the main experiments but we expect the benefit of DiFlowOpt from leveraging both zeroth- and first-order information will also apply to the warm start cases.

In trajectory optimization, DiFlowOpt achieves a lower objective with nonzero constraint violation, whereas differential evolution returns feasible solutions. The results therefore do not directly imply better solution quality at comparable feasibility.

We thank the reviewer for the observation on the trajectory optimization experiment results. We conducted additional experiments applying feasibility repair to the sampled solutions from DiFlowOpt, as shown in Appendix D.6. With repair, DiFlowOpt achieves an objective of 8.984±0.1488.984 \pm 0.148 with 0.000±0.0000.000 \pm 0.000 violation compared to DE's objective of 9.1079.107, while requiring much less average total inference time (including the inference time of the diffusion model and the feasibility repair stage). We also note that the constraint enforcement can be further enhanced by using the ALM formulation as shown in the results above, and DiFlowOpt (without feasibility repair) is expected to achieve lower violation in that setting than with the current standard Lagrangian formulation.

Which values of δ\delta and which multiplier projections are used in the implementation? If this can be satisfied the aforementioned weakness, how can equations (21)--(24) be corrected for positive δ\delta?

We use δ=0.01\delta=0.01 for Levy, Schwefel, Hartmann, and Morse, and δ=0.003\delta=0.003 for TO. The multipliers for both the inequality and equality constraints are projected to be nonnegative, i.e., λig≥0\lambda_i^g \geq 0 and λih≥0\lambda_i^h \geq 0. We have fixed the typo in the revised manuscript, as described in our response above.

In [Li and He, 2026], the authors show that xx-prediction, which directly predicts the data rather than the noise, can improve generative performance. Could this choice also improve amortized optimization or the proposed method?

Thank you for the constructive suggestion. We conducted additional experiments on the Levy problem and found that x\mathbf{x}-prediction can indeed further boost the performance of DiFlowOpt. We have added this result to Appendix D.1.

ParametrizationObj. ↓Viol. ↓
ε (used in the paper)10.533 ± 0.8850.000 ± 0.000
x₀5.013 ± 0.5870.000 ± 0.000

We will run larger scale experiments on the other problems to confirm this finding and will update the paper to include the complete x\mathbf{x}-prediction results.

Additional ablation studies

We conduct additional ablation studies on the number of posterior samples KK, the score-update step size ρs\rho_s, and self-distillation on the Levy problem. As shown in the tables below, our method is robust to the number of posterior samples once KK is moderately large (since KK affects the variance of the posterior-mean estimate), and to the score-update step size as long as it is not too large (an overly large step size leads to unstable updates). In practice, the step size can be selected based on the norm of g^τ,t\hat{g}_{\tau, t} relative to that of the current score function. We also observe that self-distillation is crucial for stabilizing the training of DiFlowOpt.

(a) Number of particles KK

KObj. ↓Viol. ↓
816.034 ± 7.3860.000 ± 0.000
3212.360 ± 0.6510.000 ± 0.000
128 (default)10.533 ± 0.8850.000 ± 0.000
51213.783 ± 3.9860.000 ± 0.000

(b) Score-update step size

Step sizeObj.Viol.
114.753 ± 0.9670.000 ± 0.000
2 (default)10.533 ± 0.8850.000 ± 0.000
410.717 ± 0.8790.000 ± 0.000
81745.718 ± 650.3444.843 ± 5.342

(c) Self-distillation

Self-distillationObj. ↓Viol. ↓
w/ (default)10.533 ± 0.8850.000 ± 0.000
w/o77.183 ± 25.5120.000 ± 0.000

Reporting learning rates, training steps, test-instance counts, and total training time would be beneficial.

We thank the reviewer for the feedback. Reporting these details is indeed helpful for the reproducibility of the paper. We have updated the manuscript to include this information.

[1] Park, Seonho, and Pascal Van Hentenryck. "Self-supervised primal-dual learning for constrained optimization." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 37. No. 4. 2023.

[2] Ding, Shutong, et al. "Diffusion-based learning framework for Constrained Nonconvex Optimization with Weighted Bootstrapped Refinement." arXiv preprint arXiv:2502.10330 (2025).

Thank you for the detailed response. The correction to the multiplier formulation and the additional baseline comparisons address my main concerns. Therefore, I will raise my score to 3.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.