acceptodds
Under review as a conference paper at ICLR 2027

Constrained Diffusion Models with Primal-Dual Inference

Abstract

This paper develops constrained diffusion models with primal-dual inference (PDI) to sample from optimal distributions of entropy-regularized optimization problems with average constraints. We formalize constrained sampling in the Lagrangian dual domain, where the optimal distribution takes the form of a Gibbs distribution indexed by the optimal dual variable. Rather than estimating this dual multiplier before sampling and freezing it throughout generation, PDI jointly infers the optimal primal distribution and its parametrizing dual variable. Each reverse diffusion step denoises using the score field associated with the current multiplier and then updates the multiplier through dual ascent using the estimated constraint violation of the denoised samples. To enable this conditional score field, we train a single dual-conditioned score network over the family of Gibbs distributions induced by the dual variables encountered during inference. We prove that the dual iterates converge to a neighborhood of the dual optimum, and that the generated distribution approaches that of an oracle sampler that knows the optimal multiplier, up to a bias floor. We evaluate PDI on constrained sampling from a mixture of Gaussians, wireless resource allocation, and portfolio management.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
constrained diffusion models, sampling, constrained sampling, primal-dual algorithms

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_5JGpOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes constrained diffusion models with primal-dual inference (PDI) as a method for sampling the solutions of entropy-regularised optimisation problems with average constraints. Under the Lagrangian formulation, the target is a Gibbs distribution indexed by an unknown optimal dual variable. Instead of first estimating the dual multiplier and then sampling the corresponding fixed target (using a diffusion), PDI runs a single reverse diffusion with a score network conditioned on the dual variable λ\lambda, whilst updating λ\lambda by projected dual ascent after each denoising step. The theoretical analysis presents a last iterate bound for the dual iterates, and an O(T−12)O(T^{-\frac{1}{2}}) KL comparison between PDI and an oracle sampler that knows the optimal Lagrange multiplier (up to a bias floor). The proposed method is evaluated on a constrained mixture of Gaussians, wireless resource allocation, and portfolio optimisation.

Critical strengths
  • The idea of conditioning a single score model on the dual variable, and then updating that variable during the reverse diffusion trajectory, is conceptually clean and practically useful. In particular, this approach avoids having to retrain a score model each time the dual variable changes.
  • The numerical section includes several useful analyses and ablations, including frozen-multiplier variants, alternative noise schedules, and shifted constraint levels. These help to separate the various mechanisms contributing to the method's overall performance.
Critical weaknesses
  • My main concern relates to the theoretical guarantees. In particular, the derivation of Theorem 2 (see, e.g. Eq (50)-(51), proof of Lemma 4) appears to rely on the assumption that CΨ(T)C_{\Psi}(T) is bounded, uniformly over TT. However, under the bτ=O(1/T)b_\tau = O(1/T) refinement (see Assumption 6), the reverse steps become increasingly close to the identity, which would imply that CΨ(T)C_{\Psi}(T) must grow at least quadratically for any fixed non-degenerate target. If "bounded" here just means finite for each fixed TT, the dependence on TT should be retained in the subsequent bounds. In any case, I do not see how the stated rate in Theorem 2 holds.
  • There are also a few issues in the numerical experiments (and their descriptions) that make the results difficult to assess. In particular, PDL is described as enforcing constraints pointwise, even though (as described in the original paper) it targets average constraints; the Gaussian mixture comparison appears to use different temperatures for PDL and PDI; and the portfolio experiment reports an effective number-of-assets violation for the equally weighted portfolio that is inconsistent with the stated minimum-diversification constraint. Further, Tables 7 and 8 appear to report very different inverse temperatures for PDI-{MC,Net}, making it unclear whether these variants target the same distribution.
Questions for authors
  • Is CΨ(T)C_{\Psi}(T) in Assumption 6 meant to be bounded uniformly in TT? Under the stated bτ=O(1/T)b_\tau = O(1/T) refinement, the reverse maps become increasingly close to the identity. This would appear to imply that CΨ(T)=Ω(T2)C_{\Psi}(T) = \Omega(T^2) for any fixed, non-degenerate target. If "bounded" here only means finite for each fixed TT, how should the TT-dependence of Theorem 2 be modified?
  • How are "mode-coverage", "average-constraint feasibility" (Figure 2), and "feasibility percentage" (Table 1) defined in Sections 5.1-5.2?
  • Why is PDL described in Sections 5.1-5.2 as enforcing pointwise constraints? PDL is formulated for constraints in expectation, while the update in Equations (122)-(123) appears to be a stochastic approximation to an expectation-based dual update (i.e., not a per-sample constraint). Could the authors clarify?
  • What temperature is used for PDL in the Gaussian-mixture experiments? Equation (122) corresponds to unit-temperature Langevin dynamics, while Table 3 reports β−1=50\beta^{-1} = 50 for PDI. Is there an additional β\beta-dependent rescaling, and are the methods being compared for the same entropy-regularised target?
  • How is the effective number-of-assets violation for the equal-weight portfolio in Table 2 being computed? Under Equations (130)-(131), equal weights give Neff=N=500N_{\mathrm{eff}}=N=500, so shouldn't we expect zero violation of any feasible minimum NeffN_{\mathrm{eff}} constraint? The reported violation of 440 seems to suggest that the implemented constraint differs from Equation (130).
  • Table 7 reports β−1=2000\beta^{-1} = 2000 for PDI-MC, while Table 8 reports β−1=1\beta^{-1} = 1 for PDI-Net training and inference. Were PDI-MC and PDI-Net in Table 2 run at different temperatures? In this case, they target different distributions, so it makes the comparison somewhat difficult to interpret.
  • Could the authors provide a low-dimensional example where λ∗\lambda^{*} and the corresponding Gibbs target can be computed accurately? Such a reference would be quite helpful for illustrating that the method is indeed approximating the intended optimal distribution.
  • The abstract contrasts PDI with approaches that first estimate the optimal multiplier and then keep it fixed during sampling. Could the authors include this baseline, using the same λ\lambda-conditioned network? E.g., estimate λ∗\lambda^* via an outer loop of full reverse-diffusion runs with dual ascent, before sampling with λ\lambda fixed. This would be useful for isolateing the benefit of updating the multiplier in the reverse diffusion trajectory.
3 comments

We thank the reviewer for a technically precise review. Its main point is correct, and it has greatly improved the paper. The general response describes the revised theory (G1) and the exactly solvable testbed the reviewer asked for (G2). We answer each point below.

  1. CΨ(T)C_\Psi(T) is not bounded uniformly in TT. The reviewer is absolutely right, and we thank them for this meticulous observation. With per-step noise variances of order 1/T1/T, every reverse map is within O(1/T)O(1/T) of the identity. The stability products Ψj,t\Psi_{j,t} of original Lemma 4 then stay bounded away from zero, CΨ(T)C_\Psi(T) grows at least like T2T^2, and the O(T−1/2)O(T^{-1/2}) rate of the original Theorem 2 does not follow. We withdraw it. Since the stability factors are worst-case Lipschitz constants that can exceed one near the data, we replaced the argument rather than patching it.
  • The revised analysis (Section 4 and Appendix C) follows the law of the samples instead of their trajectories. The key observation is that two laws that pass through the same exact reverse kernel cannot move apart in KL, so a reverse step never expands the lag of the samples behind the target. Moreover, under sufficient noise corruption, it shrinks the lag by the factor 1−ωt=exp⁡(−∫stst+1ϱ(s) ds)1-\omega_t=\exp\big(-\int_{s_t}^{s_{t+1}}\varrho(s)\,ds\big), which depends only on the range of log-SNR ss that the step covers (Lemma C.2). Here, ϱ\varrho, and thus ω\omega depends on the LSI, or in simpler cases the strong log-concavity factor, of the noisy data marginal. As TT grows, each factor tends to one, as the reviewer observes, but their product over any stretch of the reverse process does not depend on the number of steps. With the speed condition of Proposition 3 on the dual updates, which ties each dual step to ωt\omega_t, no term of Theorem 1 grows with TT.

  • We chose the symbol γt\gamma_t for the weights of Theorem 1 deliberately. In the submission, a multiplier error at step jj reached the samples through the factor Ψj,t−1γj\Psi_{j,t-1}\gamma_j in Eq.(46), damped by the stability of the later steps. The weights γt=mt∏s>t(1−ms)\gamma_t=m_t\prod_{s>t}(1-m_s) keep this idea. They discount the error of each step by the contraction of all later steps, but these factors never exceed one, and the weights add up to at most one for every TT.

  • On the lattice version of the testbed, where every quantity is exact, all inequalities of Appendix C hold over 420 runs (Appendix I.6 and Table 18).

  1. Definitions. In the submission, average-constraint feasibility is the fraction of constraints whose ensemble mean satisfies the constraint up to a tolerance. In the MoG experiment, mode coverage was the fraction of the twelve mixture modes that received at least one sample, with each sample assigned to its most likely mode under the unconstrained mixture. It recorded only which modes are visited, not how much mass each received. We no longer report it. Table 1 now compares each method with exact samples of μ⋆\mu^\star directly. It reports the regularized objective Ef0−βH^\mathbb{E}f_0-\beta\widehat H, where H^\widehat H is a nearest-neighbor estimate of the entropy, and the total-variation distance between the fractions of samples in each mode and the mode weights of μ⋆\mu^\star (Appendix E). In the wireless experiment, feasibility is the percentage of users who achieved the minimum rate requirement.
  1. Description of the PDL baseline. The reviewer is correct that PDL (Chamon et al., 2024) enforces the constraints on average. Its Algorithm 1 updates the multiplier with a one-sample stochastic gradient, i.e., the constraint value f(xt)f(x_t) of the current iterate (Eq. (68)), and its convergence guarantees are for the laws of the iterates averaged over iterations. Our baseline ran Algorithm 1 independently for each sample, kept the terminal iterate of each chain with its own multiplier. Since each multiplier remains bounded, each chain satisfies the constraints on average along its own trajectory. On a multimodal target, a chain of 500 iterations stays in one mode.Thus, in the terminal distribution every chain is feasible on its own rather than only the ensemble.
  • To see how this choice affects the objective and feasibility, we now report two variants of PDL in the mixture-of-Gaussians experiment (Table 1). The first uses one multiplier vector per chain, updated with the one-sample stochastic gradient f(xt)f(x_t) of Algorithm 1. The second evolves a batch of samples with a shared multiplier vector and updates it using the constraint values averaged over the batch, Ebatch[f(x)]\mathbb{E}_{\mathrm{batch}}[f(x)], as in (PDI-D). In the former, PDL satisfies all ten constraints on average. The largest mean constraint value is −0.0004-0.0004, just below the feasibility threshold of zero. The objective, however, is 50.7750.77, compared with 32.7032.70 under the optimal distribution μ⋆\mu^\star. With a shared multiplier, the objective improves to 35.5435.54, reducing the objective gap from 18.0718.07 to 2.842.84, while the largest mean constraint value increases to 0.01480.0148.

  • The main difference in this experiment is therefore in the objective. The per-chain updates achieve feasibility on average, but at a substantially higher objective. Sharing the multiplier allows a sample to violate a constraint when other samples compensate for it, as the average-constrained problem permits. Here, this yields an objective much closer to that of μ⋆\mu^\star, with a small remaining violation.

  • Our observation about pointwise feasibility concerns the terminal samples of independent chains of finite length, each with its own multiplier. It does not contradict the paper's own claims regarding averages across iterations.

  1. Matched temperature. The reviewer is correct. The PDL update in Eq. (122) of the submission omitted the temperature parameter β\beta. This omission affected the mixture-of-Gaussians experiment, which we repeated using the corrected update. The results are reported in Table 1. All other experiments used β−1=1\beta^{-1}=1, so no changes were needed.

  2. The equal weight NeffN_{\mathrm{eff}} violation. Thanks for spotting this typo. The constrained problem without the effective-number constraints has a deterministic solution, which is the equal-weight portfolio. That is, μ⋆\mu^\star is concentrated around this single point. The effective-number-of-assets constraint is meant to exclude this point by forcing a maximum number of effective assets. Eq. (130) in the original submission had the inequality sign reversed and it is now fixed. The problem forces a maximum number of assets Neff=60N_{eff} = 60, which matches the reported violation of the equal-weight portfolio.

  3. Temperatures in Tables 7 and 8. Tables 12 and 13 of the revision now report the correct hyperparameters. The baseline-hyperparameter table in the original submission (with β−1=2000\beta^{-1}=2000) had stale values from earlier experiments. The reported results of all methods were at β−1=1\beta^{-1}=1 and we were able to reproduce them.

  1. A low-dimensional example with accurately computed λ⋆\lambda^\star and Gibbs target. Done. We now report a comparison with μ⋆\mu^\star for the mixture-of-Gaussians (MoG), wireless resource allocation, and image generation experiments. We obtain the reference differently in each case.
  • Mixture of Gaussians (Table 1). The known mixture and constraints allow us to compute the optimal tilted distribution at λ⋆\lambda^\star and draw reference samples directly from μ⋆\mu^\star. We can therefore compare both the objective and the fraction of samples assigned to each mode. PDI-Net achieves an objective of 32.8532.85, compared with 32.7032.70 for samples from μ⋆\mu^\star. The total variation distance between the mode frequencies of PDI-Net and the optimal mode probabilities is 0.0240.024. For comparison, the same distance is 0.0190.019 for a finite sample drawn directly from μ⋆\mu^\star. Thus, PDI-Net recovers both the objective and the allocation of probability across modes to nearly the accuracy observed with direct sampling.

  • Wireless resource allocation (Figure 2). Here we use nested dual ascent (Section 3.1) to obtain a numerical reference for each of the 6464 test networks. Using the score model trained with KMC=512K_{\mathrm{MC}}=512, we hold the multiplier fixed for a full denoising pass, update it using the average constraint violations, and repeat. PDI approaches this reference in a single denoising pass, at about 1/1001/100 of the computational cost. Its mean rate is 2.792.79, compared with 2.832.83 for nested dual ascent, its fifth-percentile rate is slightly higher (0.610.61 versus 0.570.57). Increasing KMCK_{\mathrm{MC}} and its worst-user violation 0.280.28 is close to the reference value of 0.250.25.

  • Image generation (Figure 4). The three binary attributes define eight joint outcomes. We estimate their probabilities under the pretrained model ν\nu using 15,36015{,}360 generated images and solve for λ⋆\lambda^\star by dual ascent on this empirical distribution. For each outcome oo, its optimal probability is proportional to ν(o)exp⁡(λ⋆⊤o)\nu(o)\exp({\lambda^\star}^{\top}o). This gives an exact reference for the empirical attribute distribution, including how often the attributes should occur together. PDI meets all three target levels within one standard error under both judges. Its objective on the attribute labels is 1.111.11, compared with the optimum of 1.001.00, and its label KL divergence to μ⋆\mu^\star is 0.020.02, the smallest among the methods compared. This shows that PDI comes close to the optimal joint attribute distribution as well as meeting the individual frequency constraints.

  • The testbed of G2 (Appendix I). Our testbed has fifteen two-dimensional instances and twenty more in dimensions 5 to 30, each with exact λ⋆\lambda^\star, exact P⋆P^\star, and exact samples of μ⋆\mu^\star as reference. Appendix I reports the distance to λ⋆\lambda^\star and the exact dual gap along the reverse process (Figure 16 of the revision), and checks every inequality of Proposition 3 and Theorem 1 step by step on a lattice version of seven two-dimensional instances (Appendix I.6). For the mixture of Section 5.1, Table 1 now also reports exact samples of μ⋆\mu^\star. The main finding is the one in point 2. With the exact score, the only error that remains is the band created by the bias of the Tweedie plug-in.

  1. An outer-loop baseline. We implemented this approach, which we refer to as nested dual ascent (DA). It consists of a full reverse passes at frozen multiplier and a dual step between passes. We report its results for wireless allocation, shown as dashed lines in Figure 2. The figure shows that the MMD distance between nested DA and PDI is below 0.02 and decreases with the KMCK_{MC} samples used in training the score model. The two approaches achieve the same mean rates while PDI deviates from μ⋆\mu^\star (generated by nested DA) in the lower tail. PDI has a larger rate violation at the first percentile (Figure 2c) but achieves a higher rate at the fifth percentile (Figure 2b). Notably, PDI nearly achieves this performance at one hundredth of the computational cost of nested DA.
Reviewer_qiYFOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper develops constrained diffusion models with primal-dual inference for sampling from entropy-regularized distributions subject to expectation constraints, where the authors treat the dual multiplier as part of the reverse-diffusion state. Theoretically, the paper provides a last-iterate bound for the dual variable and a KL bound comparing the terminal PDI distribution. Experiments are conducted on different cases to verify the proposed method.

Critical strengths
  • The constrained diffusion modeling problem is interesting and relevant to applications involving randomized or time-sharing policies.
  • The paper proposes a primal-dual inference method that updates the dual variable throughout the reverse diffusion process, rather than estimating and fixing it before sampling.
Critical weaknesses
  • The adopted methods are not new. The adopted method, including sampling-time primal-dual updates and dual-conditioned score model representing a family of Gibbs distributions, are closely related to existing primal-dual Langevin sampling, dual training for constrained diffusion, and energy-based diffusion training.
  • Theoretical results do not establish convergence to target domain. Theorem 1 does not establish convergence to the optimal multiplier. It seems like that Theorem 1 only show convergence to an unspecified neighborhood for an idealized population update. Besides, Theorem 2 does not establish primal convergence to the target distribution. In particular, Theorem 2 bounds $ D_{\mathrm{KL}}\left(p_T\middle|\widetilde p_T^{\lambda^\star}\right), $ where$\widetilde p_T^{\lambda^\star}$ is the output of a discrete exact-score oracle sampler with fixed λ⋆\lambda^\star. This is not the target distribution μ⋆\mu^\star.
  • The assumptions are very strong and insufficiently justified. In particular, Assumption 3 requires uniform strong concavity of the dual function over Λ\Lambda, equivalently a uniformly positive-definite constraint covariance matrix. This is not implied by entropy regularization and appears to be violated in the portfolio experiment. Assumption 1 is a standard Slater condition, but strict feasibility is not verified for either the wireless or portfolio setting.
  • The experiments do not validate the theory well. The experiments report neither distances to a known λ⋆\lambda^\star, dual optimality gaps, nor numerical evidence that the assumptions hold. Even the Gaussian-mixture experiment uses only one instance and does not compare against a high-accuracy reference distribution, although such a comparison should be feasible in this setting.
  • The presentation is not very clear, which makes the theoretical claims unnecessarily difficult to assess.
Questions for authors
  1. In what precise asymptotic regime do the authors claim that Theorems 1 and 2 imply convergence rather than convergence to an error neighborhood?

  2. How can the authors ensure the equation in Theorem 2 converge?

  3. How can Assumption 3 hold in the portfolio problem when every pair of upper/lower sector constraints has a constant sum?

  4. How does Assumption 2 hold for the case NeffN_{\mathrm{eff}} on the closed probability simplex?

3 comments

We thank the reviewer. We are confident that our revised analysis summarized in the general response (G1), and its experimental verification (G2), will alleviate their concerns regarding the soundness and clarity of our theory. We elaborate on each point in detail below.

1. Novelty

The reviewer writes that the adopted methods are not new. This is factually incorrect. PDI shares ingredients with existing methods, but the method itself is new. To our knowledge, it is the first diffusion model that searches for the optimal multipliers of average constraints during sampling, within a single reverse pass and with one score network for the whole family of Gibbs targets, and our analysis bounds the distance of its output to the constrained optimum. The reviewer's own summary of our strengths names this difference: 'PDI updates the dual variable throughout the reverse diffusion process, rather than estimating and fixing it before sampling.'

None of the three lines of work the reviewer names does this. Each treats the problem PDI solves differently. We discuss the methods mentioned by the reviewer in detail below.

  • Energy-based diffusion training (e.g., Zhang and Chen, 2022; Vargas et al., 2023; Berner et al., 2024; Richter and Berner, 2024) learns to sample a Gibbs target whose energy is given. Adding the constraints to the energy with fixed weights penalizes them rather than enforcing them, and solves a different, unconstrained problem. Our network is trained from the energy too, but over the whole Gibbs family indexed by the multiplier, so that the weights are adapted during sampling, per instance, instead of being fixed beforehand.

  • Primal--dual Langevin sampling (Chamon et al., 2024) also updates the multiplier at sampling time, but its primal updates are Langevin steps. The limitations of Langevin samplers compared to diffusion samplers are well-known. Most importantly, the Langevin chain reaches its (fixed) target only as it mixes, which can be slow on multi-modal and high-dimensional targets. A reverse diffusion instead transports noise to the target in finite time, along prescribed noised marginals, given the score. Diffusion is therefore preferred when gradients are uninformative about the target, as when the target is far from the initialization or the dimension is high. In the 30-dimensional mixture of Section 5.1, PDL with a shared multiplier ends with mode weights at total-variation distance 0.33 from those of μ⋆\mu^\star, against 0.024 for PDI (Table 1 of the revision).

  • Dual training for constrained diffusion (Khalafi et al., 2024; Gutjahr et al., 2026) searches for the multiplier during training, with a score model that does not take the multiplier as input. The multipliers end up in the network's weights, and this has the drawback that they cannot be adjusted at test time to a new instance or constraint level. Indeed, we observe that our DT baseline matches PDI only after 1000 further iterations at frozen multipliers (DT+, Table 6 of the revision). PDI trains one network over the Gibbs family and solves for the multiplier at inference, per instance. Since shifting a constraint by a constant only shifts the energy by a constant, that family does not change with the constraint levels, and new levels need no retraining (Figure 3 of the revision).

\quad We pay a one-off cost by training a score network conditioned on the multiplier, which is then amortized over every later query. That is the regime our applications target, where new channel realizations and new constraint levels arrive repeatedly and a classical sampler would have to solve each of them afresh. Since the multiplier enters the energy only linearly, this conditioning is also relatively smooth and low-dimensional, and no more involved to learn than in standard conditional diffusion models.

In addition, we argue our theoretical analysis is a novel contribution in its own right. In PDI, the samples are transported toward a target that moves with the multiplier, while the multiplier reads the constraints from samples that lag behind it. This coupling is controlled with a joint potential that follows the lag and the dual gap together in the revised analysis (G1). Its main tools, e.g., the forgetting of the reverse kernels at high noise are properties of the diffusion rather than of PDI's update. Hence, we expect the analysis to carry forward to other constrained sampling algorithms that steer a diffusion during sampling, such as PDI with pointwise or chance constraints, and constraint guidance. We leave these to future work.

2. An unspecified neighborhood, and the regime of convergence (Q1).

Convergence to λ⋆\lambda^\star and μ⋆\mu^\star holds in two regimes. If each multiplier gets a full reverse pass, the outer loop is exact dual ascent and converges linearly (Proposition 2 of the revision for nested dual ascent), at the cost of one reverse pass per dual step. For PDI with a single pass, Theorem 1 bounds TV(pT,μ⋆)TV(p_T,\mu^\star) by a transient, a band and the implementation error, and each has an explicit limit. The transient decays polynomially in the starting SNR, independently of TT (Remark C.2 of the revision). The band is zero when the dual step reads the exact posterior mean of the constraints, which automatically holds in particular for affine constraints. The implementation error vanishes with the score error and the discretization (point 3). With the Tweedie plug-in and nonlinear constraints, the multiplier converges to a band of width about the Tweedie gap divided by the curvature. While the gap is a systematic bias rather than noise, and it is a gap shared by the majority of the guidance-based literature, and typically shrinks towards closer to the data end.


3. How Theorem 2 converges (Q2), and KL to an oracle.

Theorem 1 of the revision bounds TV(pT,μ⋆)TV(p_T, \mu^\star) directly, where pTp_T is the law of PDI's output, hence we do not refer to an interim oracle sampler. The first two error terms behave as in point 2. The implementation error is Δ=Δdisc+Δscore\Delta=\Delta_{\rm disc}+\Delta_{\rm score} (Lemma C.9 of the revision). The score part Δscore\Delta_ {\rm score} is ε\varepsilon times a schedule factor that tends to 12log⁡(1+1/SNR0)\frac12\sqrt{\log(1+ 1/SNR_0)} on fine grids and does not grow with TT, for ε\varepsilon-universal score parametrization (Assumption 4). The discretization error Δdisc\Delta_{\rm disc} becomes small as the grid is refined when the sampler stops at a small noise level, and the early-stopping error vanishes as that level approaches the data. Corollary 1 turns the bound into bounds on the cost gap and the expected violation. The O(T−1/2)O(T^{-1/2}) term, the condition η=κ/T\eta=\kappa/\sqrt T and the oracle's unbounded error ε⋆\varepsilon^\star no longer appear.


4. The idealized population update.

Proposition 3 and Theorem 1 are stated for the population update; Appendix C.9 discusses the finite ensemble of II particles sharing one multiplier. The one-step inequality behind Theorem 1 never uses the law of the particles, so it holds along every realization of the random multiplier path, and the ensemble enters only as an extra estimation error. For independent particles, this error would be O(1/I)O(1/I) in mean square. each particle also moves the shared multiplier, though only by O(1/I)O(1/I) with exact posterior means. A leave-one-out argument should close this gap, which we have not worked out.


5. Assumption 3 in the portfolio problem (Q3).

The reviewer has a point in that entropy regularization alone does not imply Assumption 3, but together with non-redundant constraints, it does. Every Gibbs distribution then has a nonsingular constraint covariance, its smallest eigenvalue is bounded below on the bounded box Λ\Lambda by continuity, and the revised analysis requires it only inside the region that the iterates reach (Appendix C.1 of the revision). In the portfolio problem, the reviewer is right that the constraints are redundant. Each upper/lower sector pair has a constant sum, so the covariance of the constraints is singular at every multiplier, and gg is not strongly concave. The revised analysis needs less than strong concavity. In Remark C.3 of the revision, we argue that Proposition 3 and Theorem 1 hold when gg grows quadratically away from its optimal set instead. This is plausible for the portfolio problem, and the analysis could apply to it under this assumption, which we have not verified. Even then, the portfolio experiment runs the diffusion on logits, which the analysis does not cover and is acknowledged in Appendix B.3 of the revision.

6. Slater's condition.

Strict feasibility is an assumption on the problem rather than on the method itself and is standard in the analysis of primal--dual methods. PDI's projection onto Λ=[0,λmax⁡]d\Lambda=[0,\lambda_{\max}]^d already provides a safeguard because maximizing the dual over this box is the dual of the relaxed problem in which each constraint may be violated at a linear cost λmax⁡\lambda_{\max}. The relaxed problem is strictly feasible by construction and coincides with the original whenever λ⋆∈Λ\lambda^\star\in\Lambda. In practice, a multiplier that ends at the boundary of Λ\Lambda would signal that this fails.


7. Assumption 2 for NeffN_{\rm eff} (Q4).

The reviewer is right that Neff=exp⁡(−∑jxjlog⁡xj)N_{\rm eff}=\exp(-\sum_jx_j\log x_j) has an unbounded gradient at the boundary of the simplex. The revised analysis no longer needs Lipschitz constraints. Assumption 2 now asks only that the constraints be bounded, and NeffN_{\rm eff} takes values in [1,N][1,N] on the simplex. % Lipschitz continuity appears only in an explicit rate at which the undecided part of the constraint covariance vanishes near the data (Lemma C.3(c) of the revision), which the proofs of Proposition 3 and Theorem 1 do not use. The portfolio problem, nevertheless, lies outside the analysis for the reasons of point 5.


8. Evidence on λ⋆\lambda^\star, dual gaps and the assumptions.

The testbed of G2 (Appendix I of the revision) provides this evidence. It has fifteen two-dimensional instances and twenty more in dimensions 5 to 30, each with exact λ⋆\lambda^\star, exact P⋆P^\star, and exact samples of μ⋆\mu^\star as reference. Appendix I reports the distance to λ⋆\lambda^\star and the exact dual gap along the reverse process (Figure 16 of the revision), and checks every inequality of Proposition 3 and Theorem 1 step by step on a lattice version of seven two-dimensional instances (Appendix I.6). For the mixture of Section 5.1, Table 1 now also reports exact samples of μ⋆\mu^\star. The main finding is the one in point 2. With the exact score, the only error that remains is the band created by the bias of the Tweedie plug-in.


9. Presentation.

As summarized in our general response, we rewrote the convergence analysis to make its claims easier to assess. Section 4 now follows one potential, which combines the samples' lag and the dual gap, and states one proposition and one theorem, with each error term explained after the theorem. Appendix C separates the assumptions, the setting, the facts about the exact process and the proofs, and it uses the notation of the main text.

Reviewer_1KFpOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose an algorithm that generates samples from a distribution defined as an optimal solution of an entropy-regularized distribution optimization problem with additional constraints on the distribution statistics. The solution relies on the representation of the Lagrangian solutions as a Gibbs distribution: for each \lambda the authors estimate distribution score function and extend diffusion sampling to update \lambda along with the sample throughout the denoising process. The algorithm is accompanied by two theorems. The first shows that the dual variable converges to a neighborhood of the optimal value. The second shows that the sample output distribution converges to a vicinity of the oracle distribution, where the vicinity size is bounded by the score estimation error. The experimental section covers a synthetic constrained gaussian mixture example along with two practical tasks motivating the setup: wireless resource allocation and portfolio management.

Critical strengths
  • On the one hand, the optimization setup considered in the paper seems quite general, on the other hand it has real practical applications.
  • The proposed sampling algorithm relies on a well-established score approximation approach and can be seen as a natural extension of diffusion sampling. The dual variable update procedures induce only a negligible computational overhead compared to unconstrained sampling.
  • To incorporate constraints, the authors propose training a score estimate that depends on lambda. They ablate the amortized score estimate against a Monte Carlo estimate in their experiments.
  • The proposed procedure is supported by a theoretical analysis of procedure convergence, covering the terminal lambda properties along with output sample distribution properties. It identifies different error sources such as early step Tweedie estimate errors and the score estimation error.
Critical weaknesses
  • (W1) The reported results show that the proposed method performs better than the baselines, but they do not show how close the samples are to the optimum. Since problem (1) is a minimization, every reported number is an achievable value and therefore an upper bound on the optimal objective, with nothing bounding it from below. This seems avoidable, at least in the Gaussian mixture example. An exact optimum would make theorems testable. Besides that, Figure 2 should include the objective the paper optimizes besides f_0 alone.
  • (W2) In PDI the number of denoising steps equals the number of dual ascent steps by construction. One consequence is that accelerated samplers cannot be used: reducing the number of steps would also reduce the number of dual updates. The number of sampling steps is fixed to 500 in all three experiments, independently of the dimension and the number of constraints, and it is never ablated. A sweep over T on all three problems would show whether the required number of steps tracks the number of constraints rather than the sample quality.
  • (W3) Neither the theory nor the experiments show how the method depends on the problem dimension. All the dimension dependence is hidden inside constants. This is a concern because the score estimator is an importance sampling estimate in the data space, so it may degrade as the dimension grows.
  • (W4) The number of Monte Carlo candidates is 256 for the 30-dimensional mixture and 512 for the 500-dimensional portfolio problem, but only 8 for the 200-dimensional wireless problem. With 8 candidates in 200 dimensions the importance sample estimate may be unreliable and no ablation over the number of candidates is reported for this problem.
Questions for authors
  • (W1) In the Gaussian mixture example, would it be possible to report a lower bound on the optimum or the exact optimum?
  • (W1) Could Figure 2 report the objective of problem (1) in addition to f₀ alone? As it stands, the quantity the method optimizes is not shown, and mode coverage is reported separately as a proxy for the entropy term.
  • (W2) Was T = 500 chosen for sample quality or for dual convergence? Was it re-tuned for the wireless and portfolio problems, or inherited from the Gaussian mixture?
  • (W2) Would it be possible to sweep T on all three problems? More generally, can the number of dual updates be decoupled from the number of denoising steps? For instance several dual steps per denoising step, or dual updates on a subset of steps.
  • (W3) Could you report the effective sample size of the importance weights in the score estimator? As a function of the diffusion step and the dimension, for all three problems?
  • (W4) Why is K_MC = 8 sufficient for the 200-dimensional wireless problem when the 30-dimensional mixture uses 256 and the portfolio problem uses 512?
2 comments

We thank the reviewer for the positive assessment and for concrete requests, which improved the quality of our manuscript. The general response describes the revised theory (G1), the exactly solvable testbed (G2), and the image experiment (G3). Below we address the individual questions in turn.

  1. (W1) Distance to the optimum. Done. Table 1 now reports exact samples of the optimal distribution μ⋆\mu^\star of the Gaussian mixture. It also reports the objective of Problem (1) with the entropy term. PDI-Net comes closest to the optimum. Its objective is 33.65 against 33.56 for exact samples of μ⋆\mu^\star, and its mode weights are within total variation 0.024 of those of μ⋆\mu^\star. On the exactly solvable testbed (G2), the optimal value is known exactly. Lemma 2 there splits the objective gap of any sampler into its KL divergence from μ⋆\mu^\star and its constraint violation.

  2. (W2) Dual updates and the choice of T. Done. In our new image generation experiment, we varied the number of denoising steps TT from 32 to 512 at a fixed diffusion horizon. Figure 6 shows that the KL divergence between the generated and optimal attribute distributions remained at or below 0.050.05 across the sweep. The gray-hair constraint was satisfied for every TT, while the hat constraint was more sensitive to the choice of TT and fell short on the coarsest grids. These results agree with the revised theory, whose bounds do not grow with TT. The hat frequency is the most sensitive because hats are decided at high noise, where fewer steps leave fewer dual updates before the attribute is fixed. On the exactly solvable testbed, Figure 17 sweeps TT as well. PDI improves up to about T=1000T=1000 and then levels off at the error of the Tweedie estimate. The oracle sampler at λ⋆\lambda^\star stays at the level of exact samples for every TT. We chose T=500T=500 for the mixture, wireless and portfolio problems with both the accuracy of the diffusion sampler and the convergence of the multiplier in mind. We did not sweep TT on the wireless and portfolio problems. To examine the role of the starting SNR directly, we increased σmax⁡\sigma_{\max} while fixing T=128T=128 and η=0.8\eta=0.8. This lowers the starting SNR while keeping the number and size of the dual updates fixed. Figure 12 in Appendix G.4 shows that, at σmax⁡=40\sigma_{\max}=40, all three attribute frequencies were within one standard error of their target levels. At σmax⁡≤20\sigma_{\max}\leq 20, the eyeglasses and gray-hair frequencies fell below their targets. The KL divergence on the attributes was 0.0490.049 at σmax⁡=10\sigma_{\max}=10, compared with 0.0090.009--0.0240.024 for σmax⁡≥40\sigma_{\max}\geq 40.

  • The primal and dual updates can be decoupled, as in nested dual ascent (Section 3.1). Nested dual ascent (DA) holds the multiplier fixed for a full denoising pass and uses the resulting samples to estimate the constraint violations for the next dual update. PDI interleaves the two updates within a single pass. Each dual update is followed by a denoising step, allowing the primal samples to respond to the new multiplier before the next dual update. With exact passes, nested dual ascent is exact dual ascent and converges linearly to λ⋆\lambda^\star and μ⋆\mu^\star (Proposition 2), but each dual step costs a full pass. Decoupling the updates in the opposite direction, by taking several dual steps per denoising step, is more delicate. The dual updates should be slow enough for the primal samples to track the changing target distribution. We tested this in the image experiment. Taking 11, 22, and 44 dual steps per denoising step yielded KL divergences to the optimal attribute distribution of 0.0200.020, 0.3200.320, and 0.6090.609, respectively (Figure 14).
  1. (W4). The number of Monte Carlo candidates. Done. We report new experiments on the wireless allocation problem in which we train the score model with KMC∈{8,256,512}K_{MC} \in \{ 8, 256, 512 \} Monte Carlo candidates. Figure 2 shows that increasing KMCK_{MC} brings the learned distribution closer to the optimal distribution μ∗\mu^* obtained by nested dual ascent (described in Section 3.1 and Proposition 2). The differences among the three models appear in the lower tail of the rate distribution. All three achieve the same mean rate and the same fifth-percentile rate, while increasing KMCK_{MC} from 88 to 512512 reduces the maximum violation from 0.360.36 to 0.280.28, compared with 0.250.25 for nested dual ascent. On the Gaussian mixture testbed, more candidates help up to KMC=64K_{\mathrm{MC}}=64, and the error does not improve beyond that (Figure 17, right). Our original choice of KMC=8K_{\mathrm{MC}} = 8 during training was therefore based on computational convenience, since the cost of each training target grows linearly with KMCK_{\mathrm{MC}}.
  1. (W3) Dependence on dimension. The reviewer has a point that the importance weights in the Monte Carlo score estimator can become highly concentrated in high dimensions, leaving very few effective samples. Indeed, Figure 17 (bottom) shows that, at high noise, the effective sample size is close to one from dimension 5 on. Our contribution is mainly the primal--dual inference procedure, which updates the multiplier during denoising and uses the score field corresponding to the current multiplier. Lemma 1 and the corresponding estimation procedure are adapted from Akhound-Sadegh et al. (2024). That said, the estimator in Lemma 1 is one way to provide this score field, but not the only way.
  • To address a high-dimensional problem, where the Monte Carlo estimator would fail, we added the image generation experiment to demonstrate an alternative based on standard denoising score matching. For each training multiplier λ\lambda, we draw a clean image yTy_T from a pool of pretrained samples with probability proportional to eλ⊤c(yT)e^{\lambda^\top c(y_T)}. We add Gaussian noise to the image and train a small λ\lambda-conditioned add-on module to recover yTy_T while keeping the pretrained network frozen. The weights determine how often each clean image appears in training. The clean image itself provides the regression target, so training does not require an importance-weighted score estimate at each noisy input. With this add-on network, PDI meets all three attribute levels within one standard error under both classifiers (Figure 4). The images have 12,288 dimensions (G3).

  • We also trained a standard denoising score matching model in the revised 3030-dimensional mixture-of-Gaussians (MoG) experiment, where we obtain training samples from the tilted distributions using sequential Monte Carlo (SMC). Table 1 shows the effect of this choice. PDI-MC evaluates the estimator in Lemma 1 directly, with approximately one effective sample out of 256256 candidates at every noise level (Appendix E.3). The total variation distance between its mode probabilities and those of μ⋆\mu^\star is 0.6220.622. PDI-Net uses the learned score in the same primal--dual updates and reduces this distance to 0.0240.024.

  • The Monte Carlo estimator nevertheless works reasonably well in our wireless power allocation experiments. We used it to construct score-training targets with KMC∈{8,256,512}K_{\mathrm{MC}} \in \{8, 256, 512\} candidates. As Figure 2 shows, the squared MMD between the PDI distribution and the nested dual ascent reference was below 0.020.02 and decreased as KMCK_{\mathrm{MC}} increased. Our conclusion is that more sophisticated training schemes might be warranted depending on the specific application.

Comments on the paper

4 comments
AuthorsOpenReview ↗

General Response (To All Reviewers)

We thank the reviewers for their thorough evaluation of our submission. A shared concern among all reviewers is the soundness and clarity of our convergence theory. They ask whether it holds uniformly across the diffusion steps, what exactly it guarantees, and whether it can be checked in a setting where the optimal solution is known.

The revision makes three main changes, and the first two address this concern directly. We revise the convergence theory so that it now follows the samples and the multiplier jointly through a single potential and bounds the distance of PDI's output to the optimal distribution G1. The bound consists of a transient error that decays with the starting signal-to-noise ratio, plus some well-understood approximation errors, namely the error of the Tweedie estimate shared with guidance methods and the score and discretization errors of a standard diffusion sampler. We validate it on an exactly solvable testbed in which the optimal multipliers and the optimal distribution are known G2. The third change goes beyond the reviews. We add a high-dimensional image experiment to strengthen the case for the scalability and practicality of PDI and to show its use for fine-tuning pretrained diffusion models G3.

We summarize these general points below and then answer each reviewer's individual points in turn. In the revised paper, new text is in blue.

AuthorsOpenReview ↗

G1. The Revised Convergence Theory

G1. The Revised Convergence Theory

Nested dual ascent (Section 3.1) runs a full reverse pass at a fixed multiplier before each dual step. With exact passes it is exact dual ascent and converges linearly to λ⋆\lambda^\star and μ⋆\mu^\star (Proposition 2), but every dual step costs a full pass. PDI instead updates the multiplier at every reverse step of a single pass. Each dual step then moves the noised target, so the samples lag behind it, and the lagging samples bias the next dual step. The revised analysis (Section 4 and Appendix C) follows the lag and the dual gap together through the potential Vt:=KL(p~t∥qtλt)+D⋆−g(λt)β,V_t := KL\big(\tilde p_t \| q_t^{\lambda_t}\big) + \frac{D^\star-g(\lambda_t)}{\beta}, where p~t\tilde p_t is an exact copy of PDI's marginals ptp_t that runs the exact reverse kernels along PDI's own multipliers. This potential replaces the stability products of the submission, whose weighted sum CΨ(T)C_\Psi(T), as Reviewer 5JGp pointed out, grows at least like T2T^2. An exact reverse step never expands the lag and forgets a fraction ωt\omega_t of it at high noise. Proposition 3 shows that VtV_t contracts at a rate mt≤ωtm_t\le\omega_t, up to the estimation error of the dual step, as long as each dual step is small enough for the forgetting to absorb the lag it creates. Where the forgetting outpaces the dual step, this rate is that of the exact dual ascent of Proposition 2, so PDI makes comparable dual progress within a single reverse pass.

At the balanced dual step and up to constants, Theorem 1 reads TV(pT,μ⋆)≲e−12∑tmtV0⏟transient+1βϑ(∑tγt τˉt2)1/2⏟constraint estimation band + Δdisc+Δscore⏟implementation error,TV(p_T,\mu^\star) \lesssim \underbrace{e^{-\frac12 \sum_tm_t} \sqrt{V_0}} _ {\text{transient}} + \underbrace{\frac{1}{\sqrt{\beta\vartheta}}\Big(\sum_t\gamma_t\,\bar\tau_t^2\Big)^{1/2}} _ {\text{constraint estimation band}}\ +\ \underbrace{\Delta_{\rm disc}+\Delta_{\rm score}}_{\text{implementation error}}, where τˉt\bar\tau_t is the constrained estimation error of the dual step at step tt, ϑ\vartheta is the curvature of the dual function, and the weights γt=mt∏s>t(1−ms)\gamma_t=m_t\prod _ {s>t} (1-m_s), which sum to at most one, forget past estimation errors as the transient forgets the initial potential. The transient decays with the starting signal-to-noise ratio and shrinks with a warm start. The band comes from reading the constraints through the Tweedie estimate rather than their posterior mean, the shortcut that guidance methods also take. It is a bias that a smaller dual step does not remove, but it vanishes for affine constraints and is small at the temperature of our MoG experiment (Appendix I.5). The implementation error is the usual price of an imperfect score and a finite grid (Lemma C.9). Unlike the submitted theorems, the bound is on the distance to μ⋆\mu^\star itself. Moreover, none of its terms grows with TT, the coupling η=κ/T\eta=\kappa/\sqrt T is gone, and the neighborhood is an explicit band. Corollary 1 turns it into bounds on the cost gap and the expected constraint violation, and Assumptions 5-7 are dropped.

AuthorsOpenReview ↗

G2. Empirical Validation of the Theory

To check the theory against a known answer, we built a tractable testbed on top of the MoG setup of Section 5.1 (Appendix I). Its objective is the energy of a Gaussian mixture and its constraints are linear or quadratic, so the optimal multiplier λ⋆\lambda^\star, the optimal distribution μ⋆\mu^\star and all noised scores are available in closed form, in two to thirty dimensions. The testbed lets us compare PDI with the optimum itself and look at the terms of Theorem 1 one at a time. In particular, giving PDI the exact posterior mean of the constraints instead of the Tweedie estimate switches the constraint estimation band off. On a two-dimensional lattice, the reverse process can even be propagated exactly, so every inequality of Appendix C can be evaluated at every step.

The testbed confirms the theory in two ways. First, the analysis holds on exact dynamics. Every inequality of Appendix C holds over 420 lattice runs. The forgetting inequality and the one-step contraction of Proposition 3 are nearly tight. Indeed, the weighted bound of Proposition 3 comes within 7% of the true potential at its tightest horizon (Table 18 and Figures 18 and 19). Second, the testbed separates the error terms of Theorem 1. With the band switched off, PDI matches exact samples of μ⋆\mu^\star and its multiplier ends within 0.05--0.10 of λ⋆\lambda^\star (Table 14 and Figure 16). With the Tweedie estimate, the band is the only error that remains. It grows with the dimension (Table 16), however it does not grow with the number of steps and the feasibility gap degrades more gently. In summary, the coupling of the samples and the multiplier thus converges as the analysis predicts, and the only remaining error is the bias of the Tweedie estimate, which is shared by all guidance methods already. We acknowledge that AI coding agents assisted us with the implementation of the testbed and verification.

AuthorsOpenReview ↗

G3. High-Dimensional Image Experiments

G3. High-Dimensional Image Experiments

Beyond the reviews, we add an experiment in which PDI constrains a pretrained diffusion model in a high-dimensional space (Section 5.3 and Appendix G). We take the EDM model of 64×6464\times64 faces trained on FFHQ, with 12,288 dimensions, and seek the distribution closest to it in KL divergence whose frequencies of eyeglasses, hats and gray hair are at least 0.5, 0.5 and 0.3, against 0.20, 0.11 and 0.07 under the pretrained model. We freeze the pretrained model and train a multiplier-conditioned module with 0.3% as many parameters as the original network. We use the same module across target levels, changing only the thresholds in the dual update at inference.

PDI meets all three levels within one standard error, both under the classifier that defines the attributes and under a held-out classifier, with objective 1.11 against the optimal value 1.00, and it attains the smallest divergence to μ⋆\mu^\star on the labels (Figure 4). Batch-level guidance meets the levels only under the classifier it differentiates, and its hat frequency drops to 0.37 under the held-out one. Distribution guidance falls about one standard error short of the hat and gray-hair levels. PDI adds the attributes while preserving the identity and pose of each sample (Figure 5). The ablations agree with the theory. Fewer reverse steps leave fewer dual updates before hats are decided at high noise, and the hat level is then met less closely (Figure 6). Starting from a lower signal-to-noise ratio brings eyeglasses and gray hair to their levels (Figure 12), as the transient of Theorem 1 suggests.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.