- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper proposes constrained diffusion models with primal-dual inference (PDI) as a method for sampling the solutions of entropy-regularised optimisation problems with average constraints. Under the Lagrangian formulation, the target is a Gibbs distribution indexed by an unknown optimal dual variable. Instead of first estimating the dual multiplier and then sampling the corresponding fixed target (using a diffusion), PDI runs a single reverse diffusion with a score network conditioned on the dual variable , whilst updating by projected dual ascent after each denoising step. The theoretical analysis presents a last iterate bound for the dual iterates, and an KL comparison between PDI and an oracle sampler that knows the optimal Lagrange multiplier (up to a bias floor). The proposed method is evaluated on a constrained mixture of Gaussians, wireless resource allocation, and portfolio optimisation.
Critical strengths
- The idea of conditioning a single score model on the dual variable, and then updating that variable during the reverse diffusion trajectory, is conceptually clean and practically useful. In particular, this approach avoids having to retrain a score model each time the dual variable changes.
- The numerical section includes several useful analyses and ablations, including frozen-multiplier variants, alternative noise schedules, and shifted constraint levels. These help to separate the various mechanisms contributing to the method's overall performance.
Critical weaknesses
- My main concern relates to the theoretical guarantees. In particular, the derivation of Theorem 2 (see, e.g. Eq (50)-(51), proof of Lemma 4) appears to rely on the assumption that is bounded, uniformly over . However, under the refinement (see Assumption 6), the reverse steps become increasingly close to the identity, which would imply that must grow at least quadratically for any fixed non-degenerate target. If "bounded" here just means finite for each fixed , the dependence on should be retained in the subsequent bounds. In any case, I do not see how the stated rate in Theorem 2 holds.
- There are also a few issues in the numerical experiments (and their descriptions) that make the results difficult to assess. In particular, PDL is described as enforcing constraints pointwise, even though (as described in the original paper) it targets average constraints; the Gaussian mixture comparison appears to use different temperatures for PDL and PDI; and the portfolio experiment reports an effective number-of-assets violation for the equally weighted portfolio that is inconsistent with the stated minimum-diversification constraint. Further, Tables 7 and 8 appear to report very different inverse temperatures for PDI-{MC,Net}, making it unclear whether these variants target the same distribution.
Questions for authors
- Is in Assumption 6 meant to be bounded uniformly in ? Under the stated refinement, the reverse maps become increasingly close to the identity. This would appear to imply that for any fixed, non-degenerate target. If "bounded" here only means finite for each fixed , how should the -dependence of Theorem 2 be modified?
- How are "mode-coverage", "average-constraint feasibility" (Figure 2), and "feasibility percentage" (Table 1) defined in Sections 5.1-5.2?
- Why is PDL described in Sections 5.1-5.2 as enforcing pointwise constraints? PDL is formulated for constraints in expectation, while the update in Equations (122)-(123) appears to be a stochastic approximation to an expectation-based dual update (i.e., not a per-sample constraint). Could the authors clarify?
- What temperature is used for PDL in the Gaussian-mixture experiments? Equation (122) corresponds to unit-temperature Langevin dynamics, while Table 3 reports for PDI. Is there an additional -dependent rescaling, and are the methods being compared for the same entropy-regularised target?
- How is the effective number-of-assets violation for the equal-weight portfolio in Table 2 being computed? Under Equations (130)-(131), equal weights give , so shouldn't we expect zero violation of any feasible minimum constraint? The reported violation of 440 seems to suggest that the implemented constraint differs from Equation (130).
- Table 7 reports for PDI-MC, while Table 8 reports for PDI-Net training and inference. Were PDI-MC and PDI-Net in Table 2 run at different temperatures? In this case, they target different distributions, so it makes the comparison somewhat difficult to interpret.
- Could the authors provide a low-dimensional example where and the corresponding Gibbs target can be computed accurately? Such a reference would be quite helpful for illustrating that the method is indeed approximating the intended optimal distribution.
- The abstract contrasts PDI with approaches that first estimate the optimal multiplier and then keep it fixed during sampling. Could the authors include this baseline, using the same -conditioned network? E.g., estimate via an outer loop of full reverse-diffusion runs with dual ascent, before sampling with fixed. This would be useful for isolateing the benefit of updating the multiplier in the reverse diffusion trajectory.
3 comments
- Description of the PDL baseline. The reviewer is correct that PDL (Chamon et al., 2024) enforces the constraints on average. Its Algorithm 1 updates the multiplier with a one-sample stochastic gradient, i.e., the constraint value of the current iterate (Eq. (68)), and its convergence guarantees are for the laws of the iterates averaged over iterations. Our baseline ran Algorithm 1 independently for each sample, kept the terminal iterate of each chain with its own multiplier. Since each multiplier remains bounded, each chain satisfies the constraints on average along its own trajectory. On a multimodal target, a chain of 500 iterations stays in one mode.Thus, in the terminal distribution every chain is feasible on its own rather than only the ensemble.
-
To see how this choice affects the objective and feasibility, we now report two variants of PDL in the mixture-of-Gaussians experiment (Table 1). The first uses one multiplier vector per chain, updated with the one-sample stochastic gradient of Algorithm 1. The second evolves a batch of samples with a shared multiplier vector and updates it using the constraint values averaged over the batch, , as in (PDI-D). In the former, PDL satisfies all ten constraints on average. The largest mean constraint value is , just below the feasibility threshold of zero. The objective, however, is , compared with under the optimal distribution . With a shared multiplier, the objective improves to , reducing the objective gap from to , while the largest mean constraint value increases to .
-
The main difference in this experiment is therefore in the objective. The per-chain updates achieve feasibility on average, but at a substantially higher objective. Sharing the multiplier allows a sample to violate a constraint when other samples compensate for it, as the average-constrained problem permits. Here, this yields an objective much closer to that of , with a small remaining violation.
-
Our observation about pointwise feasibility concerns the terminal samples of independent chains of finite length, each with its own multiplier. It does not contradict the paper's own claims regarding averages across iterations.
-
Matched temperature. The reviewer is correct. The PDL update in Eq. (122) of the submission omitted the temperature parameter . This omission affected the mixture-of-Gaussians experiment, which we repeated using the corrected update. The results are reported in Table 1. All other experiments used , so no changes were needed.
-
The equal weight violation. Thanks for spotting this typo. The constrained problem without the effective-number constraints has a deterministic solution, which is the equal-weight portfolio. That is, is concentrated around this single point. The effective-number-of-assets constraint is meant to exclude this point by forcing a maximum number of effective assets. Eq. (130) in the original submission had the inequality sign reversed and it is now fixed. The problem forces a maximum number of assets , which matches the reported violation of the equal-weight portfolio.
-
Temperatures in Tables 7 and 8. Tables 12 and 13 of the revision now report the correct hyperparameters. The baseline-hyperparameter table in the original submission (with ) had stale values from earlier experiments. The reported results of all methods were at and we were able to reproduce them.
- A low-dimensional example with accurately computed and Gibbs target. Done. We now report a comparison with for the mixture-of-Gaussians (MoG), wireless resource allocation, and image generation experiments. We obtain the reference differently in each case.
-
Mixture of Gaussians (Table 1). The known mixture and constraints allow us to compute the optimal tilted distribution at and draw reference samples directly from . We can therefore compare both the objective and the fraction of samples assigned to each mode. PDI-Net achieves an objective of , compared with for samples from . The total variation distance between the mode frequencies of PDI-Net and the optimal mode probabilities is . For comparison, the same distance is for a finite sample drawn directly from . Thus, PDI-Net recovers both the objective and the allocation of probability across modes to nearly the accuracy observed with direct sampling.
-
Wireless resource allocation (Figure 2). Here we use nested dual ascent (Section 3.1) to obtain a numerical reference for each of the test networks. Using the score model trained with , we hold the multiplier fixed for a full denoising pass, update it using the average constraint violations, and repeat. PDI approaches this reference in a single denoising pass, at about of the computational cost. Its mean rate is , compared with for nested dual ascent, its fifth-percentile rate is slightly higher ( versus ). Increasing and its worst-user violation is close to the reference value of .
-
Image generation (Figure 4). The three binary attributes define eight joint outcomes. We estimate their probabilities under the pretrained model using generated images and solve for by dual ascent on this empirical distribution. For each outcome , its optimal probability is proportional to . This gives an exact reference for the empirical attribute distribution, including how often the attributes should occur together. PDI meets all three target levels within one standard error under both judges. Its objective on the attribute labels is , compared with the optimum of , and its label KL divergence to is , the smallest among the methods compared. This shows that PDI comes close to the optimal joint attribute distribution as well as meeting the individual frequency constraints.
-
The testbed of G2 (Appendix I). Our testbed has fifteen two-dimensional instances and twenty more in dimensions 5 to 30, each with exact , exact , and exact samples of as reference. Appendix I reports the distance to and the exact dual gap along the reverse process (Figure 16 of the revision), and checks every inequality of Proposition 3 and Theorem 1 step by step on a lattice version of seven two-dimensional instances (Appendix I.6). For the mixture of Section 5.1, Table 1 now also reports exact samples of . The main finding is the one in point 2. With the exact score, the only error that remains is the band created by the bias of the Tweedie plug-in.
- An outer-loop baseline. We implemented this approach, which we refer to as nested dual ascent (DA). It consists of a full reverse passes at frozen multiplier and a dual step between passes. We report its results for wireless allocation, shown as dashed lines in Figure 2. The figure shows that the MMD distance between nested DA and PDI is below 0.02 and decreases with the samples used in training the score model. The two approaches achieve the same mean rates while PDI deviates from (generated by nested DA) in the lower tail. PDI has a larger rate violation at the first percentile (Figure 2c) but achieves a higher rate at the fifth percentile (Figure 2b). Notably, PDI nearly achieves this performance at one hundredth of the computational cost of nested DA.
We thank the reviewer for a technically precise review. Its main point is correct, and it has greatly improved the paper. The general response describes the revised theory (G1) and the exactly solvable testbed the reviewer asked for (G2). We answer each point below.
The revised analysis (Section 4 and Appendix C) follows the law of the samples instead of their trajectories. The key observation is that two laws that pass through the same exact reverse kernel cannot move apart in KL, so a reverse step never expands the lag of the samples behind the target. Moreover, under sufficient noise corruption, it shrinks the lag by the factor 1−ωt=exp(−∫stst+1ϱ(s)ds), which depends only on the range of log-SNR s that the step covers (Lemma C.2). Here, ϱ, and thus ω depends on the LSI, or in simpler cases the strong log-concavity factor, of the noisy data marginal. As T grows, each factor tends to one, as the reviewer observes, but their product over any stretch of the reverse process does not depend on the number of steps. With the speed condition of Proposition 3 on the dual updates, which ties each dual step to ωt, no term of Theorem 1 grows with T.
We chose the symbol γt for the weights of Theorem 1 deliberately. In the submission, a multiplier error at step j reached the samples through the factor Ψj,t−1γj in Eq.(46), damped by the stability of the later steps. The weights γt=mt∏s>t(1−ms) keep this idea. They discount the error of each step by the contraction of all later steps, but these factors never exceed one, and the weights add up to at most one for every T.
On the lattice version of the testbed, where every quantity is exact, all inequalities of Appendix C hold over 420 runs (Appendix I.6 and Table 18).