acceptodds
Under review as a conference paper at ICLR 2027

Constrained Flow Matching via Lagrangian Dual Flows

Abstract

Flow matching is a powerful tool for generative modeling, but emerging applications in robotics, planning, and control require inference-time constraints on generated outputs. Such constraints are often complex and highly nonlinear. As a result, methods designed for linear constraints like image inpainting are rarely sufficient, and projection or optimization-based alternatives can be prohibitively expensive. In this paper, we introduce Lagrangian Dual Flows, a new family of constrained generation techniques based on Lagrangian dual dynamics. By flowing a dual co-state alongside generated samples, we can guarantee nonlinear constraint satisfaction without expensive optimization subproblems, pseudoinverses, or projection steps during the denoising process. The resulting constrained generation algorithms are simple, effective, and open new theoretical connections between flow matching and primal-dual methods in numerical optimization.

Primary area:
generative models
Keywords:
Flow matching, constrained generation, enforcing inference-time constraints

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_pXoYOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Lagrangian Dual Flows, a method for enforcing nonlinear constraints on pretrained flow matching models at inference time. The main idea is to introduce a dual variable that evolves alongside the generated samples, allowing constraints to be enforced without expensive projection or optimization at every sampling step. The authors also provide theoretical convergence guarantees under certain assumptions. Experiments on synthetic distributions, MNIST inpainting, obstacle avoidance, and robot locomotion show that LDF achieves low constraint violations while being significantly faster than several existing constrained generation methods.

Critical strengths
  1. I like that the method only requires augmenting the original flow ODE with dual dynamics, without solving optimization problems at every step such as in PCFM. The runtime improvements over projection-based and CBF-based methods are quite convincing.
  2. The paper provides convergence analysis for both equality and inequality constraints and demonstrates the method across several different applications. The results suggest that LDF could be useful for constrained generation, especially in robotics and planning.
Critical weaknesses
  1. The convergence proofs require a full-rank constraint Jacobian and a small-variation assumption on its Gram matrix. It is unclear how restrictive these assumptions are for general nonlinear constraints.
  2. While the constraint satisfaction results are good, most experiments focus on violation and runtime rather than the quality of the resulting distribution or downstream performance.
Questions for authors
  1. How restrictive is the small-variation assumption in practice? Can the authors verify whether it holds for the nonlinear constraints used in the experiments, particularly obstacle avoidance?
  2. Since the experiments use the reduced dynamics in Eq. (7), is there any theoretical or empirical evidence that they retain the convergence properties of Eq. (6)?
  3. How much does the optional final projection contribute to the results? Table 1 shows that LDF without projection sometimes has noticeably larger violations. Could the authors compare the methods under matched constraint tolerances and clarify when projection is necessary?
3 comments
AuthorsOpenReview ↗

Response to Reviewer pXoY (1/2)

Thank you for your careful reading of our paper. We are glad you found the paper clear and easy to follow! Our revised paper has its changes shown in magenta. Below are our responses to your weaknesses and comments:


W1: Assumptions on the constraint Jacobian. We thank the reviewer for raising this point. Both are fair questions and we have clarified them in the revision.

Assumption (A2) is the continuous-time analog of the linear independence constraint qualification (LICQ) used in nonlinear programming. When the rows of JgJ_g become linearly dependent, the multiplier enforcing g(x)=0g(x)=0 is not unique and may be unbounded, and the feasible set {x∣g(x)=0}\lbrace x \mid g(x)=0 \rbrace need not be a smooth surface. We therefore view (A2) as a natural requirement.

We agree that the small-variation assumption (A4) deserves more discussion. It is a sufficient condition for constraint satisfaction, not a necessary one. Its role in the proof of Theorem 1 is to ensure that a single quadratic Lyapunov function decreases along trajectories. In the revised version, we have weakened the small-variation condition (A4), and its inequality-constrained counterpart (B4), so that it need only hold on a final interval [t1,1)[t_1, 1) rather than the full integration period [0,1)[0, 1). The constraint Jacobian may therefore vary arbitrarily during the early part of generation. While this is still only a sufficient condition, we would like to stress that it captures the important case of affine constraints. A more comprehensive analysis of alternative sufficient conditions is an interesting future direction.


W2: Distribution quality and downstream performance. We thank the reviewer for this suggestion. To quantify the quality of the generated distribution, we have added FID and KID metrics for the new large-scale ImageNet inpainting example (Table 2 of the revised paper, reproduced below). These metrics are computed both over the full image and over the inpainted region alone.

MethodFID (full) ↓FID (hole) ↓KID×10³ (full) ↓KID×10³ (hole) ↓Viol.
PCFM83.2181.213.574.39.8×10−9\mathbf{9.8\times10^{-9}}
ΠGDM56.5124.95.121.11.1×10−31.1\times10^{-3}
OPC89.2102.18.212.14.8×10−24.8\times10^{-2}
DPS74.099.77.110.23.6×10−23.6\times10^{-2}
LDF (ours)55.4104.03.98.74.4×10−44.4\times10^{-4}

Image inpainting consistency metrics. FID and KID compare outputs with the reference images over the whole image and over the missing region (hole). "Viol." is mean absolute error on observed pixels.

LDF achieves the best full-image FID and KID and the best hole KID among all methods, and is competitive in hole FID, while also achieving low constraint violation. Several of these baselines are explicitly designed to sample from the conditional distribution.

We emphasize that this is an empirical observation: LDF is designed for constraint satisfaction, and we do not provide theoretical guarantees on the resulting distribution. Nonetheless, these results suggest that LDF preserves the quality of the pretrained model well in practice, and we have added a discussion of this point to the revised manuscript.


Q1: Restrictiveness of the small-variation assumption. We have relaxed the small-variation assumption so that it need only hold on a final interval [t1,1)[t_1, 1) rather than on the full integration interval. For the obstacle avoidance experiment, we found that the original sufficient condition was not satisfied, but that the new one is satisfied if we let t1t_1 be sufficiently close to 11. However, in some cases the condition was only barely met, and verifying it numerically required a very small step size and 64-bit precision; LDF itself was run with the same settings as in Table 1. We provide more empirical analysis on the small-variation assumption in Appendix B.

Again, we would like to stress that this small-variation assumption is merely a sufficient condition for constraint satisfaction, and there may be alternative conditions that establish constraint satisfaction for certain structured classes of problems. The purpose of these nonlinear experiments is primarily to test if we see constraint satisfaction even when these sufficient conditions are not (immediately) satisfied.


Q2: Convergence of the reduced dynamics. Thank you for your suggestion! After we submitted our first version, we were working on proving convergence for these dynamics and we now have a proof. We have added the theorem to the main text; it is Theorem 4 there and the proof is in Appendix C. It uses the same assumptions as Theorem 3.

AuthorsOpenReview ↗

Response to Reviewer pXoY (2/2)

Q3: Contribution of the final projection. Table 1 now reports projection variants of all applicable algorithms. CBF and PCFM already project at each substep of the denoising process, so an extra projection at the end would not change their results. We reproduce the equality-constrained portion of the updated table below. The full table, including the inequality-constrained examples, is in the revised manuscript.

MethodStar: TimeStar: Viol.MNIST: TimeMNIST: Viol.
Penalty6.43.6×10−23.6\times10^{-2}102.12.8×10−12.8\times10^{-1}
+ proj.6.61.3×10−7\mathbf{1.3\times10^{-7}}103.86.9×10−8\mathbf{6.9\times10^{-8}}
TV penalty6.0failed100.9failed
+ proj.5.3failed102.4failed
PCFM25.29.2×10−8\mathbf{9.2\times10^{-8}}147.57.7×10−8\mathbf{7.7\times10^{-8}}
ΠGDM10.42.9×10−22.9\times10^{-2}327.11.0×10−11.0\times10^{-1}
+ proj.10.77.7×10−8\mathbf{7.7\times10^{-8}}329.09.6×10−8\mathbf{9.6\times10^{-8}}
OPC20.12.2×10−22.2\times10^{-2}337.83.2×10−23.2\times10^{-2}
+ proj.20.13.9×10−8\mathbf{3.9\times10^{-8}}339.42.9×10−7\mathbf{2.9\times10^{-7}}
DPS6.15.8×10−25.8\times10^{-2}200.01.5×10−11.5\times10^{-1}
+ proj.6.21.4×10−41.4\times10^{-4}202.06.3×10−8\mathbf{6.3\times10^{-8}}
LDF (ours)6.42.6×10−32.6\times10^{-3}102.44.1×10−24.1\times10^{-2}
+ proj. (ours)6.54.5×10−8\mathbf{4.5\times10^{-8}}103.76.0×10−8\mathbf{6.0\times10^{-8}}

Mean per-sample generation time (ms) and constraint violation ∥g(x1)∥\|g(x_1)\| across 20 samples. "+ proj." rows apply a final Gauss-Newton projection to the samples of the method above. Bold times are within 20% of the lowest time for that problem; bold violations are below 10−610^{-6} (near machine precision for single-precision float). failed: every sample produced a NaN error; such methods are excluded from the bold time comparison.

The "+ proj." rows also provide the matched-tolerance comparison. On the equality-constrained examples, LDF, the penalty baseline, ΠGDM, and OPC all reach very tight constraint satisfaction (≤3×10−7\leq 3\times10^{-7}) after projection. Among these, LDF + proj. is the fastest or within 20% of the fastest on every example.

Projection is not always sufficient on its own, however. The inpainting-style methods (ΠGDM, OPC, DPS) can achieve tight constraint satisfaction with the final projection on these examples, but if the unconstrained sample is too far from the constraint set, projection can struggle or fail to enforce the constraint. We see this with DPS on the star example, where the violation after projection remains at 1.4×10−41.4\times10^{-4}. Projection can also degrade sample quality even when the constraints are technically satisfied. The revised appendix includes ImageNet inpainting examples with and without projection that illustrate this effect.

In summary, LDF without projection already drives the violation to small values, and the final projection serves to reach machine precision at negligible additional cost. Because LDF samples are already close to the constraint set, the projection is reliable and does not noticeably change the samples.

Thanks for the response and the extra experiments. The ImageNet results help address my concern about sample quality, and the updated comparisons clarify how much the final projection contributes. I’ll keep my rating at weak accept as the comparison with baselines shows a similar performance result and the robotics examples leaves open on whether the generated trajectories are physically feasible and perform well when executed and I’m still unsure how broadly the theoretical conditions apply to nonlinear constraints.

Reviewer_9tpVOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Lagrangian Dual Flows (LDF) for flow model sampling with nonlinear (in)equality constraints, which addresses the limitation of existing methods that naively apply constraint violation loss gradients within the sampling process. LDF incorporates the existing primal-dual dynamics of constrained optimization into flow model sampling, which introduces Lagrange multipliers that evolve alongside the generated samples. The authors theoretically prove that constraint violations converge to zero under LDF dynamics as sampling approaches completion. Experiments on toy datasets, image inpainting, and robotic trajectory generation demonstrate that LDF achieves more tight constraint satisfaction than optimization-based baselines.

Critical strengths
  • The authors faithfully provide theoretical justifications for their motivation, overall structure for primal-dual dynamics, the convergence of constraint violation as sampling ends, and hyperparameter selection of p.
  • Qualitative examples of the samples suggest that LDF provides more efficient and stable control toward the constrained region, compared to the jittered and circuitous generation path of the baselines.
Critical weaknesses
  • Comparison over baselines in the experiment: In Table 1, it seems like LDF requires additional Gauss-Newton projection to achieve the best performance, which was not applied to the other baselines.

  • Limited experiment scope: All experiments are conducted on low-dimensional toy problems or MNIST images. Since ΠGDM and posterior sampling methods have demonstrated scalability to high-dimensional images, the method being able to tackle such domains(e.g., inverse problems) would be desirable.

  • Concern about preserving prior distribution: While the authors already stated that this work focuses only on constraint satisfaction apart from posterior sampling method, I'm concerned that focusing solely on constraint satisfaction may admit trivial solutions.

    That said, the prior seems to be empirically preserved in qualitative results, since substituting ∇f(x)\nabla f(x) into the model velocity vθv_{\theta} effectively amounts to using a timestep-conditioned NLL as the objective ff(although this isn't equal to prior distribution sampling).

Questions for authors

I'd like to revise my scores if the following concerns are addressed:

  • Could the authors also report the performance of applying Gauss–Newton projection to the baseline methods after sampling?

  • Table 1: How can applying projection to LDF require less runtime in some tasks?

  • Table 1: The value for LDF+proj in Star(eq.) is not the best performance but bolded.

  • Figure 2: Does PCFM appear to sample more accurately from the prior distribution conditioned on the constraints?

  • Could the authors report the performance of the existing posterior sampling methods[1, 2] when they're applied on the evaluated tasks?

  • Could the authors report a metric for prior preservation(e.g., FID) separately from constraint violation? While toy experiments can easily see the prior preservation from the figures, I think a quantitative metric is necessary for the robotics and image experiments.


[1] Diffusion Posterior Sampling for General Noisy Inverse Problems, ICLR 2023

[2] Improving diffusion models for inverse problems using optimal posterior covariance, ICLR 2024

3 comments
AuthorsOpenReview ↗

Response to Reviewer 9tpV (1/2)

Thank you for your careful reading of our paper. We are glad you appreciated the overall structure for primal-dual dynamics and the convergence guarantees. Our revised paper has its changes shown in magenta. Below are our responses to your weaknesses and comments:


W1: Comparison over baselines in the experiment. Table 1 now includes the extra Gauss-Newton projections for all methods. Note that PCFM and CBF methods include projection during the denoising process, and therefore do not have the extra "+proj" option. We reproduce the equality-constrained portion of the updated table below. The full table with the inequality-constrained examples, is in the revised manuscript.

MethodStar: TimeStar: Viol.MNIST: TimeMNIST: Viol.
Penalty6.43.6e-2102.12.8e-1
+ proj.6.61.3e-7103.86.9e-8
TV penalty6.0failed100.9failed
+ proj.5.3failed102.4failed
PCFM25.29.2e-8147.57.7e-8
ΠGDM10.42.9e-2327.11.0e-1
+ proj.10.77.7e-8329.09.6e-8
OPC20.12.2e-2337.83.2e-2
+ proj.20.13.9e-8339.42.9e-7
DPS6.15.8e-2200.01.5e-1
+ proj.6.21.4e-4202.06.3e-8
LDF (ours)6.42.6e-3102.44.1e-2
+ proj. (ours)6.54.5e-8103.76.0e-8

Mean per-sample generation time (ms) and constraint violation ∥g(x1)∥\|g(x_1)\| across 20 samples. "+ proj." rows apply a final Gauss-Newton projection to the samples of the method above. Bold times are within 20% of the lowest time for that problem; bold violations are below 10−610^{-6} (near machine precision for single-precision float). failed: every sample produced a NaN error; such methods are excluded from the bold time comparison.

Before projection, LDF achieves the lowest violation among methods with comparable runtime, and after projection it matches the most accurate methods at a fraction of the cost of PCFM, ΠGDM and OPC.


W2: Limited experiment scope. To address this concern, we added an example of inpainting with a 0.95B-parameter transformer model that produces 512×512512 \times 512 RGB images. We remove random 192×192192 \times 192 pixel square patches, resulting in a 786,432-dimensional sampling space with 675,840 constraints. LDF is competitive with methods designed specifically for inpainting and linear inverse problems, including ΠGDM and posterior sampling methods such as DPS, as shown in Table 2 of the revised paper. Examples of inpainted images are included in the revised appendix.

Noisy inverse problems (e.g., denoising), where the measurements do not define an exact constraint on the sample, fall outside the constrained generation setting we consider. Noiseless inverse problems such as inpainting, however, are exactly equality-constrained generation problems, and LDF achieves competitive performance on them.


W3: Concern about preserving the prior distribution. We thank the reviewer for this thoughtful comment. While our primary focus is constraint satisfaction rather than distributional guarantees, we agree that sample quality is important, and that a method focused solely on feasibility could in principle produce trivial solutions.

Empirically, we find that LDF preserves the prior well. Using the new larger image inpainting example, we perform a larger study of the conditional distribution using FID and KID metrics. The results in the new Table 2, reproduced below, indicate that LDF performs competitively with state-of-the-art inpainting methods, which are explicitly designed to sample from the conditional distribution.

MethodFID (full) ↓FID (hole) ↓KID×10³ (full) ↓KID×10³ (hole) ↓Viol.
PCFM83.2181.213.574.39.8×10−99.8\times10^{-9}
ΠGDM56.5124.95.121.11.1×10−31.1\times10^{-3}
OPC89.2102.18.212.14.8×10−24.8\times10^{-2}
DPS74.099.77.110.23.6×10−23.6\times10^{-2}
LDF (ours)55.4104.03.98.74.4×10−44.4\times10^{-4}

Image inpainting consistency metrics. FID and KID compare outputs with the reference images over the whole image and over the missing region (hole). "Viol." is mean absolute error on observed pixels.

We believe that a rigorous theoretical analysis of convergence to the conditional distribution is an important area for future research, but outside the scope of this work. Indeed, to the best of our knowledge, most methods for constrained flow matching do not establish such convergence guarantees. Nonetheless, this new experimental evidence does suggest that establishing such guarantees may be possible for LDF.

AuthorsOpenReview ↗

Response to Reviewer 9tpV (2/2)

Q1: Gauss–Newton projection for baselines. Yes, Table 1 now reports Gauss-Newton iterations for all applicable baseline methods.


Q2: Runtime of LDF with projection. This is a simple case of measurement noise: because projection is relatively cheap, it has minimal impact on runtime. We regenerated all numbers for Table 1; the new measurements do include some instances where a projected version is faster. This is simply a sign that final-step projection is very cheap, with run-to-run variation dominating the generation times.


Q3: Bolding in Table 1. In Table 1 we bold all times that are within 20% of the lowest time for that problem and all constraint violations below 10−610^{-6}. This results in multiple bold entries per example, and is described in the table caption. We believe that this is a more illustrative scheme than simply bolding the lowest numbers, which would serve only to highlight noise.


Q4: PCFM in Figure 2. In Figure 2 yes, this appears to be the case. But our experiments with MNIST inpainting and the new large-scale inpainting examples suggest the opposite: PCFM is particularly good at tight constraint enforcement, but often fails to sample accurately from the prior distribution.


Q5: Posterior sampling baselines. Yes, we added DPS and OPC to the set of baselines. These are included in the revised manuscript in both Table 1 and in the new large-scale inpainting experiment (Table 2). Because these methods are designed for noisy inverse problems rather than constrained generation, they produce constraint violations one to two orders of magnitude larger than LDF before projection. On the inpainting task, they achieve sample quality comparable to LDF (Table 2), while LDF attains lower constraint violation at lower cost.


Q6: Quantitative metric for prior preservation. Thank you for the suggestion! We added FID and KID metrics for the new large-scale ImageNet inpainting example (see our answer to W3). LDF achieves the best full-image FID and KID and the best hole KID among all methods, and is competitive in hole FID.

Of course, this is an empirical observation. LDF is designed for constraint satisfaction, and we do not provide theoretical guarantees on preservation of the prior or convergence to the conditional distribution. Nonetheless, these results suggest that LDF preserves the prior well in practice, and we have added a discussion of this point to the revised manuscript.

I thank the authors for their responses to address my concerns, and I will update my review accordingly.

Reviewer_VK5QOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes Lagrangian Dual Flows, a training-free method for enforcing nonlinear constraints in pretrained flow-matching models. It jointly evolves samples and Lagrange multipliers, providing constraint-satisfaction guarantees under specific assumptions. Experiments on image inpainting and robotic trajectories demonstrate efficient constrained generation, with optional final projection improving numerical accuracy.

Critical strengths
  • The paper is well organized and provides a thorough mathematical background, clearly connecting classical primal–dual optimization methods to flow matching.
  • Experiments on synthetic datasets, MNIST inpainting, and robotic trajectory tasks provide initial evidence of the method’s effectiveness.
Critical weaknesses
  • Incomplete attribution of foundational work. The background should acknowledge the concurrent development of flow matching by Lipman et al., rectified flow by Liu et al., and stochastic interpolants by Albergo and Vanden-Eijnden. These closely related contributions deserve appropriate credit and a discussion of their connections.

  • Scalability. The paper motivates its approach through applications involving complex nonlinear constraints, but the evaluation remains limited to synthetic examples, MNIST, and relatively small robotic trajectory tasks. Experiments on higher-dimensional data and larger pretrained models are needed to establish scalability and practical applicability.

  • Unclear presentation of Table 1. The table could distinguish the proposed methods from the baselines more clearly. Explicitly labeling the LDF variants as “Ours” and visually separating them from competing methods would improve readability, particularly when comparing results with and without final projection.

To the authors: I used an AI tool to help analyze the manuscript and improve my understanding. I formulated my own human-based concerns and used the tool to polish their wording. I carefully reviewed the revised text to ensure that it accurately reflected my original concerns.

  • Lack of distributional guarantees. The theoretical analysis establishes constraint satisfaction under stated assumptions, but it does not characterize the resulting sampling distribution or guarantee convergence to the desired conditional distribution.

  • The connection to guidance needs deeper discussion. The proposed correction is closely related to gradient-based constraint guidance, with the main distinction being an evolving dual variable that accumulates constraint violations. Although the paper discusses quadratic-penalty guidance, it should more explicitly explain the additional benefits of the dual dynamics and compare against stronger adaptive or time-dependent guidance baselines.

Questions for authors

see weakness

1 comment
AuthorsOpenReview ↗

Response to Reviewer VK5Q

Thank you for your careful reading of our paper. We are glad you found our paper well-organized and that we clearly connected primal-dual dynamics to flow matching. Our revised paper has its changes shown in magenta. Below are our responses to your weaknesses:


W1: Incomplete attribution of foundational work. We agree. Flow matching, rectified flow and stochastic interpolants were developed concurrently and are closely related. Each learns a time-dependent velocity field whose ODE transports a reference distribution to the data distribution. We have revised the introduction and Section 3 to credit all three lines of work and to describe their connection. We also note that LDF only requires access to the sampling ODE x˙=vθ(x,t)\dot x = v_\theta(x,t), so it applies without modification to pretrained models from any of these frameworks.


W2: Scalability. To address this concern, we added an example of inpainting with a 0.95B parameter transformer model. This model produces 512×512512 \times 512 RGB images. We remove random 192×192192 \times 192 pixel square patches, resulting in a 786,432-dimensional sampling space with 675,840 constraints. LDF is competitive with algorithms for inpainting and linear inverse problems, as shown in Table~2 of the revised paper. Examples of inpainted images are included in the revised appendix.


W3: Unclear presentation of Table 1. We have reorganized Table 1 in the revised manuscript. It is now a transpose of the previous version, with methods per row and examples per column. In addition to adding additional baselines, we now evaluate the impact of an extra projection step for all algorithms, separate the LDF variants from the baselines with a horizontal line, and label LDF variants as "ours."


W4: Lack of distributional guarantees. While our primary focus is constraint satisfaction rather than distributional guarantees, the issue of sample quality is an important one. While we do not yet have theoretical guarantees regarding convergence to the conditional distribution, we do find empirically that LDF produces samples that appear to be near the conditional distribution, as in Fig. 4a.

Using the new larger image inpainting example, we perform a larger study of the conditional distribution using FID and KID metrics. The results in the new Table 2, reproduced below, indicate that LDF performs competitively with state-of-the-art inpainting methods, which are explicitly designed to sample from the conditional distribution.

MethodFID (full) ↓FID (hole) ↓KID×10³ (full) ↓KID×10³ (hole) ↓Viol.
PCFM83.2181.213.574.39.8×10−99.8\times10^{-9}
ΠGDM56.5124.95.121.11.1×10−31.1\times10^{-3}
OPC89.2102.18.212.14.8×10−24.8\times10^{-2}
DPS74.099.77.110.23.6×10−23.6\times10^{-2}
LDF (ours)55.4104.03.98.74.4×10−44.4\times10^{-4}

Image inpainting consistency metrics. FID and KID compare outputs with the reference images over the whole image and over the missing region (hole). "Viol." is mean absolute error on observed pixels.

We believe that a rigorous theoretical analysis of convergence to the conditional distribution is an important area for future research, but outside the scope of this work. Indeed, to the best of our knowledge, most methods for constrained flow matching do not establish such convergence guarantees. Nonetheless, this new experimental evidence does suggest that establishing such guarantees may be possible for LDF.


W5: Connection to guidance. We thank the reviewer for raising this important point. In the revised paper, we add a time-dependent baseline that scales a quadratic penalty with 1/(1−t)21/(1-t)^2 during the denoising process.

This baseline does not perform particularly well, failing on all equality-constrained examples, in large part because the resulting ODE becomes extremely stiff. Fortunately, there are principled methods from the numerical optimization literature for obtaining an effectively infinitely stiff penalty without numerical ill-conditioning. This is none other than the idea of Lagrange multipliers, the basis of our proposed approach.

In addition to the new baseline, we added some discussion of this connection to the revised manuscript.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of changes

Dear Area Chair and Reviewers,

As the discussion period comes to a close, we would like to thank everyone for their constructive comments. We highlight the key changes that were common among several reviewers.

Scalability: we added a new large-scale ImageNet inpainting experiment with a 0.95B-parameter model.

Sample quality: we included new FID/KID results showing LDF is competitive with methods designed for conditional sampling.

Baselines: we added extra baselines including DPS, OPC, a time-varying penalty. To isolate the role of the Gauss-Newton steps, we incorporated this final projection for all applicable methods.

Theory: we have a new convergence theorem for the reduced dynamics used in our experiments, and a weaker small-variation assumption.

Detailed responses to all reviewer comments are available in individual replies. We hope that these responses address the reviewers' concerns, and we would be grateful if they could be reflected in any final assessments. We sincerely thank everyone for their time in reading our paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.