acceptodds
Under review as a conference paper at ICLR 2027

Manifold Attracting Flow Matching: Physics-Constrained Sampling via Continuous Geometric Dynamics

Abstract

Generating scientific data under previously unseen equality constraints requires satisfying the prescribed constraints without destroying the physical structure learned from data. Existing inference-time approaches often enforce physical constraints by repeatedly correcting predicted endpoints, but such discrete corrections can distort learned generative structure and introduce unstable feedback. In this work, we propose Manifold Attracting Flow Matching (MAFM), which continuously evolves a persistent anchor through an ODE whose dynamics drive it analytically toward the constraint manifold. Specifically, its geometric correction preserves model-guided tangent motion while prescribing the normal motion to contract constraint violations toward the manifold. This construction separates model tracking from constraint attraction and provides theoretical guarantees for residual contraction and local feedback stabilization. Experiments across geometric and partial differential equation benchmarks demonstrate improved generation quality over variant constraint-guided baselines while maintaining low constraint residuals. MAFM provides a flexible framework for coordinating learned physical structure and explicit constraints through controllable sampling dynamics, without retraining the underlying generative model.

Primary area:
generative models
Keywords:
Flow Matching, Manifold, PDE, Generative Model

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_aQ7VOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

In this paper, the authors propose Manifold Attracting Flow Matching (MAFM), which continuously evolves a persistent anchor through an ODE whose dynamics drive it analytically towards the constraint manifold. Specifically, its geometric correction preserves model-guided tangent motion while prescribing the normal motion to contract constraint violations towards the manifold. This construction separates model tracking from constraint attraction and provides theoretical guarantees for residual contraction and local feedback stabilization.

Critical strengths
  1. introduce a persistent-anchor sampling paradigm for imposing new equality constraints at inference time with a frozen flow model.
  2. characterize the constraint-residual dynamics induced by the persistent-anchor updates, establish conditions for convergence toward feasibility, and analyze how endpoint-prediction and adjoint estimation errors affect the anchor trajectory.
Critical weaknesses
  1. It is so sudden to introduce the function κ(t)\kappa(t)? Are there some theoretical or intuitive explanations?
  2. It is not reasonable to set γ(t)=0\gamma(t)=0 or η(t)=0\eta(t)=0 in the ablation studies paper of this paper, since this paper focuses on the problems for both the pde and the physical constraint.
  3. There is no setup for the function κ(t)\kappa(t). Could you add it?
  4. Is it slow to solve line 9 in Algorithm 1? It seems slow compared with PCFM. Could you provide some computational complexity?
Questions for authors
  1. It is so sudden to introduce the function κ(t)\kappa(t)? Are there some theoretical or intuitive explanations?
  2. There is no setup for the function κ(t)\kappa(t). Could you add it?
  3. Is it slow to solve line 9 in Algorithm 1? It seems slow compared with PCFM. Could you provide some computational complexity?
1 comment
AuthorsOpenReview ↗

Multiplier solve complexity and runtime compared with PCFM

Q3. What is the cost of the solve on line 9 of Algorithm 1 in the original submission? Is it slower than PCFM? (W4)

A3. We want to clarify that the computational cost in both methods is of the same order. We denote the constraint residuals as C\mathcal C.

PCFM [1], Eq. (4)MAFM, original Algorithm 1, line 9 (p. 6)
uproj=u1−JP⊤(JPJP⊤)−1C(u1)u_{\mathrm{proj}}=u_1-J_{\mathrm P}^{\top}(J_{\mathrm P}J_{\mathrm P}^{\top})^{-1}\mathcal C(u_1)(JkJk⊤+δIm)λk=−[Jkwk+κ(tk)C(ak)](J_kJ_k^\top+\delta I_m)\lambda_k=-[J_kw_k+\kappa(t_k)\mathcal C(a_k)]

PCFM evaluates JP=DC(u1)J_{\mathrm P}=D\mathcal C(u_1) at the predicted endpoint u1u_1 and corrects that endpoint with a Gauss–Newton projection. MAFM evaluates Jk=DC(ak)J_k=D\mathcal C(a_k) at the persistent anchor aka_k and solves for the multiplier λk\lambda_k to correct the candidate velocity wkw_k and prescribe residual contraction at rate κ(tk)\kappa(t_k). The evaluation points and right-hand sides differ. With mm scalar constraints and a dd-dimensional state, given the m×dm\times d Jacobian, both dense implementations take O(dm2)O(dm^2) to form an m×mm\times m Gram matrix and O(m3)O(m^3) to factor and solve it.

As in PCFM [1], our experiments use m≪dm\ll d, with far fewer scalar constraints than state dimensions. Fixed sensor selection gives a diagonal Gram matrix in Poisson. Mass and energy allow framewise mean and radial corrections, respectively. Imposing a strong-form PDE residual at every space-time grid point can make mm comparable to dd, bringing the generic dense cost to O(d3)O(d^3). Our main PDE experiments impose specified hard constraints during sampling and evaluate PDE consistency with an independent solver.

TaskTiming unitMAFM timePCFM time
Reaction–diffusions/sample0.043750.04249
Euler (fixed energy)s/512 samples7.256.72
Controlled NS (fixed energy)s/512 samples7.286.73
Poisson (64 fixed sensors)s/512 samples17.8815.77

MAFM was slightly slower than PCFM in these timing tests. Both methods used 100 forward evaluations, and MAFM also computed 100 input vector–Jacobian products (VJPs) on Poisson. In the fixed-energy tests, both methods used zero VJPs, and MAFM performed additional work to track the endpoint and contract the radial deviation of its persistent anchor.

In the revised manuscript, we have added an explanation of the costs of dense and structured solves.

References

  1. Utkarsh et al. Physics-Constrained Flow Matching.
Reviewer_WrHQOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
1
Soundness
3
Read review
Paper summary

The paper proposes a sampling algorithm for flow matching that satisfies the underlying physics constraints. The paper proposes an anchor process to modify the reverse ODE updates and guide them towards the feasible region induced by the constraints. The experiments show that their method improved performance over baselines in satisfying the physics-based constraints using PDE datasets.

Critical strengths
  1. The paper motivates the problem well.
  2. The approach shows improved constraint satisfaction for benchmark PDE datasets.
Critical weaknesses
  1. The paper is very hard to follow. For example, the main contribution of the paper is the anchor dynamics that guide the reverse process. However, from the description of the anchor dynamics at the end of Sec. 3.3, it is not clear how it helps the reverse process find the feasible region.

  2. Comparisons with important baselines like Chance Constrained Flow Matching (Liang et al. 2026) are missing.

  3. I would encourage the authors to add descriptions to the figures. For example, in Fig. 2, it is not clear how the proposed update moves x_k+1 towards the feasible target.

Questions for authors
  1. What is \lambda in line 172?
  2. What is JDθ (xt, t)⊤ in eq 8? Is that a Jacobian?
  3. What is VJP in line 239?
4 comments
AuthorsOpenReview ↗

Persistent-anchor sampling and the CCFM comparison

Thank you for reading our paper carefully and for recognizing the motivation for the problem and the constraint satisfaction results on the PDE benchmarks. We will mark the main-text changes made primarily in response to your comments in purple. We hope the responses below address your concerns. If any point remains unclear, we would welcome follow-up questions and be glad to discuss it further.

Q1. How do the anchor dynamics in Sec. 3.3 guide reverse sampling toward the feasible region? Why maintain a persistent anchor, and what does it do at each step? (W1, W3)

Answer 1. Physical constraints apply to the final generated object. At intermediate times, the frozen model reads a noisy query x(k)x^{(k)}. We maintain a candidate physical endpoint a(k)a^{(k)} across steps and update it with each new prediction d(k)d^{(k)}, carrying earlier corrections forward. The detailed steps are as follow.

1. Fig. 2A | Predict the full endpoint. The first endpoint prediction from source noise x0x_0 initializes the persistent anchor. At later steps, we extrapolate from the current query along the model velocity to obtain d(k)=x(k)+(1−tk)vθ(x(k),tk)d^{(k)}=x^{(k)}+(1-t_k)v_\theta(x^{(k)},t_k).

2. Fig. 2B | Compute observation guidance. We backpropagate the observation loss at d(k)d^{(k)} through the frozen model to obtain its gradient with respect to the input. In Poisson, this lets sparse sensor observations guide updates in unobserved regions of the field.

3. Fig. 2C | Form the candidate velocity. Before constraint correction, w(k)w^{(k)} combines endpoint tracking along d(k)−a(k)d^{(k)}-a^{(k)} at rate γ(tk)\gamma(t_k) with optional input adjoint guidance weighted by η(tk)\eta(t_k).

4. Fig. 2D | Correct the velocity and update the anchor. At a(k)a^{(k)}, compute J(k)=JC(a(k))J^{(k)}=J_{\mathcal C}(a^{(k)}). The multiplier correction (J(k))⊤λ(k)(J^{(k)})^\top\lambda^{(k)} preserves the tangential component of w(k)w^{(k)} and replaces its normal component to enforce contraction at rate κ(tk)\kappa(t_k). Line 9 of the original Algorithm 1 solves for λ(k)\lambda^{(k)}. Then we take a step of size hh to update the anchor a(k)a^{(k)}: a(k+1)=a(k)+h[w(k)+(J(k))⊤λ(k)]a^{(k+1)}=a^{(k)}+h[w^{(k)}+(J^{(k)})^\top\lambda^{(k)}].

5. Fig. 2E | Form the next model query. Interpolate the fixed source noise and updated anchor: x(k+1)=(1−tk+1)x0+tk+1a(k+1)x^{(k+1)}=(1-t_{k+1})x_0+t_{k+1}a^{(k+1)}. This passes the correction to the next prediction. Constraint contraction acts on the anchor, so intermediate queries can lie outside the feasible set. At t=1t=1, the query and final output equal the corrected anchor.

Q2. The paper lacks a comparison with Chance-Constrained Flow Matching (CCFM). How do the two methods compare? (W2)

Answer 2. Thank you for suggesting CCFM [1]. We compared it with MAFM at matched budgets on all five PDE benchmarks, including both Poisson sensor densities. MAFM has lower PDE relative L2L_2 error across all benchmarks for both CCFM schedules.

CCFM projects the noisy state under a time-dependent chance constraint. MAFM updates a persistent physical endpoint via tangential tracking and normal contraction.

Within each task, both methods use the same frozen model, paired source noise, and 100 forward calls per sample. CCFM uses p(s)=(s/2)np(s)=(s/2)^n and terminal projection. Cells show constraint error (CE) / PDE relative L2L_2, plus divergence RMSE for the energy tasks. Independent solvers use the generated initial field (own-IC) or, for Poisson, the generated source FF.

MethodControlled NSEulerBurgersReaction–diffusionPoisson (64 sensors)Poisson (256 sensors)
CCFM (n=1n=1)2.35e-07 / 4.05% / 0.070272.39e-07 / 4.20% / 0.070355.39e-08 / 4.08%5.92e-08 / 6.49%0 / 22.74%0 / 45.45%
CCFM (n=2n=2)2.35e-07 / 3.48% / 0.064022.33e-07 / 3.68% / 0.064995.39e-08 / 5.73%5.96e-08 / 6.83%0 / 22.78%0 / 46.47%
MAFM6.89e-06 / 2.76% / 0.047366.84e-06 / 2.94% / 0.048294.22e-08 / 0.965%1.13e-07 / 2.74%1.72e-07 / 1.70%1.75e-07 / 2.25%
MAFM + terminal projection2.43e-07 / 2.76% / 0.047362.37e-07 / 2.94% / 0.048295.32e-08 / 0.965%6.77e-08 / 2.74%0 / 1.70%0 / 2.25%

With terminal projection, MAFM matches CCFM's CE scale for mass and energy and reaches zero sensor CE on Poisson. Revised Appendix D.5 details both schedules and their results.

References

  1. Liang et al. Chance-constrained Flow Matching for High-Fidelity Constraint-aware Generation. 2025.
AuthorsOpenReview ↗

Figure 2 and notation in the sampling algorithm

Q3. In the original Fig. 2, how does the update move x(k+1)x^{(k+1)} toward the feasible target? Can the figure explain this more clearly? (W3)

Answer 3. We pass the constraint correction to the next query through the anchor. Panel D in Fig. 2 of the original submission (p. 5) produces the updated anchor a(k+1)a^{(k+1)}. Panel E interpolates it with the fixed source noise x0x_0 to give x(k+1)=(1−tk+1)x0+tk+1a(k+1)x^{(k+1)}=(1-t_{k+1})x_0+t_{k+1}a^{(k+1)}. With the source and next time held fixed, a change Δa\Delta a in the anchor shifts the next query by tk+1Δat_{k+1}\Delta a. The noisy query can still lie outside the constraint set at intermediate times. At t=1t=1, the final query equals the final anchor, so the output inherits its constraint residual. The revision retains the five geometric panels (A–E), adds a short explanation below each panel, and gives the interpolation formula in the caption.

Q4. What are λ\lambda in line 172, JDθ(xt,t)⊤J_{D_\theta}(x_t,t)^\top in Eq. (8), and VJP in line 239 of the original submission? (Questions for Authors)

Answer 4. These symbols enter two computations. The multiplier λt\lambda_t corrects the constraint residual of the current anchor. The transposed Jacobian and VJP use the error at the model's predicted endpoint to compute a guidance direction for the model input.

  • λt\lambda_t (line 172) is the constraint multiplier, with one entry per scalar constraint. Line 9 solves for λt\lambda_t at each step, and we then add JC(at)⊤λtJ_{\mathcal C}(a_t)^\top\lambda_t to the proposed anchor velocity.
  • Pseudoinverse in the constraint solve. Write J=JC(at)J=J_{\mathcal C}(a_t). For full-row-rank JJ, J†=J⊤(JJ⊤)−1J^\dagger=J^\top(JJ^\top)^{-1} is the standard Moore–Penrose pseudoinverse formula. With an exact solve and δ=0\delta=0, the normal correction is J⊤λt=−J†[Jwt+κ(t)C(at)]J^\top\lambda_t=-J^\dagger[Jw_t+\kappa(t)\mathcal C(a_t)].
  • JDθ(xt,t)⊤J_{D_\theta}(x_t,t)^\top (Eq. 8) is the transpose of the endpoint predictor's Jacobian with respect to xtx_t at fixed tt. It maps the gradient at the predicted endpoint back to the input.
  • VJP (line 239) stands for vector–Jacobian product, a standard operation for computing gradients efficiently in reverse-mode automatic differentiation. If uu is the gradient of the error energy with respect to the predicted endpoint, one backward pass gives the input gradient JDθ⊤uJ_{D_\theta}^\top u in Eq. (8) without forming the full Jacobian.

We will explain each symbol and its role where it first appears in the revision.

I thank the author for their additional experiments and for explaining the method. I still have a question about the key contribution of the paper. It is stated in the introduction that anchor sampling is the key contribution of the paper. However, I am unclear about eq 14 and how it helps achieve the improvement in constraint satisfaction.

Also, can the author clarify what the n=1 and n=2 variants of CCFM are?

AuthorsOpenReview ↗

Anchor dynamics in Eq. (14) and CCFM schedules

Dear Reviewer,

We are very grateful that you took the time to read our response and the additional experiments. We appreciate the opportunity to continue the discussion and clarify how the anchor and Eq. (14) work together. We will be very happy to continue the discussion if you have any further questions.

Core contribution. We hope to clarify that our core contribution is the constraint-control dynamics in Eq. (14) for a persistent candidate physical endpoint, the anchor ata_t. We choose the anchor as the object of control because the prescribed physical constraints are defined on the final clean sample, whereas the intermediate sampling state xtx_t still contains source noise. We therefore continuously control the anchor’s constraint residual without requiring intermediate noisy states to satisfy these constraints throughout sampling.

The anchor is our current estimate of the final physical sample. At each step, the frozen model’s one-step endpoint prediction Dθ(xt,t)D_\theta(x_t,t) updates the existing anchor, allowing earlier predictions and corrections to carry forward into subsequent denoising. Eq. (14) preserves the constraint-compatible tangential guidance in the candidate velocity while independently controlling residual decay through the normal correction.

We then feed the updated anchor into the next model query through x(k+1)=(1−tk+1)x0+tk+1a(k+1)x^{(k+1)}=(1-t_{k+1})x_0+t_{k+1}a^{(k+1)}, where x0x_0 is the fixed source noise. At t=1t=1, the source-noise term vanishes and the generated sample coincides exactly with the anchor, xgen=a1x_{\mathrm{gen}}=a_1. Thus, controlling the anchor’s terminal residual directly controls the constraint residual of the final generated sample, while allowing intermediate sampling states to evolve outside the constraint manifold.

How Eq. (14) works. You can think of Eq. (14) as splitting the candidate velocity wtw_t during denoising. Here, wtw_t is the denoising velocity we propose for the anchor using the frozen model’s one-step endpoint prediction. Then we replace the normal component of wtw_t with an attraction term derived from the Lagrange multiplier, so we can control how fast the constraint residual decays. Fig. 2 illustrates this step. Panel C uses the frozen model’s endpoint prediction to form the candidate anchor velocity wtw_t, and panel D applies Eq. (14) to update the anchor. a˙t=PT(at)wt−JC(at)⊤G(at)−1κ(t)C(at).(14)\dot a_t= P_T(a_t)w_t- J_{\mathcal C}(a_t)^\top G(a_t)^{-1}\kappa(t)\mathcal C(a_t). \qquad (14) Here PT(at)P_T(a_t) projects onto the tangent space of the local manifold through ata_t on which the constraint residual is constant at C(at)\mathcal C(a_t). We write JCJ_{\mathcal C} for the constraint Jacobian, define G=JCJC⊤G=J_{\mathcal C}J_{\mathcal C}^{\top}, and use κ(t)\kappa(t) to set the contraction rate. The first term PT(at)wtP_T(a_t)w_t keeps the tangential component of the candidate velocity, and the second supplies the normal correction that reduces the residual. Under Eq. (14), the chain rule gives ddtC(at)=−κ(t)C(at)\frac{d}{dt}\mathcal C(a_t)=-\kappa(t)\mathcal C(a_t) (Eq. 15), so the residual decays exponentially in the cumulative contraction ∫0tκ(s)ds\int_0^t\kappa(s)ds.

Why use this separation? The design is motivated by stiff fast–slow systems of the form u˙=f(u,z,ε)\dot u=f(u,z,\varepsilon) and εz˙=g(u,z,ε)\varepsilon\dot z=g(u,z,\varepsilon), with 0<ε≪10<\varepsilon\ll1. When the fast equilibria form a normally attracting manifold, a nearby invariant slow manifold persists for small ε\varepsilon [1]. Nearby trajectories approach it quickly and then evolve slowly along it. In MAFM, the constraint residual plays the role of the fast variable. The model guides tangential motion, and κ(t)\kappa(t) plays the role of the fast relaxation rate 1/ε1/\varepsilon. In the exact continuous dynamics, the constraint manifold C=0\mathcal C=0 is itself invariant.

CCFM schedules. The exponent nn is a key parameter in CCFM [2] that controls how tightly the chance constraint is enforced at intermediate sampling times. The two variants use the same sampler with n=1n=1 or n=2n=2 in the schedule p(s)=(s/2)np(s)=(s/2)^n, where ss is sampling time. At intermediate times, n=2n=2 gives a smaller probability threshold and therefore a wider projection band in our implementation. This allows the intermediate sample to deviate further before a correction is needed. Both variants use the same terminal projection to enforce the prescribed hard constraint. We report both schedules in Appendix D.5.

References

  1. Neil Fenichel (1979). Geometric singular perturbation theory for ordinary differential equations. Journal of Differential Equations, 31(1), 53–98. DOI
  2. Jinhao Liang et al. (2025). Chance-constrained Flow Matching for High-Fidelity Constraint-aware Generation. arXiv:2509.25157v1
Reviewer_Eo6bOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces Manifold Attracting Flow Matching (MAFM), an inference-time method to constrain samples from a pretrained flow-matching model. The key idea is to introduce persistent-anchor dynamics representing the estimated final sample, which is continuously updated alongside the sampling process using endpoint predictions and corrections based on the specified constraints. The method is evaluated on six benchmarks, including geometric and PDE problems.

Critical strengths

S1. The general motivation of the method seems compelling, as the constraints are enforced only during inference while trying to preserve the distributional accuracy of a pretrained generative model. The idea of maintaining a persistent endpoint estimate and gradually refining it using the constraints and flow-matching model predictions seems like a reasonable way to potentially reduce disruptions to the learned dynamics during sampling.

S2. The proposed framework is mathematically well motivated, and the authors provide theoretical analysis to explain the different components of the method.

Critical weaknesses

W1. One of my main concerns is that the paper does not sufficiently discuss or distinguish itself from several closely related methods. For example, PnP-Flow [1] also uses pretrained flow-matching models with iterative corrections and reinterpolation to solve inverse problems, while Lagrangian Dual Flows [2] formulates constrained generation using continuous-time dynamics, which seems closely related to the MAFM framework. Moreover, the authors themselves discuss the connection between MAFM and PCFM when the adjoint guidance is removed, although MAFM still retains an additional relaxation mechanism.

In particular, for linear constraints, the persistent-anchor update seems closely related to a relaxed projected-endpoint iteration, with the relaxation parameter controlling how much information is retained from previous predictions. While the proposed formulation is not mathematically identical to these existing approaches, I don't think the paper clearly establishes what additional advantage comes from the proposed persistent-anchor dynamics. This strengthens my concern that some of the underlying motivations and algorithmic ideas are quite similar to previously published approaches.

W2. The paper does not sufficiently compare against several closely related methods. In particular, methods such as D-Flow [3] and PnP-Flow [1] also leverage pretrained generative models and operate only during inference. More recently, Chance-constrained Flow Matching (CCFM) [4] and FunDPS [5] consider a similar problem of constraint-aware flow-based generation. Similarly, there are methods addressing PDE inverse problems using generative modeling approaches, such as DiffusionPDE.

I think including more of these baselines directly would help establish the advantages of the proposed method more convincingly, particularly because some of these methods have already been evaluated on PDE systems similar to those considered in this paper.

W3. I'm also quite surprised by the performance of the existing baselines, particularly ECI and PCFM, across several experiments. For example, in Table 1c, PCFM-Base achieves an own-IC relative L2 error of 13.6% on Euler and 12.3% on controlled Navier–Stokes, whereas even a simple terminal projection achieves much lower errors of 3.62% and 3.40%, respectively. These are quite substantial differences, especially considering that PCFM was explicitly designed to enforce constraints during sampling while preserving the quality of generated samples, and its published results also include evaluations on PDE systems.

Moreover, recent work such as CCFM [4] also evaluates PCFM on PDE-constrained generation problems, where it achieves relatively competitive performance. I understand that the experimental settings and constraints may not be exactly the same, and this is precisely what I would like to understand. I think the authors should clarify how these baselines were implemented or explain why there is such a large degradation in physical consistency. In particular, it is unclear whether this behavior arises from the choice of correction strengths, projection schedules, or other differences in the experimental setup.

W4. The current experiments do not sufficiently establish some of the broader claims made in the paper about preserving the learned generative distribution. For example, the main physical-consistency metric for the dynamical PDEs is the relative L2 error between a generated trajectory and an independently computed PDE solution using the generated initial condition. I understand this is a reasonable metric for evaluating physical consistency, but it does not necessarily evaluate or give an idea of the distribution of generated samples under the specified constraints.

There are some distributional metrics reported in the paper and appendix, but upon closer inspection, the results seem mixed. For example, resampled ECI achieves substantially better field SWD than MAFM on Burgers, while terminal projection achieves better field SWD on the fixed-energy benchmarks. I think these results deserve more discussion, particularly because one of the main motivations of the proposed method is to preserve the learned data distribution while enforcing constraints during sampling.

Moreover, the Euler and Navier–Stokes experiments use training distributions with a fixed energy level, and the inference-time constraint enforces the same energy. This does not demonstrate conditional generation at previously unseen energy levels, which seems particularly relevant to the motivation of the paper.

References

  1. Martin, Ségolène, et al. "Pnp-flow: Plug-and-play image restoration with flow matching." International Conference on Learning Representations. Vol. 2025. 2025.

  2. Kurtz, Vince, and Alexander Davydov. "Constrained Flow Matching via Lagrangian Dual Flows." arXiv preprint arXiv:2607.04513 (2026).

  3. Ben-Hamu, Heli, et al. "D-flow: Differentiating through flows for controlled generation." arXiv preprint arXiv:2402.14017 (2024).

  4. Liang, Jinhao, et al. "Chance-constrained flow matching for high-fidelity constraint-aware generation." arXiv preprint arXiv:2509.25157 (2025).

  5. Yao, Jiachen, et al. "Guided diffusion sampling on function spaces with applications to pdes." Advances in Neural Information Processing Systems 38 (2026): 127057-127094.

Questions for authors

See weaknesses above, but here are some additional questions:

Q1. Can the authors clarify how the reported distributional metrics are computed and whether they evaluate the distribution of generated samples conditioned on the specified constraints? In particular, the own-IC relative L2 metric evaluates whether a generated trajectory is physically consistent, but does not necessarily establish that the samples follow the desired conditional distribution. Would it be possible to compare the generated samples against reference conditional ensembles, particularly for Poisson or Burgers, where the underlying data-generating distributions are known?

Q2. How well does the proposed method generalize to conditions outside the training distribution? It would be interesting to study how the method performs when conditioning on different energy levels or other physical conditions not represented in the training distribution.

Q3. How does the computational cost of the proposed method scale with the number of constraints and the additional overhead of the persistent-anchor dynamics?

8 comments
AuthorsOpenReview ↗

Hard constraint satisfaction, PDE consistency, and Field SWD

Thank you for your careful review and for recognizing the motivation of our work. We address your questions below and have marked the corresponding changes in dark blue in the revised version. We welcome any follow-up questions and would be glad to continue the discussion.

Q1 / W4. Does trajectory-level relative L2L_2 error establish conditional distribution matching? How is Field SWD computed, and how should it be interpreted?

Answer 1. Our main metrics are hard constraint error (CE) and relative L2L_2 error, which measure each generated sample’s constraint violation and PDE consistency, respectively. For time-dependent problems, we use an independent solver to obtain a reference trajectory from each generated sample’s own initial field, then measure relative L2L_2 error over the subsequent frames.

The phrase “distribution preservation” in the original submission focuses on the manifold-supported distribution (feasible samples under constraints) rather than the full-dimensional distribution (data samples). A regular equality constraint defines a lower-dimensional feasible manifold, and a law supported on it is singular with respect to ambient Lebesgue measure [1]. For example, constraining a nondegenerate Gaussian to a single point yields a Dirac measure. A regular, invertible ODE flow cannot achieve this collapse in finite time [2]. When the available data were sampled without the target constraint, filtering for exact equality generally cannot produce a reference ensemble.

Field SWD (full-field sliced Wasserstein distance) is an auxiliary comparison of generated trajectories with a specified reference ensemble. We flatten each of the NN full trajectories per ensemble and project them onto the same LL directions. With gℓ,(i)g_{\ell,(i)} and rℓ,(i)r_{\ell,(i)} denoting the ii-th sorted projection in each ensemble along direction ℓ\ell, Field SWD=1LN∑ℓ=1L∑i=1N(gℓ,(i)−rℓ,(i))2\mathrm{Field\ SWD}=\sqrt{\frac{1}{LN}\sum_{\ell=1}^{L}\sum_{i=1}^{N}(g_{\ell,(i)}-r_{\ell,(i)})^2}. Lower values indicate closer empirical distributions along these directions. For fixed-energy Euler and controlled NS, we use held-out trajectories at the same energy. For Burgers and reaction–diffusion, we independently generate reference trajectories at each target mass.

The revision presents Field SWD as a diagnostic relative to the specified reference ensemble. Our main conclusions rely on hard constraint error and sample-level PDE relative L2L_2 error, which measure constraint satisfaction and physical consistency, respectively.

References

  1. Kim et al. SoftFlow: Probabilistic Framework for Normalizing Flow on Manifolds. NeurIPS 2020, Sec. 3.
  2. Chen et al. Neural Ordinary Differential Equations. NeurIPS 2018, Sec. 4.
AuthorsOpenReview ↗

Unseen energy targets and sampling cost

Q2. The fixed-energy tests use the training energy. How well does the frozen model respond to energy levels absent from training?

Answer 2. At the training energy, we tested whether the sampler enforces the hard constraint while preserving PDE consistency. Each frame's target is one scalar, and the sampler generates the full field and its evolution.

On Euler and controlled NS, we tested 0.8× and 1.2× the training energy, then reused the same configurations at 2×. Each setting uses the same frozen model and 128 paired sources, with 100 forward calls per method. ECI and CCFM use the presets with the lowest mean relative L2L_2 across the four 0.8×/1.2× settings. PCFM-Relaxed and MAFM use the same terminal projection (PCFM Eq. (7)). Cells give energy CE / relative L2L_2 error.

MethodEuler 0.8×Euler 1.2×Euler 2×Controlled NS 0.8×Controlled NS 1.2×Controlled NS 2×
PCFM-Relaxed + Terminal Projection2.23e-07 / 11.76%2.24e-07 / 20.44%2.20e-07 / 37.92%2.20e-07 / 11.38%1.99e-07 / 13.98%1.96e-07 / 25.20%
ECI (two mixes, noise refresh)2.09e-07 / 11.84%2.01e-07 / 18.24%1.68e-07 / 35.03%2.03e-07 / 11.20%1.96e-07 / 11.67%1.74e-07 / 25.56%
Full Gradient2.45e-01 / 12.01%1.69e-01 / 9.57%5.00e-01 / 30.72%1.77e-01 / 9.09%1.62e-01 / 8.84%4.92e-01 / 30.53%
CCFM (n=2)2.10e-07 / 4.01%2.27e-07 / 4.82%2.20e-07 / 22.96%2.04e-07 / 3.99%2.14e-07 / 4.25%2.12e-07 / 19.25%
MAFM + Terminal Projection2.11e-07 / 2.96%2.24e-07 / 3.60%1.86e-07 / 9.85%2.10e-07 / 2.87%2.12e-07 / 3.45%1.94e-07 / 10.01%

MAFM with terminal projection satisfies the hard energy constraint and gives the lowest relative L2L_2 among feasible methods in all four original settings and at 2×. Its errors at 2× rise to 9.85% (Euler) and 10.01% (controlled NS). Terminal projection still meets the energy target, while the larger shift may reduce the frozen model's accuracy in predicting the evolution. We have reported the full comparison in the revised appendix.

Q3. How does sampling cost scale with the number of constraints, and what overhead do the persistent anchor and input adjoint introduce?

Answer 3. As in PCFM [1], our experiments use m≪dm\ll d, where mm counts scalar constraints and dd is the physical state dimension. Both methods solve an m×mm\times m multiplier system. Given a dense Jacobian, forming the Gram matrix and directly factoring and solving it cost O(dm2+m3)O(dm^2+m^3). Enforcing a strong-form PDE residual at every space-time grid point can make mm comparable to dd, bringing the generic dense cost to O(d3)O(d^3). We enforce the prescribed hard constraints during sampling and evaluate PDE consistency with an independent solver. The table compares sampling times with PCFM.

Timing in original Table 10MAFMPCFM comparator
Poisson, one layout (s / 512 samples)17.8815.77 (PCFM-Relaxed)
Reaction–diffusion (s / sample)0.043750.04249 (PCFM-Base)

We discuss the overhead of the persistent anchor and input adjoint separately.

Persistent anchor. We carry one dd-dimensional anchor across steps and linearly interpolate it with the fixed source noise to form the next query, at O(d)O(d) work per step. We still use one model forward call per step.

Input adjoint. As in Full Gradient, we backpropagate the predicted endpoint's constraint loss through the frozen model to obtain the input gradient gtg_t at the current query. Since xt=(1−t)x0+tatx_t=(1-t)x_0+ta_t, the chain rule gives the anchor gradient tgtt g_t. Each guided step adds one vector–Jacobian product (VJP), and multiplication by tt is a vector scaling. In a paired ablation on two 64-sensor Poisson layouts, mean sampling time per 512 samples increased from 5.23 s with the adjoint off to 14.04 s with it on. Both variants use 100 forward calls. The guided variant uses 100 VJPs, including the first step where its guidance coefficient is zero.

References

  1. Utkarsh et al. Physics-Constrained Flow Matching. NeurIPS 2025, §3.3.
AuthorsOpenReview ↗

Baseline comparisons on five PDE benchmarks

W1/W2. Can you add direct comparisons with CCFM, FunDPS, DiffusionPDE, D-Flow, PnP-Flow, and LDF? The persistent anchor also resembles a relaxed endpoint projection. What does retaining the anchor across steps add?

Answer. A standard relaxed projection takes a candidate point and moves it partway toward its projection onto the constraint set. In a relaxed endpoint iteration that blends the old anchor with a projected prediction, one weight controls both how much of the new prediction is adopted and how much the residual is reduced. MAFM controls these separately. At the current anchor, the tangential update sets how quickly the anchor tracks new predictions, and the normal update sets how quickly the constraint residual shrinks. The anchor can therefore make a large constraint correction while incorporating only part of the latest prediction and retaining information from earlier steps.

We evaluated CCFM [1], D-Flow [6], PnP-Flow [4], and LDF [5] on all five PDE benchmarks. LDF uses the authors’ default step size and no terminal projection, with 198 model forwards per sample. CCFM uses Gaussian projection, p(s)=(s/2)2p(s)=(s/2)^2, and exact terminal projection. Mass and sensor constraints are affine. We linearize the energy equalities at the predicted endpoints. Cells give hard constraint error (CE) / PDE relative L2L_2 error, plus divergence RMSE for the fixed-energy tasks.

MethodControlled NSEulerBurgersReaction–diffusionPoisson (64 sensors)Poisson (256 sensors)
CCFM n=22.35e-07 / 3.48% / 0.064022.33e-07 / 3.68% / 0.064995.39e-08 / 5.73%5.96e-08 / 6.83%0 / 22.78%0 / 46.47%
D-Flow0.00748 / 3.27% / 0.059940.00751 / 3.54% / 0.062560.995 / 39.01%0.0317 / 3.45%0.555 / 1.03%0.567 / 1.04%
PnP-Flow0.0554 / 10.25% / 0.188680.0581 / 11.33% / 0.202790.00149 / 0.71%0.00221 / 0.98%0.538 / 1.22%0.234 / 1.00%
LDF (198 NFE)5.35e-04 / 5.86% / 0.151825.85e-04 / 7.13% / 0.165067.22e-03 / 4.54%2.69e-03 / 21.81%3.84e-02 / 14.96%6.65e-02 / 17.53%
MAFM6.89e-06 / 2.76% / 0.047366.84e-06 / 2.94% / 0.048294.22e-08 / 0.965%1.13e-07 / 2.74%1.72e-07 / 1.70%1.75e-07 / 2.25%
MAFM + Terminal Projection2.43e-07 / 2.76% / 0.047362.37e-07 / 2.94% / 0.048295.32e-08 / 0.965%6.77e-08 / 2.74%0 / 1.70%0 / 2.25%

Within each benchmark, methods share a frozen model and paired sources. Independent solvers provide the reference solutions for relative L2L_2 error. For time-dependent cases, the solver starts from each generated trajectory’s own initial field (own-IC). The table reports hard constraint satisfaction and PDE consistency. MAFM achieves the lowest PDE relative L2L_2 error and divergence RMSE on both fixed-energy benchmarks. D-Flow and PnP-Flow yield lower relative L2L_2 errors on some benchmarks, but neither meets the prescribed energy constraint on the fixed-energy benchmarks. MAFM has lower constraint and PDE relative L2L_2 errors than LDF across all six settings.

FunDPS (function-space DPS) [7] and DiffusionPDE [8] both use per-step full-gradient guidance, backpropagating a loss on the denoised prediction to the current sampling state. In the paper, our Full Gradient results evaluate this shared mechanism. To plus, DiffusionPDE also evaluates an explicit PDE residual loss, which can be computationally costly.

References

  1. Liang et al. CCFM. arXiv v1, September 2025. Accepted at NeurIPS 2026.
  2. Utkarsh et al. PCFM. NeurIPS 2025.
  3. Cheng et al. ECI. ICLR 2025.
  4. Martin et al. PnP-Flow. ICLR 2025.
  5. Kurtz and Davydov. Lagrangian Dual Flows. arXiv preprint, July 2026.
  6. Ben-Hamu et al. D-Flow. ICML 2024.
  7. Yao et al. FunDPS. NeurIPS 2025.
  8. Huang et al. DiffusionPDE. NeurIPS 2024.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response

We thank all reviewers for their constructive feedback. In response, we conducted additional experiments, expanded baseline comparisons, clarified our method and theory, and improved the presentation. The corresponding changes are highlighted in the revised manuscript. We sincerely hope that these substantial revisions and additional results will address the reviewers' concerns and encourage a renewed assessment of the contributions and merits of our work. The major revisions are summarized below.

  1. Core idea and sampling mechanism. MAFM maintains a persistent anchor as an evolving estimate of the final sample, retaining earlier corrections as it incorporates new endpoint predictions. At each step, the pretrained flow model guides the anchor's tangential motion, while a geometric correction contracts constraint violations along the normal directions. The updated anchor is interpolated with fixed source noise to form the next query, passing constraint corrections into subsequent model predictions (Section 3.3; Fig. 2; Algorithm 1).

  2. Expanded and fair baseline evaluation. We compare with CCFM, D-Flow, PnP-Flow, and LDF across five PDE benchmarks, including both Poisson sensor settings, and clarify the connection to FunDPS and DiffusionPDE through shared full-gradient guidance. We revisit PCFM and ECI in the fixed-energy experiments, document the protocols, and add comparisons using PCFM's published endpoint-projection and reinterpolation procedure. We distinguish terminal energy satisfaction from trajectory-level PDE consistency, measured by an independent solver initialized from each sample's own initial condition. MAFM achieves lower PDE relative L2 errors than CCFM across the evaluated benchmarks and strong physical consistency on fixed-energy tasks. D-Flow and PnP-Flow perform better on some individual metrics, but do not satisfy the prescribed energy constraints in those settings (Experiments; Appendix).

  3. Distributional evaluation. We clarify Field Sliced Wasserstein Distance (Field SWD) and the reference ensembles for fixed-energy and target-mass settings. We distinguish the manifold-supported distribution of feasible samples from the original full-dimensional data distribution. Field SWD is an auxiliary diagnostic; hard constraint error and sample-level PDE relative L2 error are the primary measures of constraint satisfaction and physical consistency (Evaluation Metrics; Appendix).

  4. Generalization to unseen constraint levels. We add Euler and controlled Navier–Stokes experiments at target energies of 0.8×, 1.2×, and 2× the training energy, using frozen models and paired sources. MAFM with terminal projection satisfies the energy constraints and achieves the lowest PDE relative L2 error among feasible methods at all tested levels. At 2× energy, errors are 9.85% for Euler and 10.01% for controlled Navier–Stokes (OOD Experiments; Appendix).

  5. Theoretical motivation for constraint contraction. We explain the role of κ(t): the anchor residual c(t)=C(a(t)) follows ċ(t)=−κ(t)c(t), yielding exponential contraction. We connect this design to attracting slow manifolds and clarify the integrated contraction budget K. With N equal steps, each normal update retains e(−K/N)e^{(−K/N)} of the preceding residual (Proposition 1; Section 3).

  6. Component ablations. We clarify that setting the endpoint-tracking coefficient γ(t) or adjoint-guidance coefficient η(t) to zero does not disable constraint correction; normal contraction remains active. On Poisson, removing endpoint tracking or adjoint guidance increases PDE relative L2 error from 1.70% to 29.0% and 38.0%, respectively, with essentially unchanged sensor constraint error (Ablations; Appendix).

  7. Presentation and notation. We expand Fig. 2 to show how endpoint prediction, observation guidance, anchor correction, and interpolation with fixed source noise pass corrections to subsequent queries. We explain why intermediate queries may remain infeasible while the final output inherits the corrected anchor. We also define λ, the endpoint-predictor Jacobian transpose, and the vector–Jacobian product (VJP) (Fig. 2; Algorithm 1; Eq. 8).

  8. Computational complexity and runtime. For m scalar constraints and a d-dimensional state, the multiplier solve in both MAFM and PCFM has dense complexity O(dm²+m³). We quantify the O(d) anchor update and additional VJP cost. MAFM is slightly slower in our tests; for Poisson with 64 sensors, it takes 17.88 s per 512 samples versus 15.77 s for PCFM (Complexity Analysis; Appendix).

Together, these revisions strengthen the theoretical and empirical assessment MAFM. We appreciate the reviewers' suggestions, which have improved the rigor and clarity of our paper.

Dear Reviewers,

Please review the authors' rebuttal and participate in the discussion to address any remaining questions before the period closes on Nov 18 AoE.

Thank you!

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.