- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
- The paper introduces Source Anchoring for Physical Consistency (SAPC), a Functional Flow Matching method that projects the source noise onto a constraint manifold before generation. By incorporating this projection into training and additionally projecting the estimated endpoint during sampling, the method aims to improve constraint satisfaction while reducing distributional mismatch.
- The method is evaluated on six tasks across five PDE systems, comparing against several recent baselines. The results demonstrate substantial improvements in distributional fidelity while maintaining constraint satisfaction comparable to the strongest projection-based methods.
Major strengths/weaknesses and questions are listed below; here are some further minor notes:
-
Related work: Fine-tuning approaches appear to be missing from the discussion, especially regarding the statement "Such approaches are tied to the selected system and must be redesigned for each new problem" (lines 139–140), since fine-tuning exactly tackles this problem of quickly adapting a pre-trained model to various settings. PCFT or PIRF might be suitable references. Otherwise, the related work nicely contextualizes the contribution within recent literature.
-
Physical constraints: The statement in lines 80–84 is somewhat misleading. The considered constraints enforce necessary, but generally insufficient, conditions for solving the PDE, thereby encoding partial information about the dynamics. However, this limitation is presented as a problem of existing methods, even though SAPC retains it.
-
Physical consistency: The definition of the "manifold of physically-consistent states" (Eq. 1, lines 173–187) is confusing. The set contains states satisfying the selected constraints, but not necessarily the governing PDE. This conflicts with the earlier definition of physical consistency through the full PDE residual. "Constraint-admissible states" might be more appropriate. Also, calling this set a manifold requires regularity assumptions.
-
Boundary conditions: The definition of the local constraint (lines 180–184) is narrower than suggested, as it only directly represents Dirichlet conditions rather than general boundary conditions.
-
Projection formula: The inverse in line 202 does not necessarily exist. The implementation already accounts for this through a pseudoinverse, which could also be reflected in the main text using the Moore–Penrose pseudoinverse.
-
Training and sampling: The wording in lines 207–209 could more clearly distinguish the source projection, performed during both training and sampling, from the posterior-mean projection, which is performed only during sampling.
-
Sampling algorithm: The update in Figure 2 and Algorithm 2 is called an Euler update. While technically defensible, "corrected Euler update" or a reference to the ECI-style update might be more descriptive. Furthermore, the indexing in Algorithm 2 is imprecise: the loop index is not used explicitly, and would be needed. Replacing and with their expressions in terms of could also make the algorithm more self-contained and easier to read.
Critical strengths
- S1 Clear motivation and positioning The paper addresses an interesting and relevant issue in physically constrained generative modeling: improving constraint satisfaction without sacrificing distributional fidelity. The distinction between constraint violations and distributional mismatch is compelling, and the paper is generally well motivated and nicely contextualized within recent literature.
- S2 Extensive and convincing experimental evaluation The empirical evaluation is a major strength. The authors consider several different PDE systems, compare against a broad range of recent baselines, and provide extensive ablation studies. The improvements are substantial and consistent across the considered tasks. In particular, Figure 4 nicely demonstrates the gains in both distributional fidelity and constraint satisfaction.
Critical weaknesses
-
W1 Limited methodological novelty and insight into the proposed mechanism
- The methodological contribution appears rather incremental. The proposed algorithm essentially modifies an existing Functional Flow Matching approach by projecting the source distribution onto the constraint manifold and adapting the training objective accordingly, while retaining an existing projection-based sampling strategy. While this is a sensible modification, the mechanism itself is straightforward.
- The theoretical analysis provides some results on constraint residuals along the interpolant, which are a nice addition. However, these are essentially expected interpolation-error bounds for general smooth functions. They establish that interpolation between admissible endpoints preserves affine constraints and approximately preserves nonlinear ones, but provide limited insight into the actual generative mechanism or the improved distributional fidelity.
- The central question is therefore why source projection leads to such substantial improvements in distributional fidelity. The authors provide an intuitive explanation: starting from a constrained source keeps the generative trajectory closer to the admissible set, reducing the need for large corrective projections that would otherwise distort the generated distribution.
- However, the empirical evidence does not fully establish this interpretation. In particular, the ablations with endpoint projection but without source anchoring achieve distributional fidelity comparable to unconstrained FFM, which does not directly demonstrate that projection itself introduces additional distributional bias. Furthermore, the conditional FFM baseline achieves competitive distributional metrics without source anchoring. These observations at least partially challenge the proposed explanation and suggest that other effects might play a dominant role.
- Consequently, I find the explanation of the underlying mechanism somewhat limited. A deeper understanding of when and why source anchoring improves distributional fidelity would significantly strengthen the contribution and be valuable for future developments. Relatedly, some claims regarding "distributional bias" and the mechanism behind its reduction appear stronger than what is directly supported by the experiments or theoretical results. More conservative wording would be appropriate.
-
W2 experimental details I was surprised by the relatively poor performance of PBFM, which I have found to be a competitive baseline in my own experience. After examining the provided source code and comparing it with the original implementation, I identified three potential issues that might contribute:
- 1) Loss computation There might be an issue in the loss computation for PBFM, at least regarding the heat equation. PBFM uses the callback residual_from_reference to supply the specific per-sample target, but this callback is absent for the heat IC constraint. Therefore, this baseline might use a wrong physics target during training by enforcing a fixed initial condition rather than the actual per-sample constraint. (see src/projectors/constraints/factory.py, lines 54-77)
- 2) Custom implementation The provided code uses a custom implementation of ConFIG, that at least differs by using torch.linalg.solve (plus fallback) vs. ConFIG's original torch.linalg.lstsq. This might be more of a robustness issue especially for near ill-conditioned systems, but using the official conflictfree package would at least be a simple reproducibility improvement.
- 3) Methodoogical The original PBFM paper conditions its model on physical parameters (e.g. Reynolds number). Both training and sampling pass these parameters as variables to the network. An exception is the Darcy case, in which the joint distribution over state and parameters is modeled directly. Nonetheless, this means that the original method receives or models the conditioning parameters. For comparison against SAPC, which also does not receive conditioning input, an unconditional PBFM variant might be a meaningful ablation. Generally, however, I believe it would be fair to also inform PBFM about the constraint during inference through its native mechanism instead of withholding conditioning information.
The paper introduces and evaluates a seemingly overlooked mechanism in constrained generative modeling. While the methodological contribution is rather incremental, with a straightforward implementation and limited insight into the workings of the underlying mechanism, the experimental results are convincing and suggest that the proposed approach could be of interest to the ICLR community. I encourage further analysis and discussion of the raised empirical issues as well as a more conservative phrasing around claims interpreting how the method works.
Furthermore, some minor notational/mathematical issues are flagged above. Fixing these would improve readability and accessibility.
Questions for authors
- Q1 (W1) Could the authors provide further insight into why source anchoring leads to such substantial improvements in distributional fidelity? In particular, how does the proposed explanation of projection-induced distributional bias relate to the ablation results, where endpoint projection alone achieves fidelity comparable to unconstrained FFM, and conditional FFM achieves competitive distributional fidelity without source anchoring?
- Q2 (W2) Could the authors clarify the potential implementation and experimental issues raised in W2 and verify whether these affect the reported results (also the other experiments) or overall conclusions?
- Q3 Could the authors compare SAPC against conditional FFM combined with the same sampling-time endpoint projection? Given the competitive distributional fidelity of condFFM, this would help isolate the specific benefit of source anchoring.
5 comments
The results of the three variants of PBFM and of SAPC are reported below as PCA-FD / CE, mean over 3 seeds × 4 test sets:
- PBFM (submitted): Heat 496 / 0.65 · RD 630 / 0.69 · Stokes IC 245 / 0.69 · Stokes BC 432 / 0.67 · Burgers 977 / 0.64 · NS / 0.85
- V1 (per-sample target): Heat 488 / 0.71 · RD 599 / 0.68 · Stokes IC 245 / 0.66 · Stokes BC 430 / 0.68 · Burgers 997 / 0.71 · NS / 0.81
- V2 (official ConFIG): Heat 488 / 0.63 · RD 595 / 0.67 · Stokes IC 244 / 0.68 · Stokes BC 435 / 0.65 · Burgers 981 / 0.62 · NS / 0.81
- V3 (conditioned): Heat 2.61 / 0.10 · RD 33 / 0.90 · Stokes IC 23 / 0.16 · Stokes BC 10 / 0.28 · Burgers 2.27 / 0.16 · NS / 0.25
- SAPC: Heat 0.48 / · RD 1.01 / · Stokes IC 56 / 0.061 · Stokes BC 56 / 0.036 · Burgers 1.22 / · NS /
Dear authors, thank you for the revision and the clarifying comments.
My issues regarding presentation and the minor comments are resolved. I believe that the revised paper is easier to follow and particularly appreciate the now consistent mathematical presentation.
Also, W2 seems addressed sufficiently. This resolves the questions regarding the experiments, especially with respect to PBFM.
I am happy to keep my initial positive assessment.
Regarding W1/Q1:
- I appreciate the more careful wording in the revision. However, some passages still contain strong claims which are not fully supported by the theoretical insights into the mechanism (e.g., lines 94–95).
- The new ablations (Appendix D, Table 7) and comparisons with conditional FFM (Appendix E, Table 8) provide meaningful insights, suggesting that implicit conditioning, rather than reduced projection-induced bias, is the main driver of the distributional improvements.
- However, the theoretical explanation remains somewhat limited. In principle, flow matching can transport different source distributions to the same target distribution. While Eq. (4) establishes proximity to the admissible set, it does not explain why source anchoring should improve distributional fidelity.
Overall, the revision strengthens the paper, but I still believe the remaining gap in the theoretical understanding of the central mechanism would be a valuable contribution.
We thank the reviewer. The revised paper is uploaded with changes in blue.
Q3. We added this comparison (448–459; App. E, Table8, Fig.12), evaluating conditional FFM alone, with the same endpoint projection and sampler as SAPC, and with the full SAPC procedure. Alone, condFFM is competitive in distribution, but its CEL is orders of magnitude higher than SAPC’s. With the same endpoint projection, distributional fidelity is comparable, but SAPC has a lower CEL on every task, as it needs 1.3–3× smaller endpoint corrections. Adding source anchoring to condFFM further lowers its CEL on five of six tasks and improves its distribution on Heat and RD. Finally, when the conservation law carries trajectory-specific information (App.E, Table9), SAPC exploits it better than an input channel. The specific benefit of source anchoring is thus tighter constraint satisfaction and smaller corrections without an input channel, and the two mechanisms are complementary.
Q1, W1. To clarify the mechanism, we added ablations anchoring the source on the IC/BC or the global law only (App.D, Table7, Fig.11), a comparison with condFFM (Q3), and a measurement of the endpoint corrections (Fig.12). Anchoring on the IC/BC only reproduces the gain of SAPC, whereas anchoring on the global law only leaves the distribution at the level of S1. The main driver of the distributional gain is therefore the conditional information carried by the anchored source. S1 adds no information and stays close to FFM, and condFFM is competitive because it receives the same information explicitly. Projection-induced shift is only a secondary effect, where corrections are large. We now present source anchoring as a form of conditioning that requires no input channel nor additional sampling cost, complementary to explicit conditioning (18–20, 98–107, 264–269), and revised the claims on distributional bias accordingly. Finally, Eq.4 concerns admissibility along the training interpolant rather than distributional fidelity; we now report when its assumptions hold (Table5, App.A.3).
Q2, W2. We re-ran PBFM in three variants, one for each point, on all six tasks with 3 seeds × 4 test sets. V1) On Heat, the residual_from_reference callback was missing for the IC constraint. We added the per-sample physics target on every benchmark; the effect is small, and the other tasks stay within seed variability. V2) We replaced our implementation with the official conflictfree package. Our implementation treated the complex gradients of the FNO spectral weights as complex vectors, whereas all re-runs now flatten them to real views. Results are unchanged within error on every task. Since V1) and V2) remain within seed variability of the submitted run, the conclusions of the other experiments are unaffected. V3) We now give PBFM the constraint through its native mechanism, conditioning the network on the IC/BC through an input channel during training and sampling. This improves PBFM, reducing PCA-FD by one to two orders of magnitude. CondPBFM still does not enforce the constraints: its CE stays between 0.10 and 0.90, four to six orders of magnitude above SAPC on the affine tasks and RD, and 3 to 8× above it on Stokes. In distributional fidelity, SAPC remains better on Heat, RD and Burgers, while condPBFM is better on Stokes and NS (as condFFM).
Minor points. In the Related Work, we added fine-tuning approaches, citing PCFT and PIRF (140–142). We rewrote the Introduction to state that ICs, BCs and integral conservation laws are necessary but not sufficient conditions for solving the PDE (81–86), and the new Limitations paragraph states that this limitation is shared by every projection-based method, SAPC included (521–525); a new per-sample PDE residual confirms it empirically (App.C.3, G). Eq.1 now defines the set H of constraint-admissible states, and we replaced “manifold” with “set” or “admissible set” throughout; regarding regularity, we state after Eq.1 that H is a closed affine subspace for linear constraints and a smooth surface for C2, independent non-linear constraints (192–193). After the definition of the local constraint, we state that Rloc represents Dirichlet-type conditions (184). We replaced the inverse with the Moore–Penrose pseudo-inverse in the main text (208) and in App.A.2 and C.1, consistently with the implementation, which uses a Cholesky factorisation and falls back to a pseudo-inverse when AA⊤ is rank-deficient. We rewrote lines 213–217 to separate the source projection, performed at training and sampling, from the endpoint projection, performed only at sampling. Finally, we call the update a projected Euler update citing ECI (241), corrected the caption of Figure2, and rewrote Algorithm 2 on the explicit grid ti, so that every step uses the loop index, with At and Bt written explicitly in terms of t (244).