acceptodds
Under review as a conference paper at ICLR 2027

Beyond Parameter-Space Conflict: Intrinsic Function-Space Geometry for Physics-Constrained Flow Matching

Abstract

Physics-constrained generative modeling requires balancing distribution fidelity with physical consistency, two objectives that can induce conflicting optimization directions. Existing approaches typically resolve this conflict using parameter gradients, but we show that the resulting geometry depends on the chosen parameterization: function-preserving reparameterizations can substantially alter measured conflict without changing the represented model. Because multi-objective rules use this geometry to determine the update itself, equivalent parameterizations can induce different optimization decisions for the same model function. Based on this observation, we introduce Intrinsic Geometry Flow Matching (IGFM), which defines objective conflict over realizable first-order changes in function space and constructs a common-descent direction within this intrinsic geometry. For a fixed observation map and output metric, the resulting ideal direction is invariant to locally invertible changes of parameter coordinates and admits a scale-free common-descent guarantee. Across Darcy flow, Kolmogorov flow, and Structural Topology Optimization, IGFM reduces the primary physics or engineering error by , , and , respectively, relative to the strongest PBFM baseline, while maintaining comparable task-specific fidelity. These theoretical and empirical findings support realizable function-space geometry as a principled framework for resolving physics–distribution conflicts in generative modeling.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Physics-Constrained Generative Modeling, Flow Matching, Conflicting Gradients, Function-Space Geometry

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_BShjOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Intrinsic Geometry Flow Matching (IGFM), which constructs a common-descent direction using realizable function-space gradients instead of parameter gradients. The authors establish invariance and common-descent properties and demonstrate improvements on three physics-constrained generation tasks.

Critical strengths
  1. Interesting theoretical motivation. The paper identifies the parameterization dependence of parameter-gradient conflict and motivates the use of realizable function-space geometry. The invariance and common-descent properties provide a clear theoretical basis for IGFM.

  2. Strong empirical improvements and useful ablations. IGFM reduces the primary physics or engineering error by 40.0%, 22.4%, and 55.7% compared with the strongest PBFM baseline across the three tasks. The following analysis experiments provide useful evidence for the proposed method.

Critical weaknesses
  1. Insufficient evidence of gradient conflict during training. The reparameterization audit (Sec. 4.2, Table 4) is a static audit on a single trained Darcy model. In this audit, the parameter-space cosine γθ\gamma_\theta ranges from 0.40 to 0.93, and the intrinsic cosine γT\gamma_T stays within 0.3093–0.3122. Both values are positive, so the two objectives do not conflict at this state in either geometry. The paper does not show how γθ\gamma_\theta and γT\gamma_T evolve during training on the three tasks. It also does not show whether these changes relate to the gains in physical consistency achieved by IGFM.

  2. Limited evaluation on higher-resolution physical systems. The experiments use relatively low-resolution physical fields, while the closely related PBFM[1] has already evaluated 128×128 Kolmogorov Flow. This raises two concerns:

  • (1) Performance: It is unclear whether IGFM maintains its improvements at higher resolutions.

  • (2) Computational cost: The reported 1.02× training latency is measured on the current model configuration. It is unclear whether this efficiency holds at higher resolutions, where tangent solves and JVP/VJP computations may be more expensive.

  1. Limited to two objectives. The closed-form direction d⋆d^\star (Eq. 7) and all experiments use only two objectives: distribution matching and physical consistency. The authors acknowledge that the many-objective extension (Appendix A.4) is not evaluated empirically. Many physical problems have several constraints, such as multiple PDE residuals, boundary conditions or conservation laws. It is still unclear how well IGFM works in that setting, and how much it costs.

[1]. Physics vs Distributions: Pareto Optimal Flow Matching with Physics Constraints, ICLR 2026

Questions for authors

See weakness

Reviewer_AisLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposed Intrinsic Geometry Flow Matching (IGFM), a training approach to balance distribution matching and physical consistency in flow-based generative models. It builds on PBFM but changes how the 2 objectives are combined: instead of comparing parameter gradients, IGFM projects each objective's gradient onto the realizable function space tangent, then normalizes the directions and combines them. They approximately lift this direction to the parameter space with damped conjugate-gradient solves and AdamW updates. The authors evaluated their method on common problems in Darcy flow, Kolmogorov flow, and structural topology optimization, followed by some reparameterization studies and ablations.

Critical strengths
  1. The proposed training geometry makes sense and is systematically studied. The authors investigated their method via reparameterizations and ablations of the gradient representation, normalization, damping, solver depth, etc. On Darcy flow, they demonstrated the matched control gives a residual of 14.20 compared to 6.93 for IGFM. These comparisons give ablated evidences that auxiliary update mechanics do not explain the improvements alone.

  2. Improved physical results across 3 tasks. Compared with PBFM-4, IGFM reduces the Darcy residual, Kolmogorov divergence residual, and topology compliance error. The reported distribution fidelity is also comparable and improved relative to PBFM.

Critical weaknesses
  1. My main concern is on the methodological novelty and contribution. What does IGFM add beyond existing function-space optimization and common descent updates? IGFM adopts the normalized combination principle introduced by ConFIG, and ANaGRAM already studies the projected functional gradients. Methods like physics-constrained flow matching (PCFM) also enforce constraints into flow-based models without re-training. While matched parameter space comparisons were useful, they do not fully distinguish the benefit of geometric preconditioning from IGFM's particular way of combining the objectives. A comparison with a weighted-loss baseline using the same preconditioner, or clarifications if the current results in your paper already address this, may help resolve the additional contribution.

  2. The benefit under different physical objectives is not fully explained. In Appendix F.3, earlier topological experiments that optimize the equilibrium residuals, IGFM obtained 0.715 which is worse than 0.245 from PBFM-4. The authors explained why this residual is not a useful proxy for compliance but this does not explain IGFM's worse performance since the residual is being optimized itself. Some more clarifications can help understand this result and thereby establish when does the proposed method is beneficial.

Questions for authors
  1. How does IGFM differ from combining normalized natural gradient directions under the same output metrics, for example in relation to ANaGRAM and ConFIG methods? Please clarify the relationship in order to better identify the methodological contributions.

  2. What exact update is used in the "without normalization" ablation results? If this is not done, can the authors include a small comparison over the objective weights, i.e. optimizing a weighted sum of objectives using the same preconditioner, with the same solver and optimizer settings so as to isolate the benefits of IGFM?

  3. Addressing weakness 2: why does IGFM obtain a worse equilibrium residual than PBFM when the residual is the physical training objective?

  4. Does incorporating a new physical constraint via IGFM require re-training? How does IGFM compare with inference-time projection approaches such as PCFM, which enforces constraints using pretrained flow models without re-training? I am curious to understand the relative advantages of enforcing constraints during training vs inference, across physical consistency, distribution fidelity, and computational cost tradeoffs. If feasible, a comparison on a shared PDE task with matched constraint residuals could help clarify the trade-offs and novelty of IGFM vs other methods such as PCFM.

Reviewer_pgDBOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper revisits gradient-based conflict resolution (ConFIG/PBFM) for physics-constrained flow matching. It shows that the cosine between parameter gradients is not reparameterization-invariant: any cosine in (−1, 1) can be produced by a linear chart (Prop. A.1).

IGFM instead proceeds in three steps:

  • Represent each objective by the M-orthogonal projection of its output-space gradient onto the realizable tangent range(J).
  • Form the normalized bisector (two-objective MGDA) in that space.
  • Lift the result to parameters via damped CG solves of (JᵀMJ + λI)ξ_i = r_i with k = 5, and feed it to AdamW.

The ideal direction is chart- and objective-scale-invariant, and is a common-descent direction for both objectives. Experiments on Darcy flow, Kolmogorov flow and topology optimization use a 9.7M-parameter DiT. They report 40%, 22% and 56% reductions in physics error versus PBFM-4 at similar fidelity, plus chart audits, solver-dynamics studies, and a claimed 1.02× training cost relative to PBFM-1.

Critical strengths

It identifies a genuine conceptual issue with parameter-space conflict measures, and the main text is clearly written. Strong results on Darcy and topology optimization, with distribution fidelity largely maintained. Extensive ablations covering CG depth, damping, objective scale and observation metric. The matched parameter-space control is a good idea. The authors candidly report unflattering results, including the adaptive-solver reversal (App. C) and the original topology objective (App. F.3).

Critical weaknesses
  1. The relationship to natural-gradient / Gauss–Newton methods is understated. With G = JᵀMJ, we have g_i = J G†r_i. Hence ⟨g_D, g_P⟩_M = r_Dᵀ G† r_P and ‖g_i‖²_M = r_iᵀ G† r_i. The “intrinsic cosine” is therefore the cosine of parameter gradients in the generalized Gauss–Newton (GGN) metric. The lifted update is ½(G†r_D/‖g_D‖ + G†r_P/‖g_P‖): the average of each objective’s damped natural-gradient direction, normalized in GGN norm. In other words, it is ConFIG/MGDA in the GGN metric. Chart invariance then follows from the standard covariance of the natural gradient. The theoretical content is thus modest. Prop. A.1 is elementary linear algebra, and Prop. 2 restates MGDA properties in a different inner product. The “to the best of our knowledge, new” claim in App. H.4 should be tempered. Function-space and Gauss–Newton natural gradients have been applied specifically to physics-informed losses, e.g. energy natural gradient descent (Müller & Zeinhofer, ICML 2023). This line of work needs discussion.

  2. Attribution: are the gains from the conflict geometry or from preconditioning? The matched parameter-space control removes both the GGN preconditioning and the change of normalization at once. It cannot separate better-conditioned optimization from the intrinsic conflict resolution. Needed: the same CG/GGN solve applied to a fixed-weight objective L_D + ωL_P with a small ω sweep, and to PBFM’s update. A standard second-order or adaptive optimizer (K-FAC, Shampoo/SOAP) under PBFM would also help. The “Ambient gradients” row (Table 2) needs explaining. Since JᵀMh_i = JᵀMg_i = r_i, lifting h_i or g_i through the normal equations gives the same ξ_i. The two variants should differ only in the normalization constants (‖h_i‖ vs ‖g_i‖). A gap of 19.24 vs 6.93 from reweighting two terms would indicate strong sensitivity to the relative weighting, which feeds back into the attribution question. Please specify exactly how the ambient variant is computed and lifted.

  3. The invariance property does not describe the trained algorithm. The practical update uses Euclidean damping λI in θ, a Euclidean unit normalization in the step-length rule, and AdamW’s diagonal preconditioner. All three are chart-dependent. The twin-run audit confirms non-zero divergence (12.8). Notably, the adaptive solver tracks the invariant ideal more closely at each step. Yet it is less chart-consistent (51.3) than PBFM-1 (25.2) and worse in the long run (Table 7). This undercuts invariance as the explanation for the gains. The motivating audit uses dense synthetic linear charts with κ up to 3000, and in the trained model the cosine never changes sign (B.2). There is no evidence that ConFIG makes bad decisions in the default parameterization.

  4. Several results need reconciliation.

Topology optimization. In App. F.3, under the original equilibrium-residual objective, IGFM was clearly worse than PBFM on the objective being optimized (R_eq 0.715 vs 0.231/0.245) and on compliance (1.18 vs 0.68). The task was then reformulated.

Cost. Each CG iteration needs a GGN-vector product, i.e. a JVP and a VJP through the backbone and the unrolled physics. k = 5 for two objectives means about 10 such products per step, each comparable to a forward-plus-backward pass. A 2% overhead (or 18% uncompiled) is hard to reconcile with this. Please give matvec counts and a profiling breakdown.

  1. Protocol details and baselines are missing. Unspecified details: dataset sizes, splits and resolutions per task

Missing baselines: cheap multi-task methods (PCGrad, CAGrad, Nash-MTL). A sampling-time constraint method, such as the cited DiffusionPDE line of work, would add context.

  1. Scale. The model is a 9.7M-parameter DiT at 64×64. Please discuss how GGN-vector products scale to larger backbones.
Questions for authors
  1. What happens with GGN-preconditioned weighted-sum training, and with PBFM using GGN-preconditioned gradients?
  2. What exactly is in O_B, and what is its dimension relative to the parameter count? Is J numerically full row rank on the velocity block, and if so, what does the projection change?
  3. How is the ambient variant computed?
  4. Can you provide the timing breakdown and clarify Table 16?
  5. What unrolling depth does IGFM use?

Minor

There are “Eq. equation 3/8/64” reference artifacts. Much of the appendix prose has redundant trailing clauses (e.g. end of A.4, A.6, B.3, and the intro to App. C). Trimming would help readability. Is the 1.5×10⁻⁶ learning rate in D.6 a typo for 1.5×10⁻⁵? Please specify the Fig. 1 toy problem. Please avoid bolding differences that are not significant.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.