acceptodds
Under review as a conference paper at ICLR 2027

Equation-Coordinate-Invariant Residual Metrics for Physics-Informed Neural Networks

Abstract

Physics-informed neural networks (PINNs) can learn different solutions from algebraically equivalent formulations of the same partial differential equation (PDE). For coupled systems, invertible scaling and mixing preserve the physical constraints but change standard residual losses and their optimization geometry. We formalize this dependence as equation-coordinate bias and introduce ECI-PINN, which replaces the algebraic residual norm with the minimum correction required to satisfy the locally linearized PDE constraints in a dimensionless jet space. The resulting full residual metric accounts for coupling between equations rather than normalizing each equation independently. When the residual Jacobian with respect to the jet has full row rank, we prove that the unregularized metric preserves loss values and detached parameter gradients under pointwise invertible residual-basis transformations independent of network parameters and jet state. Matched deterministic training therefore follows the same optimization trajectory. We evaluate ECI-PINN using PDE-Rep, a protocol spanning Wave, Reaction-Diffusion, and Kovasznay benchmarks, five equivalent residual representations, and three random seeds. ECI-PINN achieves the lowest identity, average, and worst-case relative L2 errors among the compared methods on all three tasks, reducing the Wave worst-case error by more than 50-fold relative to Vanilla PINN. Comparisons with diagonal metrics and residual normalization support the importance of modeling the full coupled residual geometry. These results establish equation-coordinate invariance as a design principle for reliable physics-informed learning.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
physics-informed neural networks, scientific machine learning, equation-coordinate invariance, residual geometry, partial differential equations

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_tT4vOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The manuscript proposes a iteratively reweighed least squares-like modification of the PDE term in a PINN loss. The idea is interesting, but the manuscript has two significant weaknesses: (i) It does not use Gauss-Newton type (or natural gradient in ML parlance) methods for optimization, although it is by now well-established that these are orders of magnitude more accurate than Adam or L-BFGS, (ii) it tests the method on much too simple problems. These problems are fundamental and lead me to recommend rejection. I give more details below, and a suggestion on how to use the paper's idea in a Gauss-Newton context.

Technical summary The plain PINN loss is

L(θ)=∣∣rPDE(θ)∣∣2+λBCLBC(θ)+λICLIC(θ).L(\theta) = ||r_{\text{PDE}}(\theta) ||^2 + \lambda_{\text{BC}}L_{\text{BC}}(\theta) + \lambda_{\text{IC}}L_{\text{IC}}(\theta).

The manuscript proposes to re-weight the PDE term to be

$ ||r_{\text{PDE}}(\theta) ||_{W(\text{sg}(\theta))}^2 $

where WW is a block-diagonal matrix (with small blocks) depending on θ\theta through the jet of uθu_\theta and sg(θ)\text{sg}(\theta) is the stop gradient operator. The re-weighting matrix is obtained in the following way:

For a fixed collocation point xix_i the point-wise satisfaction of the PDE is encoded via the function FF taking spatial coordinate and the jet as input and returning in Rq\mathbb R^q

riPDE(θ)=F(xi,uθ(xi),Duθ(xi),D2uθ(xi),…)=:Fxi(ξi)∈Rqr_i^{\text{PDE}}(\theta) = F\big(x_i, u_\theta(x_i), Du_\theta(x_i), D^2u_\theta(x_i), \dots\big) =: F_{x_i}(\xi_i) \in \mathbb{R}^q

The plain PINN loss uses (the sum over) ∣∣Fxi(ξi)∣∣22||F_{x_i}(\xi_i)||^2_2. There are however several choices of F that lead to the same PDE. This redundancy leads the authors to propose a geometric variant in place of the squared L2 loss based on the distance to the zero set of the PDE

dist⁡(ξi,Mxi)2\operatorname{dist}(\xi_i, \mathcal M_{x_i})^2

where Mxi=ξ∣Fxi(ξ)=0\mathcal M_{x_i} = \\{ \xi \mid F_{x_i}(\xi)=0 \\}. To make this tractable the constraint is linearized at the current jet which turns the problem into a quadratic minimization over an affine linear subspace (the linearization of the manifold) and this has a closed form solution. Denoting by JiJ_i the Jacobian of FxiF_{x_i} with respect to ξ\xi at ξi\xi_i, this approximation can be computed as (assuming full rank of JiJ_i)

$ F_{x_i}(\xi_i)^T (J_i J_i^\top)^{-1} F_{x_i}(\xi_i) $

So setting Wi=(JiJi⊤)−1W_i = (J_i J_i^\top)^{-1} as a block in the block diagonal weighting matrix WW is the formula above.

Critical strengths

Addressing freedom in the choice of FF for the PDE is an interesting idea.

Critical weaknesses

There are two critical weaknesses:

1.) The manuscript does not use state of the art optimizers for PINNs. Gauss-Newton/natural gradient type methods perform orders of magnitude better for essentially all known instances in PINN training, while at the same time remaining scalable and efficient. GN optimization often takes a fraction of the time of Adam [ENGD, DeepMind, Karniadakis]. Benchmarking a new loss function with an outdated training protocol is therefore not meaningful, and unfortunately weakens the empirical findings significantly.

2.) The paper uses very easy benchmark problems that can be solved to high accuracy with Gauss-Newton and the plain PINN formulation. I prompted Claude Opus 5.5 with reasoning on high through the web API in the following way:

"Can you take the examples of this paper, use JAX in double precision and a kernel formulation of Levenberg Marquardt with constant damping of 1e-5 and backtracking line search with Armijo and quickly run their experiments and give me the relative L2 error?"

The results were: Wave relative L2 3.68e-5, Reaction-Diffusion relative L2 1.89e-4, Kovasznay relative L2 3.3e-4.

While I do not intent to check Claude's code carefully, this is exactly what I expected and more accurate than all runs in this paper.

Nevertheless, I think there might be something interesting to the core idea of the paper and I give the details here: Applying Levenberg-Marquardt (=Gauss-Newton with damping) in kernel formulation to the weighted PDE residual (dropping other terms for simplicity) gives the update formula (after some sorting of terms)

θk+1=θk−ηkJ⊤(JkJk⊤+λWk)−1rk\theta_{k+1} = \theta_k - \eta_k J^\top(J_kJ_k^\top + \lambda W_k )^{-1}r_k

where in this case JkJ_k is the Jacobian of the residual with respect to the trainable parameters (sorry for the clash with your notation). The WkW_k is your weighting matrix. So what this yields is a principled way to substitute the unmotivated identity based damping that is very often used. I leave it to the authors to pursue this idea---it is certainly not ment as a suggestion for what has to be done in a revision, as it will lead to a significantly different paper.

Literature:

[ENGD] https://proceedings.mlr.press/v202/muller23b/muller23b.pdf

[DeepMind] https://arxiv.org/html/2509.14185v1

[Karniadakis] https://arxiv.org/pdf/2604.05230

Questions for authors

I have no questions.

Reviewer_1nrXOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies how invertible changes of a coupled PDE residual basis, r↦A(z)rr\mapsto A(z)r, affect PINN training. It proposes ECI-PINN. The method measures the minimum correction needed to satisfy locally linearized PDE constraints in a fixed dimensionless jet space. Its metric is r⊤(JJ⊤)−1rr^\top(JJ^\top)^{-1}r. Under stated rank and transformation assumptions, the paper proves invariance of the unregularized metric value and detached update. It also derives a deterministic-trajectory invariance result. Experiments cover Wave, Reaction-Diffusion, and Kovasznay PDEs, five residual representations. The paper reports loss-value diagnostics, solution-error comparisons, and ablations. It claims improved accuracy and robustness across equivalent residual representations.

Critical strengths

Equation-coordinate invariance is an interesting perspective on PINN training. Equivalent PDE constraints can induce different residual objectives. The proposed minimum-correction metric provides a precise invariance criterion under explicit assumptions. This concept could motivate useful PDE reformulations.

Critical weaknesses

1. The submitted implementation and evaluation protocol do not support the empirical claims.

Equation (6) requires the residual Jacobian with respect to the dimensionless jet. The code in the submitted Supplementary Material instead uses an identity matrix in the PDE training path. It also uses the known transformation AA to recover the canonical residual. Thus, its unregularized ECI computation reduces to ∥r∥2\|r\|^2, not the proposed jet-space metric. This discrepancy is substantive for the nonlinear Reaction-Diffusion and Kovasznay systems, whose true jet Gram matrices are generally state-dependent and non-diagonal.

The evaluation is also not controlled. The final ECI configuration uses 5,000 epochs and evaluates every 25 epochs. It selects the checkpoint with the lowest reference relative L2L^2 error. The main Vanilla and MultiAdam configurations use 2,000 epochs, evaluate every 100 epochs, and report the final checkpoint. The ECI selection uses the exact solution on the evaluation grid. Appendix C.2 discloses the selection rule, but the baselines do not receive an equivalent procedure. Optimizers and schedules differ as well. These confounds prevent attributing the reported improvements to the proposed residual metric.

The empirical claims therefore require a faithful implementation and new controlled experiments. Clarifying the existing code alone would not resolve the evaluation problems.


2. The practical contribution, distinction from prior work, and presentation remain insufficiently established.

Synthetic invertible transformations are valid tests of representation invariance. However, undoing an artificial transformation with its known inverse does not demonstrate the practical value of the proposed jet metric. The evaluation should separate raw transformed residuals, known-AA unmixing, equation-wise balancing, and the actual diagonal and full jet metrics. These comparisons require the same training pipeline. Tests on canonical and practically motivated PDE formulations are also needed. Correcting the Jacobian alone would not address this gap.

The equal-weight coupled residual norm is a valid baseline. However, it does not represent the range of established equation-wise weighting and gradient-based optimization methods. ConFIG [6] and HARMONIC [7] are relevant alternatives. PINNacle [8] provides a broader benchmarking context. These methods need not satisfy arbitrary equation-basis invariance to be relevant to the paper's claims about solver accuracy.

The novelty also needs sharper positioning. Multi-constraint Sampson approximations already use a full-Gram minimum-correction metric [2]. The paper should distinguish its jet formulation and detached update from this established construction. Operator preconditioning [3], energy natural gradients [4], and PINN loss-landscape analysis [5] address training-relevant geometry. Invariance of a jet-space metric alone does not imply better parameter-space conditioning or lower solution error. The paper does not establish this connection or a clear practical advantage.

The presentation compounds these issues. Some symbols, including zz, appear before their definitions. Supplementary-table references are inconsistent. The connections among the geometric metric, detached update, stabilization, and representation-specific optimizer portfolio are also difficult to follow. The paper needs a clearer account of the method actually evaluated.

Questions for authors

1. Implementation and result provenance. Which implementation produced the reported results? Where does the submitted training code compute SξS_\xi and J=∂r/∂ξ^J=\partial r/\partial\widehat\xi? How does this agree with the identity Jacobian in the submitted Supplementary Material? Which reported configurations use the ordinary residual loss? Please provide the corresponding per-run configurations and result provenance.

2. Evaluation fairness. Did all methods use identical training budgets, optimizer schedules, and checkpoint-selection rules? Please report matched final-checkpoint comparisons without reference-based checkpoint selection. Please also provide a faithful MultiAdam [1] comparison. How much of the gain remains when the optimizer portfolio and selection protocol are held fixed?

3. Practical contribution. Does the true jet metric improve accuracy beyond known-AA unmixing, equation-wise reweighting, and optimizer refinement? Please compare these alternatives in a controlled pipeline. Are there results on canonical or practically motivated PDE formulations where the metric is computed directly from the PDE, without recovering a known canonical residual?


References

[1] Yao et al., "MultiAdam: Parameter-wise Scale-invariant Optimizer for Multiscale Training of Physics-informed Neural Networks." ICML 2023.

[2] Rydell et al., "Revisiting Sampson Approximations for Geometric Estimation Problems." CVPR 2024.

[3] De Ryck et al., "An operator preconditioning perspective on training in physics-informed machine learning." ICLR 2024.

[4] Müller et al., "Achieving High Accuracy with PINNs via Energy Natural Gradient Descent." ICML 2023.

[5] Rathore et al., "Challenges in Training PINNs: A Loss Landscape Perspective." ICML 2024.

[6] Liu et al., "ConFIG: Towards Conflict-free Training of Physics Informed Neural Networks." ICLR 2025.

[7] Bu et al., "Harmonized Cone for Feasible and Non-conflict Directions in Training Physics-Informed Neural Networks." ICLR 2026.

[8] Hao et al., "PINNacle: A Comprehensive Benchmark of Physics-Informed Neural Networks for Solving PDEs." NeurIPS 2024, Datasets and Benchmarks Track.

Reviewer_NviDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses the issue arising from the fact that the squared-residual loss function used when solving coupled Partial Differential Equations (PDEs) using Physics-Informed Neural Networks (PINNs) can be different for algebraically identical formulations of said PDEs. Even though mathematically the solutions one would aim to recover should not be altered by certain transformations of the equations, including certain rescalings or mixings, the training runs along with the final obtained loss and accuracy values can be altered as a result of the modified landscape that the new representation of the PDEs has caused. The authors call this the equation-coordinate bias. The proposed ECI-PINN methodology approaches the issue working in the context of jets, namely the Taylor data of the numerical solution u at a point in coordinate space z. It replaces the residualsquared term in the usual PINNs loss function by the minimum squared correction of the (dimensionless) jet ˆξ = S−1 ξ ξ needed to satisfy the linearized constraint r + Jδˆξ = 0, with J = ∂r/∂ˆξ. Thus, at each collocation point, this evaluates how far the network’s local solution data would have to move to satisfy the linearized equations. The authors prove a series of results in the proposed construction: the ECI loss value is invariant (Thm. 1), the detached gradient is invariant (Thm. 2), and hence the matched deterministic runs coincide (Cor. 1). These hold under the conditions that J is full-row-rank, the jet scale is fixed and that and the transformations considered, such as some recombination of the equations, does not depend either on the network or its current output. When the above conditions hold true, a rewriting of the equations in a new formulation has no effect on the loss or gradient in the runs, such that two otherwise identical training runs follow the same trajectory. The results presented in the paper are shown for experiments on three analytic benchmarks: a first-order Wave system, reaction–diffusion with manufactured forcing, and Kovasznay flow at Re = 40. Each uses a four-layer width-64 MLP, 512 collocation points and three seeds, under five representations (identity, diagonal scaling, constant dense, position-dependent and anisotropic mixing). As stated in the manuscript, the trained configuration adds whitening, a warm-up blended with the vanilla loss, a condition-gated fallback, a per task/representation optimizer portfolio with L-BFGS refinement, and best-checkpoint selection. ECI-PINN has the lowest Identity, Average and Worst relative L2 errors in Table 1; for example, its Wave worst case is 1.92e-2 vs. 9.79e-1 for a vanilla PINN. It also beats a diagonal-Gram variant, at about twice the runtime.

Critical strengths
  1. A real question with a cheap remedy. Equivalent formulations of a coupled system induce different PINN objectives, and this is rarely made explicit. Definition 1 and Proposition 1 state it cleanly. The remedy costs one q × q solve per point.

  2. Careful theory. The closed form and Thms. 1–2 are correct. The paper clearly separates the geometric value from the detached surrogate update actually used (Eq. 12 vs. gt), which is often glossed over.

  3. A sensible connection and ablation axis. The link to Sampson-type geometric errors is natural, and full vs. diagonal Gram is the right ablation axis.

  4. A reusable protocol. The representation-stress protocol keeps evaluation in canonical variables while changing only the training residual basis. Gram diagnostics are reported.

  5. Candour. Undocumented settings, withheld aggregates and the limits of the code package are openly stated.

Critical weaknesses

Appendix tables and figures are cited by their printed captions; the text’s cross-references are offset (text Tables S1–S4 = printed S3–S6; text Figures S1–S9 = printed S6–S14).

  1. The invariance principle is largely known, narrow in scope, and not operative in the method that produced the results.

Novelty. The metric is not new, and neither is the premise.

• r⊤(JJ⊤)−1r is the multi-constraint Sampson (first-order geometric) error, with the jet in the role of the measurement (Sampson 1982; Taubin 1991; Hartley & Zisserman 2004; Rydell et al. 2024). Its invariance under r ↦→Ar, J ↦→AJ is immediate because the value depends only on the linearized constraint set. Thm. 2 then follows in one line from the stop-gradient.

• For q = 1 with unit jet scales, ECI coincides exactly with the Jacobian normalization r2/(1+J2 y) of Bento et al. (2026, Eq. 25). ECI-Diag is that normalization applied per equation.

• The premise has already been shown and exploited. Cao & Zhang (2025, cited) demonstrate that equivalent systems with the same solution but differently conditioned Jacobians train differently. Song, Cao & Zhang (2025) scale the PDE residual with a preconditioner built from the PDE-system Jacobian.

• Invariance under F ↦→AF is the classical affine covariance of Newton-type methods (Deuflhard 2004). In the paper’s overparameterized regime (∼103 residual rows, ∼104 parameters), the undamped minimum-norm Gauss–Newton/natural-gradient step −J+ θ R of ENGD/GNNG-type PINN optimizers (Müller & Zeinhofer 2023; Jnini et al. 2024; Jnini & Vella 2025) is invariant to any invertible residual transformation, with no jet metric needed. None of this is discussed. What remains new is the coupled (q ≥2) full-Gram extension of Jacobian normalization in a scaled jet space.

Scope. The guarantee covers only pointwise transformations independent of the parameters and the jet.

• Reformulations that occur in practice are either diagonal (units, nondimensionalization), which ECI-Diag already handles exactly, or outside the theory: first- vs. second-order forms, auxiliary variables, velocity–pressure vs. vorticity formulations, conservative forms, state-dependent factors.

• No naturally occurring non-diagonal, state-independent mixing is given. R2–R4 are synthetic, and their matrices and severity are undocumented.

• The arbitrariness is relocated to the hand-chosen jet scale Sξ, to which ECI is not invariant (J ↦→JD−1).

• On Wave, J is constant with disjoint row supports, so JJ⊤is a constant diagonal matrix fixed by Sξ. ECI-PINN is then a vanilla PINN with two fixed weights on the canonical residual under every Rk, and equals ECI-Diag on R0 and R1. The headline 51-fold Wave worst-case gain is thus the gain from undoing a synthetic transformation that the method ignores by construction.

Not operative. The configuration behind Table 1 violates the assumptions of Cor. 1. It adds:

• ridges and whitening;

• a warm-up blended with the non-invariant vanilla loss;

• a fallback gated on the condition number of AGA⊤, which depends on A;

• an optimizer portfolio chosen per task–representation pair. The paper’s own numbers confirm that the evaluated method is not invariant:

• With matched seeds, an invariant method has Average = Identity. On Kovasznay, Table 1 gives 3.74 × 10−3 vs. 6.07 × 10−4.

• The Worst of 1.92 × 10−2 exceeds the largest possible R0 run (≤3 × 6.07 × 10−4) more than tenfold.

• On Kovasznay, errors grow with the condition-number diagnostic (Fig. S12); on Wave they are flat.

• Whitening applied consistently to r and J cannot change the unregularized ECI value or update. Yet removing it more than doubles the Kovasznay Worst error (1.92 →4.22 × 10−2, Table S6). The robustness therefore comes from the non-invariant components.

Why this is consequential: the central claim is that equation-coordinate invariance is a design principle validated by the experiments. That requires the invariance to be new, to matter for real formulation choices, and to be operative in the experiments. None of the three is established.

  1. The experiments do not isolate the effect of the metric.

Unequal protocols. App. C.1 describes a 2000-epoch common-grid registry and a 5000-epoch “task–representation portfolio” registry with evaluation every 25 steps. ECI-PINN’s R0 errors in Table 1 are 2.5–20× lower than the same method’s R0 errors in the common-grid Table S4:

• Kovasznay: 6.07 × 10−4 vs. 1.19 × 10−2;

• RD: 3.48 × 10−3 vs. 1.79 × 10−2;

• Wave: 1.32 × 10−2 vs. 3.31 × 10−2. By contrast, the MultiAdam and RMSNorm entries are identical in the two tables, and Vanilla differs by only 10–25%. This suggests that only ECI-PINN received the larger, per-representation-tuned budget.

Test-set model selection. Checkpoints are selected by the lowest relative L2 error against the reference solution (App. C.2). Without this selection, ECI-PINN on Kovasznay gives 4.87 × 10−3 / 6.74 × 10−3 / 3.88 × 10−2 (Identity/Average/Worst, Table S6). That is:

• worse at Identity than every L-BFGS-based baseline in Table 1 (4.87 × 10−3 vs. 4.47, 2.01, 1.17 × 10−3);

• worse than NAdam+L-BFGS on Average;

• worse than L-BFGS PINN on Worst. Removing any single component raises the Identity error above every L-BFGS-based baseline in Table 1.

Small gains where the comparison is fair. At R0, the improvement over standard optimizer baselines is 4% on Wave (1.32 vs. 1.38×10−2, AdamW+L-BFGS) and 13% on RD (3.48 vs. 4.02×10−3, NAdam+L-BFGS). These are three-seed results with no intervals.

Confounded full-vs-diagonal ablation. ECI-Diag and ECI-PINN are mathematically identical on Wave at R0 (see above), yet Table S4 reports Identity errors of 6.62 × 10−2 vs. 3.31 × 10−2. The two arms must differ in something other than the metric.

Baselines and reporting.

• Failing baselines. RMSNorm is far worse than Vanilla PINN even at R0 (Wave 1.01, RD 0.25, Kovasznay 0.695; on Kovasznay the standard deviation across nine runs is 1.42 × 10−6). MultiAdam fails at R0 on Wave and Kovasznay (0.65 and 0.67), where Vanilla PINN reaches 0.04–0.06.

• Missing comparisons. Per-equation loss weighting and Gauss–Newton/natural-gradient optimizers are not compared.

• Absolute accuracy. For Kovasznay at Re = 40 on the same domain, Jnini & Vella (2025) report median errors of 9.5 × 10−5 (L-BFGS) and 5.2 × 10−7 (natural gradient), under a larger wall-clock budget, against 4.47 × 10−3 (L-BFGS PINN) and 6.07 × 10−4 (ECI-PINN) here.

• Withheld but plotted. Three baseline aggregates are withheld from Table 1 as inconsistent with the stated aggregation, but they are still plotted in Fig. 4.

• Irreconcilable figure. The Fig. S7 values cannot be reconciled with the Table 1 averages (Question 8).

• Undocumented settings. The transformation matrices, severity, jet scales and optimization settings are not documented (p. 20), and the code package does not claim an end-to-end rerun of every reported experiment.

Why this is consequential: the headline result, lowest errors on all three tasks, cannot be attributed to the ECI metric rather than to budget, tuning and test-set checkpoint selection.

Questions for authors

Clarification requests (answers could change my assessment)

  1. Protocol parity in Table 1. For each method, please state the number of epochs, the L- BFGS iterations, the evaluation frequency, the checkpoint-selection rule, the hyperparameter search and the wall-clock time. Did the baselines also use the 5000-epoch registry, evaluation every 25 steps and best-checkpoint selection? Please re-report Table 1 with a single budget for all methods and with checkpoints chosen without the reference solution (final iterate, or a residual-based criterion).

  2. Table 1 vs. Table S4. Why are ECI-PINN’s R0 errors 2.5–20× lower in Table 1 than in Table S4, while the MultiAdam and RMSNorm entries coincide?

  3. The rule as analyzed. Please report the unregularized ECI rule (no ridge, whitening, warmup, switching or portfolio) with one fixed configuration across R0–R4. Corollary 1 predicts essentially identical errors across representations: is that observed? How does the rule then compare with Vanilla PINN and L-BFGS PINN at R0?

  4. Wave Gram matrix. With r1 = ut −w, r2 = wt −uxx and a diagonal Sξ, J is constant with disjoint row supports. Hence JJ⊤is constant and diagonal, and ECI coincides with ECI-Diag on R0 and R1. Is this correct? If so, why do the two have different Wave Identity errors in Table S4 (6.62 × 10−2 vs. 3.31 × 10−2)? Which three representations does Table S4 use?

  5. Stabilization and L-BFGS. Please define the following precisely:

• whitening: which statistics, and is it applied to r only or also to J?

• the warm-up blend;

• the safe-switch fallback: which loss is used, and is the gate computed on AGA⊤? Applied consistently to r and J, whitening cannot change the unregularized ECI value or update. How, then, does removing it double the Kovasznay Worst error? Which objective does L-BFGS refine? The returned value is r⊤G−1r but the gradient is (∂θr)⊤y; how is the strong-Wolfe line search made consistent?

  1. Jet scales. Please give the numerical Sξ for each task, together with the sensitivity of the results to it (e.g. one jet component rescaled by a factor of 10). How should Sξ be chosen when solution amplitudes are unknown? Please also give the matrices Ak and the severity σ.

  2. Baselines. How is RMSNorm defined (is the normalizer detached?), and how was MultiAdam tuned? Both fail on the identity representation. Please also add:

• ECI-Diag inside the same full pipeline;

• one per-equation adaptive weighting scheme (e.g. learning-rate annealing or NTK weights);

• one Gauss–Newton/natural-gradient optimizer, which is invariant to residual transformations in the overparameterized regime.

  1. Reporting.

• What does each cell of Fig. S7 show? If the cells are seed means or medians, the Table 1 Average for Vanilla PINN on Wave would have to be at least ≈0.28, not 0.212. Similarly, the Kovasznay ECI-PINN Average would have to be at least ≈5.3×10−3, not 3.74×10−3.

• What was inconsistent in the three withheld aggregates, and could the same issue affect other entries?

• Why are the withheld entries still plotted in Fig. 4?

  1. Relevance. Is there a pair of formulations used in practice that differ by a non-diagonal, state-independent A, and on which the full Gram outperforms ECI-Diag?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.