- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes a foundation model known as TERM that incorporates state-local governing equation information into its latent representations. The authors argue that global PDE structure is not sufficient for distinguishing different parts of the state space, since mathematical phenomena such as term cancellation may unduly unite very different states. By exploiting a DAG representation of governing equations, TERM informs each token in the context window with its contributions from each term in the governing equations. Results and ablations on standard benchmarks are presented along with some very light theoretical analysis.
Critical strengths
There are many benchmarks and it is clear that the method improves on prior art. The use of generative AI is (mostly) appropriately subtle and I trust that the core contributions come from the authors. Overall I feel that the "pros" outweigh the "cons" and (weakly) recommend acceptance.
Critical weaknesses
The biggest differentiator of this method is also its biggest drawback: it requires exact knowledge of governing equations. This is arguably too restrictive for the majority of situations where foundation models are applied, although I can see a use case as a surrogate for high-fidelity PDE codes. Given this requirement, I think the framing against benchmarks is not 100% transparent; this method is more performant, but also requires strictly more information. The benefit of the analysis in section 4 is also unclear. The first result is a tautology given how the are constructed, and the second result is a straightforward application of the triangle inequality given strong assumptions on the system and observations. There is no experimental verification of either result, so I do not feel that they add anything to the paper. Additionally, TERM relies on Fourier features and is restricted to regular grids, heavily limiting its applicability to realistic computational scenarios.
Questions for authors
- Why does appear in eqn 3 when no other parts of this equation use it?
- in eqn 4 is not defined.
- There are various LLM-isms like "binding these views" that are linguistically jarring and obviously machine-generated. I recommend reading the prose carefully and rewriting things in a more human way.
4 comments
Question 1 — Role of the log response scale in Equation 3
Q1: The log response scale in Eq. 3 is the logarithm of the term's response-window RMS scale, . The same equation defines the normalized response field and this separate scale feature. The normalized field communicates the spatial pattern while retains amplitude information; the response encoder receives the scale, and the gate in Eq. (4) also uses it. We now call out this subsequent use near the definition.
Question 2 — Definition of the response tokens in Equation 4
Q2: Lowercase denotes the numerical response field at frame . The response encoder processes the normalized fields, their coordinates, relative times and scale; learned queries then resample the resulting features into a fixed-size token set . After projection to a common attention width, has shape . The symbolic term embedding is the query, while supplies keys and values in its term-local cross-attention. We have added the distinction and dimensions in Sec. 3.2.
Question 3 — Unnatural wording and language
Q3: We agree that abstract phrases can obscure the operation and have revised them in the manuscript. For example, the revised conclusion states that TERM enriches symbolic PDE terms with their state-dependent responses before the shared Transformer. The revised method opening also gives a Burgers example before the module-level description. Thank you for pointing out that the presentation should make the operations directly traceable to the equations.
Thanks to the authors for providing additional details and clarifications. Given that I still have concerns about the utility of the theory and fairness of the comparisons, I will maintain my score of "weak accept".
Thank you for reviewing the revision and for clarifying your remaining concerns.
Comparison fairness. TERM and Origo use exactly the same train/test trajectories on all ten benchmarks, the same 10-frame history, a 10-step training rollout, and relative evaluation. They also share 1,000 joint pretraining epochs, a total batch size of 24, and 200 adaptation epochs per task. Section 5.1 and App. A.3.4 state these matched settings and document the different learning rates and schedules. Tables 1 and 3 identify the external values as published references and explain TERM's additional equation and response inputs. Tables 2 and 4 provide the controlled comparisons of response organization.
Use of the theory. We shortened Sec. 4 and tied it to two design choices. With the other response inputs held fixed, term-local fusion gives each response a direct path to its own embedding before the Transformer mixes terms. The pooled-response objective supervises that embedding, and App. A.6 states when reconstruction error and target separation can bound misattribution. The discussion of evaluator sensitivity motivates the TERM-Direct comparison in Sec. 5.5. Those experiments report prediction error under perturbations. The analysis describes the assumptions for the bounds, including contraction of the full predictor for the horizon-uniform result in App. A.2.5.
Weaknesses: First, we agree that TERM's dependence on a specified equation is a substantive restriction. Its intended setting is an equation-conditioned surrogate: the equation, coefficients and relevant boundary/domain information are available, and the model predicts future fields for different states and problem instances. Unseen equation means absent from the applicable training split but supplied at inference; it does not mean unknown to the system. Cases where the law is unknown, essential variables are unavailable, or the supplied equation or coefficients are inaccurate are outside what the present transfer experiments establish.
Second, TERM computes the numerical term responses from the supplied equation and observed state. A baseline that is not given the equation or these features has less physical conditioning. In the revised manuscript, we have clarified that the main benchmark compares complete methods under their respective input configurations. Its performance gaps cannot by themselves attribute the improvement to architectural design under identical information. The shared-backbone controls provide a more controlled analysis, but the original Unbound Response does not include explicit response-source labels. New source-labeled variants in Sec. 5.4, Table 4 help assess that distinction; their remaining limitations are explained in our reply to Reviewer 8KvM.
Third, for the theoretical analysis in Sec. 4, we agree that the response-sum identity is definitional and that non-injectivity of summation does not by itself prove a prediction gap. The revised Sec. 4 explains the locality of term enrichment and a conditional pooled-response attribution bound. App. A.6 proves the block-diagonal local Jacobian, the attribution and representation-separation bounds, and a filtered linear Fourier-evaluator perturbation bound. These results concern local updates, pooled targets or the stated linear periodic evaluator; they do not prove full-field recovery or stability of the complete model. In particular, the horizon-uniform rollout estimate still requires LTERM<1, which the architecture does not enforce. Existing ablations and noise results provide empirical context but do not verify the bounds or their constants.
Finally, considering the Fourier features and regular-grid limitations, TERM consists of three different stages:
To clarify this, the revised Sec. 5.5 summarizes the irregular-query comparison. These results support off-grid output querying given structured-grid histories, not a demonstrated unstructured-mesh input pipeline. Adapting the input representation and constructing reliable derivatives on irregular geometries require further validation. We now state that boundary in the conclusion and implementation discussion (Secs. 3.2 and 3.4; App. A.4.4).