- Rating
- 2(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2(+1)
Read review
Paper summary
This paper proposed a general framework for neural simulation that aims to preserve local details while performing autoregressive rollouts using coarse spatial patch states. The framework is based on patchwise Galerkin projection, patch-state evolution, and partition-of-unity reconstruction. The authors instantiated this framework as SPANS, which uses a DeepSet encoder to aggregate input observations into spatial patch states, a processor with short- and long-range attention to evolve these states, and a patch-conditioned decoder to reconstruct a continuous physical field through partition-of-unity blending.
Critical strengths
-
A general framework for latent-space simulation: The paper presented a conceptual framework that interprets autoregressive simulation in latent space through Galerkin projection, latent-state evolution, and field reconstruction.
-
Accuracy and efficiency on long rollouts: The method demonstrated strong predictive accuracy with relatively low computational cost on tasks involving long autoregressive rollouts.
Critical weaknesses
- The correspondence between SPANS and the general framework in Section 3 was not sufficiently established: SPANS was presented as a concrete instantiation of the general framework, but there are major differences that make the correspondence unclear.
- Encoder: The general framework motivated the latent representation through the Galerkin projection in Eq. (4), but its connection to the DeepSet encoder in SPANS is unclear. When the basis and projection rule are independent of the input field, projecting the input field gives coefficients that depend linearly on the input field, whether the basis is prescribed or learned. The MLP-pooling-MLP encoder does not enforce this linearity. Although the DeepSet encoder could approximate such a projection during training, the paper did not specify the corresponding projection or establish that the encoder approximates it. The authors should clarify how the learned initial patch states relate to the Galerkin coefficients introduced in Section 3.
- Decoder: Partition-of-unity in Eq. (9) reconstructs the field using local basis functions b_(p,j) (x) that are independent of the latent states z_(p,j) (t). However, partition-of-unity decoder of SPANS reconstructs the field using the local basis functions A^((1) ) (x,t) which get z_p (t) as an input. Although Eq. (12) generalizes the reconstruction to learnable local functions, the paper does not explain how this generalization preserves the Galerkin interpretation of the latent states as projection coefficients. Especially, original separation between spatial basis functions and evolving coefficients became unclear in partition-of-unity decoder of SPANS. The authors should justify this transition and explain whether and how the resulting architecture retains the projection–evolution–reconstruction interpretation introduced in Section 3. These gaps also leave the applicability of the theoretical analysis in Section 4 to the implemented SPANS architecture insufficiently justified. For example, the proof relies on reconstructing both the reference and learned fields using the same state-independent spatial basis b_p (x), as shown in Eqs. (25) and (54). The authors should establish the required correspondence or clearly delimit the scope of the theoretical result.
- The theoretical analysis did not sufficiently substantiate the proposed model design: Theorem 1 provides a conditional error-propagation bound in terms of approximation errors and stability constants, but it did not establish how the proposed architectural choices improve these quantities.
- The role of patch count in preserving local details was insufficiently justified. As noted in lines 197-199, for any number of patches P, the partition-of-unity expansion in Eq. (9) can be written as the global expansion in Eq. (4) by choosing ψ_((p,j) ) (x)=w_p (x) b_(p,j) (x), with the same coefficient count M=PJ. Thus, at a fixed total latent budget M=PJ, since increasing P reduces the number of coefficients J available per patch, a reduction in local approximation error does not follow automatically. Beyond the qualitative discussion in lines 170-176 and the empirical evidence in Table 11, the theoretical analysis should quantify how patch count affects local approximation error and identify conditions under which it provides an advantage over a specified global representation. Otherwise, the theoretical justification for using a sufficiently large number of spatial patches, as suggested in lines 265–267, remains incomplete.
- The role of an expressive decoder was insufficiently justified. By the definition of tilde{u}^* in Eq. (26), Assumption 3 bounds the discrepancy between the reference Galerkin field y^* and its reconstruction h_(θ_rec ) (x,y^). This discrepancy can become zero by choosing the identity map h_(θ_rec ) (x,y^)=y^*. Consequently, this reconstruction error alone does not explain why an expressive reconstruction map is needed or why increasing its expressivity should improve accuracy as suggested in lines 267-268.
- The practical informativeness of Theorem 1 for long rollouts remains unclear. The third term in Eq. (14) contains an exponential factor e^(L_dyn T), which may make the bound excessively loose as the rollout horizon increases. For experimental horizons such as T=30, the paper did not estimate the relevant stability factors or provide architecture-specific control of the error terms, leaving the quantitative informativeness of the bound unclear. Although Table 2 and the ablations in Appendix H provided evidence on end-to-end performance and the effects of individual architectural choices, they did not directly verify reductions in the corresponding theoretical error terms or isolate the claimed representational advantage over a specified global representation with the same latent budget. Architecture-specific analysis or targeted experiments addressing these gaps are needed to substantiate the connection between the theoretical results and the proposed design choices.
Questions for authors
-
Consistency of the Smoke experiments with the problem formulation: Could the authors clarify how the SmokeGrid and SmokeObs tasks fit the PDE formulation in Eq. (1)? Since density evolution depends on velocity, what assumptions or additional state information make the density-only prediction setting in Table 3 consistent with this formulation?
-
Processor specification and comparison with GraphGPS [1]: Could the authors explicitly define the short- and long-range attention operations in Eq. (16), including how attention scores and local neighborhoods are computed? Please also clarify how this processor differs from GraphGPS’s combination of local message passing and global attention, and identify the specific methodological novelty.
9 comments
W1: encoder, and W2: patch count [post 2/5]
W1, Encoder
"[...] the paper did not specify the corresponding projection or establish that the encoder approximates it. The authors should clarify how the learned initial patch states relate to the Galerkin coefficients introduced in Section 3."
Section 3 (paragraph "Encoder") now specifies this projection. The initial latent state is given by an analogous least-squares problem for the state instead of the velocity,
For a fixed basis, this fit gives the Galerkin coefficients of the initial field. The encoder is a learned approximation of this fit. The Galerkin trajectory that serves as the reference of Theorem 1 starts from this fit (Section 3, paragraph "Encoder"), and the encoding error is the distance between the encoder output and the fit.
"When the basis and projection rule are independent of the input field, projecting the input field gives coefficients that depend linearly on the input field [...]. The MLP-pooling-MLP encoder does not enforce this linearity."
We agree for a fixed basis, where the normal equations of this fit are linear in the input field. SPANS uses the same least-squares fit, but on the manifold of its nonlinear decoder (Section 3, paragraph "Encoder"). The fit is then a nonlinear function of the input field, so the encoder has to be nonlinear to approximate it. DeepSet is a simple and efficient choice for this, and cross-attention and continuous convolution reach a similar MSE while using more memory (Figure 6).
W2, The role of patch count
"[...] at a fixed total latent budget , since increasing reduces the number of coefficients available per patch, a reduction in local approximation error does not follow automatically."
We agree, and Theorem 1 now makes this trade-off explicit. Its first term, the global approximation error , is the error of a classical PUM with the same latent size, i.e., patches with local spaces of dimension . This error can be reduced in two classical ways, by smaller patches (-refinement, larger ) or by richer local spaces (-refinement, larger ). At a fixed budget , the two compete, since more patches leave a smaller for each patch. More patches therefore do not always reduce this error.
"[...] the theoretical analysis should quantify how patch count affects local approximation error and identify conditions under which it provides an advantage over a specified global representation."
Lemma 1 (Appendix B) bounds the error of the reference solution by , with the stability constant of Assumption 1, so the approximation error of the classical PUM analysis carries over to the first term of the bound (Melenk & Babuška, 1996). For example, with local polynomials of degree , the classical error estimate is , where is the patch diameter in dimensions and is the regularity of the solution, i.e., the number of bounded derivatives it has (Brenner & Scott, 2008). More patches thus shrink , while richer local spaces raise , which helps only up to . Which refinement pays off depends on the local regularity of the solution.
- Smooth solutions favor richer local spaces or a global representation.
- Localized features of low regularity, such as the filaments, wrinkles and contact in our tasks, favor smaller patches, which can represent their fine detail and are easier to learn.
Table 11 shows this on SmokeGrid at a fixed latent budget, and the table in our answer to Reviewer S4Hh (Q2) lists the settings side by side.
- At a budget of 4,096, more patches (256 × 16) give a lower MSE than wider patch states (64 × 64).
- At a budget of 16,384, 256 patches (256 × 64) give a lower MSE than 16 large patches (16 × 1024). The latter is the setting closest to a global representation, and its MSE is as large.
W2: expressive decoder [post 3/5]
W2, The role of an expressive decoder
"[...] This discrepancy can become zero by choosing the identity map [...]. Consequently, this reconstruction error alone does not explain why an expressive reconstruction map is needed or why increasing its expressivity should improve accuracy [...]"
We agree, and Theorem 1 now treats the complete decoder (Eq. (14)). In the submission, the reconstruction error compared the reference field with its own reconstruction, so the bound could not show what an expressive map contributes. Now measures how well the decoder reproduces the blend of a classical PUM with the same latent size (Assumption 3(iii)),
Comparing the identity map with a nonlinear decoder shows what an expressive map contributes.
- With the identity map and local maps that are linear in the patch states, the decoder is a classical PUM with a fixed basis, so and . The approximation error is then of this basis, which decreases only through - or -refinement, i.e., with a larger latent state for the dynamics to learn and evolve. This choice is sufficient when a fixed basis of the given size already captures the solution.
- With a nonlinear decoder, the local functions change with the latent state (Eq. (15)), so they can follow the solution as it evolves. An expressive map can reproduce the classical approximation closely, which keeps small. However, its sensitivity to the latent state must stay bounded, since otherwise it would amplify small latent errors. We therefore assume a Lipschitz decoder (Assumption 3(i) and (ii)), whose constants scale these errors in the bound.
Our ablation further shows the benefit of expressivity. The shared decoder, a special case of the more expressive patch-conditioned one, has the rollout MSE on Cylinder and Airfoil and on Teddy (Table 13).
W1: decoder [post 1/5]
We thank the reviewer for the detailed and precise reading of our framework and theory. The comments led us to restructure the analysis in two places. Section 3 now derives the Galerkin projection for the nonlinear decoder, and Theorem 1 (Section 4, proof in Appendix B) now applies to this decoder directly. We answer each point below.
W1, Decoder
We agree that the step from Eq. (9) to Eq. (12) of the submission was not fully derived, and the revised Section 3 now derives it, starting from the linear case. It replaces the fixed basis functions of the linear expansion with the nonlinear decoder (Eq. (6)), so the approximate field lies on the manifold induced by the decoder. The velocity of this field is still the solution of a Galerkin approximation. By the chain rule (Eq. (7)), the tangent functions ψm=∂Dθdec/∂zm span the tangent space of the manifold at the current state and take the role of the basis functions. Galerkin projection onto this tangent space is therefore analogous to the least-squares problem in Eq. (5) and again yields a system of ODEs (Eq. (8)). The difference to the linear case lies in how the basis functions are formed.
The latent state is thus no longer the coefficient vector of a fixed basis, but its velocity is still the solution of a Galerkin approximation at every time.
The resulting architecture retains this interpretation, and each of the three steps has a direct counterpart in Section 3.
For a fixed basis, ψp,j=wpbp,j, and the classical PUM-Galerkin system of the submission is recovered as a special case.
Theorem 1 now applies to the nonlinear decoder of SPANS directly, and its proof no longer uses a state-independent basis. Two changes make this possible.