acceptodds
Under review as a conference paper at ICLR 2027

Moving-Latent Neural PDE Surrogates

Abstract

Neural surrogates offer data-driven alternatives to computationally expensive numerical solvers of partial differential equations (PDEs) by learning to approximate PDE solution operators directly from simulated trajectories. Among various approaches, latent-space surrogates further reduce the cost of long-horizon prediction by evolving compact representations instead of dense physical fields. However, existing approaches typically evolve latent features on fixed spatial or static abstract supports, limiting their ability to adapt where representational capacity is allocated as the solution moves and deforms. We introduce Moving-Latent Neural PDE Surrogates (MoLa-PDE), where latent features evolve together with their spatial coordinates, allowing a compact latent representation to adapt not only what information is stored but also where it is represented as the PDE dynamics evolve. Making such a moving representation effective requires both modeling interactions on evolving spatial supports and learning how the support itself should deform. To address the former, we develop a geometry-structured continuous operator that explicitly combines feature-dependent interactions with local neighborhood geometry through localized polynomial–Fourier bases. To address the latter, we introduce a structured latent motion mechanism that infers potential fields from global interactions and local differential features, and converts them into support deformation through a Hodge-inspired construction. Experiments across controlled synthetic problems and diverse PDE benchmarks show that the proposed moving-latent representation consistently improves prediction accuracy over fixed-support latent surrogates, while revealing interpretable latent-motion patterns that reflect evolving spatial structures.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Neural PDE Surrogates, Latent-Space Modeling, Geometry-Aware Learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_9dj9OpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Paper proposes MoLa-PDE, latent-space PDE surrogate that jointly evolves latent features and their spatial coordinates. It combines geometry-aware interaction operator with potential-based motion, allowing a compact representation to follow moving and deforming structures.

Critical strengths
  1. Moving latent coordinates is well-motivated idea for transport problems, where fixed representations may struggle to follow evolving structures.

  2. Extensive experiments.

Critical weaknesses
  1. Experiments use only 8 latent points in 1D and 16 in 2D. It is unclear whether the gains remain substantial when the static model receives more latent points under similar computational budget. How does it scale??

  2. Results do not report variability across training seeds making the smaller improvements difficult to assess.

  3. The stability of the moving support needs more discussion. Point clustering or nearly degenerate neighborhoods could affect the local least-squares derivative estimates.

  4. Moving supports add noticeable computational overhead, while the model remains less accurate than some baselines on Navier–Stokes and cylinder flow. The accuracy–cost tradeoff needs stronger evaluation.

  5. Missing Baselines. Can author compare against Transolver and Latent Mamba operator as it seems to be relevant recent baselines.

Questions for authors
  1. Does moving the support still help when the static model uses more latent points at a matched inference cost? What's the tradeoff??

  2. How does method prevent latent points from collapsing together or leaving poorly covered regions? Is the local least-squares system regularized when geometry becomes ill-conditioned?

  3. Could you separately ablate global translation, scalar-potential motion and rotational motion to show which parts are needed and when they are useful??

  4. Would learning an input-dependent initial support recover much of the gain without moving the points at every step? it seems particularly relevant to the cylinder case.

  5. Could you report results across multiple seeds and evaluate beyond the training horizon?

  6. Can you quantify how the method differs from existing adaptive-mesh and moving-point approaches, beyond applying movement in latent space [1, 2] and compare against them and with SSM based operator [3]??

[1] Wen, Shizheng, et al. "Geometry aware operator transformer as an efficient and accurate neural surrogate for pdes on arbitrary domains." Advances in Neural Information Processing Systems 38 (2026): 155423-155501.

[2] Mousavi, Sepehr, et al. "RIGNO: A Graph-based framework for robust and accurate operator learning for PDEs on arbitrary domains." Advances in Neural Information Processing Systems 38 (2026): 150039-150101.

[3] Tiwari, Karn, Niladri Dutta, and N. M. Krishnan. "Latent mamba operator for partial differential equations." arXiv preprint arXiv:2505.19105 (2025).

7 comments
AuthorsOpenReview ↗

General Response and Summary of Revisions

We thank the reviewer for the detailed and constructive feedback. To address the questions, we conducted additional experiments and analyses to further evaluate the accuracy, efficiency, robustness, and stability of MoLa-PDE along six dimensions:

  1. Latent scaling and computational cost (W1/W4/Q1): We evaluated static and moving supports with both original and doubled latent sizes, measuring prediction accuracy and full-rollout inference latency. The benefits of latent motion persist at larger latent sizes.
  2. Robustness and temporal extrapolation (W2/Q5): We repeated experiments with three random seeds on all six PDE benchmarks and evaluated predictions beyond the supervised training horizon. Motion achieves lower extrapolation errors on five of six benchmarks. We further analyzed the Navier–Stokes exception and its association with a distribution shift.
  3. Numerical stability and spatial coverage (W3/Q2): We clarified the ridge regularization used in local WLS estimation and evaluated Gram-matrix eigenvalues, near-collision rates, and spatial coverage across six PDE benchmarks. The results show that moving supports maintain numerically stable local systems without collapse.
  4. Additional baselines and methodological comparison (W5/Q6): We added comparisons with Transolver, Latent Mamba Operator (LaMO), GAOT, and RIGNO. We also clarified how MoLa-PDE differs from existing adaptive-mesh and moving-point approaches through the joint evolution of compact latent features and spatial coordinates.
  5. Motion-component analysis (Q3): We quantified the relative contributions of global translation, scalar-potential motion, and rotational motion across all six PDE benchmarks. The results reveal different component preferences associated with the underlying PDE dynamics.
  6. Input-dependent initial support (Q4): We introduced an additional baseline that learns input-dependent initial latent positions while keeping them fixed during rollout. The results show that this strategy can be particularly effective for Cylinder Flow, but does not generally recover the benefits of temporal latent motion across the other benchmarks.

The detailed experimental results and methodological clarifications are provided in our responses below. We have incorporated these additional analyses into the revised manuscript and expanded the discussion of the corresponding tradeoffs and limitations.

AuthorsOpenReview ↗

Response to W1 / W4 / Q1

Response to W1 / W4 / Q1

We thank the reviewers for highlighting the tradeoffs between prediction accuracy, latent capacity, and computational cost. To investigate these questions, we double the number of latent points from 8 to 16 in 1D and from 16 to 32 in 2D, comparing static and moving supports at both latent sizes and measuring their full-rollout inference latency.

First, the benefits of latent motion persist as the latent size increases. At the doubled latent size, MoLa-PDE continues to outperform its static counterpart across all six PDE benchmarks, suggesting that the advantage of adaptive latent geometry is not limited to highly compressed representations.

Second, increasing the number of fixed latent points does not necessarily compensate for the absence of motion. On four of the six benchmarks (Burgers, Diffusion–Reaction, Advection, and Cylinder), the original moving model achieves lower prediction errors than the static model with twice as many latent points. This suggests that adapting where latent information is represented can be more effective than simply increasing the number of fixed spatial supports.

PDEStatic (1x)Motion (1x)Static (2x)Motion (2x)
Burgers2.161.741.901.62
Diffusion–Reaction0.1780.1630.1710.155
Advection1.430.4341.360.223
Navier–Stokes9.929.048.758.06
Shallow Water0.2250.1980.1820.173
Cylinder15.012.913.112.2

Comparison of relative L2L_2 error (×10−2\times 10^{-2}) for static and moving supports using the original and doubled latent sizes.

Third, the accuracy–cost comparison reveals different tradeoffs across spatial dimensions and PDE dynamics. In 1D, Motion-8 outperforms Static-16 on all three benchmarks, with only 26% higher full-rollout latency (30.83 vs. 24.41 ms). In 2D, Motion-16 and Static-32 have comparable latency (27.62 vs. 28.59 ms), with Motion-16 achieving lower error on Cylinder. Increasing the moving support to 32 points further improves accuracy over Static-32 on all three 2D benchmarks, at approximately 34% higher latency. These results indicate that latent motion offers an alternative to increasing static resolution, although its accuracy–cost advantage depends on the underlying dynamics.

ConfigurationStatic (ms)Motion (ms)
1D, 8 points17.3230.83
1D, 16 points24.4137.19
2D, 16 points19.4627.62
2D, 32 points28.5938.21

Full-rollout inference latency (ms) of static and moving models at original and doubled latent sizes.

Overall, these experiments show that adaptive latent geometry provides benefits beyond increasing static latent capacity, while introducing additional computational cost. Rather than claiming universal accuracy–efficiency cost optimality, our results demonstrate that evolving the latent support offers a complementary way to improve compact PDE representations. We have incorporated these scaling and efficiency comparisons into the revised manuscript.

AuthorsOpenReview ↗

Response to W2 / Q5

Response to W2 / Q5

We thank the reviewer for this suggestion. To assess the robustness of our results across random seeds, we repeat the experiments for both MoLa-PDE and its static counterpart using three random seeds on all six benchmarks.

PDEStaticMotionRel. Diff. ↑\uparrow
Burgers2.217±0.0672.217 \pm 0.0671.787±0.0471.787 \pm 0.04724.06%
Diffusion–Reaction0.176±0.0020.176 \pm 0.0020.159±0.0030.159 \pm 0.00310.69%
Advection1.457±0.0591.457 \pm 0.0590.459±0.0360.459 \pm 0.036217.4%
Navier–Stokes9.916±0.0509.916 \pm 0.0509.026±0.0239.026 \pm 0.0239.860%
Shallow Water0.217±0.0060.217 \pm 0.0060.195±0.0040.195 \pm 0.00411.28%
Cylinder14.47±0.84614.47 \pm 0.84613.35±0.63913.35 \pm 0.6398.390%

Relative L2L_2 error (×10−2\times 10^{-2}) over three random seeds for moving and static supports, reported as mean ±\pm standard deviation. Relative difference is normalized by the moving mean.

For the beyond-horizon evaluation, we retrained Motion and Static with shortened supervision horizons and evaluated their free-running predictions over the remaining unseen timesteps. As shown in Table 9, Motion achieves lower extrapolation errors on five of the six PDE benchmarks, suggesting that the benefits of adaptive latent geometry generally extend beyond the supervised horizon. The exception is Navier–Stokes (NS), where Static achieves a lower extrapolation error than Motion (28.63 vs. 45.32). To better understand this difference, we further examine the prediction behavior and underlying flow dynamics.

On NS, Motion maintains lower errors within the supervised interval and comparable accuracy at the first extrapolation step, with the performance gap emerging over subsequent unseen steps. Interestingly, we find that these later steps also involve a shift in the underlying flow regime. The mean velocity RMS reconstructed from the vorticity fields increases by approximately 25% from frame 14 to frame 19, and by the final frame, all test trajectories exceed the maximum velocity RMS observed during training. Thus, this evaluation involves not only temporal extrapolation but also a shift toward higher flow velocities beyond the training distribution.

This distribution shift provides a plausible explanation for the observed behavior. In MoLa-PDE, latent features and support coordinates evolve jointly, allowing the representation to adapt to changing spatial structures. However, when extrapolating to unseen dynamical regimes, errors in the predicted motion may affect subsequent feature interactions through the evolving geometry, leading to additional error accumulation. The static model, in contrast, does not involve this coordinate–feature feedback. These observations suggest that the NS performance gap is associated with the additional challenge of extrapolating coupled latent dynamics under a shifted flow regime, rather than indicating a general limitation of moving supports in modeling transport and deformation. These observations are consistent with the benefits of adaptive latent geometry demonstrated across our standard benchmarks and controlled transport experiments, while also highlighting the challenges of generalizing learned support motion to dynamical regimes beyond the training distribution. We have incorporated these additional experimental results and analyses, together with a discussion of the corresponding limitations, into the revised manuscript.

DatasetTraining HorizonExtrapolation HorizonStaticMotion
Burgers1–3031–402.6472.274
Diffusion-reaction1–1516–200.2310.208
Advection1–3031–404.0170.539
Navier–Stokes1–1415–1928.6345.32
Shallow water1–1516–201.1370.707
Cylinder1–1011–1415.7114.64

Relative L2L_2 error (×10−2\times 10^{-2}) beyond the training horizon.

Reviewer_j3Z9OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

The paper proposes a latent-space neural PDE surrogate in which the latent points have spatial coordinates that evolve over time together with their features, instead of staying on a fixed or abstract support. The model uses a classical encoder-predictor-decoder design built on a shared point-to-point operator. In this operator, source features predict the coefficients of a fixed, localized polynomial-Fourier basis evaluated on relative positions. At each step, the predictor first moves the latent points by applying a velocity comes from scalar and rotational potentials plus a global translation. Features are then updated on the new positions. The decoder can be queried at arbitrary locations.

Critical strengths
  • Making the latent coordinates part of the dynamic state, such that latents can adapt where they represent information and not only what, is a natural extension of latent PDE surrogates. It suits transport-dominated dynamics especially well.
  • Motion comes from scalar and rotational potentials which keeps neighburing points moving coherently. The ablation shows it beats direct MLP displacement prediction on all benchmarks.
Critical weaknesses
  • Notation in the method section is unclear and sometimes inconsistent, which makes the model hard to verify or reproduce.
    • Missing definitions for core operations. The relevance score aijha_{ijh} (Eq. 3), the boundary-aware displacement and projection ΔΩ\Delta_\Omega/ΠΩ\Pi_\Omega, the map from vlocv^{\text{loc}} to the bounded displacement ΔQloc\Delta Q^{\text{loc}}, and the global translation ΔQtrans\Delta Q^{\text{trans}} are described only in words. The local-context feature ViV_i is never defined (presumably it is CiC_i).
    • Sign convention. Does ξij\xi_{ij} point from target ii to source jj, or the reverse? Eq. 5 needs ξij≈Qj−Qi\xi_{ij} \approx Q_j - Q_i to give the correct Jacobian.
    • Inconsistent definitions. ℓ\ell is Ns−1/dN_s^{-1/d} in Eq. 1 but M−1/dM^{-1/d} in App. B.3. The basis counts disagree between Eq. 2, Eq. 8 and the ablation (25 vs. 24 Fourier terms in 2D), and the ablation's capacity argument relies on those counts.
    • Confusing and overloaded symbols. Q0/Z0Q^0/Z^0 (physical input) vs. Q0/Z0Q_0/Z_0 (initial latent state) differ only in sub/superscript, and Fig. 1, Sec. 2.2 and Sec. 3 each use different parameter notation. Many symbols are reused for unrelated quantities (cc, vv, KK, TT, MM, ϵ\epsilon).
Questions for authors

See Weaknesses. Please improve the writing quality accordingly.

3 comments
AuthorsOpenReview ↗

General Response and Summary of Revisions

We thank the reviewer for the careful reading and detailed feedback. We agree that several definitions, notation conventions, and methodological details in the original submission were insufficiently explicit, making the proposed method harder to follow and reproduce. To address these concerns, we have revised the method section, clarified the mathematical formulations, and standardized the notation throughout the manuscript.

In particular, we made the following revisions:

  1. Explicit definitions of core operations (W1(a)): We added mathematical formulations for the relevance score aijha_{ijh}, boundary-aware displacement ΔΩ\Delta_\Omega, coordinate projection ΠΩ\Pi_\Omega, bounded local displacement ΔQloc\Delta Q^{\mathrm{loc}}, and global translation ΔQtrans\Delta Q^{\mathrm{trans}}. We also corrected the inconsistent local-context notation ViV_i to CiC_i.

  2. Clarification of displacement conventions (W1(b)): We explicitly defined the direction of relative displacements, ensuring that ξij\xi_{ij} points from target ii to source jj, consistent with the local Taylor approximation and Jacobian estimation.

  3. Consistency of support spacing and basis definitions (W1(c)): We clarified the relationship between the support-spacing definitions involving NsN_s and MM, corrected the missing zero-order polynomial term in the basis formulation, and reconciled the Fourier ablation counts (24 oscillatory terms plus one zero-order term).

  4. Systematic notation cleanup (W1(d)): We distinguished physical inputs, encoder-stage representations, and latent rollout states; unified the learnable parameter notation as θ\theta; and reorganized symbols throughout the manuscript. We also added a comprehensive notation table (Table G1, Appendix G) to facilitate reference and reproducibility.

These revisions are incorporated into the updated manuscript, with changes highlighted in red. Below, we provide point-by-point responses to each concern, including the precise mathematical definitions and explanations of the corresponding corrections.

Response to W1(a)

We thank the reviewer for highlighting these missing definitions. In the submission, we described several operations at the conceptual level without explicitly specifying their mathematical formulations. Some of these operations, such as periodic wrapping, coordinate clipping, and query-key similarity, follow standard formulations, whereas others, including the bounded local displacement and global translation, involve method-specific parameterizations that should have been stated explicitly. We acknowledge that our description was not sufficiently clear and was missing some definitions. We provide the precise formulations below and have incorporated them into the revised manuscript.

Relevance score aijha_{ijh}. The relevance score in Eq.~(3) depends on whether the interaction is cross-support or same-support:

\begin{cases} w_h^\top Z_j, & \text{cross-support}, \\\\ \displaystyle \frac{(W_h^q Z_i)^\top(W_h^k Z_j)} {\sqrt{d_h}}, & \text{same-support}, \end{cases}$$ where $w_h$, $W_h^q$, and $W_h^k$ are learnable parameters, and $d_h$ is the head dimension. Cross-support relevance depends only on source features, whereas same-support relevance uses query--key similarity. **Boundary-aware displacement and projection.** For coordinates normalized to $[0,1]^d$, the boundary-aware displacement is defined as $$\Delta_\Omega(Q,P) = \begin{cases} ((P-Q+\frac{1}{2})\bmod 1)-\frac{1}{2}, & \text{periodic}, \\\\ P-Q, & \text{non-periodic}. \end{cases}$$ The corresponding coordinate projection is $$\Pi_\Omega(q) = \begin{cases} q\bmod 1, & \text{periodic}, \\\\ \operatorname{clip}(q,0,1), & \text{non-periodic}. \end{cases}$$ These operations implement periodic wrapping or coordinate-wise clipping within the normalized domain. **Bounded local displacement $\Delta Q^{\mathrm{loc}}$.** Given the potential-induced local velocity $v_i^{\mathrm{loc}}$, we compute the bounded local displacement as \$ \\Delta Q _i^{\\mathrm{loc}} = \\delta\\tanh(v _i^{\\mathrm{loc}}). \$ **Global translation $\Delta Q^{\mathrm{trans}}$.** To capture coherent translation of the entire latent support, we predict a shared displacement from the mean-pooled latent features and global-context features: \$ \\Delta Q^{{trans}} = \\delta\\tanh( {MLP}([ \\frac{1}{M}\\sum _{i=1}^{M} Z _i,\\, \\frac{1}{M}\\sum _{i=1}^{M}G _i ])). \$ The resulting translation is shared by all latent locations. **Local-context notation $V_i$.** We confirm that $V_i$ is a notation inconsistency and should be $C_i$, the neighborhood contrast feature already defined in Sec. 3.2. Accordingly, the local context should consistently be written as $L_i=[C_i,D_i]$. These definitions have been incorporated into Sections 3.1 and 3.2 of the revised manuscript. Thanks again for this advice to help us improve the manuscript! **Response to W1(b)** We thank the reviewer for pointing out this ambiguity. In the revised manuscript and as clarified in W1(a) response, we explicitly define $\Delta_\Omega(x,y)$ as the displacement from target $x$ to source $y$, using $y-x$ for non-periodic boundaries and the corresponding wrapped displacement for periodic boundaries. Following this definition, the normalized relative displacement in Eq. (5) is computed as $\xi_{ij}=\Delta_\Omega(Q_i,Q_j)/\ell$, which points from target $i$ to source $j$. This direction is consistent with the local Taylor approximation $Z_j-Z_i\approx J_i^Z\xi_{ij}$, where $J_i^Z$ is the Jacobian with respect to normalized local coordinates. We have made this displacement convention explicit in Sec. 3.1.

Response to W1(c)

We thank the reviewer for pointing out these ambiguities. We clarify the spacing definition, basis formulation, and Fourier ablation counts as follows.

Support spacing. In Eq. (1), ℓ=Ns−1/d\ell=N_s^{-1/d} denotes the estimated average spacing determined by the source support size NsN_s. In Appendix B.3, the coordinate perturbation is applied to the latent support, which contains MM latent states. Since Ns=MN_s=M for interactions within the latent support, the corresponding average spacing is ℓ=M−1/d\ell=M^{-1/d}. The two definitions of ℓ\ell are consistent. We have clarified this relationship in the revised paper.

Basis formulation. We thank the reviewer for careful reading and agree that Eq. (2) missed the zero-order polynomial term, which is correctly included in Eq. (8). The complete basis formulation is

= \{e _p(\xi)\} \cup \{e _p(\xi)\xi _a\} _{a=1}^{d} \cup \{e _p(\xi)\xi _a\xi _b\} _{1\leq a\leq b\leq d} \cup \{ e _{\kappa}(\xi)\sin(2\pi\kappa u^\top\xi),\; e _{\kappa}(\xi) [\cos(2\pi\kappa u^\top\xi)-1] \} _{\kappa\in\mathcal K,\;u\in\mathcal U _d}.$$ We have already corrected Eq. (2) to make it consistent with Eq. (8) and the implemented bases. **Fourier ablation counts.** The reported 25 basis functions in the 2D Fourier-only ablation include one zero-order term in addition to the 24 oscillatory Fourier terms. Specifically, the three frequency scales and four spatial directions produce $3\times4\times2=24$ sine and centered-cosine terms, giving a total of $1+24=25$ basis functions. Thus, 24 refers to the oscillatory Fourier components, whereas 25 denotes the complete basis retained in the Fourier-only ablation. We have made this distinction explicit in the ablation discussion and clarified the corresponding basis counts. **Response to W1(d)** We thank the reviewer for highlighting these notation issues. We have revised the notation throughout the manuscript, with particular attention to the method section, and collected all symbols in Table G1 in Appendix G. **Physical inputs vs. latent states.** The physical inputs are denoted by $(\mathbf U_0,\mathbf X)$, encoder-stage coordinates and features by $(\mathbf Q_{\mathrm{enc}}^{(l)},\mathbf Z_{\mathrm{enc}}^{(l)})$ with $\mathbf Q_{\mathrm{enc}}^{(0)}=\mathbf X$, and latent states at prediction step $t$ by $(\mathbf Q_t,\mathbf Z_t)$. Physical inputs and initial latent states are therefore no longer distinguished only by the placement of an index. **Parameter notation across Fig. 1, Sec. 2.2, and Sec. 3.** We now use $\theta$ for all learnable parameters throughout. **Symbols listed by the reviewer.** - $c$ denotes the channel index. - $v$ denotes velocities: $\mathbf v$ for the physical velocity, $v_i^{\mathrm{loc}}$ for the local latent velocity, and $v_\perp$ for the interface-normal speed. - $\mathcal K$ denotes the set of Fourier frequency scales in the basis. - $T$ denotes the terminal physical time, and $N_t$ is the number of prediction steps. - $M$ denotes the number of latent points. - $\epsilon$ appears with subscripts for numerical constants: $\epsilon_\lambda$ (locality parameterization), $\epsilon_l$ (loss denominator), and $\epsilon_{\mathrm{ridge}}$ (ridge regularization). All changes are included and highlighted in red in the revised manuscript.
Reviewer_Hx53OpenReview ↗
Rating
4
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes MoLa-PDE, a latent-space neural surrogate for time-dependent PDEs. It follows the usual encoder–evolver–decoder pattern. The encoder compresses the physical field onto a small set of latent points. The evolver advances the latent state autoregressively. The decoder maps the latent state back to any query location. The key difference from prior latent surrogates is that each latent point has spatial coordinates, and these coordinates move at every time step. The motivation is that, for transport-dominated problems, moving the points lets the model follow moving structures instead of representing them through feature changes alone.

Critical strengths

The core claim is tested with a clean, controlled comparison. The static and moving versions share the same architecture, the same parameter count, and the same training protocol. The only difference is whether the latent points move. Moving supports win on all six benchmarks and in all synthetic settings. The synthetic results also behave the way the mechanism predicts. In rigid transport, the static error grows with transport speed, while the moving error stays nearly flat, so the gain grows with speed.

Critical weaknesses

The paper does not isolate why motion helps. The static control places its latent points on a uniform grid that never changes. The moving model can place its points anywhere. The gain could therefore come from non-uniform placement alone, not from movement over time. A static model with learned but fixed positions, which is what CALM-PDE uses, would separate the two effects, and the paper does not include one.

Questions for authors
  1. Can you compare against a static model whose latent positions are learned during training but fixed during rollout, as in CALM-PDE?
  2. How many random seeds were used for each result? Can you report the mean and standard deviation, especially for small gains such as Reaction–Diffusion (0.178 versus 0.163)?
  3. For T and ST, the true velocity fields are known. Can you report a quantitative comparison between the learned latent-point velocity and the true transport velocity?
  4. In 1D, the antisymmetric potential vanishes, and any velocity field is the gradient of some scalar potential. In 2D on a periodic domain, gradient plus rotational plus constant translation can represent any smooth field. What constraint does the Hodge-style construction actually impose, compared with predicting velocity directly?
  5. With only 8 latent points in 1D and 16 in 2D, how accurate are the weighted least-squares gradient estimates? Did you check them against analytic gradients on a known potential?
  6. Can you compare the geometry-structured operator against a learned MLP kernel, as in CALM-PDE, inside the moving model? This would test the claim that factorizing features and geometry matters when the points move.
  7. On cylinder flow, most motion happens in the first few steps and then stops. Does this suggest that the benefit there comes from adapting the initial layout rather than from tracking dynamics?
4 comments
AuthorsOpenReview ↗

General Response and Summary of Revisions

General Response and Summary of Revisions

We thank the reviewer for the detailed and constructive feedback. To address the questions, we conducted additional experiments and refined the interpretation of our method along seven dimensions:

  1. Motion vs. learned placement (W1/Q1): We added a CALM-PDE-style baseline with learned but rollout-fixed latent coordinates. MoLa-PDE outperforms both uniform-static and learned-static supports on all six benchmarks.
  2. Robustness (Q2): We repeated the static and moving models with three random seeds on all six PDEs and report mean and standard deviation. The moving model has lower mean error in every case.
  3. Physical interpretation (Q3): On synthetic transport datasets, we compared learned latent-point velocities with known transport velocities using directional similarity, relative error, and speed ratio. We find that learned latent motion nearly has strong directional alignment with the real velocity field.
  4. Role of the Hodge-inspired formulation (Q4): We clarify that it is a structured inductive bias, not a hard restriction on the space of smooth velocity fields.
  5. Gradient estimation (Q5): We evaluated weighted least-squares (WLS) gradients against analytical gradients of 1,024 synthetic potentials per dimension, using sparse supports sampled from trained rollouts.
  6. Spatial operator (Q6): We compared our geometry-structured operator with an MLP kernel under both static and moving supports. The structured operator makes the advantage of motion substantially more consistent.
  7. Cylinder-flow behavior (Q7): We clarify that the benefit in this case appears to arise mainly from early adaptive reconfiguration, rather than continuous motion throughout the rollout.

The detailed results and methodological clarifications are given in the responses below. We have incorporated the additional experimental results into the revised manuscript and appendix and clarified the interpretive ambiguities in the revision.

AuthorsOpenReview ↗

Response to W1/Q1 - Q3

We thank the reviewer for these valuable suggestions.

Response to W1 / Q1

To separate latent motion from learned spatial placement, we add a static baseline using CALM-PDE's learnable query point strategy. The latent coordinates are optimized during training but fixed throughout the autoregressive rollout, testing whether learned non-uniform placement alone accounts for the gains from latent motion.

As shown in the table below, learned static supports improve prediction accuracy over uniform static supports on four of the six benchmarks, but yield slightly higher errors on Advection and Navier--Stokes. In contrast, MoLa-PDE with moving supports consistently outperforms both static variants across all six PDEs. These results suggest that the benefits of moving latent supports cannot be explained by fixed spatial optimization alone. Allowing the latent coordinates to evolve with the PDE dynamics provides additional flexibility to adapt where information is represented as spatial structures move and deform. We have included this comparison in the revised manuscript.

PDEUniform StaticLearned StaticMotion
Burgers2.162.101.74
Diffusion--Reaction0.1780.1730.163
Advection1.431.630.434
Navier--Stokes9.929.989.04
Shallow Water0.2250.2070.198
Cylinder15.014.212.9

Comparison of relative L2L_2 error (×10−2\times 10^{-2}) across PDE benchmarks for uniform static, learned static, and moving latent supports. Learned static positions remain fixed during rollout.

Response to Q2

To assess the robustness of our results across random seeds, we repeat the experiments for both MoLa-PDE and its static counterpart using three random seeds on all six benchmarks.

The table below reports the mean and standard deviation of the relative L2L_2 error.

The moving model achieves lower mean prediction errors than its static counterpart across all six PDEs. In particular, on Diffusion--Reaction, where the improvement is relatively modest, the error decreases from 0.176±0.0020.176 \pm 0.002 to 0.159±0.0030.159 \pm 0.003, with the difference between the means exceeding the observed seed-to-seed variability. These results suggest that the benefits of latent motion are not limited to a particular random seed. We have included the multi-seed results in the revised appendix.

PDEStaticMotionRel. Diff.
Burgers2.217±0.0672.217 \pm 0.0671.787±0.0471.787 \pm 0.04724.06%
Diffusion--Reaction0.176±0.0020.176 \pm 0.0020.159±0.0030.159 \pm 0.00310.69%
Advection1.457±0.0591.457 \pm 0.0590.459±0.0360.459 \pm 0.036217.4%
Navier--Stokes9.916±0.0509.916 \pm 0.0509.026±0.0239.026 \pm 0.0239.860%
Shallow Water0.217±0.0060.217 \pm 0.0060.195±0.0040.195 \pm 0.00411.28%
Cylinder14.47±0.84614.47 \pm 0.84613.35±0.63913.35 \pm 0.6398.390%

Relative L2L_2 error (×10−2\times 10^{-2}) over three random seeds for moving and static supports, reported as mean ±\pm standard deviation. Relative difference is normalized by the moving mean.

Response to Q3

We quantitatively compare the learned latent-point velocities with the ground-truth transport velocities using directional cosine similarity, relative L2L_2 error, and mean speed ratio.

Mark learned and true velocities of a latent point jj as v^j\hat{\mathbf v}_j and vj\mathbf v_j, cosine similarity averages their directional agreement and could be calculated as v^j⊤vj/(∥v^j∥2∥vj∥2)\hat{\mathbf v}_j^\top\mathbf v_j/ (\|\hat{\mathbf v}_j\|_2\|\mathbf v_j\|_2).

Relative L2L_2 error measures the discrepancy of the full velocity vectors, formulated as Erel=∑j∥v^j−vj∥22∑j∥vj∥22E_{\mathrm{rel}} = \sqrt{\frac{\sum_j\|\hat{\mathbf v}_j-\mathbf v_j\|_2^2}{\sum_j\|\mathbf v_j\|_2^2}}. The mean speed ratio averages ∥v^j∥2/∥vj∥2\|\hat{\mathbf v}_j\|_2/\|\mathbf v_j\|_2, indicating whether the learned motion is faster or slower than the true transport. Each metric is computed separately per configuration, and ST results are averaged equally over nine configurations.

The table below shows that learned latent motion nearly recovers rigid transport and has strong directional alignment on T and ST-Shear, although magnitude errors remain in these spatially varying flows. The model receives no explicit velocity supervision; latent motion emerges solely from optimizing solution prediction. This suggests that the learned motion implicitly captures meaningful aspects of the underlying transport dynamics, allowing the latent support to follow evolving spatial structures rather than relying entirely on feature updates to represent their displacement. We have included this comparison in our revised manuscript.

ConfigCosine ↑\uparrowRel L2L_2 ↓\downarrowRatio
T0.999037924.15%0.839
ST-Rigid0.99999960.339%1.002
ST-Shear0.999682218.46%0.867

Comparison of learned latent-point and ground-truth transport velocities on synthetic datasets.

AuthorsOpenReview ↗

Response to Q4 - Q5

Response to Q4

We agree that the Hodge decomposition can represent general smooth vector fields under appropriate domain conditions. Our goal is not to constrain motion-field expressivity, but to introduce a structured inductive bias for latent motion. Rather than predicting independent pointwise displacements, MoLa-PDE predicts scalar and antisymmetric potentials and derives motion from their spatial derivatives on the current latent support. Thus, motion depends on neighboring potential values and relative geometry, encouraging spatially coherent deformation. The construction also explicitly separates gradient and rotational components, together with a shared global translation, allowing the model to represent different motion patterns within a unified formulation.

Although the continuous decomposition does not restrict representable smooth fields, our discrete potential-based construction spatially couples learned motion. Fig. 3 supports this inductive bias: replacing potential-based motion with direct MLP displacements increases errors on all six PDEs. We will clarify this distinction and the motivation for our Hodge-inspired construction in the revised manuscript.

Response to Q5

Because analytical gradients are unavailable in latent space, we test WLS on synthetic smooth periodic potentials:

\frac{1}{Z_f}\sum_{\mathbf{k}\in\mathcal{K}} \frac{ a_{f,\mathbf{k}}\sin(2\pi\mathbf{k}^{\top}\mathbf{x}) +b_{f,\mathbf{k}}\cos(2\pi\mathbf{k}^{\top}\mathbf{x}) }{\|\mathbf{k}\|_2^2},$$ where $a_{f,\mathbf{k}},b_{f,\mathbf{k}}\sim\mathcal{N}(0,1)$ are sampled independently, and $Z_f^2=\sum_{\mathbf{k}\in\mathcal{K}} (a_{f,\mathbf{k}}^2+b_{f,\mathbf{k}}^2)/\|\mathbf{k}\|_2^2$ normalizes each field. We used $\mathcal{K}=\{1,2,3\}$ in 1D and $\mathcal{K}=\{(1,0),(0,1),(1,1),(1,-1)\}$ in 2D, generating 1,024 test potentials per dimension. We sampled moving latent-point configurations from trained-model rollouts as latent supports. We used 8 latent points from Burgers rollouts in 1D and 16 from Navier--Stokes rollouts in 2D. For each synthetic potential, we evaluated its values at these points, reconstructed its gradients using WLS, and compared the estimates with the analytical gradients at the same points. Let $\hat{\mathbf{g}} _{fj},\mathbf{g} _{fj}$ denote the estimated and analytical gradients at latent point $j$ for potential $f$. Relative $L _2$ error is defined as $E _f=\sqrt{\frac{\sum _j\lVert\hat{\mathbf{g}} _{fj}-\mathbf{g} _{fj}\rVert _2^2}{\sum _j\lVert\mathbf{g} _{fj}\rVert _2^2}}$. Field cosine similarity is defined as $C _f=\frac{\sum _j\hat{\mathbf{g}} _{fj}^{\top}\mathbf{g} _{fj}}{\sqrt{\sum _j\lVert\hat{\mathbf{g}} _{fj}\rVert _2^2}\sqrt{\sum _j\lVert\mathbf{g} _{fj}\rVert _2^2}}$. Pointwise directional agreement measures the fraction of observations with a positive estimated-versus-analytical gradient inner product and energy coverage measures the fraction of this energy located at directionally aligned points. With $\mathcal{I}=\\{(f,j):\lVert\mathbf{g} _{fj}\rVert _2 > 0\\}$ and $a _{fj}=\mathbf{1}[\hat{\mathbf{g}} _{fj}^{\top}\mathbf{g} _{fj} > 0]$, agreement and gradient-energy coverage are computed as $$A=\frac{\sum _{(f,j)\in\mathcal{I}}a _{fj}}{|\mathcal{I}|}, \qquad G=\frac{\sum _{(f,j)\in\mathcal{I}}a _{fj}\lVert\mathbf{g} _{fj}\rVert _2^2}{\sum _{(f,j)\in\mathcal{I}}\lVert\mathbf{g} _{fj}\rVert _2^2}.$$ The table below summarizes the results. WLS preserves strong directional alignment with the analytical gradients despite the sparse latent supports. The field cosine similarity reaches 0.940 in 1D and 0.956 in 2D, with over 90% pointwise directional agreement in both settings. Moreover, over 98% of the true gradient energy lies at locations where the estimated gradients have a positive inner product with the analytical gradients. Although the relative $L _2$ errors remain around 30%, indicating imperfect gradient recovery, these results suggest that WLS can retain meaningful gradient directions even when their magnitudes are not accurately estimated. This supports the use of local WLS to provide directional information for constructing potential-induced latent motion from sparse spatial supports. The practical benefit of this structured motion is further supported by the ablation in Fig. 3 in the manuscript, where replacing potential-driven motion with direct MLP displacements increases prediction error across all six PDEs. We have included these diagnostics in the revised manuscript. | **Configuration** | **Rel $L _2$ Error** $\downarrow$ | **Cosine** $\uparrow$ | **Agreement** $\uparrow$ | **Energy Coverage** $\uparrow$ | | --- | --- | --- | --- | --- | | 1D (8 points) | 32.16% | 0.940 | 90.89% | 98.93% | | 2D (16 points) | 29.28% | 0.956 | 97.87% | 99.81% | *WLS gradient estimates on moving latent supports compared with analytical gradients of synthetic periodic potentials.*
AuthorsOpenReview ↗

Response to Q6 - Q7

Response to Q6

Thank you for the suggestion. To isolate the contribution of our geometry-structured operator, we introduce an MLP kernel within the same MoLa-PDE architecture, evaluating both operators with static and moving latent supports.

The table below shows the results. Two operators achieve comparable performance with static supports, with neither consistently outperforming the other. However, their behavior differs substantially when latent motion is introduced. The geometry-structured operator consistently benefits from moving supports across all six PDEs, whereas the MLP kernel improves only on Burgers and Advection, and even degrades performance on the remaining four. This suggests that simply allowing latent coordinates to move is insufficient; the spatial operator must also effectively account for the changing geometry. By explicitly separating feature-dependent coefficients from geometric basis evaluation, our operator enables feature interactions to adapt more effectively to evolving latent supports, making the benefits of latent motion more consistent across different PDE dynamics.

We have included this ablation analysis in the revised paper.

PDEStructured (Static)MLP (Static)MLP (Motion)Ours (Motion)
Burgers2.162.302.151.74
Diffusion--Reaction0.1780.1730.1820.163
Advection1.431.450.5430.434
Navier--Stokes9.9211.211.39.04
Shallow Water0.2250.2230.2300.198
Cylinder15.014.814.912.9

Comparison of relative L2L_2 error (×10−2\times 10^{-2}) for geometry-structured and learned MLP kernels with static or moving latent supports.

Response to Q7

We thank the reviewer for this observation. We agree that, on Cylinder Flow, most latent-support displacement occurs during the early rollout steps, after which the support remains relatively stationary, as shown in Appendix E.8. This suggests that latent motion in this case primarily serves to adapt the initial spatial configuration rather than continuously track the flow dynamics.

To further examine this behavior, we introduce additional learned-static and input-dependent static baselines. On Cylinder Flow, learned-static supports reduce the error from 15.0 to 14.2, while input-dependent static supports achieve 12.7, slightly outperforming MoLa-PDE (12.9). These results further support the importance of initial spatial adaptation in this setting. This behavior may be related to the varying obstacle geometries and complex boundary interactions, which make the initial spatial configuration particularly important for representing the flow. In contrast, on the other five PDE benchmarks, moving supports consistently outperform input-dependent static supports, suggesting that continuous spatial adaptation provides additional benefits when solution structures evolve throughout the rollout.

We have clarified these observations in the revised appendix and expand our limitations discussion regarding complex boundary geometries and uniform support initialization.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.