acceptodds
Under review as a conference paper at ICLR 2027

Learning Modulated Spectral-Spatial State-Space Model for Solving Dynamic PDEs

Abstract

Forecasting the evolution of systems governed by partial differential equations (PDEs) requires capturing interactions across scales and effectively conditioning on temporal history. We propose -dyn, a spectral-spatial state-space neural operator that unifies cross-frequency modeling with history-conditioned spatial modulation. Its spectral-spatial backbone combines selective state-space modeling along bidirectional radial-angular scans of Fourier modes with local spatial processing, capturing complementary global and local interactions. A parallel modulation branch evolves historical observations in a latent spectral space and uses the resulting context to adaptively guide the forecasting backbone. Tri-plane factorization extends the selective state-space modeling in 2D Fourier space to 3D systems with reduced length of state-space sequence. We further establish an input-adaptive spatial kernel interpretation that characterizes the operator’s cross-frequency interactions. Across 15 diverse 2D and 3D physical systems, -dyn demonstrates strong dynamic prediction performance, achieving the lowest error in most evaluations on The Well and on PDEArena-NS, while remaining comparable to the strongest baseline on FNO-NS. Ablation studies support the effectiveness of combining structured spectral modeling with adaptive temporal conditioning for PDE forecasting.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Dynamic PDE Forecasting, Neural operator, Spectral-Spatial Modeling

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_6CGrOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces S⁴-dyn, a method that combines frequency-ordered scanning—a technique derived from Mamba—with spatial modulation based on historical physical fields to predict the evolution of physical fields governed by dynamic partial differential equations (PDEs) in an autoregressive manner. The authors extend this approach to three dimensions using tri-plane decomposition and evaluate its performance across 15 benchmark tasks, covering single-step prediction, multi-step rolling prediction, and the specific contributions of each individual module.

Critical strengths
  • Each component of the architecture serves a distinct purpose. A frequency-domain Mamba module models cross-frequency dependencies, local convolutions handle local spatial structures, and a historical branch generates modulation parameters—which vary according to spatial location—to assist in predicting the next time step.

  • The experimental scope is extensive. The paper evaluates the model across 15 2D and 3D benchmark tasks, encompassing various dynamical systems such as acoustic wave propagation, reaction-diffusion, fluid dynamics, and magnetohydrodynamics.

  • Key results provide quantitative validation. Across 13 systems in "The Well" benchmark—comprising 39 distinct single-step and multi-step prediction tasks—the proposed method achieved the lowest error among all compared approaches in 28 of these tasks.

  • Ablation studies confirm the utility of specific design elements. For instance, in the "Active Matter" task, removing the historical spatial modulation caused the average prediction error over the first 20 steps to increase from 0.5466 to 0.8040; furthermore, comparisons of different scanning strategies also validate the effectiveness of the proposed design.

Critical weaknesses

The methodological novelty appears limited. FourierMamba has already combined Fourier-domain modeling with Mamba in a closely related framework. The present paper primarily introduces component-level modifications and extends the evaluation to 3D data. In my view, these contributions are insufficient to establish a substantial methodological advance. Moreover, the experiments compare only FourierMamba’s scanning strategy, rather than the complete model, leaving the added value of the proposed extensions insufficiently demonstrated.

Questions for authors

Please address the concerns raised in the Weaknesses above.

5 comments
AuthorsOpenReview ↗

Response to 6CGr (1/3): Differences from FourierMamba in spectral-spatial modeling

We thank the reviewer for the thorough review. Below, we clarify how S⁴-dyn differs from FourierMamba in spectral scanning, history-conditioned modulation, and 3D spectral modeling. We also provide a full-model comparison with our FourierMamba reproduction on TRL 2D and Active Matter under identical training settings, together with additional controlled scanning and history-conditioning ablations on Active Matter and TRL 2D.

Q1: Methodological novelty beyond FourierMamba.

The novelty of S⁴-dyn lies in three components tailored to dynamic PDE forecasting: the Spectral-Spatial Operator, which combines BDRA scan based spectral state-space modeling with local convolution; the Spatial Feature Modulation Branch, which uses latent spectral evolution to generate history-conditioned Spatial-FiLM; and the Tri-Plane Factorized Spectral Module, which extends spectral state-space modeling to 3D physical fields.

1.Spectral-Spatial Operator. Whereas FourierMamba operates on amplitude and phase spectra, our Bidirectional Spectral State-Space Operator updates the real and imaginary components of the retained Fourier modes in ΛR\Lambda_R while preserving the complementary modes. Moreover, unlike FourierMamba's progressive and bilateral zigzag scanning, our BDRA scan groups the retained modes into radial rings from low to high frequencies and traverses each ring in two opposite angular directions. For spatial processing, FourierMamba uses a Spatial Mamba branch, whereas S⁴-dyn combines its Bidirectional Spectral State-Space Operator with a Local Convolutional Operator for local spatial interactions.

In Table R3-1, the component ablations w/o C(ℓ)C^{(\ell)} and w/o GΛR(ℓ)G_{\Lambda_R}^{(\ell)} show that removing either operator increases both rollout errors on Active Matter. For the scanning-path ablations, compared with the parameter-matched 2-dir raster scan, BDRA reduces the 1:20 rollout VRMSE by 20.2% on Active Matter and 24.4% on TRL 2D. Compared with the parameter-matched Reverse BDRA, which reverses only the radial scanning order from low-to-high to high-to-low, BDRA further reduces this error by 6.5% on Active Matter and 5.5% on TRL 2D. BDRA also outperforms w/o π−\pi_- (unidirectional angular scan), w/ radial reversal (which adds two high-to-low radial paths), 4-dir raster scan, and FourierMamba zigzag in both rollout intervals on both datasets. The full-model comparison with FourierMamba is presented in Q2.

Table R3-1. Component and scanning-path ablations on Active Matter and TRL 2D. Metrics are VRMSE.

DatasetVariantParamsNext-step ↓1:20 ↓21:60 ↓
Active MatterS⁴-dyn (BDRA)6.59M0.01260.54661.3731
w/o C(ℓ)C^{(\ell)}3.04M0.02981.56981.5724
w/o GΛR(ℓ)G_{\Lambda_R}^{(\ell)}4.64M0.03330.66031.9075
w/o π−\pi_-5.81M0.01521.20041.6495
w/ radial reversal8.14M0.01870.64381.4554
Reverse BDRA6.59M0.01470.58491.4132
2-dir raster scan6.59M0.01370.68471.5061
4-dir raster scan8.14M0.01660.60081.4599
FourierMamba zigzag8.14M0.01980.96781.4431
TRL 2DS⁴-dyn (BDRA)6.55M0.13420.41150.9263
w/o π−\pi_-5.78M0.13600.49361.1125
w/ radial reversal8.09M0.13450.43811.0421
Reverse BDRA6.55M0.14730.43540.9881
2-dir raster scan6.55M0.13230.54441.4830
4-dir raster scan8.09M0.13470.44171.0486
FourierMamba zigzag8.09M0.13510.44170.9768
AuthorsOpenReview ↗

Response to 6CGr (2/3): History-conditioned modulation and 3D spectral modeling

Q1: continued

2.Spatial Feature Modulation Branch. Our Spatial Feature Modulation Branch uses the Spectral Update Layer to evolve historical observations in latent spectral space and predict the next-step latent state. The Spatial-FiLM Modulation Head then uses this predicted state to generate layer-wise, spatially varying modulation fields for the Spectral-Spatial State-Space Branch. This differs from FourierMamba's spatial and frequency feature processing, which does not use a predicted future latent state to generate modulation.

As shown in Table R3-2, removing the entire branch (w/o SFM) increases rollout errors on both datasets. Retaining the frame-wise encoder and Spatial-FiLM Modulation Head, we also replace the Spectral Update Layer with three approximately parameter-matched alternatives: Spatial update + FiLM (channel-wise temporal projection and local convolutions in latent space without FFT/IFFT), ConvGRU + FiLM (3×3 convolutional gates across frames; Ballas et al., 2016), and Temporal Transformer + FiLM (single-layer, four-head temporal self-attention per location). S⁴-dyn achieves lower 1:20 and 21:60 rollout VRMSE than all three alternatives on both datasets. On Active Matter and TRL 2D, it reduces the 1:20 VRMSE by 73.3% and 14.2% over Spatial update + FiLM, and by 20.0% (vs. Temporal Transformer + FiLM) and 9.5% (vs. ConvGRU + FiLM) over the stronger temporal encoder for 1:20 rollout on each dataset.

Table R3-2. Ablations of spatial modulation and latent state updates. Metrics are VRMSE.

DatasetVariantParamsNext-step ↓1:20 ↓21:60 ↓
TRL 2DS⁴-dyn6.55M0.13420.41150.9263
w/o SFM6.01M0.13610.60811.0027
Spatial update + FiLM6.55M0.14260.47970.9469
ConvGRU + FiLM6.54M0.13250.45471.0324
Temporal Transformer + FiLM6.56M0.14070.48890.9896
Active MatterS⁴-dyn6.59M0.01260.54661.3731
w/o SFM6.04M0.01590.80401.7271
Spatial update + FiLM6.59M0.09772.04621.7898
ConvGRU + FiLM6.58M0.02571.59781.3919
Temporal Transformer + FiLM6.59M0.01940.68361.4748

3.Tri-Plane Factorized Spectral Module. We extend spectral state-space modeling to 3D physical fields through BDRA processing on three orthogonal spectral planes and Tri-Plane Volumetric Fusion. This reduces the sequence length per scanning direction from mxmymzm_x m_y m_z to mxmy+mxmz+mymzm_x m_y + m_x m_z + m_y m_z, avoiding dense volumetric scanning. On 3D Rayleigh–Taylor and TGC, the complete S⁴-dyn model reduces the 1:20 rollout VRMSE by 84.6% and 62.3% compared with Flower (Table R3-3).

Table R3-3. Prediction performance on 3D physical systems. The metric is VRMSE.

DatasetModelNext-step ↓1:20 ↓21:60 ↓
MHD 64Flower0.11650.75801.2827
S⁴-dyn0.08340.74251.2160
Rayleigh-TaylorFlower0.04910.58621.4880
S⁴-dyn0.02830.09010.7511
TGCFlower0.17001.26363.7477
S⁴-dyn0.07770.47581.0062

[1] Nicolas Ballas, Li Yao, Chris Pal, and Aaron Courville. Delving Deeper into Convolutional Networks for Learning Video Representations. ICLR, 2016.

AuthorsOpenReview ↗

Response to 6CGr (3/3): Full-model comparison with FourierMamba on TRL 2D and Active Matter

Q2: Full-model comparison with FourierMamba.

To address the concern regarding the lack of a full-model comparison, we reproduced the complete FourierMamba architecture based on its original paper, rather than comparing only its zigzag scanning strategy. Since the original implementation is not publicly available, our reproduction retains the Fourier Residual State-Space Blocks (FRSSBs), the Fourier Spatial Interaction SSM (FSI-SSM), the Fourier Channel Evolution SSM (FCE-SSM), and the original scanning strategies. We only reshaped the input from H×W×T×CH \times W \times T \times C to H×W×(TC)H \times W \times (TC) and adjusted the output channels to CC to accommodate autoregressive dynamic PDE forecasting.

For the full-model comparison on TRL 2D and Active Matter, we trained the reproduced FourierMamba using the same data splits and training settings as S⁴-dyn in the original submission, including relative L2 loss, AdamW optimizer, OneCycleLR scheduler, batch size of 4, and 100 epochs on an NVIDIA A100 GPU. As shown in Table R3-4, S⁴-dyn achieves lower VRMSE than the reproduced full FourierMamba model across all three metrics on both datasets, reducing the next-step, 1:20, and 21:60 rollout errors by 44.2%, 43.6%, and 27.5% on TRL 2D (6.55M vs. 10.03M parameters), and by 96.7%, 50.6%, and 8.5% on Active Matter (6.59M vs. 10.05M parameters).

Table R3-4. Full-model comparison on TRL 2D and Active Matter. Metrics are VRMSE.

DatasetModelParamsNext-step ↓1:20 ↓21:60 ↓
TRL 2DFlower17.33M0.19300.54911.0516
FourierMamba10.03M0.24070.73021.2770
S⁴-dyn6.55M0.13420.41150.9263
Active MatterFlower17.34M0.02491.39051.5251
FourierMamba10.05M0.37811.10681.5003
S⁴-dyn6.59M0.01260.54661.3731

In addition to the full-model comparison in Table R3-4, the controlled scanning-path comparison in Table R3-1 (keeping the S⁴-dyn framework unchanged) shows that BDRA reduces the 1:20 rollout VRMSE compared with FourierMamba zigzag from 0.9678 to 0.5466 on Active Matter and from 0.4417 to 0.4115 on TRL 2D, with fewer parameters.

Reviewer_QkQmOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces a neural operator for dynamic PDE forecasting, which captures cross-scale interactions and leverages historical observations to improve future predictions. By effectively modeling interactions between large-scale and small-scale physical structures and incorporating temporal information, it enhances forecasting accuracy.

Critical strengths
  • BDRA scan organizes fourier modes according to their radial frequencies. This allows the state-space model to process spectral information in a low-to-high frequency order.
  • It introduces a separate history-conditioned module, which allows the model to adapt its predictions based on historical data.
  • Its evaluation is across 15 physical systems, showing promising results.
Critical weaknesses
  • The proposed method combines several existing techniques, so the novelty is not that significant.
  • The evaluation does not sufficiently demonstrate that the advantages of BDRA primarily arise from its proposed radial frequency organization.
Questions for authors
  • Could you provide a more controlled comparison between BDRA and alternative frequency scanning strategies?
  • How does the proposed method ensure that the updated fourier coefficients keep conjugate symmetry?
  • Could you compare against alternatives such as using the full historical sequence, a standard temporal encoder, or other branches with a matched parameter budget?
4 comments
AuthorsOpenReview ↗

Response to QkQm (1/3): Methodological contributions and supporting ablations

We thank the reviewer for the helpful comments. Below, we clarify the contributions and conjugate symmetry of S⁴-dyn, and add controlled ablations on Active Matter and TRL 2D distinguishing the contributions of radial ring organization, radial scanning order, and alternative history-conditioning branches.

Q1:Limited methodological novelty of S⁴-dyn

The novelty of S⁴-dyn lies in three components tailored to dynamic PDE forecasting: the Spectral-Spatial Operator, which combines BDRA scan based spectral state-space modeling with local convolution; the Spatial Feature Modulation Branch, which uses latent spectral evolution to generate history-conditioned Spatial-FiLM; and the Tri-Plane Factorized Spectral Module, which extends spectral state-space modeling to 3D physical fields.

1.Spectral-Spatial Operator. We combine a Bidirectional Spectral State-Space Operator for cross-frequency interactions with a Local Convolutional Operator for local spatial processing. The key design is the BDRA scan, which organizes Fourier modes into radial rings, processes them from low to high frequencies, and uses bidirectional angular scanning within each ring to model cross-scale and angular interactions. On Active Matter, removing the Local Convolutional Operator (w/o C(ℓ)C^{(\ell)}) increases the 1:20 rollout VRMSE from 0.5466 to 1.5698, while removing the Bidirectional Spectral State-Space Operator (w/o GΛR(ℓ)G_{\Lambda_R}^{(\ell)}) increases the 21:60 rollout VRMSE from 1.3731 to 1.9075 (Table R2-1). Further controlled comparisons isolating radial ordering and alternative scanning paths are presented in Q2 below.

Table R2-1. Component ablations of the Spectral-Spatial Operator on Active Matter. Metrics are VRMSE.

DatasetVariantParamsNext-step ↓1:20 ↓21:60 ↓
Active MatterS⁴-dyn6.59M0.01260.54661.3731
w/o C(ℓ)C^{(\ell)}3.04M0.02981.56981.5724
w/o GΛR(ℓ)G_{\Lambda_R}^{(\ell)}4.64M0.03330.66031.9075

2.Spatial Feature Modulation Branch. The Spectral Update Layer predicts the next-step latent state from historical observations in the spectral domain, and the Spatial-FiLM Modulation Head converts it into layer-wise, spatially varying scale and shift fields to modulate the Spectral-Spatial State-Space Branch. As shown in Table R2-2, removing the entire branch (w/o SFM) increases the 1:20 rollout VRMSE from 0.4115 to 0.6081 on TRL 2D and from 0.5466 to 0.8040 on Active Matter. Unlike Global FiLM, which shares modulation parameters across spatial locations, Spatial-FiLM provides location-dependent modulation. On 3D Rayleigh–Taylor, replacing Spatial-FiLM with Global FiLM increases the 1:20 rollout VRMSE from 0.0901 to 0.1828 and the 21:60 rollout VRMSE from 0.7511 to 1.2893, as shown in Table R2-2. Controlled comparisons replacing the Spectral Update Layer while retaining the Spatial-FiLM Modulation Head are presented in Q4.

Table R2-2. Ablations of spatial modulation on TRL 2D, Active Matter, and 3D Rayleigh–Taylor. Metrics are VRMSE.

DatasetVariantParamsNext-step ↓1:20 ↓21:60 ↓
TRL 2DS⁴-dyn6.55M0.13420.41150.9263
w/o SFM6.01M0.13610.60811.0027
Active MatterS⁴-dyn6.59M0.01260.54661.3731
w/o SFM6.04M0.01590.80401.7271
3D Rayleigh–TaylorS⁴-dyn6.98M0.02830.09010.7511
w/o SFM6.18M0.02980.09210.7686
Global FiLM6.89M0.03150.18281.2893

3.Tri-Plane Factorized Spectral Module. For 3D physical fields, we project the retained cuboidal spectral region onto three orthogonal planes (XY, XZ, and YZ), apply BDRA scan based state-space modeling on each plane, and combine the results through Tri-Plane Volumetric Fusion. This avoids constructing dense volumetric state-space sequences, reducing the total number of tokens processed per scan direction from mxmymzm_x m_y m_z to mxmy+mxmz+mymzm_x m_y + m_x m_z + m_y m_z. S⁴-dyn also achieves competitive forecasting performance across multiple 3D physical systems (Table R2-3).

Table R2-3. Prediction performance on 3D physical systems. The metric is VRMSE.

DatasetModelNext-step ↓1:20 ↓21:60 ↓
MHD 64Flower0.11650.75801.2827
S⁴-dyn0.08340.74251.2160
Rayleigh-TaylorFlower0.04910.58621.4880
S⁴-dyn0.02830.09010.7511
TGCFlower0.17001.26363.7477
S⁴-dyn0.07770.47581.0062

Overall, S⁴-dyn achieves the lowest prediction error in 28 of 39 evaluations across 13 physical systems from The Well, as well as the best performance on PDEArena-NS, while remaining competitive on FNO-NS. Together with the component and controlled ablations, these results support the effectiveness of the proposed designs for dynamic PDE forecasting.

AuthorsOpenReview ↗

Response to QkQm (2/3): Controlled scanning comparisons and conjugate symmetry

Q2:BDRA's radial frequency organization and controlled scanning ablations.

To evaluate the contribution of BDRA's radial frequency organization, we conducted controlled scanning ablations on Active Matter and TRL 2D (Table R2-4). All scanning variants are evaluated within the same S⁴-dyn framework, using identical retained Fourier modes and training settings.

Compared with the parameter-matched 2-dir raster scan (which does not use radial rings), BDRA reduces the 1:20 rollout VRMSE by 20.2% on Active Matter and 24.4% on TRL 2D. To separate radial ring organization from radial direction, we evaluate a parameter-matched Reverse BDRA that preserves the radial rings and bidirectional angular scans but reverses the radial order to high-to-low. Even with this reversed order, Reverse BDRA reduces the 1:20 VRMSE by 14.6% on Active Matter and 20.0% on TRL 2D over 2-dir raster scan, while scanning from low to high frequencies (BDRA) further reduces this error by 6.5% and 5.5%. BDRA also outperforms w/o π−\pi_- (unidirectional angular scan), w/ radial reversal (which adds two high-to-low radial paths), 4-dir raster scan, and FourierMamba zigzag in both rollout intervals on both datasets.

Table R2-4. BDRA scanning-path ablations on Active Matter and TRL 2D. Metrics are VRMSE.

DatasetVariantParamsNext-step ↓1:20 ↓21:60 ↓
Active MatterS⁴-dyn (BDRA)6.59M0.01260.54661.3731
w/o π−\pi_-5.81M0.01521.20041.6495
w/ radial reversal8.14M0.01870.64381.4554
Reverse BDRA6.59M0.01470.58491.4132
2-dir raster scan6.59M0.01370.68471.5061
4-dir raster scan8.14M0.01660.60081.4599
FourierMamba zigzag8.14M0.01980.96781.4431
TRL 2DS⁴-dyn (BDRA)6.55M0.13420.41150.9263
w/o π−\pi_-5.78M0.13600.49361.1125
w/ radial reversal8.09M0.13450.43811.0421
Reverse BDRA6.55M0.14730.43540.9881
2-dir raster scan6.55M0.13230.54441.4830
4-dir raster scan8.09M0.13470.44171.0486
FourierMamba zigzag8.09M0.13510.44170.9768

Q3:Conjugate symmetry of updated Fourier coefficients

In our implementation, we obtain a real-valued spatial output by taking the real part after the inverse Fourier transform using torch.fft.ifftn(...).real. By the linearity of the inverse Fourier transform, this operation is mathematically equivalent to enforcing Hermitian symmetry on the merged spectrum before the inverse transform (as stated in Appendix D.2) through the Hermitian projection:

(PHv^)(k)=v^(k)+v^(−k)‾2.(\mathcal P_{\mathrm H}\widehat{\mathbf v})(\mathbf k)=\frac{\widehat{\mathbf v}(\mathbf k)+\overline{\widehat{\mathbf v}(-\mathbf k)}}{2}.

Re⁡[F−1(v^)]=F−1[PHv^].\operatorname{Re}\left[\mathcal F^{-1}(\widehat{\mathbf v})\right]=\mathcal F^{-1}\left[\mathcal P_{\mathrm H}\widehat{\mathbf v}\right].

(PHv^)(−k)=(PHv^)(k)‾.(\mathcal P_{\mathrm H}\widehat{\mathbf v})(-\mathbf k)=\overline{(\mathcal P_{\mathrm H}\widehat{\mathbf v})(\mathbf k)}.

Therefore, although the intermediate updated spectrum may not satisfy conjugate symmetry, the effective spectrum corresponding to the final real-valued output does. We will clarify Appendix D.2 to state this implementation and equivalence explicitly.

AuthorsOpenReview ↗

Response to QkQm (3/3): Controlled comparisons of history-conditioning methods

Q4:Comparisons with alternative history-conditioning methods.

We conducted controlled comparisons on Active Matter and TRL 2D (Table R2-5), where all variants take the same four historical frames as input: (1) feeding the full four-frame sequence directly into the main branch without modulation (w/o SFM); and (2) retaining the frame-wise encoder and Spatial-FiLM Modulation Head while replacing the Spectral Update Layer with three parameter-matched alternatives—a spatial-domain update branch applying channel-wise temporal projection and local convolutions in latent space without FFT/IFFT (Spatial update + FiLM), a recurrent temporal encoder (ConvGRU + FiLM, with 3×3 convolutional gates; Ballas et al., 2016), and a full-sequence attention encoder (Temporal Transformer + FiLM, with single-layer, four-head temporal self-attention per location).

As shown in Table R2-5, S⁴-dyn achieves the lowest 1:20 and 21:60 rollout VRMSE among all variants on both datasets. On Active Matter and TRL 2D, it reduces the 1:20 VRMSE by 32.0% and 32.3% over w/o SFM, and by 73.3% and 14.2% over Spatial update + FiLM. Compared with the stronger temporal encoder for 1:20 rollout on each dataset, S⁴-dyn further reduces the 1:20 VRMSE by 20.0% on Active Matter (vs. Temporal Transformer + FiLM) and 9.5% on TRL 2D (vs. ConvGRU + FiLM).

Table R2-5. Ablations of spatial modulation and latent state updates. Metrics are VRMSE.

DatasetVariantParamsNext-step ↓1:20 ↓21:60 ↓
TRL 2DS⁴-dyn6.55M0.13420.41150.9263
w/o SFM6.01M0.13610.60811.0027
Spatial update + FiLM6.55M0.14260.47970.9469
ConvGRU + FiLM6.54M0.13250.45471.0324
Temporal Transformer + FiLM6.56M0.14070.48890.9896
Active MatterS⁴-dyn6.59M0.01260.54661.3731
w/o SFM6.04M0.01590.80401.7271
Spatial update + FiLM6.59M0.09772.04621.7898
ConvGRU + FiLM6.58M0.02571.59781.3919
Temporal Transformer + FiLM6.59M0.01940.68361.4748

[1] Nicolas Ballas, Li Yao, Chris Pal, and Aaron Courville. Delving Deeper into Convolutional Networks for Learning Video Representations. ICLR, 2016.

AuthorsOpenReview ↗

Follow-up on Our Response

Dear Reviewer QkQm,

Thanks for your valuable comments. As the discussion period ends on November 18, we would like to follow up on our responses. We have added controlled comparisons of scanning strategies and alternative history-conditioning methods on Active Matter and TRL 2D, and explained the Hermitian projection corresponding to our real-valued output.

Could you please share whether these results and explanations address your concerns about BDRA’s radial organization, history conditioning, and conjugate symmetry? We would be happy to clarify any remaining questions.

Thank you for your time.

The Authors

Reviewer_G5YmOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies autoregressive forecasting of dynamic PDEs and proposes a spectral-spatial state space neural operator, S⁴-dyn. The model organizes Fourier modes through radial rings and bidirectional angular scanning, uses Mamba to model inter-frequency dependencies, and incorporates local spatial convolution. Another branch performs latent spectral evolution on historical observations to generate layer-wise spatial modulation parameters; for three-dimensional problems, a triplane decomposition is used to process spectral features. The paper evaluates single-step and rollout prediction on 13 systems from The Well and two Navier–Stokes datasets, reports error improvements in most comparisons, and validates design choices through ablations on modules, scanning paths, and operator replacement.

Critical strengths
  1. The experiments cover multiple physical systems. The paper evaluates the model on 15 systems in 2D and 3D. On 3D Rayleigh–Taylor, the error over rollout steps 21–60 is approximately 49.5% lower than that of the second-best method. This positive result suggests that the overall model merits further investigation.

  2. The ablations include some targeted comparisons. Table 2(a) compares scanning paths and spectral operators, in addition to removing modules. These comparisons are informative about the design choices on Active Matter.

Critical weaknesses
  1. The substantive contribution beyond existing methods is not sufficiently supported. The paper needs to establish the additional value of radial rings, bidirectional angular scanning, and history-conditioned modulation. The supporting evidence remains limited to specific settings. The theoretical analysis does not resolve this issue either.

  2. The main comparisons do not establish a sufficiently reliable performance advantage. It is unclear how much of the improvement comes from the architecture and how much may be affected by differences in training budgets. Table 7 also shows that single-step inference on 3D Rayleigh–Taylor takes approximately five times as long as Flower, indicating that fewer parameters cannot substitute for a comparison of actual computational costs.

Questions for authors
  1. What does BDRA add beyond existing spectral SSM methods? Has its advantage been demonstrated beyond Active Matter?
  2. Do the gains from history-conditioned modulation come from latent spectral evolution or from additional spatial modulation? How do the ablations distinguish the two?
  3. Are the training budgets comparable to those of Flower and the other baselines? Please report the actual training costs and explain how budget differences affect the comparison.
  4. Are the main results and key ablations stable across independent training runs? Please report means and standard deviations.
4 comments
AuthorsOpenReview ↗

Response to G5Ym (1/3): BDRA contributions and cross-system scanning ablations

We thank the reviewer for the constructive feedback. Below, we clarify the contribution of BDRA and provide additional scanning ablations on TRL 2D, controlled comparisons distinguishing the contributions of spatial modulation and latent spectral evolution, matched 12-hour training comparisons with Flower on A100 GPUs, and evaluations across three independent training runs.

Q1: BDRA's contribution and evidence across systems

The BDRA scan organizes 2D Fourier modes into radial rings to model cross-scale spectral dependencies. Specifically, BDRA scans the radial rings from low to high frequencies while traversing each ring in two opposite angular directions. Under the lower-triangular state-space transfer (Theorem 3.1 / Appendix B), this low-to-high ordering ensures that inner-ring modes do not depend on outer-ring modes within each State-Space Operator, while the bidirectional paths capture complementary angular contexts.

To validate BDRA scan across different physical systems, we conducted additional scanning ablations on TRL 2D and combined them with the Active Matter results in Table R1-1. All scanning strategies are compared using the same S⁴-dyn framework and Fourier modes. BDRA scan achieves the lowest 1:20 and 21:60 rollout VRMSE on both datasets. Compared with the parameter-matched 2-dir raster scan, BDRA reduces the 1:20 rollout VRMSE by 24.4% on TRL 2D and 20.2% on Active Matter. Compared with the parameter-matched Reverse BDRA, which reverses only the radial scanning order from low-to-high to high-to-low, BDRA further reduces this error by 5.5% on TRL 2D and 6.5% on Active Matter. BDRA also outperforms w/o π−\pi_- (unidirectional angular scan), w/ radial reversal (which adds two high-to-low radial paths), 4-dir raster scan, and FourierMamba zigzag in both rollout intervals on both datasets.

Table R1-1. BDRA scanning-path ablations on Active Matter and TRL 2D. Metrics are VRMSE.

DatasetVariantParamsNext-step ↓1:20 ↓21:60 ↓
Active MatterS⁴-dyn (BDRA)6.59M0.01260.54661.3731
w/o π−\pi_-5.81M0.01521.20041.6495
w/ radial reversal8.14M0.01870.64381.4554
Reverse BDRA6.59M0.01470.58491.4132
2-dir raster scan6.59M0.01370.68471.5061
4-dir raster scan8.14M0.01660.60081.4599
FourierMamba zigzag8.14M0.01980.96781.4431
TRL 2DS⁴-dyn (BDRA)6.55M0.13420.41150.9263
w/o π−\pi_-5.78M0.13600.49361.1125
w/ radial reversal8.09M0.13450.43811.0421
Reverse BDRA6.55M0.14730.43540.9881
2-dir raster scan6.55M0.13230.54441.4830
4-dir raster scan8.09M0.13470.44171.0486
FourierMamba zigzag8.09M0.13510.44170.9768
AuthorsOpenReview ↗

Response to G5Ym (2/3): Contributions of latent spectral evolution and spatial modulation

Q2: Contributions of latent spectral updates and spatial modulation

To distinguish the contributions of spatial modulation and latent spectral evolution, we conducted controlled ablations on Active Matter, TRL 2D, and 3D Rayleigh–Taylor (Table R1-2). First, removing the Spatial Feature Modulation Branch (w/o SFM) increases the 1:20 rollout VRMSE from 0.5466 to 0.8040 on Active Matter and from 0.4115 to 0.6081 on TRL 2D. Moreover, replacing Spatial-FiLM with spatially shared Global FiLM under the same latent spectral evolution increases the 1:20 VRMSE from 0.0901 to 0.1828 on 3D Rayleigh–Taylor (Table R1-2).

Second, to isolate the effect of latent spectral evolution, we retain the frame-wise spatial encoder and Spatial-FiLM Modulation Head and replace only the Spectral Update Layer with three approximately parameter-matched alternatives: Spatial update + FiLM, ConvGRU + FiLM (Ballas et al., 2016), and Temporal Transformer + FiLM. All three alternatives use the same four encoded historical frames. Spatial update + FiLM applies channel-wise temporal projection and local convolutions directly in the latent spatial domain without FFT/IFFT. ConvGRU processes the frames sequentially using 3×3 convolutional gates. Temporal Transformer applies single-layer, four-head temporal self-attention at each spatial location.

As shown in Table R1-2, S⁴-dyn achieves lower 1:20 and 21:60 rollout VRMSE than all three alternatives on both TRL 2D and Active Matter. Compared with Spatial update + FiLM, spectral evolution reduces the 1:20 VRMSE by 73.3% on Active Matter and 14.2% on TRL 2D. Compared with the stronger temporal encoder for 1:20 rollout on each dataset, S⁴-dyn further reduces the 1:20 VRMSE by 20.0% on Active Matter (vs. Temporal Transformer + FiLM) and 9.5% on TRL 2D (vs. ConvGRU + FiLM).

Table R1-2. Ablations of spatial modulation and latent state updates. Metrics are VRMSE.

DatasetVariantParamsNext-step ↓1:20 ↓21:60 ↓
TRL 2DS⁴-dyn6.55M0.13420.41150.9263
w/o SFM6.01M0.13610.60811.0027
Spatial update + FiLM6.55M0.14260.47970.9469
ConvGRU + FiLM6.54M0.13250.45471.0324
Temporal Transformer + FiLM6.56M0.14070.48890.9896
Active MatterS⁴-dyn6.59M0.01260.54661.3731
w/o SFM6.04M0.01590.80401.7271
Spatial update + FiLM6.59M0.09772.04621.7898
ConvGRU + FiLM6.58M0.02571.59781.3919
Temporal Transformer + FiLM6.59M0.01940.68361.4748
3D Rayleigh–TaylorS⁴-dyn6.98M0.02830.09010.7511
w/o SFM6.18M0.02980.09210.7686
Global FiLM6.89M0.03150.18281.2893

[1] Nicolas Ballas, Li Yao, Chris Pal, and Aaron Courville. Delving Deeper into Convolutional Networks for Learning Video Representations. ICLR, 2016.

AuthorsOpenReview ↗

Response to G5Ym (3/3): Matched training budgets and stability across independent runs

Q3: Training budgets and actual computational costs

In the original experiments, S⁴-dyn was trained for 100 epochs on an A100 80GB GPU, while Flower and the other baselines used H200 GPUs with a training limit of 100 epochs or 24 hours, whichever came first. To further assess the fairness of the comparison, we retrained S⁴-dyn and Flower on TRL 2D and Active Matter using the same A100 80GB GPU type, batch size of 4, and a fixed 12-hour training budget. On TRL 2D, S⁴-dyn completed 121 epochs, compared with 218 epochs for Flower. On Active Matter, S⁴-dyn completed 60 epochs, compared with 129 epochs for Flower.

As shown in Table R1-3, despite Flower completing more epochs within the same training time, S⁴-dyn achieves lower errors across all reported metrics on both datasets. Specifically, it reduces the 1:20 and 21:60 rollout VRMSE by 36.8% and 33.3% on TRL 2D, and by 48.5% and 11.1% on Active Matter. We also report test VRMSE across intermediate training-time budgets for TRL 2D (https://anonymous.4open.science/r/Re-C186/trl2d_matched_budget.pdf) and Active Matter (https://anonymous.4open.science/r/Re-C186/am_matched_budget.pdf).

Table R1-3. Results after 12 hours of training on the same A100 model with the same batch size. Metrics are test VRMSE of the best validation checkpoint within the budget.

DatasetModelNext-step ↓1:20 ↓21:60 ↓
TRL 2DS⁴-dyn0.12260.42170.8957
Flower0.17860.66691.3435
Active MatterS⁴-dyn0.01700.67171.4560
Flower0.01831.30371.6372

Q4: Consistency across independent training runs

We repeated the training with two additional independent random seeds on TRL 2D, Active Matter, and MHD 64. As shown in Table R1-4a, S⁴-dyn exhibits consistent performance across three runs, with relatively small standard deviations across the reported metrics. Compared with Flower's results in Table 1 of the original submission, the three-run mean VRMSE of S⁴-dyn remains lower across all evaluated settings (and comparable within one standard deviation on MHD 64 at 1:20).

Table R1-4a. Stability of main results across three independent training runs. S⁴-dyn results are reported as mean ± sample standard deviation.

DatasetModelNext-step ↓1:20 ↓21:60 ↓
TRL 2DS⁴-dyn0.1358 ± 0.00230.4198 ± 0.00780.9231 ± 0.0259
Active MatterS⁴-dyn0.0133 ± 0.00080.5493 ± 0.01031.3565 ± 0.0277
MHD 64S⁴-dyn0.0810 ± 0.00210.7542 ± 0.01021.2233 ± 0.0221

We also repeated three key ablations on Active Matter using independent random seeds. As shown in Table R1-4b, removing SFM (w/o SFM), removing π−\pi_-(w/o π−\pi_-) , or replacing BDRA with the 2-dir raster scan consistently increases rollout errors. These results support the stability of our key design choices.

Table R1-4b. Stability of key ablations on Active Matter across three independent training runs. Results are reported as mean ± sample standard deviation.

DatasetVariantParamsNext-step ↓1:20 ↓21:60 ↓
Active MatterS⁴-dyn6.59M0.0133 ± 0.00080.5493 ± 0.01031.3565 ± 0.0277
w/o SFM6.04M0.0159 ± 0.00070.8051 ± 0.01151.7254 ± 0.0218
w/o π−\pi_-5.81M0.0152 ± 0.00041.2002 ± 0.01551.6507 ± 0.0256
2-dir raster scan6.59M0.0137 ± 0.00040.6853 ± 0.01031.5075 ± 0.0246
AuthorsOpenReview ↗

Follow-up on Our Response

Dear Reviewer G5Ym,

Thanks for your valuable comments. As the discussion period ends on November 18, we would like to follow up on our responses. We have added scanning ablations on TRL 2D, controlled SFM ablations, matched 12-hour training comparisons with Flower, and three-run evaluations of the main results and key ablations.

Could you please share your assessment of this additional evidence and indicate any remaining concerns? We would appreciate the opportunity to clarify them before the discussion closes.

Thank you for your time.

The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank all three reviewers for their feedback and for recognizing our evaluation across 15 physical systems (G5Ym, QkQm, 6CGr), the 49.5% 21:60 rollout reduction on 3D Rayleigh–Taylor (G5Ym), and the targeted component and scanning ablations (G5Ym, 6CGr). Below, we summarize the new controlled experiments and clarifications addressing the main concerns.

1. Methodological novelty (G5Ym, QkQm, 6CGr). S⁴-dyn is designed for autoregressive PDE forecasting through three components: (a) a Spectral-Spatial Operator combining a Bidirectional Spectral State-Space Operator (updating retained Fourier modes ΛR\Lambda_R via Bidirectional Radial-Angular [BDRA] scanning) with a Local Convolutional Operator; (b) a Spatial Feature Modulation Branch (SFM) evolving history in latent spectral space for layer-wise Spatial-FiLM fields; and (c) a Tri-Plane Factorized Spectral Module extending BDRA to 3D via three orthogonal planes (scan length mxmymzm_x m_y m_z→mxmy+mxmz+mymzm_x m_y+m_x m_z+m_y m_z). Removing either operator degrades Active Matter rollouts (w/o C(ℓ)C^{(\ell)}: 1:20 VRMSE 0.5466→1.5698; w/o GΛR(ℓ)G_{\Lambda_R}^{(\ell)}: 21:60 VRMSE 1.3731→1.9075; Table R2-1, QkQm). On The Well (13 systems), S⁴-dyn achieves the lowest VRMSE in 28 of 39 evaluations (Paper Table 1).

2. Controlled BDRA ablations across systems & conjugate symmetry (G5Ym, QkQm, 6CGr). We extended scanning ablations from Active Matter to TRL 2D under the same framework, retained Fourier modes, and training settings to evaluate radial-angular organization and isolate radial scanning direction (Table R1-1, G5Ym). Over the parameter-matched 2-dir raster scan, Reverse BDRA (radial rings + bidirectional angular scans, high-to-low) cuts 1:20 VRMSE by 14.6% (Active Matter) and 20.0% (TRL 2D). Compared with Reverse BDRA, low-to-high BDRA further reduces it by 6.5% and 5.5%; relative to 2-dir raster, BDRA’s reductions are 20.2% and 24.4%. BDRA also outperforms w/o π−\pi_- (0.5466 vs. 1.2004 on Active Matter; 0.4115 vs. 0.4936 on TRL 2D), w/ radial reversal, 4-dir raster, and FourierMamba zigzag on both rollout intervals and datasets. For QkQm’s conjugate-symmetry question, torch.fft.ifftn(...).real is algebraically equivalent to Hermitian projection PH\mathcal{P}_{\mathrm{H}} on the merged spectrum before inversion (to be stated in Appendix D.2).

3. Disentangling SFM and latent spectral evolution (G5Ym, QkQm, 6CGr). Removing SFM raises 1:20 VRMSE on Active Matter (0.5466→0.8040) and TRL 2D (0.4115→0.6081); under the same latent spectral evolution on 3D Rayleigh–Taylor, Spatial-FiLM lowers 1:20 VRMSE from 0.1828 (Global FiLM) to 0.0901. When we retain the frame-wise spatial encoder and Spatial-FiLM Modulation Head and replace only the Spectral Update Layer with approximately parameter-matched alternatives—Spatial update + FiLM (latent temporal projection + local conv without FFT/IFFT), ConvGRU + FiLM, and Temporal Transformer + FiLM—S⁴-dyn achieves the lowest 1:20 and 21:60 errors on Active Matter and TRL 2D, cutting 1:20 VRMSE by 73.3% / 14.2% over Spatial update + FiLM and 20.0% / 9.5% over the stronger temporal encoder for 1:20 rollout on each dataset (Temporal Transformer + FiLM on Active Matter; ConvGRU + FiLM on TRL 2D; Table R1-2, G5Ym).

4. Equal training-time fairness and multi-run stability (G5Ym). We retrained S⁴-dyn and Flower using the same A100 80GB GPU model, batch size 4, and a 12-hour budget. S⁴-dyn achieves lower VRMSE on all metrics across Active Matter and TRL 2D—cutting 1:20 and 21:60 VRMSE by 48.5% / 11.1% and 36.8% / 33.3%—despite Flower completing nearly twice as many epochs (129 vs. 60 and 218 vs. 121; Table R1-3, G5Ym). On 3D Rayleigh–Taylor inference, S⁴-dyn takes 1.465 s vs. 0.293 s/step for Flower while achieving 84.6% and 49.5% lower 1:20 and 21:60 VRMSE (Paper Tables 1 and 7). Across three independent runs (mean ± std) on TRL 2D, Active Matter, and MHD 64, S⁴-dyn shows small stds and lower mean VRMSE than Flower across all settings—with MHD 64 1:20 close to Flower’s published value (0.7542 ± 0.0102 vs. 0.7580)—while repeated Active Matter ablations (w/o SFM, w/o π−\pi_-, 2-dir raster) increase mean error across all three metrics (Tables R1-4a/b, G5Ym).

5. Full-model comparison with FourierMamba (6CGr). We reproduced the complete FourierMamba architecture for four-frame autoregressive PDE forecasting. Under identical data splits and training settings, S⁴-dyn achieves lower VRMSE across all three metrics on TRL 2D and Active Matter, reducing next-step, 1:20, and 21:60 VRMSE by 44.2%, 43.6%, and 27.5% on TRL 2D (6.55M vs. 10.03M) and 96.7%, 50.6%, and 8.5% on Active Matter (6.59M vs. 10.05M; Table R3-4, 6CGr).

We hope this summary and our detailed responses assist the Area Chair and reviewers in their final evaluation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.