acceptodds
Under review as a conference paper at ICLR 2027

WaveOperator RNN: Learning Structured World Dynamics with Trainable Local Wave Propagation

Abstract

Many sequential data are not only temporally ordered but also organized by an underlying topology in which neighboring variables interact locally and information propagates across structured states. Existing sequence models capture temporal or global interactions effectively but rarely encode externally defined local topology explicitly, while neural operators remain largely field-to-field models. We introduce WaveOperator RNN (WO-RNN), which uses local wave propagation as a recurrent operator to combine structured spatial interaction with temporal persistence. Instead of the staggered-grid discretization commonly used for wave equations, WO-RNN adopts a backward-gradient/forward-divergence scheme that preserves local wave coupling in an aligned tensor form. Spatially varying propagation and damping parameters enable heterogeneous, location-dependent dynamics and task-adaptive state evolution. This inductive bias is well suited to sequential and spatiotemporal data with meaningful internal topology. We demonstrate the same operator on tonotopically organized spiking audio and PDE forecasting, illustrating its applicability to sequence modeling and structured world-state evolution. Here, we use world dynamics in the broad sense of predicting how structured latent or physical states evolve through local interactions over time.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Structured World Dynamics, Wave Dynamics, Recurrent Neural Networks, Topology-Aware Sequence Modeling, Neural Operators

Reviews

Among:
Shape:
1234this paper, Bottom 9%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.50 over 4 reviews, above 9% of the others.
Reviewer_xaRROpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents a well-motivated recurrent architecture that models spatial information exchange and temporal memory through a discretized first-order damped wave system on a grid. Its hidden state consists of a primary scalar field and two auxiliary velocity fields. Each transition combines backward-gradient/forward-divergence propagation with implicit damping, followed by input injection and a pointwise nonlinearity on the primary field. Spatially varying propagation and damping coefficients are trained end-to-end.

The block-operator formulation cleanly separates fixed local topology from trainable dynamical fields, giving the physical formulation a concrete architectural role. The evaluation covers three sequence-classification tasks and three PDEBench forecasting tasks, supported by component ablations and an analysis of the linear transition operator. The strongest reported results occur on SHD, reaction diffusion, and shallow-water prediction, while other tasks show more limited benefits. The authors provide substantial experimental work and generally careful qualification of their findings. The main unresolved issue is whether the observed advantages can be attributed specifically to the proposed local wave dynamics and their alignment with meaningful data topology.

Critical strengths
  1. Interpretable and substantively structured design. WO-RNN separates a fixed sparse gradient divergence topology from trainable propagation and damping fields. The explicit recurrence and block-operator formulation make this architectural constraint transparent and provide a useful framework for studying structured recurrent dynamics. Retaining explicit neighbor interactions is a meaningful design choice, even though its practical advantage over closely related alternatives requires further evidence.

  2. Substantial empirical scope and careful claim calibration. The evaluation spans sequence classification and spatial forecasting, with useful ablations showing that both spatial coupling and modulation affect performance. The paper also reports less favorable results on visual sequences and compressible CFD. It explicitly acknowledges that the experiments do not establish topology alignment, isolate the benefit of coefficient heterogeneity, or prove global nonlinear stability. These qualifications strengthen the credibility of the reported observations.

Critical weaknesses
  1. The evidence does not sufficiently isolate the distinctive contribution of the proposed dynamics. The paper lacks a direct comparison with BioOSS, a closely related wave-inspired state-space model. For PDE forecasting, the reported implementation also includes a multiscale architecture whose contribution is not separated from that of the wave update. Removing propagation establishes that coupling matters within WO-RNN, but does not demonstrate an advantage specific to its wave formulation. This limits the evidence for the paper’s central methodological contribution.

  2. The topology-alignment interpretation is insufficiently supported, and the key SHD comparison remains uncertain. The unrestricted input projection does not explicitly preserve cochlear-channel adjacency, and no evidence establishes that meaningful alignment emerges during training. Both modulation and spatial coupling substantially affect SHD performance, so the gains cannot be attributed specifically to topology alignment. Moreover, the reported advantage over the strongest listed LSTM configuration is only 0.70 percentage points, all experiments use one seed, and the validation procedure for SHD hyperparameter selection is unspecified. These limitations weaken confidence in the result most directly used to motivate the proposed benefit on structured sequences.

Questions for authors
  1. How does the implemented WO-RNN recurrence differ from BioOSS beyond a change of state coordinates, particularly in its parameterization and placement of nonlinearities? Is there comparative evidence that these differences improve performance?

  2. Does the SHD input mapping impose any locality constraint, or is there evidence that meaningful channel-to-grid alignment emerges during training? Please also clarify how the random-topology ablation was constructed and whether coupling scales and the gradient–divergence negative-adjoint relationship were preserved.

  3. How were SHD hyperparameters and checkpoints selected, given that no validation split is reported? Are repeated-run results available to assess the reliability of the improvement over the strongest listed LSTM configuration?

  4. Could you specify the multiscale PDE architecture and how it incorporates the wave update? If no corresponding control is available, a targeted comparison that retains the surrounding architecture while replacing the wave update would materially help establish the operator’s contribution.

  5. Could you reconcile the settings in Appendices C and D, distinguishing initialization values from learned parameters? Under the stated discretization, c = 80, dt = 0.1, and dx = dy = 1 imply a strongly expansive autonomous linear operator. Is additional scaling used, or does training move the coefficients into a different regime? This concerns the interpretation of the implemented dynamics, rather than a claim that the complete nonlinear recurrence is unstable.

6 comments

We thank the reviewer for recognizing WO-RNN's design and scope. We provide five-seed ablations, input-projection analyses, and clarifications.

W1 & Q1: Beyond BioOSS

WO-RNN shares BioOSS's damped two-field wave system but differs in state evolution, beyond coordinate changes.

BioOSS uses approximate diagonal modal recurrence for parallel scans, omitting cross-mode interactions under heterogeneous propagation. WO-RNN retains these via local gradient–divergence updates with spatially varying coefficients.

Within-transition nonlinear feedback yields state-dependent Jacobians beyond BioOSS's linear core, enabling nonlinear evolution at the cost of scan parallelism. Learned input/output mappings support recurrent modeling.

Our contribution is a general recurrent backbone combining heterogeneous local propagation and nonlinear feedback, extending wave-based sequence modeling to broader applications, including spatiotemporal PDE forecasting.

W2 & Q2: Topology, mapping, and coupling

Dense input projection does not enforce tonotopy. Grid topology constrains latent interactions, while input organization is learned.

Across five checkpoints, adjacent-channel cosine similarity rises from −0.0005±0.0013 to 0.6958±0.0225, versus ≈0.063 under permutations. Adjacent centroid distances beat the null (0.70 vs. 1.09). This supports learned channel-order structure, not proven tonotopic or causal alignment.

Five-seed SHD ablations (accuracy %, mean±std):

VariantCleanBN+Dropout
Full + mod.90.98±.5991.49±.42
Full, no mod.89.80±.4591.48±.77
No coupling + mod.79.39±.6980.76±1.20
Neither78.43±1.3876.69±.87
Single field + mod.77.89±1.4479.87±.37

Removing coupling costs 11.59 points versus 1.18 for modulation. With BN+Dropout, losses are 10.73 and 0.01. Coupling dominates within WO-RNN, without proving topology alignment.

The exploratory random-topology ablation uses a 10%-sparse Gaussian matrix of matched density. Degrees vary (mean≈102.4, SD≈9.6), and negative adjointness is lost. It is not degree/operator matched. We claim no grid optimality or isolated topology effect, only superiority over this random variant.

Q3: SHD selection and reliability

Official SHD has 8,156 train / 2,264 test samples without validation. We use 7,340 train / 816 validation (split seed 42) for tuning, scheduling, early stopping, and checkpointing.

The original 135-configuration sweep used test feedback. We repeated all configurations with validation-only selection and five-seed evaluation (42–46).

The original 90.59% vs. 89.89% comparison was single-seed. Under the revised protocol, unmodulated WO-RNN achieves 89.80±.45%, versus LSTM 86.66±.89% and GRU 86.34±.40%. Modulated results appear in W2. Full results are in the revised PDF and our reply to 6SXY.

Q4: PDE architecture and attribution

The PDE forecaster uses three scales (64/32/16). CNNs lift frames into four latent channels per scale. Wave cells, shared across time, evolve primary and auxiliary states via local gradient–divergence coupling with trainable propagation and damping.

Tanh, 1×1 channel mixing, and cross-scale fusion follow each update. A residual CNN decoder combines multiscale primary states and the previous frame for autoregressive prediction. Auxiliary states contribute through coupling.

CNNs handle representation/reconstruction, while WO-RNN provides structured temporal evolution. Multiscale processing is not novel. Our SSM baselines use multilayer architectures, and published PDE baselines use established designs. The key distinction is the recurrent operator, modeling heterogeneous local propagation.

Q5: CFL, discretization, and stability

Linear stability differs from nonlinear training behavior. For c=80, dt=0.1, dx=dy=1, the directional Courant number is 8, beyond linear CFL bounds, but linear instability need not imply nonlinear divergence.

Tanh can suppress local amplification, while learned heterogeneous damping alters dynamics. Some configurations beyond CFL limits remained trainable, without guaranteeing stability.

Paired forward/backward differences avoid spurious high-frequency null modes of central–central stencils and trained more robustly in preliminary tests. We claim neither unconditional stability nor dissipation from truncation-error cancellation.

Original Appendix settings involved exploratory test feedback. Validation-only search now selects c=80, dt=0.005, dx=dy=1, giving an initial Courant number of 0.4, within conventional bounds.

We will update Appendices C/D to distinguish historical settings, selected initializations, and learned coefficients. Linear stability analysis does not guarantee stability of the trained nonlinear system.

Thank you for the additional experiments and clarifications. The rebuttal substantially addresses my concerns about SHD reliability and the role of input modulation. However, two central concerns remain: the benefits of explicit spatial coupling and nonlinear feedback relative to BioOSS have not been established through a controlled comparison, and the PDE experiments do not isolate the wave operator’s contribution within the multiscale architecture. These gaps still limit the evidence for the method’s distinctive contribution.

We thank the reviewer for the constructive follow-up. We conducted two additional controlled comparisons to address the remaining concerns regarding the contributions of explicit wave recurrence, nonlinear feedback, and the wave operator within the multiscale PDE architecture.

1. Controlled comparison with BioOSS

We evaluated three recurrent variants on SHD using identical data splits, surrounding architecture, training protocol, and validation-based checkpoint selection across three paired seeds (42–44).

Recurrent backboneSHD accuracy (%)
BioOSS core84.23±0.58
WO-RNN without nonlinear feedback88.72±0.67
Full WO-RNN91.20±0.71

Replacing the BioOSS modal recurrence with the explicit WO-RNN implementation improves accuracy by 4.49±0.76 percentage points across paired seeds. This comparison retains differences in parameterization and scan direction, so the improvement cannot be attributed exclusively to spatial coupling.

More directly, introducing within-transition nonlinear feedback improves accuracy by 2.47±0.61 percentage points under otherwise matched conditions. This isolates the contribution of nonlinear recurrent state evolution and provides evidence for a key architectural distinction from BioOSS.

The three-seed full WO-RNN results are a subset of our previously reported five-seed evaluation, using the same checkpoints and protocol.

2. Isolating the wave operator in PDE forecasting

We conducted a controlled operator-replacement comparison on RD2D, retaining the same multiscale encoder, recurrent-state dimensions, channel mixing, cross-scale fusion, decoder, autoregressive rollout, and training protocol.

We replaced the structured gradient–divergence wave update with a learnable 3×3 local convolutional recurrence while keeping the surrounding architecture unchanged. Both models have nearly identical parameter counts.

Recurrent operatorRD2D test nRMSE ↓Parameters
WO-RNN wave operator0.050549±0.01459218,123,676
Local convolution0.063620±0.01945918,123,919

Across three paired seeds (42–44), the wave operator achieves 20.5% lower mean test nRMSE, with a mean paired difference of 0.013070±0.021057.

WO-RNN outperforms the convolutional recurrence in two of three seeds. The improvement is seed-dependent, and three seeds are insufficient to establish statistical significance. The convolutional baseline is approximately 4.3% faster at inference.

Nevertheless, this comparison provides direct empirical evidence that the wave-structured recurrent operator can improve forecasting performance independently of the surrounding multiscale architecture.

We will incorporate these results into the revised manuscript, including the controlled configurations, per-seed results, and methodological limitations. We appreciate the reviewer's suggestions, which helped us strengthen the experimental attribution of WO-RNN's contributions.

Reviewer_J6A3OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes WaveOperator RNN (WO-RNN), a recurrent model whose hidden state lives on a 2D grid and is updated by a discretized damped wave equation with trainable and spatially varying propagation and damping fields. The model is evaluated on sequence classification and PDEBench forecasting. The authors report competitive results on SHD and 2 of the 3 PDE tasks, and claim that the model works best when the data has meaningful local topology.

Critical strengths

The idea of wave dynamics is quite natural and interpretable. Evaluating on both sequence classification and PDE forecasting is a reasonable way to probe the scope of the inductive bias, and the authors are transparent about where it does not help and which claims their experiments do not establish.

Critical weaknesses
  • The performance gain in the SHD experiment seems to come from input modulation and not really from the wave operator. For instance, the "no spatial coupling" case still achieves 81.89%, which is about the same as full wave without modulation. This appears to contradict the statement that the tonotopic and local inductive bias drives the performance.

  • The PDE model is underspecified, and therefore it is difficult to judge whether the forecasting gain is attributed to the wave operator or the surrounding architecture.

  • The key difference from prior fixed-wave models, in particular FDTDESN, is that the propagation and damping fields are trainable and spatially varying. However, this main contribution is not isolated experimentally and no experiment tests this directly.

  • All results come from a single random seed, and the authors state that no claims of statistical significance are made. Since many differences between models are small, the experimental results, especially the rankings, may not be reliable.

Questions for authors
  • What is the relation to the paper [1] below?

  • The SHD case has no validation split, how were hyperparameters selected?

  • Also concerning this SHD experiment, how does 87.94% (best grid-search run reported in Appendix C.2) become 90.59% in the main paper?

[1]: "Wave physics as an analog recurrent neural network", Hughes et al. 2019, Science Advances

3 comments

We thank the reviewer for recognizing WO-RNN's interpretability and cross-task evaluation. We provide five-seed results, SHD ablations, validation-based selection, and architectural and prior-work clarifications.

W1: SHD coupling vs. modulation

Five-seed (42–46) ablations use identical splits, settings, budgets, and validation-selected checkpoints (accuracy %, mean±std):

VariantCleanBN+Dropout
Full + mod.90.98±.5991.49±.42
Full, no mod.89.80±.4591.48±.77
No coupling + mod.79.39±.6980.76±1.20
Neither78.43±1.3876.69±.87
Single field + mod.77.89±1.4479.87±.37

Removing coupling costs 11.59 SHD points versus 1.18 for modulation. With BN+Dropout, losses are 10.73 and 0.01 points. Thus, coupling dominates within WO-RNN, without proving superiority over other recurrent operators.

W2: PDE architecture and attribution

The PDE forecaster uses a three-scale (64/32/16) encoder–recurrent evolution–decoder. Per-scale CNNs lift frames into four latent channels. Wave cells, shared across time, evolve primary and auxiliary states via local gradient–divergence coupling with trainable spatial propagation and damping. Tanh, channel mixing, and cross-scale fusion follow each update. A convolutional residual decoder combines multiscale primary states and the previous frame for autoregressive prediction.

CNNs handle spatial representation and reconstruction, while WO-RNN provides structured temporal evolution. Multiscale processing is not our novelty. Our SSM baselines use multilayer architectures, while published PDE baselines use established designs. The key distinction is the recurrent operator, modeling heterogeneous local propagation rather than generic state transitions.

RD2D/SW2D results support structured wave recurrence as an effective PDE forecasting backbone alongside SSMs and Transformers, without attributing all gains solely to the operator or claiming universal PDE SOTA.

W3: Trainable heterogeneity

Unlike fixed-wave FDTDESN, WO-RNN learns spatial propagation and local damping within fixed gradient–divergence topology, enabling task-adaptive dynamics. PDE experiments do not isolate trainability or heterogeneity, distinct from measured coupling effects.

W4: Reliability

W1's five-seed results show unmodulated WO-RNN outperforms unmodulated LSTM (86.66±.89%) and GRU (86.34±.40%) on SHD. Modulated WO-RNN and BN+Dropout are reported separately. Full five-seed results appear in the revised PDF and our response to reviewer 6SXY.

Stronger baselines lead on sMNIST and noisy CIFAR-10, indicating task-dependent advantages. Single-seed PDE results do not establish significance.

Q1: Hughes et al. (2019)

Hughes et al. demonstrated trainable wave recurrence via differentiable FDTD/backpropagation, including nonlinear media. We claim no priority for these ingredients.

WO-RNN combines coupled fields, learned input/output mappings, heterogeneous damping, and within-transition nonlinear feedback in a reusable recurrent backbone. Its state-dependent Jacobian can attenuate local amplification without guaranteeing global stability.

Unlike BioOSS's approximately diagonal modal recurrence for parallel scans, WO-RNN retains explicit local gradient–divergence coupling and nonlinear feedback at every step, trading scan efficiency for nonlinear recurrence without claiming superiority. We will cite Hughes and clarify these distinctions.

Q2–Q3: SHD validation and reporting discrepancy

Official SHD contains 8,156 train / 2,264 test examples without a validation split. We partition training into 7,340 train / 816 validation (split seed 42). Validation determines hyperparameters, scheduling, early stopping, and checkpoints.

Original and revised evaluations differ. The original 135-configuration exploratory sweep involved test feedback and was not validation-only. We repeated all 135 configurations using validation-only selection, followed by five-seed evaluation.

Appendix C.2's 87.94% came from exploratory grid search, while the main table's 90.59% came from a separate final-model run. Their provenance was unclear, and records do not establish identical selection protocols.

We will clarify both results and use W1's consistent five-seed, validation-selected results as primary evidence, without retroactively assigning validation-only selection to the original experiments.

The new evidence supports coupled-field recurrence as WO-RNN's main SHD contributor. We will clarify novelty claims and limitations.

I thank the authors for the detailed answer and the additional experiments, which address partially my concerns. However, a few concerns still remain: PDE experiments are still single seed and the evidence for the distinctive contribution of the approach over others remains weak.

We thank the reviewer for the follow-up and for acknowledging that our additional experiments partially address the original concerns. We have since completed further controlled comparisons that provide new evidence regarding PDE reliability and the distinctive contribution of WO-RNN.

1. PDE forecasting: multi-seed operator comparison

We conducted a new five-seed (42–46) experiment on RD2D, comparing WO-RNN against a parameter-matched local convolutional recurrent operator.

Both models retain the same multiscale encoder, recurrent-state dimensions, channel mixing, cross-scale fusion, decoder, autoregressive rollout, training protocol, and validation-based checkpoint selection.

Recurrent operatorRD2D test nRMSE ↓Parameters
WO-RNN wave operator0.050549±0.01459218,123,676
Local convolution0.063620±0.01945918,123,919

WO-RNN achieves 20.5% lower mean test nRMSE across five paired seeds, with a mean paired difference of 0.013070±0.021057.

This controlled comparison provides direct evidence for the contribution of the wave-structured recurrent operator within the shared multiscale forecasting architecture.

We are also conducting a comprehensive five-seed evaluation across all PDEBench tasks and baselines to further assess forecasting reliability. However, due to the substantial computational cost of PDE training and limited available GPU resources, we do not expect all runs to finish before the discussion period ends. We therefore report only completed results and leave the five seeds results for the final version of manuscript.

2. Distinctive contribution relative to existing wave-based models

We have already included FDTDESN as a fixed-wave baseline in our five-seed classification evaluation. The unmodulated results are:

ModelsMNIST (%)nsCIFAR-10 (%)SHD (%)
FDTDESN87.15±0.4143.82±1.205.97±0.14
WO-RNN97.91±0.0949.32±0.6689.80±0.45

These results demonstrate that WO-RNN outperforms the fixed-wave FDTDESN baseline across all three classification tasks. While this comparison does not independently isolate coefficient trainability or spatial heterogeneity, it provides empirical evidence for the effectiveness of the proposed trainable recurrent architecture relative to an existing fixed-wave reference.

To further distinguish WO-RNN from closely related trainable wave-based models, we conducted a controlled three-seed comparison with BioOSS on SHD:

Recurrent backboneSHD accuracy (%)
BioOSS core84.23±0.58
WO-RNN without nonlinear feedback88.72±0.67
Full WO-RNN91.20±0.71

The explicit WO-RNN implementation improves accuracy by 4.49±0.76 percentage points relative to the BioOSS core, although differences in parameterization and scan direction remain.

Introducing within-transition nonlinear feedback yields a further 2.47±0.61 percentage-point improvement under otherwise matched conditions, directly isolating this architectural contribution.

Together with our previous five-seed coupling ablations, these comparisons provide complementary evidence for the contributions of structured local recurrence and nonlinear feedback, beyond input modulation and the surrounding forecasting architecture.

We will incorporate these additional results into the revised manuscript and clarify the distinctions between fixed-wave recurrence, trainable wave dynamics, and the specific contributions established by our controlled experiments.

We appreciate the reviewer's constructive feedback, which has helped us strengthen the experimental evidence and clarify the scope of our methodological claims.

Reviewer_6SXYOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposed a network called WaveOperator RNN (WO-RNN), which aims to bridge sequence modeling and structured predictive dynamics using a state evolution dynamic that is constrained by topology. The authors used a recurrent network based on trainable local wave dynamics, which defines its hidden state on a 2D grid and updates its states through wave transition. The network is validated on both classification and forecasting tasks.

Critical strengths

None

Critical weaknesses

Overall, experimental results are too weak for me to evaluate the quality of the proposed method.

  • The experimental evaluation is too limited in scale to support strong conclusions. The authors should evaluate the method on larger datasets, including some of the many available large-scale classification benchmarks.
  • The PDEBench results require clarification. In particular, the statement “The separately published FNO value on SW2D is 0.0044 and is included only as context” leaves unclear why FNO is treated as contextual information rather than a competing baseline. If the results are not directly comparable, the authors should explain why. Moreover, the proposed method outperforms the baselines on only one of the three datasets and underperforms ResNet on another. These results do not provide convincing evidence of the proposed method’s effectiveness.
  • The overall methodological novelty appears modest.
Questions for authors

NA

2 comments

We thank the reviewer. We provide five-seed results, ablations, and clarification of PDE evaluation and methodological contributions.

W1: Experimental scale and reliability

WO-RNN targets structured sequence modeling through local propagation. We evaluate serialized images, spiking audio, and three PDE forecasting tasks. Larger benchmarks would further establish scalability, while our experiments cover distinct sequence structures.

We repeated classification experiments over 5 seeds (42–46) with validation-based checkpoint selection (accuracy %, mean±std, without → with modulation).

ModelsMNISTNoisy CIFAR-10SHD
ESN78.28±.89→64.15±.7231.50±.27→30.03±.415.19±.12→5.30±.23
FDTDESN87.15±.41→67.09±.1443.82±1.20→43.57±.675.97±.14→6.46±.29
LSTM98.17±.21→96.90±.1558.63±.54→58.15±.6986.66±.89→86.74±.80
GRU98.60±.15→96.91±.2561.02±.71→59.81±.4486.34±.40→86.51±.71
LRU98.68±.18→98.21±.1063.14±.68→65.41±.5681.07±.89→79.82±.61
S4D98.68±.19→97.58±.0968.43±.23→64.41±.4480.10±1.36→80.59±1.07
Mamba99.24±.11→98.61±.0668.16±.60→64.37±.4468.82±3.91→75.10±6.04
coRNN98.22±.16→96.73±.1653.17±3.62→55.59±4.4966.77±1.36→66.85±1.71
NWM95.13±1.46→94.64±.6056.35±.81→57.30±.4171.40±3.89→71.82±2.85
wRNN96.56±.29→95.04±.3730.92±1.78→35.84±7.7767.23±.80→68.66±.76
WO-RNN97.91±.09→97.44±.2349.32±.66→51.11±.6089.80±.45→90.98±.59

WO-RNN reaches 91.49±.42% on SHD with BN+Dropout. Full configurations are in the revised PDF. Its advantage is most evident on SHD; S4D/Mamba remain stronger on serialized visual tasks.

For SHD, we repeated 135 hyperparameter configurations, selecting exclusively by validation accuracy on a fixed 7,340/816 train/validation split. The original optimum was recovered: (c,Δt,kp,ko,G)=(80,.005,.05,.05,32)(c,\Delta t,k_p,k_o,G)=(80,.005,.05,.05,32). The original exploratory sweep considered test results, but the repeated sweep excluded test accuracy from selection. Five-seed reruns reproduced previous accuracies.

W2: PDEBench evaluation and FNO comparison

All PDE experiments follow the standardized PDEBench dataset splits, autoregressive rollout, normalization, and nRMSE evaluation protocol, aligned with published benchmark results. The published FNO results are directly comparable under this evaluation setting. Superscripts distinguish published (†) and locally evaluated (*) results by provenance.

ModelRD2DSW2DCFD
Best other local*.6647.0128.3450
WO-RNN*.0522.0045.3990
FNO†.1200.0044.3600

WO-RNN achieves the lowest nRMSE among locally evaluated models on RD2D and SW2D, outperforming published FNO on RD2D (.0522 vs. .1200) and nearly matching it on SW2D (.0045 vs. .0044). ResNet and FNO perform better on CFD.

Due to the substantial computational cost of PDE forecasting, no extensive hyperparameter search was conducted. These experiments test the applicability of the same recurrent operator across physical systems. We claim neither universal PDE-solver SOTA nor statistical superiority from single-seed results.

W3: Methodological novelty and evidence

WO-RNN develops explicit local wave propagation into a trainable nonlinear recurrent backbone. Hughes et al. (2019) studied analog inverse design using trainable media and fixed source/probe operators. WO-RNN combines learned input/output projections, coupled pressure–velocity states, heterogeneous damping, and within-step tanh feedback.

BioOSS uses an approximate diagonal modal recurrence for parallel scans, neglecting cross-mode interactions under heterogeneous propagation. WO-RNN directly computes local gradient–divergence updates and introduces nonlinear recurrent feedback with state-dependent Jacobians.

Five-seed SHD ablations isolate the contribution of coupling:

VariantAccuracy (%)
Full + modulation90.98±.59
Full, no modulation89.80±.45
No coupling + modulation79.39±.69
No coupling, no modulation78.43±1.38
Single field + modulation77.89±1.44

Removing coupling reduces accuracy by 11.59 points, compared with 1.18 points for removing modulation. This demonstrates the importance of coupled-field propagation within WO-RNN.

Across five checkpoints, adjacent-channel projection cosine similarity increases from approximately zero at initialization to 0.696±0.023 after training, versus 0.063 under random channel permutations (empirical p≈0.001p≈0.001 per seed). This supports emergent channel-order-related representations, although explicit tonotopic localization remains unproven.

Our contribution combines trainable heterogeneous propagation, structured local recurrence, and nonlinear feedback in a reusable backbone. The results reveal task-dependent advantages and motivate further large-scale evaluation.

Thank you, I won't change my score if there is no experiment and performance gain on larger datasets

Reviewer_o7v8OpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes WaveOperator RNN (WO-RNN), a recurrent model whose hidden state lives on a 2D grid and evolves according to a discretized damped first-order wave system with a primary field p and an auxiliary velocity field o. Each step applies an explicit backward-gradient/forward-divergence propagation update, followed by implicit damping. The projected input is then added to p and passed through a tanh nonlinearity. The propagation speed and the damping coefficients are trainable fields that can vary across the grid, and an optional learnable sinusoidal input modulation is added. The authors argue that this gives an inductive bias suited to data with meaningful local topology. They evaluate it on sequence classification (row-wise sMNIST, noisy CIFAR-10 and SHD) and on PDEBench forecasting (reaction–diffusion, shallow water and 2D compressible CFD). They report competitive results on the visual tasks, their strongest relative result on SHD (90.59%), and the lowest error among locally trained models on RD2D and SW2D, with weaker results on CFD. Ablations removing spatial coupling, the velocity field plus modulation, or the grid topology are used to argue that local propagation and topology contribute to performance.

Critical strengths

The paper is clearly written and easy to follow. The model is described step by step through the continuous equations, the discretization and Algorithm 1, so the core recurrence is easy to understand. The underlying idea, using local wave propagation on a grid as a recurrent operator, is intuitive and well motivated, although it builds closely on prior work (notably BioOSS) rather than being new to this paper.

Critical weaknesses

1: The experimental comparison is not reliable enough to support the paper’s performance claims.

All results in Tables 1–3 come from a single training seed with no variance estimates (Appendix B). The authors acknowledge this, but the main text still draws ranking conclusions from it. For example, WO-RNN is said to outperform LSTM, GRU, S4D and LRU on SHD, yet the gap to LSTM is 0.7 points (90.59 vs 89.89). That is within the test-set sampling error alone, about 0.6 pp for 2,264 test samples, before seed-to-seed variance is even considered. The comparison is also uneven in two ways:

WO-RNN received a 135-run grid search on SHD, apparently selected on the test set since no validation split is used (Table B.1). Each baseline is reported with a single configuration and no description of how it was chosen. For coRNN, NWM and wRNN on sMNIST and nsCIFAR-10, the appendix gives no hyperparameters, parameter counts or resource statistics (Tables D.2, B.2). Their unmodulated values in Table 1 are identical to published results from Keller et al. (2024): coRNN 99.30 / 59.00 and wRNN 97.60 / 55.00. Those results were obtained on 784-step pixel-wise sMNIST and 1000-step noise-padded CIFAR-10, not on the 28-step row-wise sMNIST and label-noise nsCIFAR-10 used here. The modulated values for the same models must come from local runs, so the reported modulation deltas may compare results from different benchmarks.

The paper’s contribution is mainly empirical. Without repeated runs, comparable tuning and clearly sourced baselines, none of the claimed advantages can be judged.

2: The ablations do not support the central claim that WO-RNN’s benefit comes from aligning local latent interactions with feature locality.

The authors attribute the SHD result to the model exploiting tonotopic locality (line 359). The ablations point the other way:

Replacing the grid with a random topology causes by far the largest drops on sMNIST (−25.25) and nsCIFAR-10 (−29.91). The authors themselves describe these tasks as lacking meaningful feature locality. The drop on SHD (−9.54) is the smallest. On SHD, removing inter-cell coupling (c = 0: 81.89, Table 2), randomizing the topology (81.05, Table 2) and removing input modulation (81.32, Table 1) all give essentially the same accuracy, about 9 points below the full model. So the SHD gain appears only when coupling and modulation are combined, and the ablations cannot separate a locality effect from a modulation effect. Input is projected through a dense learned matrix W_in, so the architecture never enforces that adjacent cochlear channels map to adjacent grid cells. The claimed benefit of spatially heterogeneous coefficients rests on “exploratory runs” that are not reported anywhere.

The paper’s framing as a topology-aware inductive bias depends on this claim. Without ablations that isolate locality, over multiple seeds, the paper doesn’t show why or when the method works.

3: The novelty relative to prior work is overstated.

BioOSS (Yuan et al., NeurIPS 2025) already uses the identical two-field system (Eq. 2) on a 2D grid, with trainable propagation speed and damping and spatially heterogeneous dynamics. Trainable, spatially varying wave speed in an FDTD-discretized RNN was introduced by Hughes et al. (Science Advances, 2019), which is not cited. The statement that existing approaches do not combine gradient–divergence structure with trainable propagation and attenuation (lines 61–63) is therefore not accurate. The main differences from BioOSS appear to be the pointwise nonlinearity, the explicit stencil recurrence instead of a diagonalized scan, the input modulation and the PDE application.This delta should be explicitly stated and the BioOSS should be included as a baseline in Table 1.

Questions for authors
  1. Difference from BioOSS. BioOSS (Yuan et al., NeurIPS 2025) uses the same two-field system as Eq. 2 on a 2D grid, with trainable propagation speed and damping. Please state exactly what WO-RNN changes relative to BioOSS (e.g., the pointwise nonlinearity, the explicit stencil recurrence instead of a diagonalized scan, the input modulation), and which of these changes the claimed contributions rest on. Moreover the authors mentioned “FDTDESN (Yuan et al., 2025), the closest fixed-wave reference”, while BioOSS seems to be the closest model and should be added to the comparison as a baseline..

  2. Multiple seeds. All results come from a single seed (Appendix B). Many of the claimed differences, such as SHD 90.59 vs LSTM 89.89, are smaller than the test-set sampling error alone. Please report mean ± std over at least 3–5 seeds for all models in Tables 1–3, including the ablations. Without this, the ranking claims cannot be assessed.

  3. Hyperparameter tuning for all models. Appendix C describes the search for WO-RNN, but for the baselines only a single final configuration is given (Table D.2), and for coRNN, NWM and wRNN on sMNIST and nsCIFAR-10 no configuration is given at all. Please report, for every model, the search space, the number of runs, and the split used for selection. Since SHD has no validation split (Table B.1), please also clarify whether hyperparameters were selected on the test set.

  4. Random-topology ablation. The paper argues that the local bias helps most when the input has meaningful feature locality, as in SHD. However, replacing the grid with a random topology causes much larger drops on sMNIST (−25.25) and nsCIFAR-10 (−29.91), which the paper describes as lacking such locality, than on SHD (−9.54). How do the authors reconcile this with their hypothesis? Please also describe how the random topology is constructed (degree, symmetry, whether the gradient/divergence adjoint structure is preserved), and whether the resulting operator remains stable.

  5. How the input mapping relates to locality. The input is projected onto the grid through a dense learned matrix W_in, so nothing in the architecture forces adjacent cochlear channels to map to adjacent grid cells. How does the model make use of the tonotopic ordering of SHD? Could the authors show whether the learned W_in places neighbouring channels near each other on the grid, or compare against a fixed tonotopic input mapping?

3 comments

We thank the reviewer for the feedback. We conducted five-seed experiments, validation-based selection, controlled ablations, and input-projection analyses.

W1 & Q2–Q3: Reliability, baselines, and tuning

We reran classification experiments with 5 seeds (42–46) and validation-based checkpointing (accuracy %, mean±std).

ModelsMNISTNoisy CIFAR-10SHD
LSTM98.17±.2158.63±.5486.66±.89
GRU98.60±.1561.02±.7186.34±.40
S4D98.68±.1968.43±.2380.10±1.36
Mamba99.24±.1168.16±.6068.82±3.91
WO-RNN97.91±.0949.32±.6689.80±.45
+ modulation97.44±.2351.11±.6090.98±.59

ESN, FDTDESN, LRU, coRNN, NWM, and wRNN were also evaluated. Full experimental results and configurations are provided in the revised PDF. WO-RNN excels on SHD but trails stronger baselines on serialized visual tasks; we make no universal SOTA claim.

SHD selection: The official training set (8,156) was split into train/validation (7,340/816; seed 42), separate from the official test set (2,264). Scheduling, early stopping, and checkpoint selection used validation accuracy. We repeated all 135 hyperparameter configurations, selecting the best exclusively by validation accuracy, independently recovering (c,Δt,kp,ko,G)=(80,.005,.05,.05,32)(c,\Delta t,k_p,k_o,G)=(80,.005,.05,.05,32). We acknowledge that the original exploratory sweep used test-set feedback. In the repeated sweep, test accuracies were recorded but excluded from hyperparameter selection. Subsequent five-seed runs reproduced the previous per-seed results exactly.

PDEBench: No extensive grid search was performed. The three tasks assess applicability to spatial forecasting, without claiming PDE-solver SOTA. Single-seed results cannot establish statistical significance; we will limit conclusions accordingly.

W2 & Q4: Coupling, modulation, and random topology

Five-seed ablations used identical splits, hyperparameters, training budgets, and validation-based selection.

SHD variantCleanBN+Dropout
Full + mod.90.98±.5991.49±.42
Full, no mod.89.80±.4591.48±.77
No coupling + mod.79.39±.6980.76±1.20
Neither78.43±1.3876.69±.87
Single field + mod.77.89±1.4479.87±.37
Single field, no mod.76.10±1.0374.01±.40
Complete ablation results are also available in the revised PDF.

Removing coupling costs 11.59 points on clean SHD, versus 1.18 for modulation; with BN+Dropout, modulation changes accuracy by 0.01 points. An audit of 80 runs confirmed protocol consistency.

The random-topology variant replaces grid operators with a 10%-sparse Gaussian matrix shared across both spatial directions and gradient/divergence. Degree varies (mean≈102.4, SD≈9.6), adjointness is lost, and grid-based stability arguments no longer apply. This confounds geometry and operator structure; we will interpret it as disruption of structured propagation, without claiming isolated topology effects or guaranteed stability.

W2 & Q5: Learned input organization

Across five trained checkpoints and matched initializations, adjacent-channel cosine similarity increased from −0.0005±0.0013 to 0.6958±0.0225, versus ≈0.063 under random channel permutations (1,000 permutations; empirical p≈0.001p≈0.001 per seed). Similarity decreases with channel separation (ρ=−0.808±0.007\rho=-0.808±0.007); adjacent centroid distance is below the permutation null (0.70 vs. 1.09).

This establishes training-emergent channel-order-related structure. Projections remain diffuse, so explicit tonotopy and causal effects on accuracy are unproven.

W3 & Q1: Novelty versus Hughes et al. (2019) and BioOSS

Hughes et al. (2019) demonstrated trainable wave recurrence, including nonlinear responses, through analog inverse design: a second-order scalar wave equation, trainable medium, fixed source/probe operators, and intensity-dependent wave speed. WO-RNN uses jointly trained input/output projections, coupled pressure–velocity states, trainable damping, and within-step tanh saturation. Nonlinear feedback yields state-dependent Jacobians and can attenuate local amplification, without guaranteeing global stability. We do not claim priority for nonlinear wave recurrence.

BioOSS uses an approximate diagonal modal recurrence for parallel associative scans, neglecting cross-mode interactions under heterogeneous propagation. WO-RNN retains explicit local gradient–divergence updates with heterogeneous coefficients and within-step nonlinear feedback, extending beyond BioOSS's linear wave core.

WO-RNN studies the expressivity–parallelism trade-off through nonlinear local propagation and end-to-end task adaptation. Five-seed ablations support the role of coupling. Direct comparisons with Hughes and BioOSS would quantify performance and computational trade-offs; we do not claim established superiority.

Reviewer_o7v8OpenReview ↗

Response to rebuttal

Thank you for the additional experiments. However, most of my concerns remain unresolved, and I therefore maintain my rating.

  • Seeds: The additional seeds were applied only to the classification benchmarks, not to PDEBench. The PDE results remain single-seed, so the forecasting claims cannot be assessed statistically.
  • Tuning and validation/test separation: The validation-based hyperparameter search was performed only on SHD. For sMNIST and nsCIFAR-10, no validation-based search is reported for WO-RNN, and none of the baselines received any search on any task.
  • Novelty: BioOSS and Hughes et al. (2019) were not added as baselines, as requested. Input locality: The growing cosine similarity between neighbouring input channels during training does not show that locality causes the accuracy gain. As the authors acknowledge, a causal link is not established. A comparison with a fixed tonotopic input mapping would address this.
  • Ablations: Without modulation, removing coupling reduces accuracy by a similar amount on SHD (89.80 → 78.43, −11.4%) and nsCIFAR-10 (49.32 → 37.79, −11.5%). Removing the velocity field (single field) likewise gives comparable drops on SHD (−13.7 %) and nsCIFAR-10 (−14.3 %). This does not support a locality effect specific to SHD.

We thank the reviewer for the follow-up. We would like to clarify how our additional experiments address the original concerns and distinguish the scope of our methodological claims.

Experimental reliability and tuning. We acknowledge that the original request for multiple seeds covered Tables 1–3. However, the detailed concerns focused primarily on classification, particularly the small SHD performance gap, hyperparameter selection, and baseline comparability. We therefore prioritized five-seed classification experiments, controlled ablations, and a repeated 135-configuration SHD search using validation-only selection.

PDE forecasting requires substantially longer training runs, and the remaining discussion period does not permit comprehensive multi-seed evaluations across all forecasting tasks. We acknowledge this limitation and will avoid statistical claims based on single-seed PDE results.

Regarding tuning, the original review requested documentation of search spaces, run counts, and selection splits. We have clarified the available configurations and repeated the SHD search without test-based selection. We agree that exhaustive, equally budgeted tuning across every model and task would provide stronger comparability, but this is infeasible within the rebuttal period.

Novelty and prior work. The original review requested explicit differentiation from BioOSS and Hughes et al. (2019), and specifically suggested adding BioOSS to Table 1. Our previous response explained the architectural differences. We have now also completed a controlled three-seed comparison:

ModelSHD accuracy (%)
BioOSS core84.23±0.58
WO-RNN without nonlinear feedback88.72±0.67
Full WO-RNN91.20±0.71

The paired gains are 4.49±0.76 and 2.47±0.61 percentage points. The latter isolates nonlinear recurrent feedback, while the former retains documented parameterization and scan-direction differences.

Hughes et al. (2019) establishes important prior work on trainable physical wave recurrence, which we now acknowledge and cite. Its analog physical implementation does not provide a directly comparable digital baseline under our classification and PDEBench protocols.

Local recurrent coupling and input alignment. Our central contribution is a structured recurrent operator with explicit local gradient–divergence coupling, heterogeneous propagation, and nonlinear feedback. The coupling ablations test the contribution of this recurrent mechanism.

The reviewer notes similar absolute accuracy reductions on SHD and noisy CIFAR-10. However, these tasks have substantially different baseline accuracies. Removing coupling increases SHD error from 10.20% to 21.57% (a 111.5% relative increase), compared with 50.68% to 62.21% (22.7%) on noisy CIFAR-10. Thus, similar absolute percentage-point reductions alone cannot establish equivalent task-dependent effects.

More importantly, coupling benefits on both datasets are consistent with our broader claim that local recurrent dynamics provide a useful inductive bias across sequential tasks. We do not require the benefit to be exclusive to tonotopically organized inputs.

The original review suggested examining learned input organization or comparing against a fixed tonotopic mapping. We performed the former: adjacent-channel cosine similarity increased from approximately zero at initialization to 0.6958±0.0225 after training, with permutation-based controls. This provides evidence of learned channel-order-related structure.

We agree that this observational evidence does not establish the causal contribution of tonotopic alignment. We will distinguish that open question from the experimentally demonstrated contribution of recurrent coupling and revise any statements implying a proven causal advantage from input alignment.

We appreciate the reviewer's assessment and will incorporate these clarifications and new experimental results into the revised manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.