acceptodds
Under review as a conference paper at ICLR 2027

DS4D-FFNO: Hidden-State Retrieval for PDEs with Fixed-Delay Boundary Feedback

Abstract

Partial differential equations (PDEs) with fixed-delay boundary feedback arise in fields such as wave propagation, structural dynamics, electromagnetics, and flow control, where boundary feedback depends on past states. In recent years, neural operators have been widely used to solve PDEs, but most lack a dedicated mechanism for modeling fixed-delay boundary feedback. We propose DS4D-FFNO, combining factorized Fourier neural operator (FFNO) spatial mixing with DS4D, an S4D extension that models delayed dynamics by retrieving interpolated past hidden states via a delayed-feedback branch. DS4D-FFNO consists of an encoder followed by two FFNO layers, one DS4D layer, two further FFNO layers, and an output head. Compared with seven other models, DS4D-FFNO achieves the lowest error in solving each of six such PDEs, which describe wave propagation, structural vibrations, thermoelastic coupling, boundary dynamics, and reaction–diffusion processes. To assess its applicability beyond delayed systems, we compare DS4D-FFNO with S4-FFNO on six classical PDEs without physical delay and find comparable predictive accuracy. To test whether its learned delay indices contribute to the gains on delayed PDEs, we permute or randomly replace them, and both interventions increase error, indicating that learned delayed access is essential for maintaining the trained model's predictive accuracy.

Primary area:
learning on time series and dynamical systems
Keywords:
Partial differential equations, Delayed Boundary Feedback, Neural Operator, State Space Model

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_PMPeOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces DS4D-FFNO, which combines FFNO’s spatial modeling with DS4D’s retrieval of past hidden states at learnable delays to predict PDE dynamics with fixed-delay boundary feedback. The authors evaluate its autoregressive forecasting performance on six PDE benchmarks with delayed boundary feedback and six without physical delays. They also examine the model’s reliance on the learned delays by permuting them across channels or replacing them with random values.

Critical strengths
  • The architecture is well aligned with the task requirements. FFNO models spatial interactions, while DS4D stores and retrieves historical hidden states to capture delayed dependencies. Interpolation enables retrieval at non-integer delays and allows the delay parameters to be learned through end-to-end training.

  • The experiments cover multiple physical systems and evaluation criteria. The authors compare the method with seven baselines on six PDE tasks with delayed boundary feedback and conduct supplementary evaluations on six tasks without physical delays. Results are reported over three training seeds and include full-field and boundary-region errors, alongside predicted-field visualizations and error curves over the forecast horizon.

  • The study includes targeted comparisons and diagnostic experiments. DS4D-FFNO achieves lower mean full-field errors than S4-FFNO on all six delayed tasks under a controlled setup with the same spatial backbone, initialization of shared components, and training batch order. Delay permutation and random replacement experiments assess the trained model’s reliance on its learned delays, while additional experiments examine sensitivity to spatial resolution and model parameters.

Critical weaknesses
  1. Limited differentiation from prior work. MemNO [1] already places a temporal memory layer between two pairs of FFNO layers, while hidden-state caching, learnable delays, and interpolated retrieval originate from DSSM [2]. The contribution beyond replacing the temporal module and adapting it to PDEs with delayed boundary feedback needs clarification. Comparisons with known physical delays or fixed delay banks would help establish the value of learning delay locations.

  2. Baseline comparisons partly confound history access with retrieval design. The proposed model uses 192 historical frames for 1D tasks and 96 for 2D tasks, whereas FNO, FFNO, and Transolver use only the current frame. Although S4-FFNO provides a more direct comparison, simpler baselines using the same history window, such as frame concatenation or temporal convolutions, are needed to isolate the benefits of DS4D’s retrieval mechanism.

[1] R. Buitrago Ruiz, T. Marwah, A. Gu, and A. Risteski, “On the Benefits of Memory for Modeling Time-Dependent PDEs,” in International Conference on Learning Representations (ICLR), 2025.

[2] Y. Qian, S. Chen, J. Liu, S. Cai, and C. Xu, “DSSMs: State Space Models with Explicit Memory via Delay Differential Equations,” arXiv preprint arXiv:2607.10244, 2026.

Questions for authors

see Weakness.

2 comments
AuthorsOpenReview ↗

Response to Reviewer PMPe

We thank the reviewer for asking us to clarify the contribution and to test simpler temporal models with the same history window.

The contribution beyond replacing the temporal module and adapting it to PDEs with delayed boundary feedback needs clarification. Comparisons with known physical delays or fixed delay banks would help establish the value of learning delay locations.

Thank you for this comment. We agree that our contribution needs clearer explanation. The suggested comparisons with the known physical delay and a fixed delay bank are reported in Tables 1 and 2, respectively. DS4D-FFNO has lower mean error than the fixed physical-delay variant on all six tasks, and it outperforms the fixed uniform-bank variant on five of the six.

Contribution. Building on MemNO and DSSM, we adapt and evaluate learned hidden-state retrieval for PDEs with delayed boundary feedback. Our contributions are as follows:

  1. Modeling delayed dependencies explicitly. DS4D-FFNO retrieves past hidden states within its temporal update, and each channel learns its retrieval locations jointly with the memory dynamics. This gives the model an explicit mechanism for capturing delayed dependencies.

  2. Forecasting without a supplied physical delay. In practice, the physical delay is often unknown or hard to estimate accurately. DS4D-FFNO is designed for this setting, as it forecasts PDEs with delayed boundary feedback directly from observed field histories without taking the physical delay τ\tau as an input.

  3. An architecture aligned with PDEs with delayed boundary feedback. Such PDEs combine spatial interactions with feedback from past boundary states. DS4D-FFNO reflects this structure by coupling FFNO spatial mixing with delayed retrieval from grid-wise hidden-state histories. The retrieved information enters the temporal update and is then mixed across the field, so the model can represent how delayed feedback affects later field evolution.

Together, these contributions provide a framework for forecasting delayed PDEs from observed field histories and characterize how retrieval design affects prediction.

Although S4-FFNO provides a more direct comparison, simpler baselines using the same history window, such as frame concatenation or temporal convolutions, are needed to isolate the benefits of DS4D’s retrieval mechanism.

Thank you for requesting comparisons with simpler baselines using the same history window. We evaluate three temporal-convolution variants with comparable parameter counts and report the results in Table 12 below. We believe these are meaningful controls for assessing the value of learned retrieval and will include them in the paper.

Table 12. Temporal baselines with matched parameter counts. TemporalConv-FNO, TemporalConv-FFNO, and TemporalConv-Transolver are evaluated under the same protocol as DS4D-FFNO, with parameter counts differing by at most 0.15%, 0.05%, and 1.12%, respectively. All runs use 120 epochs for 1D tasks and 60 for 2D tasks.

TaskDS4D-FFNOTemporalConv-FNOTemporalConv-FFNOTemporalConv-Transolver
KdV–KdV system0.018558 ± 0.0253980.405903 ± 0.1018660.299701 ± 0.0869300.081791 ± 0.003465
Microbeam system0.017189 ± 0.0192770.203830 ± 0.2117460.047028 ± 0.0225500.140469 ± 0.004961
Kawahara equation0.010614 ± 0.0039761.122236 ± 0.0238650.434300 ± 0.6644140.042196 ± 0.025617
Thermoelastic system0.008553 ± 0.0001570.114652 ± 0.0473430.044544 ± 0.0172210.092515 ± 0.034206
Delayed linear problem0.015332 ± 0.0009580.038167 ± 0.0055540.029848 ± 0.0043070.048767 ± 0.001660
Reaction–diffusion model0.011010 ± 0.0002840.013647 ± 0.0005210.011931 ± 0.0004830.014113 ± 0.000214

DS4D-FFNO has lower mean error than all three temporal variants on all six tasks. Even when simpler temporal variants are given the same history window, DS4D-FFNO remains more accurate.

Thank you for the helpful clarifications, which have adequately addressed my concerns. I will maintain my current score.

Reviewer_CaszOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces a Ds4D-FFNO, a factorized Fourier Neural Operator designed to predict the evolution of PDEs with fixed-delay boundary feedback. The proposed architecture combines Factorized Fourier Neural Operators (FFNOs) for spatial modeling with an existing delayed state-space model (DS4D), which explicitly retrieves past hidden states at learned delays. The authors show experimentally that Ds4D-FFNO achieves lower errors on six considered 1D and 2D- fixed-delay boundary feedback systems, among eight competing architectures, with improvements ranging from approximately 9% to 83% over the strongest baselines.

Critical strengths
  1. The paper provides a neural operator based model for modelling systems with fixed-delay boundary feedback, where the solution depends explicitly on historical states, and proposes an hybrid FFNO-based architecture to do so.

  2. The proposed architecture outperforms a number of other architectures on the selected set of six PDE benchmarks.

Critical weaknesses
  1. The novelty of the paper is pretty weak, as the proposed architecture essentially combines two existing methods, FFNO and DS4D, without introducing a fundamentally new mechanism for modeling delayed dynamics. More importantly, the physical delays are explicitly known for all the PDEs considered. It is therefore unclear why learning delays in the latent space is necessary, rather than directly providing the corresponding historical boundary quantities to a standard neural operator. The ablation experiments in Figure 5 do not adequately address this question, as they merely demonstrate that modifying learned delays after training degrades performance. A comparison against an FFNO conditioned on the known delayed boundary values, or a DS4D model trained with fixed delays, would be essential to establish the actual benefits of the proposed approach.

  2. The experiments are restricted to relatively simple 1D and low-resolution 2D PDEs, with fixed physical delays prescribed separately for each benchmark. The authors do not investigate whether the proposed architecture generalizes to unseen delay values, different feedback coefficients, or more complex physical systems and geometries.

Questions for authors
  1. The ablation experiments in Figure 5 show that modifying the learned delays after training degrades prediction accuracy. However, this does not establish the necessity of learning these delays. Could the authors compare against models trained from scratch with fixed delays, particularly delays corresponding to the known physical feedback times of the governing PDEs?

  2. Could the authors compare how DS4D-FFNO performs against a standard FFNO that explicitly receives the relevant delayed boundary quantities as additional inputs, given that the physical delay is already known?

3 comments
AuthorsOpenReview ↗

Response to Reviewer Casz(1/2)

We thank the reviewer for pressing on mechanism validation and generalization.

The ablation experiments in Figure 5 do not adequately address this question, as they merely demonstrate that modifying learned delays after training degrades performance. A comparison against an FFNO conditioned on the known delayed boundary values, or a DS4D model trained with fixed delays, would be essential to establish the actual benefits of the proposed approach.

Thank you for emphasizing the need for direct comparisons to assess the value of learned retrieval. We designed Figure 5 to test whether the learned retrieval locations work together with channel-specific memory dynamics. Permutation preserves the set of learned locations but changes their assignment to channels, while random replacement changes the locations themselves. The increased errors support the importance of this learned pairing for the trained model's predictions.

We conduct the two suggested experiments by training DS4D-FFNO from scratch with retrieval locations fixed at the known physical lag (Table 1) and providing FFNO with the corresponding delayed boundary quantities (Table 4). DS4D-FFNO achieves lower mean error on all six tasks in both comparisons.

The authors do not investigate whether the proposed architecture generalizes to unseen delay values, different feedback coefficients, or more complex physical systems and geometries.

Thank you for emphasizing the importance of evaluating generalization. We agree that evaluating unseen parameters and more complex domains is important for understanding the method's applicability. We therefore test unseen delay and feedback values (Table 10) and evaluate a Schrödinger equation with delayed boundary feedback on a noncircular domain (Table 11). We will include generalization experiments in the revised manuscript to provide a more complete description of our approach’s generalization capabilities.

Generalization to unseen physical parameters. To assess the generalization ability of DS4D-FFNO, we perturb τ\tau and β\beta separately to construct training and test data for the Kawahara equation. In the delay-generalization experiment, we randomly perturb around τ=0.64\tau=0.64 to select five values for training and validation, and then randomly perturb again to select one value for testing. In the feedback-generalization experiment, we randomly perturb around β=0.5\beta=0.5 to select five values for training and validation, and then randomly perturb again to select one value for testing. Each training condition contributes 256 training and 32 validation trajectories, giving 1,280 training and 160 validation trajectories in total. We evaluate on 128 test trajectories at the unseen value. The recorded time interval is 0.005, with 128 integration substeps per recorded step. Table 10 reports the results.

Table 10. Kawahara evaluation at unseen values.

Kawahara equation evaluationDS4D-FFNOS4-FFNORelative error reduction
Unseen τ\tau0.207695 ± 0.0570460.269230 ± 0.11974722.86%
Unseen β\beta0.130988 ± 0.0057040.140481 ± 0.0140106.76%

Generalization to the tested unseen parameter values remains challenging for both models, although DS4D-FFNO achieves lower mean error in both settings.

AuthorsOpenReview ↗

Response to Reviewer Casz (2/2)

Noncircular geometry. We evaluate DS4D-FFNO using the delayed-boundary Schrödinger equation of Nicaise and Rebiai [1]. The annular domain, with a circular inner boundary and a noncircular outer boundary, is defined by

0.3 \lt r \lt 1+0.12\cos(3\theta),\quad 0\leq\theta \lt 2\pi\right\rbrace.$$ Let $\Gamma_0=\lbrace r=0.3\rbrace$ and $\Gamma_1=\lbrace r=1+0.12\cos(3\theta)\rbrace$ denote the inner and outer boundaries. The complex field $u$ satisfies $$\begin{cases} \partial_t u(x,t)-i\Delta u(x,t)=0, & x\in\Omega,\quad t>0,\\\\[3pt] u(x,t)=0, & x\in\Gamma_0,\quad t>0,\\\\[3pt] \partial_\nu u(x,t)=i\mu_1u(x,t)+i\mu_2u(x,t-\tau), & x\in\Gamma_1,\quad t>0,\\\\[3pt] u(x,0)=u_0(x), & x\in\Omega,\\\\[3pt] u(x,s)=f_0(x,s), & x\in\Gamma_1,\quad -\tau\leq s\leq 0. \end{cases}$$ Here $\nu$ is the outward normal, $u_0$ the initial field, and $f_0$ the boundary history, with $f_0(x,0)=u_0(x)$ on $\Gamma_1$. We set $\mu_1=0.20$, $\mu_2=0.15$, and $\tau=0.32$, or 32 recorded steps at $\Delta t=0.01$, and use a conforming Ritz–Galerkin discretization. **Table 11. Full-field nRMSE on the annulus.** Training and testing are conducted on the same noncircular domain using a 64×64 sampling grid, with 128 training, 32 validation, and 32 test trajectories. The observed and forecast lengths are 96 frames ($L=96$) and 64 frames ($H=64$), respectively. All runs use 120 training epochs. | DS4D-FFNO | S4-FFNO | Relative error reduction | | ---: | ---: | ---: | | **0.035347 ± 0.001075** | 0.037501 ± 0.001530 | 5.74% | DS4D-FFNO attains a lower full-field nRMSE than S4-FFNO, which demonstrates the architecture's ability to handle curved, non-periodic domains with irregular boundaries. **Reference** [1] S. Nicaise and S.-E. Rebiai. Stabilization of the Schrödinger equation with a delay term in boundary feedback or internal feedback. Portugaliae Mathematica, 68(1):19–39, 2011.

I thank the authors for their detailed response and for conducting the additional experiments. These comparisons are useful and provide stronger evidence for the benefits of learned hidden-state retrieval.

However, I am still not entirely convinced by the argument that the method is particularly suited to situations where the physical delay is unknown. The experiments show that learning latent retrieval locations can improve prediction accuracy, but do not demonstrate the ability to identify unknown physical delays.

More importantly, my concerns regarding the limited methodological novelty remain largely unchanged. The proposed approach essentially combines two existing architectures, FFNO and DS4D, without introducing a substantially new mechanism or methodological insight.

Overall, I have decided to increase my score to Weak Reject.

Reviewer_JtdMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces DS4D-FFNO, a neural operator architecture built for forecasting PDEs that feature fixed-delay boundary feedback. I see the core idea is to marry factorized Fourier neural operators (for spatial mixing) with a delayed state-space module that grabs interpolated past hidden states using learnable, channel-specific delays. The model processes observed fields sequentially and then generates future predictions autoregressively.

The authors evaluate the approach on six delayed PDE systems across 1D and 2D, comparing it against seven baselines. DS4D-FFNO achieves the best mean full-field prediction error across all the benchmarks. I note they also show the model holds its own on PDEs without physical delays compared to S4-FFNO, and demonstrate that the network actually relies on its learned delay indices.

Critical strengths
  1. In my opinion, bringing explicit temporal memory into PDE forecasting makes sense. Fixed-delay boundary feedback is a good testbed where past information is strictly needed for future dynamics. Combining spatial operators with delayed hidden-state retrieval is intuitive. I also appreciate the diverse set of PDEs evaluated, covering dispersive waves, structural dynamics, thermoelasticity, and reaction-diffusion. It proves this is not just a one-trick pony.

  2. I find the comparison to S4-FFNO sufficient. Because both share the exact same spatial backbone and initialization, it highlights the benefit of the temporal mechanism. The performance bump is consistent, and honestly pretty massive on some of the 1D systems. The addition ablation on non-delayed PDEs and the delay-index interventions is also useful.

Critical weaknesses
  1. My main concern is whether the model actually benefits from learning the delay, rather than just having explicit access to the history. To my understanding, the physical delay in these benchmarks is fixed and known. In the 1D tasks, there are 192 observation steps, a delay of 128, and a horizon of 24 (with similar coverage in the 2D setups). So, the required boundary history is entirely contained within the observed window. While S4-FFNO is a good baseline, it forces history into a compressed recurrent state, whereas DS4D-FFNO can just reach back and grab it directly. In order for the authors to really prove the value of the learnable retrieval, I believe we need a control baseline that simply feeds the known physical boundary delay directly to the operator, or a version of DS4D-FFNO where the delay is hardcoded to the true physical value.

Right now, it is hard to tell if the gain comes from the learnable mechanism or just the simple fact that the network is finally allowed to explicitly look at the past.

  1. I feel that the architecture is essentially a combination of the DSSM framework [1] and S4-FFNO, following the direction of [2]. That is fine, applying these to delayed boundary-feedback PDEs is a valid contribution, but the paper does not dive deep enough into what the model actually learns about the physics.

To be more concrete: do the learned delays actually match the known physical delays? what happens if you tweak the physical delay or the feedback strength? the permutation tests in Section 4.4 show the model breaks if you mess with the learned delays after training, which is good, but it does not prove that the learned delays are physically meaningful, or that learning them is actually better than just setting them to the right physical values.

References

[1] DSSMs: State Space Models with Explicit Memory via Delay Differential Equations

[2] On the Benefits of Memory for Modeling Time-Dependent PDEs

Questions for authors

Q1. Can you compare DS4D-FFNO to a version where the delays are strictly fixed to the known physical delay, trained from scratch? Alternatively, what about an S4-FFNO variant that is explicitly given the true delayed boundary observations? From my perspective, this would help clarify if the learned aspect of the hidden-state retrieval is doing the heavy lifting.

Q2. To my understanding, what is the actual relationship between the learned channel delays and the ground-truth physical delay for a given PDE? Do you see a consistent pattern across different random seeds? An experiment that showing how these learned delays shift when you change the physical delay would go a long way in connecting the architecture to the underlying dynamics.

Q3. Could you include paired, per-seed comparisons with S4-FFNO? I think this is especially important for the 2D benchmarks where the performance gap is a bit more moderate. Also, practically speaking, what is the inference-time and memory overhead of storing the hidden-state history at every single spatial grid point? This would help me assess the real-world utility of the improvements.

3 comments
AuthorsOpenReview ↗

Response to Reviewer JtdM(1/2)

We thank the reviewer for the questions on retrieval locations, physical interpretation, and computational cost.

Can you compare DS4D-FFNO to a version where the delays are strictly fixed to the known physical delay, trained from scratch? Alternatively, what about an S4-FFNO variant that is explicitly given the true delayed boundary observations?

Thank you for these helpful suggestions. We agree that both comparisons are important for assessing the benefit of learned retrieval and have included them in Tables 1 and 3. DS4D-FFNO achieves lower mean error than fixed physical-delay DS4D-FFNO on all six tasks. It also outperforms S4-FFNO with exact-lag boundary inputs on five of the six tasks.

To my understanding, what is the actual relationship between the learned channel delays and the ground-truth physical delay for a given PDE? Do you see a consistent pattern across different random seeds?

To be more concrete: do the learned delays actually match the known physical delays?

We agree that this relationship needs clarification. We conduct the analysis reported in Table 5 to examine whether the learned channel delays match the physical lag and whether this relationship is consistent across random seeds. For each seed, we independently initialize the retrieval lags of the 32 channels using the same uniform sampling rule, with different random draws across seeds.

Table 5. Learned retrieval locations relative to the physical lag. For the 32 learned retrieval locations of each trained default model, we report across seeds the min–max of the final channel median and of the number of channels within 4 steps of the physical lag (in recorded steps).

TaskPhysical lag (steps)Final channel median (steps)Channels within 4 steps of physical lag
KdV–KdV system12878.50–105.590–3/32
Microbeam system12880.57–105.220–2/32
Kawahara equation12876.04–105.251/32
Thermoelastic system6438.95–53.631–4/32
Delayed linear problem6439.84–53.271–5/32
Reaction–diffusion model6439.08–53.261–4/32

As shown in Table 5, across all three seeds and all six tasks, the channel medians remain below the physical lag, ranging from approximately 59% to 84% of its value. In every run, at most 5 of the 32 channels lie within four recorded steps of the physical lag. Although these medians do not match the physical lag, learned retrieval achieves lower mean prediction error than retrieval fixed at that lag, as reported in Table 1.

This behavior can be understood from the information stored in the hidden states. We express both delays in recorded steps, denoting the physical lag by m=τ/ΔTm=\tau/\Delta T and the retrieval lag of channel cc by ℓc\ell_c. For integer retrieval lags, the channel update is

=A_c h_{n-1}^{(c)} +A_c^d h_{n-\ell_c}^{(c)} +B_c v_n^{(c)}.$$ Here, $h_n^{(c)}$ is the channel’s hidden state at step $n$. The complex diagonal matrices $A_c$ and $A_c^d$ weight the previous state and the retrieved historical state, respectively, while $B_c$ maps the spatially mixed channel input $v_n^{(c)}$ into the hidden state. For fixed model parameters, zero initial hidden history, and an input sequence starting at $n=0$, the retrieved state can be written as $$h_{n-\ell_c}^{(c)} =\sum_{r=0}^{n-\ell_c}K_c^h(r)\\,v_{n-\ell_c-r}^{(c)},$$ where $K_c^h$ is the state response kernel of the full delayed recurrence, incorporating both the previous-state and delayed-state branches. For $n\geq m\geq\ell_c$, this sum includes $K_c^h(m-\ell_c)v_{n-m}^{(c)}$. Thus, information from the input at the physical lag can be encoded in the retrieved hidden state, even when $\ell_c<m$. At step $n$, the delayed branch brings this stored information into the current state through $A_c^d h_{n-\ell_c}^{(c)}$. Information from the physical-lag input can therefore contribute to the current prediction through a stored historical state, without depending solely on its retention in the most recent state $h_{n-1}^{(c)}$. This additional access to earlier information may help explain the observed reduction in prediction error. The physical lag refers to how far back the boundary feedback quantity lies, while the retrieval lag picks a hidden state that already summarizes earlier inputs. Such a state can still carry information from the physical-lag input, and so the two lags need not coincide.
AuthorsOpenReview ↗

Response to Reviewer JtdM (2/2)

An experiment that showing how these learned delays shift when you change the physical delay would go a long way in connecting the architecture to the underlying dynamics.

We agree that varying the physical delay helps clarify the relationship between learned retrieval and the underlying dynamics. We conduct the experiment reported in Table 6 to examine how the learned retrieval locations change across physical delays.

Table 6. Learned retrieval locations under different physical delays. On Kawahara with fixed β=0.500\beta=0.500, we reuse the original-paper checkpoints at τ=0.640\tau=0.640 and train from scratch at the other delays, using the same initialization across delay settings within each seed. We report the mean ± sample SD across seeds of the median retrieval location over the 32 channels.

Physical τ\tauPhysical lag (steps)Final channel median (steps)
0.489685.87 ± 8.52
0.6412885.33 ± 11.41
0.8016084.96 ± 4.73

The mean channel median remains in the 85–86 step range even as the physical lag increases from 96 to 160 steps, which suggests that it does not directly track the physical delay.

What happens if you tweak the physical delay or the feedback strength?

We conduct the experiment in Table 7 below to examine how changing the physical delay or feedback strength affects the trained model's prediction accuracy without retraining.

Table 7. Frozen Kawahara model error under changed τ\tau and β\beta. Using the same three Kawahara DS4D-FFNO checkpoints as in the main paper (τ=0.640\tau=0.640, β=0.500\beta=0.500), we vary one physical parameter at a time by no more than 1%. We report full-field nRMSE on 96 test trajectories per condition as mean ± sample SD across three random seeds.

Physical τ\tauFeedback β\betaFrozen source DS4D-FFNO
0.6400 (source)0.50000.010614 ± 0.003976
0.63680.50001.701548 ± 0.080972
0.64320.50002.050263 ± 0.213247
0.64000.49750.016524 ± 0.003765
0.64000.49500.026539 ± 0.008550

Changing the parameters by no more than 1% raises the frozen model's mean error to 160–193 times the source error for τ\tau and 1.56–2.50 times for β\beta, which shows that its predictions depend strongly on both the physical delay and the feedback strength.

Could you include paired, per-seed comparisons with S4-FFNO?

Sure. We present the complete paired results for all three seeds on the 2D tasks in Table 8 below.

Table 8. Paired per-seed comparison with S4-FFNO on the 2D tasks.

TaskSeedDS4D-FFNOS4-FFNO
Thermoelastic system00.0084180.010509
Thermoelastic system10.0085150.009773
Thermoelastic system20.0087250.010486
Delayed linear problem00.0163790.018164
Delayed linear problem10.0144990.015071
Delayed linear problem20.0151190.017065
Reaction–diffusion model00.0106850.013757
Reaction–diffusion model10.0112060.011427
Reaction–diffusion model20.0111400.014223

DS4D-FFNO has lower error in all nine pairs.

Also, practically speaking, what is the inference-time and memory overhead of storing the hidden-state history at every single spatial grid point?

Thank you for raising this meaningful question. We agree that understanding the inference-time and memory overhead is important for assessing practical applicability. We quantify this overhead relative to S4-FFNO in Table 9 below.

Table 9. Inference cost, full-model latency, and retained state. Each checkpoint is timed in a fresh process on one RTX 4090 at batch size 1, using five warmup and 30 timed requests. Timing includes state initialization, history conditioning, and forecasting, but excludes checkpoint loading and device transfers.

TaskModelTotal request (ms)Retained state/cache (MiB)
Microbeam system (1D)DS4D-FFNO726.325 ± 8.41736.954
Microbeam system (1D)S4-FFNO637.282 ± 9.9460.190
Delayed linear problem (2D)DS4D-FFNO551.928 ± 9.6291176.010
Delayed linear problem (2D)S4-FFNO426.208 ± 11.75912.000

Compared with S4-FFNO, DS4D-FFNO adds moderate latency overhead, along with a substantially larger memory cost for retained hidden states. This memory difference comes from the additional hidden-state history buffer. S4-FFNO stores only the current hidden state at each spatial grid point. DS4D-FFNO stores the current state, LL historical states, and one additional buffer holding their sum. This additional buffer has the same size as one hidden state. For Microbeam, L=192L=192, so the retained memory is approximately 192+1+1=194192+1+1=194 times that of S4-FFNO. For the delayed linear problem, L=96L=96, giving approximately 96+1+1=9896+1+1=98 times the retained memory.

I thank the authors for their detailed rebuttal with multiple experiments. These experiments make the empirical case stronger. However I am still not inclined to increase my score. The reason is that as shown in the results in the rebuttal, the proposed method is very expensive on memory, by several orders. This is not very attractive in my opinion for these applications. Also, the results dont seem to really track changes in the true physical delay, as asked in my original review.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response(1/2)

We thank all reviewers for their careful reading and constructive suggestions. All three reviews converge on one question.

The physical delays are explicitly known for all the PDEs considered. It is therefore unclear why learning delays in the latent space is necessary, rather than directly providing the corresponding historical boundary quantities to a standard neural operator.

To answer this question, we first clarify that the motivation for our approach is to forecast PDE dynamics when the physical delay is unknown, since the delay is often difficult to estimate accurately in practice. We therefore propose DS4D-FFNO to learn directly from observed data which past hidden states to retrieve. We will make this motivation explicit in the revised manuscript. To address this concern directly, we further add controlled comparisons with fixed retrieval and with standard operators given delayed boundary quantities (Tables 1–4).

Common experimental setup. Unless specified in the individual responses, experiments follow the protocol of the paper. Full-field NRMSE is reported as the mean ± SD over three random seeds.

Table 1. Learned and fixed physical-delay retrieval. We train DS4D-FFNO from scratch with all 32 channel retrieval locations fixed at the true physical lag.

TaskDS4D-FFNOFixed physical-delay DS4D-FFNO
KdV–KdV system0.018558 ± 0.0253980.033361 ± 0.046264
Microbeam system0.017189 ± 0.0192770.259184 ± 0.425384
Kawahara equation0.010614 ± 0.0039760.026189 ± 0.026879
Thermoelastic system0.008553 ± 0.0001570.010135 ± 0.002800
Delayed linear problem0.015332 ± 0.0009580.018457 ± 0.003421
Reaction–diffusion model0.011010 ± 0.0002840.011107 ± 0.000396

DS4D-FFNO attains lower mean error on all six tasks.

Table 2. Learned and fixed uniform-bank retrieval. We train DS4D-FFNO from scratch, with each of its 32 channels' retrieval location fixed to a uniform bank spread over the history window. The uniform bank uses 32 locations dj=1+(L−2)(j+0.25)/32d_j=1+(L-2)(j+0.25)/32 for j=0,…,31j=0,\ldots,31. Within each seed, these locations are randomly assigned to channels and held fixed during training.

TaskDS4D-FFNOFixed uniform-bank DS4D-FFNO
KdV–KdV system0.018558 ± 0.0253980.030798 ± 0.040111
Microbeam system0.017189 ± 0.0192770.035426 ± 0.049414
Kawahara equation0.010614 ± 0.0039760.014898 ± 0.003752
Thermoelastic system0.008553 ± 0.0001570.008839 ± 0.000642
Delayed linear problem0.015332 ± 0.0009580.017207 ± 0.002011
Reaction–diffusion model0.011010 ± 0.0002840.010560 ± 0.000567

DS4D-FFNO attains lower mean error on five of six tasks.

Table 3. S4-FFNO with explicit delayed boundary inputs. We train a separate S4-FFNO variant for each specified lag, τ\tau, 0.75τ0.75\tau, and 1.125τ1.125\tau, with the latter two representing uncertainty in the estimated lag. In each setting, boundary features extracted or reconstructed from the field history at the specified lag serve as additional inputs to S4-FFNO.

TaskDS4D-FFNOS4-FFNOS4-FFNO+τ\tauS4-FFNO+0.75τ0.75\tauS4-FFNO+1.125τ1.125\tau
KdV–KdV system0.018558 ± 0.0253980.041844 ± 0.0637980.004330 ± 0.0006710.026673 ± 0.0309290.034157 ± 0.041096
Microbeam system0.017189 ± 0.0192770.034064 ± 0.0468980.049802 ± 0.0403450.329125 ± 0.3632850.109457 ± 0.146643
Kawahara equation0.010614 ± 0.0039760.063751 ± 0.0798820.022656 ± 0.0136620.012207 ± 0.0036320.030375 ± 0.008066
Thermoelastic system0.008553 ± 0.0001570.010256 ± 0.0004190.010454 ± 0.0033770.010519 ± 0.0035260.010520 ± 0.003593
Delayed linear problem0.015332 ± 0.0009580.016767 ± 0.0015680.019077 ± 0.0038280.018725 ± 0.0041650.018812 ± 0.003338
Reaction–diffusion model0.011010 ± 0.0002840.013136 ± 0.0014980.011024 ± 0.0004880.011157 ± 0.0004880.011032 ± 0.000425

DS4D-FFNO achieves lower mean error than S4-FFNO with exact-lag boundary inputs on five of the six tasks. It also achieves lower mean error on all six tasks at each of the two reported lag offsets.

AuthorsOpenReview ↗

General Response(2/2)

Table 4. FFNO with delayed boundary inputs. We train a separate FFNO variant for each specified lag, τ\tau, 0.75τ0.75\tau, and 1.125τ1.125\tau, with the latter two representing uncertainty in the estimated lag. In each setting, boundary features extracted or reconstructed from the field history at the specified lag serve as additional inputs to FFNO.

TaskDS4D-FFNOFFNOFFNO+τ\tauFFNO+0.75τ0.75\tauFFNO+1.125τ1.125\tau
KdV–KdV system0.018558 ± 0.0253980.362001 ± 0.0883750.026485 ± 0.0220920.384031 ± 0.1593690.296583 ± 0.022646
Microbeam system0.017189 ± 0.0192770.955917 ± 0.0016110.935004 ± 0.0010860.961289 ± 0.0006680.955747 ± 0.000309
Kawahara equation0.010614 ± 0.0039760.997786 ± 0.0007850.936250 ± 0.0325591.000202 ± 0.0044100.999161 ± 0.001333
Thermoelastic system0.008553 ± 0.0001570.009802 ± 0.0005170.009957 ± 0.0005550.009993 ± 0.0005790.009929 ± 0.000546
Delayed linear problem0.015332 ± 0.0009580.466166 ± 0.0133810.084461 ± 0.0176010.146377 ± 0.0092590.126912 ± 0.013715
Reaction–diffusion model0.011010 ± 0.0002840.018870 ± 0.0000750.017114 ± 0.0001220.018242 ± 0.0000800.018747 ± 0.000137

DS4D-FFNO achieves the lowest mean error on all six tasks among the models compared in Table 4.

In general, DS4D-FFNO achieves lower mean error than each baseline in Tables 1–4 on at least five of the six tasks. This shows the benefit of learning which past hidden states to retrieve from observed data, without taking the physical delay as an input.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.