- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper studies Direct-Time Neural Operators (DTNO) for predicting a future PDE state from an observed history and a requested lead time in a single forward pass. The implementation combines an FNO backbone, Fourier time embeddings, FiLM conditioning, and residual decoding, with an optional latent semigroup regularizer. Across five datasets, the authors compare direct prediction with a separately trained autoregressive counterpart and examine accuracy–horizon trade-offs, inference cost, seed variability, and area-relative performance. DTNO outperforms the autoregressive FNO baseline at intermediate horizons, although its long-horizon predictions often approach long-term average and its advantage depends substantially on the backbone.
Critical strengths
- The paper addresses the accuracy and computational cost of long-horizon forecasting for spatiotemporal systems, an important problem in scientific machine learning.
- The proposed model uses a straightforward architecture to support predictions at multiple lead times within a single model, with one forward pass per requested future state.
- The experiments cover five datasets including reaction–diffusion, fluid dynamics, and reacting flows, providing a reasonably comprehensive evaluation of the approach
Critical weaknesses
-
The methodological contribution is limited, and the value of the particular combination is not established. The method combines several existing components: an FNO (Li et al., 2021), Fourier time embeddings (Xu et al., 2019, Tancik et al., 2020), FiLM (Perez et al., 2018), residual connections, and an optional consistency loss. Time-conditioned direct prediction is already central to Poseidon (Herde et al., 2024) and TC-UNet (Ovadia et al., 2025), as the manuscript itself acknowledges in Section 2. Furthermore, Section 3.2 states that the pointwise encoder and decoder contain little capacity, so conditioning the propagator effectively conditions the entire network. It is therefore unclear what substantive benefit follows from the proposed factorization. The reported ablations do not isolate its advantage over simpler time conditioning, or establish the contributions of the history window and residual decoder. The main comparison establishes a difference between temporal prediction strategies, but offers limited evidence for DTNO-specific design choices.
-
The long-horizon results provide limited evidence of useful dynamical prediction. On several datasets, DT-FNO performs similarly to or worse than time-averaged reference predictions (Tables 2 and 8). Although these references use test-trajectory information, the results raise questions about whether DT-FNO preserves meaningful dynamics beyond avoiding rollout instability. Stronger validation on challenging, non-periodic systems would help establish its practical forecasting value.
-
The connection between the latent semigroup constraint and physical evolution is insufficiently justified. Equation (3) encourages , but substituting this into Equation (2) gives which does not ensure the required zero-time identity unless the decoded increment vanishes. Likewise, latent composition consistency does not by itself establish consistency of the decoded physical predictions. The manuscript does not establish that represents the encoding of an appropriately advanced history, and the residual decoder remains referenced to the original observed state. These issues weaken the interpretation of the regularizer as enforcing physical temporal structure. Moreover, increasing latent RMS rules out collapse to zero, but does not rule out loss of sample-dependent information or other degenerate representations as shown in experimental section. Consequently, the null accuracy result is difficult to interpret as evidence about the usefulness of physically meaningful semigroup consistency.
-
The reported results contain inconsistencies. Tables 2 and 9 report different RealPDEBench results for DT-FNO and SG-DT-FNO without explaining the discrepancy. Section 5.3 also claims that DT-FNO outperforms the (R=16) autoregressive model at every tabulated horizon, yet Table 3 includes a Gray–Scott autoregressive seed with lower one-step error. These inconsistencies should be clarified to support the experimental conclusions.
-
Table 4 contains the placeholder “[place tab_physics.tex next to main.tex]” or improper importing tex file.
References
- Li, Z., et al. “Fourier Neural Operator for Parametric Partial Differential Equations.” ICLR, 2021.
- Perez, E., et al. “FiLM: Visual Reasoning with a General Conditioning Layer.” AAAI, 2018.
- Herde, M., et al. “Poseidon: Efficient Foundation Models for PDEs.” NeurIPS, 2024.
- Ovadia, O., et al. “Real-time inference and extrapolation with Time-Conditioned UNet: Applications in hypersonic flows, incompressible flows, and global temperature forecasting.” Computer Methods in Applied Mechanics and Engineering.
- Xu, D., et al. “Self-attention with Functional Time Representation Learning.” NeurIPS, 2019.
- Tancik, M., et al. “Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains.” NeurIPS, 2020.
Questions for authors
- What measurable benefit does the proposed encoder–propagator–decoder factorization provide over a conventional time-conditioned FNO? Can the authors isolate the effects of FiLM versus simpler conditioning, the history window, and residual decoding?
- What hypothesis explains the crossover and its disappearance with a U-Net? Can the authors test it through a controlled within-dataset change in sampling cadence or dynamical difficulty?
- How does Equation (3) imply zero-time identity or composition consistency for the physical prediction in Equation (2)? Can the authors evaluate decoded consistency and alignment with the encoding of an advanced history, beyond latent RMS?
- Can the authors provide the missing Table 4, explain the different values in Tables 2 and 9, and correct the statement that the (R=16) direct model achieves better performance at every tabulated horizon?
3 comments
I thank the authors for the response and acknowledging the limitations. I do have several concers:
-
I thank the authors for their explanation and honest. However, the response does not address my concern; rather, it verifies it, as the authors explicitly acknowledge the limited methodological novelty.
-
As the contribution seems mainly controlled empirical comparisons. But from the manuscript, I can see DT introduces extra components and specific tuning for considered settings. And there is also no clear report of difference in training cost, potentially different hyperparameter choices, while differences in training costs and tuning budgets are not clearly reported. Given the raining-protocol difference, how can the authors attribute the improvements to the direct prediction instead of architectural or training differences?
-
I recommand that the authors provide the missing Table 4.
- Prior work and scope of the contribution.
Although we cited Poseidon, we did not adequately account for its time-conditioned FNO baseline (Appendix C.4) and its direct-versus-autoregressive comparison (Appendix D.6.2, Figure 47) when positioning our contribution. We apologize for this oversight. These experiments already include direct and autoregressive FNO inference and demonstrate task-dependent differences in their performance.
Accordingly, we do not claim priority for time-conditioned FNO or for the general idea of comparing direct and autoregressive forecasting. Our study extends this line of empirical investigation to history-conditioned forecasting with separately rollout-trained baselines, horizon-resolved evaluation, and backbone and climatology controls. The results characterize intermediate-horizon benefits and settings in which the advantage disappears or predictions approach climatology.
These differences specify the scope of our empirical contribution. They do not, by themselves, establish methodological novelty or resolve the attribution limitations discussed below. 2. Architectural controls, training differences, and attribution.
Several components mentioned in the review are shared by both models. Section 3.3 specifies the same history encoder, Fourier backbone, LayerScale residual blocks, and decoder. AR-FNO-R removes the time embedding and FiLM heads and is trained on its own multistep rollouts. The optional semigroup penalty is not used by the main DT-FNO model.
For Gray–Scott, the parameter counts are 8,414,786 for AR and 8,565,698 for DT; the difference is the 150,912 parameters in the time embedding and FiLM heads (approximately 1.8%). Appendix D.2 also reports common settings for width, depth, history length, optimizer, and learning-rate schedule, while explicitly listing dataset-dependent mode counts, padding, and horizon limits. These details specify the extent of matching; they do not establish identical architectures or equal tuning effort.
The width control in Appendix D.4 further shows that increasing the AR-FNO parameter count to approximately 15.9M does not reproduce the U-Net's accuracy and stability. This weakens a parameter-count-only explanation, but does not isolate the effects of time conditioning or temporal supervision.
Your attribution concern is therefore valid at the level of an isolated causal claim. Direct prediction changes both the conditioned model and its temporal training objective. Our experiments compare these forecasting configurations; they do not separate the accuracy contribution of avoiding recursive inference from that of horizon conditioning and multihorizon supervision. Accordingly, the supported claim is that the tested direct configuration achieves better intermediate-horizon field accuracy under the reported protocols. Attributing the entire improvement to direct inference alone would be stronger than our evidence permits.
Likewise, shared final hyperparameters do not establish equal hyperparameter-search budgets, and equal epoch counts do not establish equal training compute. Table 3 reuses the main-protocol DT result, so it cannot establish a fully compute-matched DT-versus-AR comparison or a precise end-to-end training-efficiency advantage. The existing controls narrow alternative explanations, but do not eliminate all architectural and training differences. We acknowledge this limitation explicitly.
One inference property can be attributed directly to the formulation: a single requested future state requires one DT evaluation instead of a sequence of AR steps. This evaluation-count advantage is separate from claims about accuracy, training cost, or tuning efficiency.
- Table 4.
Table 4 has now been included in the updated PDF. We apologize for the omission and thank you for identifying it.
Thank you for the detailed review. We clarify the contribution and acknowledge the mathematical and reporting limitations below.
1. Contribution and factorization.
Our contribution is an empirical comparison of separately trained direct and autoregressive models with matched spatial backbones, trajectory-held-out splits, and horizon-wise evaluation. We acknowledge prior time-conditioned methods, including Poseidon's comparison of inference modes.
The encoder–propagator–decoder factorization is an implementation choice, not a new operator class. Our ablations do not establish separate gains from FiLM, history length, or residual decoding. The contribution we defend is the controlled comparison and its boundaries, including backbone dependence and sparse-horizon interpolation failures. Missing external direct-time baselines limit broader comparative claims.
2. Useful prediction versus long-horizon averaging.
We agree that bounded error does not establish meaningful dynamics; Section 5.5 and Table 8 explicitly document this limitation. Nevertheless, Table 2 reports intermediate-horizon benefits: on Gray–Scott at h=32, DT-FNO has field error 0.3683, versus 0.5882 for AR-FNO-R and 0.5639 for trajectory climatology.
This supports field prediction at that horizon, not sustained chaotic trajectory tracking. Cylinder also has a favorable interval, but its quasi-periodicity limits that interpretation. The test-trajectory mean is an oracle diagnostic, not a deployable predictor or a rigorous information limit. Similarity to it does not prove that the observations contain no remaining predictive information.
3. Crossover and the U-Net result.
A plausible explanation is that multihorizon direct training trades some short-horizon accuracy for later-horizon performance, while autoregressive prediction accumulates approximation error through model-generated inputs. The disappearance of the advantage with an accurate, stable U-Net rollout is consistent with this hypothesis.
However, the width, mode-count, and precision controls do not establish a unique causal mechanism. Appendix D.5 provides no completed controlled cadence test. The 8–20-step crossover is therefore an observation for the tested FNO settings, not a universal physical timescale.
4. Latent versus physical consistency.
Your mathematical objection is correct. Even if Φθ(Eθ(Xt),0)=Eθ(Xt), Equation (2) gives Gθ(Xt,0)=U(t)+Dθ(Eθ(Xt)), which need not equal U(t). Equation (3) therefore does not enforce physical zero-time identity.
Likewise, latent composition does not establish alignment with an appropriately advanced history. The residual decoder remains anchored to the original observation, so latent composition is not equivalent to physical advancement followed by history re-encoding.
Increasing latent RMS cannot exclude sample-independent or other nonzero degenerate representations. The submitted diagnostics do not replace the decoded-consistency and alignment tests you request. The experiment concerns a particular latent penalty; it does not establish that physically consistent semigroup constraints are ineffective. Unregularized DT-FNO does not depend on a physical semigroup guarantee.
5. Reporting issues.
Table 4 is missing, and we apologize for the omission. Its caption cannot serve as quantitative evidence; we do not rely on those unreported physical metrics here.
We also acknowledge the inconsistent RealPDEBench summaries in Tables 2 and 9. We do not offer an unverified explanation or replacement value. These entries cannot support a precise estimate of the semigroup regularizer's effect without reconciliation.
The statement that DT-FNO outperforms R=16 at every tabulated horizon is incorrect. The Gray–Scott AR seed with one-step error 0.0136 outperforms DT's 0.0169. The reported advantage at h=32 and h=128 does not imply dominance at h=1, and the different DT training protocol remains a qualification.
Our defense rests on the explicitly identified empirical results and their limited scope, rather than missing metrics, unreconciled summaries, or a physical-consistency guarantee.