- Rating
- 3(+1)
- Confidence
- 1
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Test-Time Forecasting (TTF), a framework that improves dynamical system forecasting through recurrent computation and test-time optimization. It introduces two variants, iTTF and eTTF, which update predictions or forecaster parameters, respectively. Experiments on ODEs and PDEs show improved out-of-distribution forecasting with increased iterations, while highlighting additional computational costs and limitations in generalization.
Critical strengths
- The paper establishes a theoretically motivated connection between recurrent linear attention and Dynamic Mode Decomposition (DMD), providing a principled foundation for the proposed framework.
- It unifies implicit and explicit test-time forecasting into a flexible framework that supports different forecasters and optimization methods.
- Experiments demonstrate improved out-of-distribution forecasting as recurrent iterations increase across several dynamical systems.
Critical weaknesses
- Experiments are primarily conducted on synthetic ODEs and PDEs, leaving performance on real-world time series unverified.
- Increasing recurrent iterations introduces additional inference costs, while explicit TTF also incurs substantial training overhead.
- The novelty of the proposed TTF framework is somewhat limited, as test-time optimization and adaptation have been extensively explored in other domains.
Questions for authors
N/A
3 comments
Thanks for your feedback, which has addressed most of my concerns. I have raised my score in response.
We thank the referee for their review which has allowed us to improve the related works and experiments.
Response to reviewer
[W1.] In our revised text, we train iTTF models on the Eigenworms dataset [1] which consists of real experimental measurements of C. elegans posture motion under different stimuli and demonstrate the efficacy of basic TTF models in Figure. 5,16,17 in the revised text. Additionally, we train eTTF models on real PIV data of a 2D fluid flow and show in Figure. 8 that minimal eTTF models can beat the standing FNO baseline for this task (see response to reviewer BHyf). We prefer to focus our paper's main experiments on nonlinear dynamics since they better illustrate the connection of the TTF framework to Koopman theory.
[W2.] For iTTF models, L recurrent iteration costs the same as an analogous L layer fixed depth model since the recurrent model structurally mirrors the fixed depth model but with tied layer weights. Additional inference cost is only incurred if the user chooses to spend more recurrent iterations to achieve lower error. For eTTF models however, we agree that both training and inference is more expensive and recognize this limitation. Despite the cost and performance discrepancy between eTTF and iTTF models, we choose to present eTTF since it is the natural theoretical generalization of the idea that relying on Koopman operator theory to forecast via regression in-context directly corresponds to looped linear attention; this finding is the core idea we intend to highlight in this work rather than presenting evaluations of production-ready surrogate models.
Relatedly, it is also well known that TTT and looped transformers typically require higher pretraining overhead [2] but gain 1) adaptive test-time compute via loop iterations, and 2) reduced model size (parameters); which we demonstrate in our work.
[W3.] We emphasize that our novelty comes from the link between Koopman operator theory and test-time training in a forecasting setting, and also the theoretical connections to linear attention design. We briefly discuss the broader test-time training field in the related works section which we have updated to include a more detailed discussion.
Our Appendix D theoretically motivates multiple common transformer architecture design choices such as short convolutions, sharpness control, gated linear units, etc. which are implemented in open frontier models due to their empirical benefits, but which have previously eluded clear motivation [3,4,5]. Moreover, prior works primarily justify the use of transformer layers in dynamics forecasting and TSFMs based on their empirical performance, rather than principled motivation. Thus, TTF represents a novel framework that both guides the design of forecasting models capable of in-context learning, and theoretically motivates modern transformer architecture design more broadly, using the lens of optimization dynamics.
References
[1] Greg J. Stephens, Bethany Johnson-Kerner, William Bialek, and William S. Ryu. Dimensionality and Dynamics in the Behavior of C. elegans. PLoS Computational Biology, 4(4):e1000028, 2008.
[2] Kimi Team. Kimi Linear: An Expressive, Efficient Attention Architecture. arXiv:2510.26692, 2025.
[3] Kimi Team. Kimi K3: Open Frontier Intelligence. arXiv:2607.24653, 2026.
[4] William Merrill et al. Olmo Hybrid: From Theory to Practice and Back. arXiv:2604.03444, 2026.
[5] Hayden Prairie, Zachary Novack, Taylor Berg-Kirkpatrick, and Daniel Y. Fu. Parcae: Scaling Laws For Stable Looped Language Models. COLM, 2026.