acceptodds
Under review as a conference paper at ICLR 2027

Learning to Forecast on Convolution-Induced Geometry

Abstract

Delay-coordinate methods provide a principled way to expose the geometry of dynamical systems from observed time series. However, classical constructions typically choose coordinates to recover or characterize the dynamics of observed trajectories, rather than to optimize a downstream forecasting objective. We ask whether such coordinates can instead be learned directly for prediction. Building on the connection between convolutional coordinates and finite-dimensional Koopman representations, we show that a causal convolution provides a learnable projection of a delay embedding, with Hankel-SVD convolutional coordinates arising from a particular choice of filters. This motivates a neural forecaster that learns delay coordinates end-to-end for prediction. We further use a Koopman regularizer to encourage approximately linear evolution in the learned coordinate space while retaining efficient direct multi-horizon prediction. Experiments show that the model achieves strong performance on standard forecasting benchmarks. More importantly, ablations and representation analyses reveal that forecasting supervision gives the learned coordinates predictive structure, while Koopman regularization helps organize their temporal evolution. Overall, our results show that strong neural forecasting and an explicit, analyzable dynamical representation can be achieved within the same learned coordinate space, connecting predictive accuracy with the study of what the forecaster learns.

Primary area:
learning on time series and dynamical systems
Keywords:
Dynamical System, Embedology, Time Series Forecasting

Reviews

Among:
Shape:
1234this paper, Top 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.67 over 3 reviews, above 99% of the others.
Reviewer_EdqeOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes FCIG, a time-series forecasting framework that learns delay-coordinate representations through causal convolutions and uses Koopman regularization to encourage approximately linear latent dynamics. It retains direct multi-horizon prediction rather than relying on recursive latent-state evolution.

Critical strengths
  1. Provides representation analyses and ablation studies to investigate how convolutional coordinates and Koopman regularization affect predictive performance and latent dynamics.
  2. The method is well motivated by established theoretical foundations in delay-coordinate embedding and Koopman operator theory. In particular, the paper establishes a formal connection between causal convolutional projections and classical Hankel-SVD coordinates, providing a principled basis for learning forecasting-oriented representations.
  3. The paper goes beyond evaluating forecasting accuracy by examining the geometric and spectral properties of learned representations, providing insights into how predictive supervision shapes latent dynamics and improves the consistency of learned state-space representations.
Critical weaknesses
  1. The methodological novelty may be limited, as the framework primarily combines existing techniques, including convolutional encoders, Koopman regularization, and direct forecasting.
  2. The benefits of Koopman regularization are task-dependent; improved latent dynamical consistency does not always translate into better forecasting accuracy, as demonstrated by the Lorenz-63 experiments.
  3. Improvements over strong general-purpose baselines are often modest, and FCIG underperforms competing methods on the Traffic dataset, suggesting that its advantages are not universal.
Questions for authors

N/A

3 comments
AuthorsOpenReview ↗

Response to Reviewer Edqe

We thank the reviewer for recognizing the connection to delay-coordinate methods and the value of analyzing the learned representations. We have clarified the contribution and added controlled comparisons addressing the roles of coordinate learning and regularization. We hope these revisions and additional experiments help address your concerns.


W1. The contribution: forecasting-oriented coordinate learning

First, we would like to clarify what we set out to study. Our aim was to make forecasting representations more analyzable and bring dynamical coordinates into the learning process, rather than to design another prediction architecture. We therefore believe that assessing the contribution solely in terms of architectural novelty would overlook the central idea of this work.

In fact, we initially tried replacing a delay embedding with Hankel-SVD coordinates. In our early implementation, incorporating SVD into training was cumbersome and computationally demanding. The connection to causal convolution gave us a more natural way to learn delay-coordinate projections through standard neural-network operations, with Hankel-SVD included as a linear special case (Section 3.1‾\underline{\text{Section 3.1}}). This changed our approach from supplying precomputed coordinates to letting forecasting supervision shape the coordinate construction itself. The stacked nonlinear encoder came later as an extension of this idea.

That is also why we use ReNF's compact, strong MLP backbone: it lets us test the coordinates without introducing a new prediction architecture at the same time. The added controls keep the downstream architecture fixed and compare fixed SVD, learnable linear, and nonlinear coordinates with KAR disabled. Across six datasets and two horizons, learning the linear projection improves both mean MSE and MAE in seven of twelve settings; nonlinear coordinates improve MSE in eleven and MAE in ten relative to fixed SVD (Appendix D.1, Table 10‾\underline{\text{Appendix D.1, Table 10}}). These comparisons directly test the coordinate-learning idea that motivated the work.


W2. Task dependence of Koopman regularization

The Lorenz63 result actually highlights a distinction we consider important: coordinates can become more locally consistent without producing better forecasts. Understanding when a dynamical prior helps, and when it does not, is part of the question we wanted to investigate. We therefore report forecasting and geometric metrics together and discuss their disagreement explicitly (Section 4.4 and Appendix E.1‾\underline{\text{Section 4.4 and Appendix E.1}}).

To examine this more directly, we have added comparisons with No KAR and an identity-transition regularizer under the same forecasting architecture (Appendix D.2‾\underline{\text{Appendix D.2}}).

Across seven datasets and two horizons, KAR lowers mean MSE in 12/14 settings and MAE in 10/14 relative to Identity (Appendix D.2, Table 12‾\underline{\text{Appendix D.2, Table 12}}). It also improves MidBlock KNN-FC and LTC in 11/14 settings each (Appendix D.2, Table 13‾\underline{\text{Appendix D.2, Table 13}}). We see these results as evidence that KAR can contribute useful structure beyond simple temporal smoothing. The Lorenz63 result adds an important qualification: better geometric consistency alone is not enough to judge whether that structure helps prediction.


W3. Gains over strong baselines and the Traffic result

FCIG achieves the lowest or tied-lowest average MSE and MAE on six of the seven LTSF datasets. Relative to ReNF, it reduces both average errors on all four ETT datasets and Weather, while matching both reported averages on Electricity. These gains also extend across forecasting horizons. In our experience, this consistency across datasets and horizons is non-trivial on mature forecasting benchmarks, where strong models have closely clustered errors.

Traffic is indeed an exception to FCIG's leading LTSF results (Section 4.2, Table 1‾\underline{\text{Section 4.2, Table 1}}). Although we adopt ReNF's BDO strategy, the two models differ in scale, including their block counts and predictor widths. More fundamentally, FCIG forecasts in learned coordinate space (Section 3.2‾\underline{\text{Section 3.2}}). The delay-coordinate construction and KAR introduce a dynamical prior into this representation, whose temporal geometry and frequency content can be explicitly analyzed (Sections 3.1, 3.3, and 4.3‾\underline{\text{Sections 3.1, 3.3, and 4.3}}).

The controlled Traffic comparison at H=96H=96 further shows that KAR improves forecasting MSE/MAE and both neighborhood metrics relative to Identity within the same architecture (Section 4.5, Table 4‾\underline{\text{Section 4.5, Table 4}}). This supports the value of the dynamical constraint in this setting. Taken together, our aim is to combine competitive forecasting with a structured, analyzable representation, rather than improve prediction accuracy alone.

We have revised the Introduction and Sections 3.1–3.3 and 4.5‾\underline{\text{Introduction and Sections 3.1–3.3 and 4.5}} to make this research goal and the evidence supporting it clearer.

Thanks for your feedback. I have raised my score.

Thank you for your careful review and for taking the time to reconsider your rating. We sincerely appreciate your support, which means a great deal to us.

Reviewer_DTX8OpenReview ↗
Rating
4(+2)
Confidence
2
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This paper introduces Forecasting on Convolution-Induced Geometry, a time-series forecasting architecture motivated by the observation that classical delay-coordinate methods construct representations to recover or characterize underlying dynamical systems rather than directly optimize them for forecasting. Rather than relying on predetermined delay-coordinate projections, FCIG uses learnable causal convolutions to construct coordinates optimized end-to-end for prediction and introduces Koopman regularization to encourage approximately linear evolution in the learned coordinate space. Forecasting is then performed by directly mapping historical coordinates to future representations and then decoding for predictions. Experiments on several long-term forecasting benchmarks show that this design improves forecasting accuracy, achieving the lowest average MSE and MAE on 6 of the 7 datasets.

Critical strengths
  1. The paper provides a mathematically grounded formulation for learning forecasting-oriented delay-coordinate representations.

    Equation (8) and Proposition 3.1 establish the connection between causal convolutions and classical delay-coordinate projections. Combined with auxiliary Koopman regularization, this provides a coherent framework for incorporating dynamical structure into neural forecasting without requiring recursive linear evolution for prediction.

  2. The paper provides extensive analysis of the learned representations beyond forecasting accuracy.

    The experiments combine phase-space and spectral analyses, quantitative measures of predictive and temporal consistency, multi-seed ablations, and regularization-sensitivity studies. Particularly valuable is the observation that improved dynamical consistency can worsen forecasting accuracy on Lorenz63, revealing a meaningful limitation of the proposed inductive bias.

The paper is generally well-organized and intuitive to follow.

Critical weaknesses
  1. The theoretical contribution is limited relative to the proposed geometric interpretation.

    Equation (8) expresses causal convolution as a linear projection of delayed observations, while Proposition 3.1 follows directly from the SVD identity urTH=σrvrTu_r^T H = \sigma_r v_r^T. Although this establishes a valid connection to classical delay-coordinate methods, it does not characterize the dynamical properties of the full nonlinear encoder EE. The formulation therefore provides a useful architectural motivation, but limited theoretical insight into why the learned representation should possess meaningful dynamical structure.

  2. The Koopman regularization objective admits a scale degeneracy.

    The objective in Eq. (14) satisfies LK(cZ,K)=c2LK(Z,K)\mathcal{L}_K(cZ,K) = c^2 \mathcal{L}_K(Z,K), allowing coordinate contraction to reduce the regularization loss without improving linear dynamical consistency. Appendix D.3 acknowledges this behavior. Although the paper also provides scale-independent neighborhood and spectral analyses, these do not directly establish that the learned operator KK accurately models latent evolution. Without normalized operator-consistency measurements or explicit controls for representation collapse, the extent to which KAR induces meaningful approximately linear dynamics remains unclear.

  3. The empirical evaluation does not sufficiently isolate the proposed mechanisms.

    FCIG adopts the Boosted Direct Output architecture from ReNF, making it important to establish which improvements arise specifically from learned convolutional coordinates and Koopman regularization. However, the w/o Dyn ablation also removes KAR and changes the temporal receptive field, while the comparison against w/o KooReg still does not isolate the claimed advantage of learned delay-coordinate projections. The absence of matched fixed-projection baselines, alternative temporal regularizers, and direct operator evaluations leaves the distinctive contributions insufficiently validated, particularly given the small improvements over ReNF on several benchmarks.

Questions for authors
  1. Does KAR induce meaningful approximately linear dynamics independently of coordinate scaling? Given the scale degeneracy of Eq. (14), can the authors report normalized held-out one-step and multi-step prediction errors under the learned operator K, ideally compared with a fixed identity transition, alongside quantitative coordinate variance or effective-rank measurements? This would help distinguish improved dynamical consistency from coordinate contraction or trivial persistence.

  2. What improvement is specifically attributable to learning forecasting-oriented delay-coordinate projections? Can the authors compare against fixed Hankel-SVD projections within the same forecasting architecture, using matched temporal receptive fields and comparable downstream prediction capacity? This would help isolate the benefit of learning the projection itself from that of temporal convolution more generally.

  3. Does Koopman regularization offer advantages over simpler temporal regularization objectives? Can the authors compare KAR against fixed-identity transition regularization or first-order temporal smoothness penalties under the same ReNF-based forecasting architecture, with appropriately tuned regularization weights? This would help determine whether the gains arise specifically from learning a nontrivial linear transition operator rather than encouraging smooth coordinate evolution.

4 comments
AuthorsOpenReview ↗

Response to Reviewer DTX8 (1)

We thank the reviewer for recognizing the formulation and representation analyses. The suggested controlled experiment has been included and has significantly improved the solidness of the manuscript. In particular, we have added matched coordinate and regularization controls, direct transition diagnostics, and a clearer statement of the theoretical scope.

W1. Scope and role of the theoretical connection

Proposition 3.1 identifies fixed Hankel-SVD coordinates as a special case of the linear convolutional parameterization. Its role is to connect classical coordinate construction to a representation that can be optimized jointly with forecasting. Section 3.1 explicitly separates this linear result from the full nonlinear encoder. The latter's predictive and temporal properties are assessed empirically rather than inferred from the SVD identity.

This distinction leads to a testable question: does adapting the coordinate projection to forecasting improve upon keeping it fixed? The new matched comparison indicated below directly examines this question within the same prediction architecture.


W2 & Q1. Scale-independent transition accuracy and persistence

We agree that the raw KAR loss alone cannot separate transition improvement from coordinate contraction. Appendix D.3 (Table 14)‾\underline{\text{Appendix D.3 (Table 14)}} now evaluates the saved matrices on 512 held-out test windows, using unpadded history positions and no operator refitting. For one- and two-step transitions, we report

\frac{\sum \lVert A^q z_t-z_{t+q\Delta}\rVert_2^2} {\sum \lVert z_{t+q\Delta}-\bar{z}_{\mathrm{train}}\rVert_2^2}, \qquad q\in\lbrace 1,2\rbrace,$$ where the sums pool valid test transitions and the reference mean is estimated from training data. Uniform rescaling multiplies both numerator and denominator by the same factor, leaving this metric unchanged. $\underline{\text{Appendix D.3, Table 14}}$ provides two distinct comparisons: - **KAR versus No KAR:** mean normalized transition error is lower with KAR in all 56 comparisons: seven datasets, two horizons, two blocks, and two transition steps ($\underline{\text{Appendix D.3, Table 14}}$). - **The saved matrix versus persistence on the same KAR coordinates:** $K$ outperforms $I$ in 10/14 MidBlock settings and 4/14 ForeBlock settings, at both transition steps ($\underline{\text{Appendix D.3, Table 14}}$). The four ForeBlock settings are precisely the Electricity and Traffic configurations with operator updates; in these configurations, $K$ outperforms persistence in both blocks ($\underline{\text{Appendix D.3, Table 14}}$). The ETT and Weather configurations instead train the coordinates against fixed matrices, as specified in $\underline{\text{Appendix B.2, Table 6}}$. Thus, the evidence supports improved relative transition consistency and identifies where the matrix also improves upon persistence. To examine changes in coordinate scale and whether the variation becomes concentrated in fewer directions, we also measured coordinate variance and effective-rank fraction on the same diagnostic checkpoints. The latter is the entropy-based effective rank of the centered covariance divided by coordinate dimension $D$. Representative MidBlock results at $H=96$, averaged over variates and three seeds, are: | Dataset | Variance: No KAR → KAR | Effective rank / $D$: No KAR → KAR | |---|---:|---:| | ETTh1 | 0.44016 → 0.03528 | 0.1592 → 0.2758 | | Weather | 0.23994 → 0.00548 | 0.0403 → 0.1504 | | Traffic | 0.08489 → 0.01718 | 0.5097 → 0.3602 | Coordinate amplitudes contract, while effective rank can increase or decrease. These measurements distinguish contraction from changes in dimensional diversity; our evidence for improved transition consistency rests on the normalized errors, not on smaller coordinate variance.
AuthorsOpenReview ↗

Response to Reviewer DTX8 (2)

W3 & Q2. Isolating forecasting-oriented coordinate learning

Section 4.5 (Table 3) and Appendix D.1 (Table 10)‾\underline{\text{Section 4.5 (Table 3) and Appendix D.1 (Table 10)}} add fixed Hankel-SVD and learnable linear projections within the same ReNF-based forecasting architecture. Both start from the same training-set SVD filters and share the delay window, coordinate dimension, downstream architecture and initialization, and training sample order. KAR is disabled. The fixed variant freezes only the SVD filters; shared preprocessing and forecasting modules remain trainable.

Across six datasets and two horizons, learning the linear projection lowers both mean MSE and MAE in seven of twelve settings (Appendix D.1, Table 10‾\underline{\text{Appendix D.1, Table 10}}). On Weather, MSE decreases by 3.45% and 2.80% at H=96H=96 and H=336H=336 (Section 4.5, Table 3‾\underline{\text{Section 4.5, Table 3}}). The nonlinear convolutional reference improves mean MSE in eleven settings and MAE in ten relative to fixed SVD (Appendix D.1, Table 10‾\underline{\text{Appendix D.1, Table 10}}).

The matched linear comparison shows that forecasting supervision can improve coordinate construction without changing its temporal receptive field or downstream prediction capacity. It supplies evidence separate from the original temporal-convolution ablation.


W3 & Q3. KAR versus a matched temporal-smoothness control

We added an identity-transition regularizer that fixes K=IK=I, encouraging coordinates at the matched temporal offsets to remain similar. It retains KAR's offsets, warm-up, and gradient-detachment schedule. The forecasting architecture and coordinate dimensions are unchanged within each comparison. For each setting, both regularizers are independently tuned over {0.005,0.05,0.5}\{0.005,0.05,0.5\} using validation MAE at seed 2021, with the selected weights then fixed across the repeated runs (Appendix D.2‾\underline{\text{Appendix D.2}}).

Across seven datasets and two horizons, KAR achieves lower mean MSE in 12/14 settings and MAE in 10/14 compared with Identity (Appendix D.2, Table 12‾\underline{\text{Appendix D.2, Table 12}}). It also improves MidBlock KNN-FC and LTC in 11/14 settings each (Appendix D.2, Table 13‾\underline{\text{Appendix D.2, Table 13}}). At H=96H=96, ETTh2, ETTm1, and Traffic improve on all four metrics (Section 4.5, Table 4‾\underline{\text{Section 4.5, Table 4}}).

These comparisons support the value of KAR's non-identity transition constraint beyond simply encouraging temporal smoothness. Here Identity is a separately trained regularization control; in Appendix D.3, Table 14‾\underline{\text{Appendix D.3, Table 14}}, identity is instead evaluated on the same saved coordinates. Together, the two tests distinguish the usefulness of the training constraint from the accuracy of the matrix as a transition model.

I appreciate the authors' efforts in addressing my comments and revising the manuscript.

I still have two questions. First, what motivates the use of fixed transition matrices for ETT and Weather, but alternating coordinate/operator updates for Electricity and Traffic? Second, since K remains fixed at its initialization in the fixed-mode configurations, how sensitive are the resulting forecasting performance and coordinate geometry to the particular initialization of K? Do independently initialized matrices produce comparable results?

Regarding presentation, the revisions have improved the manuscript's technical clarity and experimental rigor. However, Section 4.5 feels somewhat disconnected from the original narrative, and the main text could better summarize the aggregate findings of the additional controlled experiments rather than relying so heavily on the appendices. I encourage the authors to address these minor presentation issues in the final version.

Nevertheless, the additional experiments substantially strengthen the empirical justification for the proposed mechanisms and address my primary concerns regarding the original evaluation. I am therefore raising my recommendation to 4. Although the questions above remain, they do not change my overall positive assessment. I believe the revised manuscript meets the standard for acceptance.

Thank you for your careful review and for taking the time to reconsider your rating. We sincerely appreciate your support and your suggestions for improving the final manuscript.

1. Choice of fixed versus alternating updates

To be honest, the choice grew out of our experiments. We first tried fixed K and found that it already gave good forecasting results. While working with the code, we then explored what happened when we detached K or the future coordinate targets in the constraint. The motivation for alternation came from Traffic: the two update modes showed different convergence behavior, and their relative behavior varied with the forecasting horizon. This led us to try alternating the coordinate-update and target-detached operator-update modes.

We then tested this idea across datasets, selecting fixed or alternating updates according to validation performance. This led to fixed matrices for ETT and Weather and alternating updates for Electricity and Traffic.

2. Initialization of the fixed matrices

We initially used orthogonal initialization for both blocks. Our motivation was to give the coordinate constraint a norm-preserving reference, so repeated application of the linear map would not itself amplify or shrink the state. As the experiments progressed, however, we found that Xavier uniform initialization sometimes gave better forecasting results in the MidBlock.

These observations encouraged us to give the two blocks more distinct roles, rather than require the same configuration in both. We retained orthogonal initialization in the ForeBlock and Xavier uniform initialization in the MidBlock, while allowing them to learn complementary temporal patterns from their different inputs.

As for geometry, our experience has been that changing network settings, including K initialization, can change the shape of the learned coordinate projection even when forecasting errors remain close. We therefore do not expect independently trained models to produce one unique projected shape. Instead, we compare their predictive relevance and local transition consistency through the geometric metrics.

3. Integration of Section 4.5

We agree that Section 4.5 should connect more clearly to the preceding analysis. We will revise it to explain how the component ablations lead to two further questions: whether forecasting should shape the coordinate basis, and whether KAR adds value beyond an identity transition.

We will also consider how to summarize the findings of full coordinate comparison and regularization comparison, including the aggregate forecasting, geometry, and scale-normalized transition results. We really appreciate your careful reading and instructive suggestions, this will make the experimental argument easier to follow in the main text.

Reviewer_XJDaOpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes FCIG, which learns causal convolutional coordinates directly from the forecasting objective and adds a Koopman regularizer (KAR) that encourages approximately linear transitions between them, while an MLP on ReNF's two-block structure produces the forecasts directly. It evaluates on seven standard forecasting benchmarks, against dynamics-oriented forecasters, and on controlled dynamical systems, with analyses of the learned coordinates. FCIG is the best or tied-best model on most benchmarks and outperforms the dynamics-oriented baselines in nearly all settings.

Critical strengths
  1. The forecasting results are strong. FCIG achieves the best or tied average MSE and MAE on most of the datasets, and leads in majority of settings against the dynamics-oriented baselines.
  2. The connection between convolution and delay-coordinate projection is clearly explained. The paper correctly identifies Hankel-SVD coordinates as a special case of the linear construction.
  3. The ablations and neighborhood metrics provide useful evidence beyond forecasting accuracy. The metrics exclude temporally adjacent neighbors to reduce correlations from overlapping windows.
  4. The analysis acknowledges that better geometric metrics need not improve forecasts, including the adverse effect of KAR on Lorenz63.
Critical weaknesses
  1. Eq. 14 can decrease simply by shrinking all ztz_t: halving them divides LK\mathcal{L}_K by four without improving relative transition accuracy. Appendix D.3 acknowledges this effect. The discussion should distinguish changes in coordinate size from evidence of more accurate linear dynamics.

  2. The w/o Dyn ablation removes temporal convolution, so it supports the value of temporal aggregation. It does not establish that learning the coordinate basis improves on fixed Hankel-SVD coordinates. Qualifying the interpretation of this ablation would make the contribution clearer.

  3. Section 3.3 describes both fixed and updated KK, but the main results do not identify which mode was used. Please state the mode, sampling interval Δ\Delta, and update schedule so the reported results can be better understood and reproduced.

Minor presentation issue: Many referenced papers including some of the famous ones are cited using Arxiv citation. Please replace them with their proper citations (journal/conference).

Questions for authors

See weaknesses

1 comment
AuthorsOpenReview ↗

Response to Reviewer XJDa

We thank the reviewer for the encouraging assessment of our work and the constructive suggestions. We have added controlled experiments and clarified the implementation details to address the three concerns.

W1. Coordinate scale versus transition accuracy

We agree that the raw KAR loss depends on coordinate scale: uniformly rescaling the coordinates by cc multiplies the loss by c2c^2. A smaller raw loss therefore does not, by itself, demonstrate more accurate relative transitions.

To examine this distinction, we added a dedicated held-out evaluation in Appendix D.3, Table 14‾\underline{\text{Appendix D.3, Table 14}}. We normalize the squared transition residual by the target coordinates’ squared deviation from their training-set mean. Both terms scale by c2c^2, making the resulting metric invariant to uniform coordinate rescaling. We evaluate the saved Koopman matrices without refitting them.

Across seven datasets, two forecasting horizons, both blocks, and one- and two-step coordinate transitions, KAR achieves lower mean normalized transition error than No KAR in all 56 comparisons (Appendix D.3, Table 14‾\underline{\text{Appendix D.3, Table 14}}). These results support improved relative linear-transition consistency, beyond a reduction in coordinate magnitude. The revised discussion explicitly distinguishes this evidence from changes in the raw regularization loss.


W2. Learning the coordinate projection versus using temporal convolutions

We agree that the original temporal-convolution ablation does not isolate the benefit of learning a projection over using a fixed Hankel-SVD basis. We have clarified its interpretation and added a separate coordinate-construction comparison in Section 4.5, Table 3‾\underline{\text{Section 4.5, Table 3}} and Appendix D.1, Table 10‾\underline{\text{Appendix D.1, Table 10}}.

The key comparison uses fixed Hankel-SVD coordinates and learnable linear coordinates initialized from the same training-set SVD basis. The two variants share the delay window, coordinate dimension, downstream architecture and initialization, and training sample order for each seed. KAR is disabled. In the fixed variant, only the SVD filters are frozen; the shared preprocessing and forecasting modules remain trainable.

Across six datasets and two horizons, learning the linear projection improves both mean MSE and MAE in seven of twelve settings (Appendix D.1, Table 10‾\underline{\text{Appendix D.1, Table 10}}). On Weather, MSE decreases by 3.45% and 2.80% at H=96H=96 and H=336H=336, respectively (Section 4.5, Table 3‾\underline{\text{Section 4.5, Table 3}}). The nonlinear convolutional coordinates additionally improve mean MSE in eleven settings and MAE in ten relative to fixed SVD (Appendix D.1, Table 10‾\underline{\text{Appendix D.1, Table 10}}). These comparisons provide direct evidence for adapting coordinate construction to forecasting supervision, separately from the effect of KAR.


W3. Fixed and alternating Koopman updates

We have clarified the update rules in Section 3.3‾\underline{\text{Section 3.3}}. Appendix B.2, Table 6‾\underline{\text{Appendix B.2, Table 6}} specifies which datasets use fixed or alternating updates, the temporal offsets, and the horizon-specific regularization warm-up schedules.


Minor comment about bibliography

Thank you for pointing this out. We have updated the bibliography to cite the published conference or journal versions where available.

Comments on the paper

1 comment

We would like to thank the reviewers and chairs for their time and thoughtful feedback on our work.

This paper asks: Can we bring the coordinate-based view of dynamical systems into neural forecasting? Can these coordinates be learned for prediction while remaining analyzable? Inspired by the connection between delay embedding and Koopman operator theory, we view causal convolution as a learnable projection of delay observations, with Hankel-SVD as a linear special case. Forecasting supervision adapts these coordinates to the task, while an auxiliary Koopman constraint encourages approximately linear temporal evolution.

The distinctive contribution is the connection between how coordinates are constructed, how they are trained for prediction, and how their temporal structure can be examined. The resulting framework provides a simple and effective bridge between neural forecasting networks and dynamical-systems methods. Its explicit coordinates provide a basis for analyzing and interpreting the temporal structure learned by the model. More broadly, this perspective may inform future work on state-space reconstruction from observations and on how prediction objectives shape the geometry and topology of neural representations.

We thank the reviewers for recognizing the following strengths of our work:

  1. A principled connection to classical coordinate methods. DTX8 described the work as providing “a mathematically grounded formulation for learning forecasting-oriented delay-coordinate representations.” XJDa and Edqe also recognized the connection between causal convolution and Hankel-SVD coordinates.

  2. Evaluation beyond forecasting accuracy. XJDa highlighted the ablations and neighborhood metrics as useful evidence beyond forecasting errors. DTX8 also valued the combination of predictive and temporal consistency measures, multi-seed ablations, and regularization-sensitivity studies.

  3. Insights into learned representations. Edqe and DTX8 recognized the geometric and spectral analyses for examining the temporal structure learned by the model. XJDa and DTX8 further appreciated the distinction between geometric consistency and predictive accuracy, which helps assess when dynamical structure is useful for forecasting.

Our responses and changes during rebuttal

  • Forecasting-oriented coordinate learning (XJDa, DTX8): We added matched fixed-SVD and learnable-linear comparisons with KAR disabled. Improvements in most settings support adapting coordinate construction to forecasting supervision (Section 4.5, Table 3; Appendix D.1, Table 10‾\underline{\text{Section 4.5, Table 3; Appendix D.1, Table 10}}).

  • KAR beyond temporal smoothing (DTX8, Edqe): We added a matched identity-transition control with independently tuned regularization weights. KAR improves forecasting and MidBlock geometry metrics in most settings for each metric, supporting benefits beyond temporal smoothing (Section 4.5, Table 4; Appendix D.2, Tables 12–13‾\underline{\text{Section 4.5, Table 4; Appendix D.2, Tables 12–13}}).

  • Transition consistency beyond coordinate shrinkage (XJDa, DTX8): Held-out, scale-normalized diagnostics show lower mean transition errors with KAR than without it in both blocks across all evaluated settings. These results support improved relative transition consistency beyond uniform coordinate contraction (Appendix D.3, Table 14‾\underline{\text{Appendix D.3, Table 14}}).

We also clarified the update settings (Section 3.3; Appendix B.2, Table 6‾\underline{\text{Section 3.3; Appendix B.2, Table 6}}) and improved the connection between the controlled experiments and the main narrative in Section 4.5‾\underline{\text{Section 4.5}}.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.