acceptodds
Under review as a conference paper at ICLR 2027

Insight Precedes Action: A Geometric Rethinking of Time Series Forecasting

Abstract

Time series forecasting has advanced rapidly, yet most approaches rely on end-to-end optimization to discover predictive structure in the training data, leaving its potential as an explicit prior underexplored. Meanwhile, attention-based models typically determine cross-channel interaction through query–key similarity, without directly considering the value representations being aggregated. These considerations lead to two key questions: (i) how can predictive structure be extracted directly from the training data and provided to the model as prior knowledge, and (ii) should the information being aggregated also contribute to determining cross-channel interaction strength? To address these questions, we introduce Navigator, which integrates Foreseer and BendingFormer to rethink forecasting through two complementary geometric structures. A Predictability Basis is first constructed from past–future statistical relationships in the training data by identifying linear directions in the historical observation space with strong predictive relevance. Foreseer then uses this basis to re-express historical observations in a predictability-oriented coordinate system for subsequent modeling. BendingFormer introduces Geometric Bending Attention (GBA), which measures the directional alignment between query-to-key and key-to-value transitions, allowing value representations to directly inform cross-channel interaction strength. Together, these components embody the principle that insight precedes action: predictive structure extracted before training guides temporal representation, while the joint query–key–value geometry guides information aggregation. Extensive experiments on twelve real-world datasets demonstrate state-of-the-art forecasting performance with consistent results across random seeds, while comprehensive analyses support the effectiveness of its two core designs.

Primary area:
learning on time series and dynamical systems
Keywords:
Time Series Forecasting, Predictability Basis

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_oa89OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper introduces a time series forecasting framework, Navigator. Its idea is that, rather than relying solely on end-to-end optimization to discover predictive patterns, a forecasting model should first extract prior knowledge about which historical information is most predictive of future observations directly from the training data. When modeling information exchange across variables, the model should not rely solely on pairwise similarity between variables, but should also consider whether the information being propagated is geometrically aligned with the underlying relational structure.

Critical strengths

s1: The authors conduct extensive experiments and analyses to demonstrate the effectiveness and robustness of Navigator.

s2: The experimental section is presented in considerable detail, providing a comprehensive evaluation of Navigator.

Critical weaknesses

w1: The paper emphasizes explicitly extracting 'predictive structure'' from the training data and using it as prior knowledge for forecasting. However, the notion of `predictive structure'' does not appear to be clearly and precisely defined. More importantly, the current experiments do not convincingly demonstrate that the performance gains are specifically attributable to exploiting this prior knowledge. In particular, the ablation studies and plug-in experiments demonstrate the effectiveness of Foreseer, but they do not sufficiently isolate the contribution of the proposed predictive structure itself.

w2: The paper appears to address two relatively independent problems: extracting predictive temporal directions from the training data and designing a geometry-aware mechanism for cross-channel interaction. However, the connection between these two problems is not sufficiently clear. It is unclear why they should be addressed jointly within a single framework, or what fundamental relationship motivates their integration. As a result, the overall motivation and narrative of the paper are somewhat difficult to follow.

w3: The performance gains of Geometric Bending Attention (GBA) over existing attention mechanisms appear to be relatively modest in the reported experiments. More importantly, the intuition behind GBA is not entirely clear. In particular, the paper proposes to assess the relationship between the query and key representations while also determining whether the information carried by the value representation is directionally compatible with the corresponding interaction. It is not immediately clear (lines 444) how such directional compatibility should be interpreted or why the geometric relationship between the query--key and key--value transitions provides a meaningful criterion for determining interaction strength.

w4: The paper discusses several limitations of the proposed approach, including the reliance on second-order statistics, the fixed nature of the Predictability Basis after training, and the limited modeling of higher-order joint relationships. However, it is unclear whether these limitations are supported by substantive empirical evidence.

Questions for authors

See the Weaknesses.

5 comments
AuthorsOpenReview ↗

Response to Reviewer oa89 (Part 1)

Dear Reviewer oa89:

We sincerely thank you for your constructive feedback. We have carefully addressed your concerns and revised the manuscript accordingly. Our point-by-point responses are provided below.

W1. Clarification of Predictive Structure and Additional Experimental Evidence

A1:

(i) Clarification of Predictive Structure.

In our paper, predictive structure refers to the statistical relationships between historical and future observations, particularly which temporal directions in the historical observation space are more informative about future observations. To characterize this structure, we formally define Temporal Predictability in Definition 1 (Section 3.2) and establish its closed-form characterization in Proposition 1. Based on this theoretical foundation, Section 4.2.1 provides a detailed procedure for constructing the Predictability Basis by identifying and retaining temporal directions with stronger predictive relevance from the training data. Therefore, the Predictability Basis serves as an explicit, data-derived representation of the predictive structure, which Foreseer subsequently incorporates into the forecasting model as prior knowledge.

(ii) Additional Experimental Evidence.

To more directly isolate the contribution of the extracted predictive structure, we have conducted additional controlled experiments, including comparisons with alternative temporal bases and analyses of the learnable biases.

First, we compare the proposed Predictability Basis with four alternative bases, including Random Orthogonal Basis, PCA Basis, DCT Basis, and Identity Basis, under the same forecasting architecture and experimental settings. Unlike the proposed Predictability Basis, these alternatives do not explicitly exploit historical–future statistical relationships: Random Orthogonal Basis uses randomly generated orthogonal directions, PCA Basis captures variance-dominant directions, DCT Basis employs predefined frequency-domain directions, and Identity Basis preserves the original temporal representation. The results show that the proposed Predictability Basis consistently achieves better forecasting performance, providing further evidence of the benefits of explicitly exploiting predictive structure from the training data. For detailed results, please refer to A1(iii) to Reviewer EM36 and A1(v) to Reviewer fvpJ.

Second, we further investigate the role of the learnable biases in Foreseer to distinguish their contribution from that of the Predictability Basis. The corresponding analysis and experimental results are provided in A1(iv) to Reviewer fvpJ.

Taken together, these additional experiments further support that Foreseer's performance improvements arise from the forecasting-relevant information captured by the Predictability Basis, rather than merely from temporal projection or additional learnable parameters, directly addressing your concern.

W2. Motivation for Integrating Foreseer and BendingFormer

A2:

(i) Motivation and Connection.

The key motivation for integrating Foreseer and BendingFormer is to enable geometry-aware cross-channel interaction over forecasting-relevant temporal representations. Specifically, Foreseer constructs a shared Predictability Basis from historical–future statistical relationships, providing a common predictability-oriented representation across channels. BendingFormer then employs GBA to model interactions among these representations based on the joint query–key–value geometry. Thus, Foreseer determines what temporal information is used for cross-channel modeling, while GBA determines how that information is exchanged across channels.

(ii) Additional Experimental Evidence.

To further examine whether Foreseer and GBA provide complementary benefits, we conduct additional controlled experiments comparing Full Attention and GBA under two temporal representations: a linear projection of raw inputs and the representation produced by Foreseer. The results (MSE/MAE), averaged over four prediction horizons, are summarized below.

Temporal RepresentationFull AttentionGBA
Linear Projection of Raw InputsAB
Foreseer RepresentationCD
Model variantsABCDΔGBAForeseer =C−D\Delta_{\mathrm{GBA}}^{\text {Foreseer }}=C-DΔGBAOriginal=A−B\Delta_{\mathrm{GBA}}^{\text {Original}}=A-B
Weather0.244/0.2660.242/0.2630.243/0.2640.237/0.2590.006/0.0050.002/0.003
Solar-Energy0.213/0.2250.203/0.2200.213/0.2210.192/0.2120.021/0.0090.010/0.005

As shown, GBA achieves larger improvements over Full Attention when combined with Foreseer, as evidenced by ΔGBAForeseer >ΔGBAOriginal \Delta_{\mathrm{GBA}}^{\text {Foreseer }}>\Delta_{\mathrm{GBA}}^{\text {Original }}. These results provide empirical evidence of the inherent complementarity between Foreseer and BendingFormer, demonstrating that their integration yields greater benefits than their individual contributions alone.

AuthorsOpenReview ↗

Response to Reviewer oa89 (Part 2)

W3. Performance Improvements and Geometric Motivation of GBA

A3:

(i) Performance Improvements.

As shown in Table 4, GBA outperforms standard Full Attention across all four evaluated datasets, achieving an average relative MSE reduction of 3.66%. These improvements are obtained under the same forecasting architecture and experimental settings, with only the attention mechanism replaced. Although the magnitude of improvement varies across datasets, the results consistently support the effectiveness of GBA.

(ii) Geometric Interpretation and Motivation.

Standard attention determines interaction strengths based on query–key similarity, while the information ultimately aggregated is carried by the value representation. Consequently, query–key similarity alone does not account for the geometric relationship between the matching process and the information being aggregated.

GBA addresses this by considering the ordered configuration qi→kj→vj\boldsymbol{q}_i \to \boldsymbol{k}_j \to \boldsymbol{v}_j, where the query-to-key transition represents the relationship between the target and source channels, and the key-to-value transition describes the displacement toward the information being aggregated. Directional compatibility measures whether the key-to-value transition continues or deviates from the direction established by the query-to-key transition. By favoring directional continuity, GBA introduces a geometric inductive bias that jointly considers Q–K, Q–V, and K–V relationships when determining interaction strengths.

(iii) Additional Experimental Evidence.

To further examine whether GBA's improvements arise from its specific geometric scoring criterion rather than simply using cosine similarity or incorporating value representations into attention scoring, we conduct additional controlled experiments comparing GBA with Q–K Cosine Attention and Q–V Cosine Attention under the same architecture and experimental settings.

ETTm1WeatherSolar-Energy
GBA0.373/0.3760.237/0.2590.192/0.212
Q–K Cosine Attention0.380/0.3810.245/0.2710.204/0.220
Q–V Cosine Attention0.377/0.3800.243/0.2680.204/0.219

(Results are reported as MSE/MAE, averaged over four prediction horizons.)

As shown, GBA consistently outperforms both alternatives across all three datasets in terms of MSE and MAE. These results indicate that GBA's improvements cannot be attributed solely to cosine-based scoring or the inclusion of value representations in attention scoring, providing direct empirical support for the proposed directional compatibility criterion.

Overall, the geometric interpretation and controlled experiments support GBA as an effective value-aware attention mechanism, with its advantages arising from modeling the ordered query–key–value configuration rather than relying on conventional pairwise similarity alone.

W4. Clarification of the Limitations

A4:

(i) Reliance on Second-Order Statistics.

As described in Sections 3.2 and 4.2.1, the Predictability Basis is constructed from covariance and cross-covariance matrices, which characterize linear statistical relationships between historical and future observations. Therefore, the basis does not explicitly capture nonlinear predictive dependencies that cannot be characterized by second-order statistics. However, this limitation concerns the basis construction rather than the entire Navigator model, which incorporates nonlinear transformations in subsequent layers.

(ii) Fixed Predictability Basis.

As described in Section 4.2.1, the Predictability Basis is constructed by aggregating historical–future statistical relationships across the training data, which may already reflect diverse non-stationary patterns. Although the basis remains fixed, it produces input-dependent representations for different historical windows. Its potential limitation arises when previously unseen predictive relationships emerge, as the basis cannot explicitly update its temporal directions to accommodate such changes.

(iii) Higher-Order Cross-Channel Relationships.

As defined in Eq. (4), GBA computes geometric compatibility scores from individual query–key–value configurations associated with channel pairs. However, the subsequent softmax normalization and weighted aggregation involve all channels, while stacked BendingFormer layers can further capture complex cross-channel dependencies. Therefore, the limitation concerns the absence of an explicit higher-order scoring mechanism involving three or more channels simultaneously, rather than an inability to model higher-order dependencies.

Overall, these limitations reflect the theoretical and architectural boundaries of our current design, rather than experimentally demonstrated shortcomings. We will clarify this distinction in the revised manuscript to avoid overstating their practical implications.

Thanks for the detailed response and the additional experiments. I retain Weak rejection.

Reviewer_fvpJOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Navigator combines Foreseer, a fixed temporal basis estimated from training-set past–future covariance, with Geometric Bending Attention (GBA), which scores channel interactions using query–key–value chord alignment. Learned offsets adapt the basis-based representation and reconstruction. The paper reports results on twelve datasets, component and plug-in experiments, five-seed comparisons, and efficiency measurements.

Critical strengths
  1. The design gives a clear role to both statistical structure and value-aware interaction. Foreseer supplies a precomputed prior without an auxiliary pretraining network, and GBA lets the content being aggregated influence attention. Both components can be added to existing forecasters.

  2. The evaluation is broad, with twelve datasets, multiple horizons, plug-in experiments and five-seed results. The reported improvements across backbones make this more convincing than a gain confined to one architecture.

Critical weaknesses
  1. The novelty and contribution of the predictability basis are not sufficiently isolated. The single-channel correlation objective is closely related to classical CCA; the pooled, regularized construction and its use in forecasting need a clearer distinction. OLinear (NeurIPS 2025) is especially relevant because it also forecasts in a data-derived orthogonal temporal domain. It is cited for the loss, but the methodological distinction and a direct comparison are missing. The ablations leave a related attribution gap: in Table 2, removing the learned offsets is worse than removing Foreseer on four of six datasets. Figures 6–7 also include full-rank settings, where no direction is discarded. These observations do not invalidate Foreseer, but they do not establish that selecting predictive directions explains its gains. Identity, random-orthogonal or PCA/DCT controls should retain the same offsets and training setup.

  2. The experimental protocol is not fully traceable from the supplied materials. The code loads precomputed bases without their generation procedure and defaults to horizon-weighted L1 training. Appendix B.3 mentions a weighted loss, but the main and plug-in comparisons need explicit objective-matched configurations. The channel-wise code branch also omits the two offsets; if used for Table 2, it changes more than the basis construction. Please specify basis preprocessing, ranks and split boundaries, and clarify which comparisons preserve the loss and offsets. These are unresolved controls, not evidence of leakage or invalid results.

References

[1] Yue et al. OLinear: A Linear Model for Time Series Forecasting in Orthogonally Transformed Domain. NeurIPS 2025. https://proceedings.neurips.cc/paper_files/paper/2025/file/571c7e164fb1ffbcf2f84a63784451ec-Paper-Conference.pdf

Questions for authors
  1. How does Foreseer differ from CCA-based predictive projections and OLinear? Can you provide basis-matched controls that retain both offsets, particularly for full-rank settings, and confirm whether the channel-wise ablation retained them?

  2. What preprocessing and training split generated each basis, and what exact loss and horizon weighting were used for each main and plug-in comparison? Which gains remain when these settings are matched?

5 comments
AuthorsOpenReview ↗

Response to Reviewer fvpJ (Part 1)

Dear Reviewer fvpJ:

Thank you for your careful review and insightful comments. We appreciate your constructive suggestions, which have helped us improve our paper. We have carefully considered the issues you raised and revised the manuscript accordingly. Please find our detailed responses below.

W1&Q1.

A1:

(i) Connection to CCA and Novelty of Predictability Basis. Please refer to A1 to Reviewer EM36.

(ii) Beyond CCA: What Else Does Foreseer Do for Forecasting? Beyond the CCA-related formulation, Foreseer incorporates three additional designs: (1) Ridge regularization to ensure numerical stability; (2) a channel aggregation strategy to construct a shared Predictability Basis for multivariate time series, enabling the model to capture common temporal predictive patterns; and (3) two learnable channel-specific bias terms, Δ1\boldsymbol{\Delta}_{1} and Δ2\boldsymbol{\Delta}_{2}, introduced in Prospector and Reconstructor, respectively, to enhance modeling flexibility. The ablation results in Table 2 further demonstrate the effectiveness of the channel aggregation strategy and the learnable bias terms.

(iii) Distinction from OLinear.

(1) Motivation and Methodology. OLinear aims to decorrelate temporal dependencies by eigendecomposing the historical Pearson correlation matrix. In contrast, Navigator explicitly exploits historical–future statistical relationships to construct a Predictability Basis, identifying temporal directions that are informative about future observations. Furthermore, OLinear employs separate orthogonal matrices for input and output transformations, whereas Foreseer uses a shared Predictability Basis in Prospector and Reconstructor, together with learnable channel-specific biases.

(2) Forecasting Performance. We compare Navigator with OLinear under the same forecasting settings, using the results reported in Table 15 of the original OLinear paper. As shown in the table below (MSE/MAE), Navigator outperforms OLinear on 16 out of 24 metrics, compared with 6 for OLinear, with the remaining 2 being ties.

ETTh1ETTh2ETTm1ETTm2ElectricityTrafficWeatherSolar-EnergyPEMS03PEMS04PEMS07PEMS08
Navigator (ours)0.422/0.4240.364/0.3870.373/0.3760.271/0.3120.161/0.2500.440/0.2490.237/0.2590.192/0.2120.091/0.1930.081/0.1800.069/0.1560.123/0.220
OLinear0.424/0.4240.367/0.3880.374/0.3770.270/0.3130.159/0.2480.451/0.2470.237/0.2600.215/0.2170.095/0.1990.091/0.1900.077/0.1640.113/0.194

(Results are averaged over 4 prediction horizons. Full comparisons are provided in Tables 17 and 18 of the revised manuscript.)

(3) Visualization of Orthogonal Bases. As shown in Figure 7 of OLinear paper, its orthogonal matrices (Q Mat) exhibit noticeably different patterns from the Predictability Bases visualized in Figure 7 of our paper. This qualitative difference is consistent with their distinct construction objectives: temporal decorrelation in OLinear versus historical–future predictability in Navigator.

We have added Appendix C.8, "Comparison with OLinear," to the revised manuscript to provide a detailed discussion of the differences between Navigator and OLinear.

AuthorsOpenReview ↗

Response to Reviewer fvpJ (Part 2)

A1:

(iv) Contribution of Learnable Biases and Their Interaction with the Predictability Basis.

The shared Predictability Basis captures common temporal predictive patterns across channels, while the learnable channel-specific biases allow the model to accommodate individual channel characteristics. Without these biases, the shared basis may not fully adapt to channel-specific variations, which could explain why removing the biases sometimes causes a larger performance drop than removing Foreseer entirely.

To further investigate this, we additionally evaluate the "r Channel-wise" variant with both learnable biases retained. As shown in the table below (MSE/MAE), with the lookback length fixed at T=96T=96, the complete Foreseer (original) consistently outperforms both channel-wise variants (with and without learnable biases) across all four evaluated datasets. These results demonstrate that the choice of Predictability Basis contributes to forecasting performance beyond the learnable biases alone. These results support our design of combining a shared Predictability Basis with channel-specific adjustments.

Prediction Horizonr Channel-wise (w/o Learnable Biases)r Channel-wise (w Learnable Biases)original
Weather960.155/0.1940.156/0.1940.153/0.192
1920.205/0.2410.205/0.2410.200/0.235
3360.261/0.2820.265/0.2830.258/0.279
7200.340/0.3330.340/0.3330.337/0.331
Solar-Energy960.160/0.1950.168/0.1990.158/0.193
1920.185/0.2200.188/0.2210.187/0.210
3360.209/0.2290.208/0.2280.205/0.220
7200.221/0.2320.219/0.2310.218/0.226
PEMSD3120.079/0.1830.074/0.1760.065/0.162
240.098/0.2060.090/0.1970.078/0.179
480.131/0.2380.116/0.2210.096/0.201
960.162/0.2690.148/0.2520.123/0.228
PEMSD4120.075/0.1760.072/0.1710.068/0.164
240.085/0.1890.080/0.1810.076/0.173
480.099/0.2030.090/0.1920.085/0.184
960.128/0.2310.107/0.2110.096/0.197

We have added this analysis to Appendix C.9, "Effect of Channel-specific Biases in Foreseer," of the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer fvpJ (Part 3)

A1:

(v) Full-rank Settings and Controlled Basis Comparisons.

Regarding the choice of rank rr, please refer to A7(i) to Reviewer EM36. Among the 12 datasets, only Electricity and Traffic use a full-rank Predictability Basis (r=T=96r=T=96), while the remaining ten datasets employ low-rank bases (r<Tr<T). We acknowledge that the full-rank setting does not discard any temporal directions. However, it still provides a predictability-oriented coordinate system derived from historical--future statistical relationships. As shown in Figure 6, forecasting performance generally improves as rr increases from 12 to 48, with the four ETT datasets achieving their best or near-best performance before reaching full rank. These results indicate that retaining all temporal directions is not necessary for effective forecasting.

To further isolate the contribution of the Predictability Basis, we compare it with alternative bases while retaining both learnable channel-specific biases and the same training configurations. Comparisons with Random Orthogonal and PCA bases are provided in A1(iii) to Reviewer EM36. We additionally introduce Identity and DCT controls, with results reported in the table below. Replacing the Predictability Basis with an Identity Basis leads to a substantial deterioration in forecasting performance, while our Predictability Basis also consistently outperforms DCT across all six evaluated datasets. Overall, these results further support the effectiveness of our Predictability Basis and learnable channel-specific biases.

Our BasisDCT BasisIdentity Basis
Weather0.237/0.2590.242/0.2640.252/0.276
Electricity0.161/0.2500.165/0.2550.167/0.255
Solar-Energy0.192/0.2120.197/0.2180.235/0.236
Traffic0.440/0.2490.447/0.2600.446/0.263
PEMSD30.091/0.1930.099/0.2030.132/0.242
PEMSD40.081/0.1800.090/0.1910.145/0.238

(Results are reported as MSE/MAE, averaged over four prediction horizons.)

W2&Q2.

A2:

(i) Construction of Predictability Basis.

The detailed procedure for constructing the Predictability Basis is provided in Section 4.2.1. As stated in our Reproducibility Statement, the complete implementation, including the code for constructing the Predictability Basis, will be publicly released upon acceptance.

(ii) Basis Rank rr and Dataset Splits.

Regarding the choice of rr, please refer to A7(i) to Reviewer EM36. We follow the same dataset splitting protocols as those reported in Table 11 of OLinear. Importantly, the Predictability Basis is constructed exclusively from the training split, without using validation or test data.

(iii) Loss Functions and Objective-matched Comparisons.

For Navigator, all main experiments (e.g. Tables 1, 7, 8 and 9) and ablation studies (e.g. Tables 2, 4, 10, and 11) use the weighted loss function described in Appendix B.3. The baseline results in the main comparisons follow their respective original papers, consistent with the evaluation protocol adopted by OLinear.

For the plug-in experiments in Table 3, all experiments are conducted using the open-source Time Series Library (TSLib)[1] framework, with the default MSE loss applied consistently to both the original baselines and their plug-in variants. These plug-in comparisons demonstrate that the improvements brought by Navigator's components persist under identical optimization objectives, rather than relying on the weighted loss function.

(iv) Clarification of the Channel-wise Ablation.

The original "r Channel-wise" variant in Tables 2 and 10 omits the learnable channel-specific biases. To address your concern, we additionally evaluate this variant with both biases retained, as detailed in A1(iv). The complete Foreseer continues to outperform this controlled variant, demonstrating that its advantage cannot be attributed solely to the learnable biases.

[1] https://github.com/thuml/Time-Series-Library

Reviewer_EM36OpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes Navigator, which combines Foreseer and BendingFormer. Foreseer projects the input onto a fixed basis of "predictive" temporal directions obtained from a generalized eigenproblem on training-set past/future covariances, and BendingFormer replaces the attention score with the cosine between the displacements kj−qi\mathbf{k}_j-\mathbf{q}_i and vj−kj\mathbf{v}_j-\mathbf{k}_j. Experiments on 12 datasets against 23 baselines show the best average results in most cases, with ablations and a plug-in study of Foreseer.

Critical strengths

Foreseer is a cheap, training-free prior that gives consistent gains as a plug-in on iTransformer, PatchTST, and PAttn and stays stable when built from only 10% of the training data. Regardless of how it is framed, that is a reusable trick, and the evaluation around it is thorough.

Critical weaknesses
  1. Definition 1 and Proposition 1 are canonical correlation analysis between the past and future windows: Eq. (3) is the CCA eigenproblem with λ\lambda the squared canonical correlation, and the proof in A.1 is the standard derivation (Hotelling, 1936). The paper only cites linear-algebra textbooks for the Rayleigh-quotient step and never identifies the result as CCA, so the Predictability Basis is a ridge-regularized, channel-averaged CCA basis presented as a new characterization. The novelty claim should be scoped accordingly.

  2. The claim that the value "participates in determining interaction strength" does not survive expanding Eq. (4). With aij=q^i⊤k^ja_{ij}=\hat{\mathbf{q}}_i^\top\hat{\mathbf{k}}_j, bij=q^i⊤v^jb_{ij}=\hat{\mathbf{q}}_i^\top\hat{\mathbf{v}}_j, gj=k^j⊤v^jg_j=\hat{\mathbf{k}}_j^\top\hat{\mathbf{v}}_j, one gets cij=aij−bij+gj−12(1−aij)(1−gj)c_{ij}=\frac{a_{ij}-b_{ij}+g_j-1}{2\sqrt{(1-a_{ij})(1-g_j)}}: since kj,vj\mathbf{k}_j,\mathbf{v}_j are both linear in hj\mathbf{h}_j, the numerator is an ordinary dot-product score with effective key k^j−v^j\hat{\mathbf{k}}_j-\hat{\mathbf{v}}_j, and the only new part is the distance-based denominator. Table 4 has no plain cosine-attention baseline to isolate that part, and the denominator vanishes when vj→kj\mathbf{v}_j\to\mathbf{k}_j.

  3. The measured contribution of each component is small: removing Foreseer or GBA changes average MSE by roughly 0.002 to 0.008 on most datasets (Tables 2 and 4), with only PEMS and Solar as exceptions. The "15.05% MSE reduction" in the introduction is the single Solar number and is not representative.

  4. Figure 1(a) and Figure 7 do not convey what the Predictability Basis captures beyond "it differs across datasets". Since this is the central object of the paper, a more interpretable view (e.g., leading directions as filters or spectra) is needed.

  5. Lastly, a formatting note: the spacing around subsection and subsubsection headings in Section 4 is noticeably tighter than the official ICLR 2027 template (about 6 to 8pt less per boundary, compared against a compile of the unmodified style file).


[1] Hotelling, H. (1992). Relations between two sets of variates. In Breakthroughs in statistics: methodology and distribution (pp. 162-190). New York, NY: Springer New York.

Questions for authors
  1. (Clarification) Figure 7 shows 96×9696\times96 bases for Electricity and Traffic, i.e., r=Tr=T, where Prospector is an orthogonal change of basis that the following linear layer can absorb exactly. Why does Table 2 still show a gap on these datasets?

  2. (Clarification) How are rr and ϵ\epsilon chosen? They are not among the tuned hyperparameters in Appendix B.3.

For other points, please refer to the weaknesses.

7 comments
AuthorsOpenReview ↗

Response to Reviewer EM36 (Part 1)

Dear Reviewer EM36:

We sincerely thank you for your time and thoughtful feedback on our paper. Your comments have been very helpful in improving the quality and clarity of our work. We have carefully considered your concerns and revised the manuscript accordingly. Below, we provide detailed responses to each of your comments.

W1. Connection to CCA and Novelty of Predictability Basis

A1:

(i) Connection to CCA. We acknowledge that the optimization in Definition 1 and the generalized eigenvalue problem in Proposition 1 are closely related to classical Canonical Correlation Analysis (CCA) [1]. We do not claim that the underlying eigenvalue formulation or its derivation is mathematically new. We have clarified this connection in the revised manuscript.

(ii) Clarification of our contribution. The contribution of Navigator does not lie in introducing a new formulation of CCA, but in leveraging classical statistical principles to construct forecasting-relevant prior knowledge for neural models. This is analogous to Fourier Transform-based forecasting methods, such as FreTS [2], FITS [3] and FilterNet [4], where the innovation lies not in the Fourier transform itself, but in how it is utilized for forecasting. Similarly, Navigator builds upon CCA-related principles to develop a data-driven prior construction mechanism for neural forecasting.

(iii) Additional experiments on basis construction. To assess the contribution of historical–future relationships, we compared Predictability Basis with two alternatives under identical settings: (a) Random Orthogonal Basis, constructed without training data, and (b) PCA Basis, derived solely from historical variance. The results are shown below. Predictability Basis (ours) consistently outperforms both, supporting the benefit of incorporating historical–future relationships rather than relying on arbitrary orthogonal projections or variance-based representations.

Our BasisRandOrtho BasisPCA Basis
Weather0.237/0.2590.245/0.2680.242/0.266
Electricity0.161/0.2500.172/0.2570.167/0.258
Solar-Energy0.192/0.2120.209/0.2280.210/0.226
Traffic0.440/0.2490.452/0.2640.448/0.262
PEMSD30.091/0.1930.107/0.2050.105/0.208
PEMSD40.081/0.1800.099/0.1980.094/0.191

(Results are reported as MSE/MAE, averaged over four prediction horizons.)

W2. Mathematical Formulation and Effectiveness of Geometric Bending Attention

A2:

(i) Mathematical Clarification. We agree that the algebraic expansion of Eq. (4) is correct. However, rewriting the numerator using an effective key does not remove the explicit dependence of the score on Value, since the effective key itself depends on Value. Moreover, the complete GBA score jointly depends on Q–K, Q–V, and K–V relationships through the directional consistency of two consecutive geometric transitions. At the scoring-function level, changing Value while holding Query and Key fixed can alter the GBA score, unlike standard QK attention. Therefore, the expansion does not invalidate our claim that Value participates in determining interaction strength.

(ii) Additional Comparison with Cosine Attention. We conducted additional experiments comparing GBA with Plain Cosine Attention under the same forecasting architecture and training settings. The results are presented below. GBA consistently outperforms Plain Cosine Attention, providing further evidence that its geometry-aware formulation offers benefits beyond conventional query–key cosine similarity.

ETTm1WeatherSolar-EnergyTraffic
GBA (ours)0.373/0.3760.237/0.2590.192/0.2120.440/0.249
Plain Cosine Attention0.380/0.3810.245/0.2710.204/0.2200.448/0.259

(Results are reported as MSE/MAE, averaged over four prediction horizons.)

(iii) Numerical Stability. The implementation already includes safeguards against numerical degeneracy. Specifically, we apply a lower bound of 10−810^{-8} before taking the square root in both edge-length calculations, add 10−810^{-8} to the denominator, and clamp the resulting cosine scores to [−1,1][-1,1]. These operations prevent division by zero and keep the computed scores within the valid cosine range. We have added these implementation details to Appendix B.3 in the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer EM36 (Part 2)

W3. Effectiveness of Individual Components

A3:

(i) Effectiveness of Foreseer and GBA. Based on the comparison between “w Foreseer” and “w/o Foreseer” in Table 2, we further calculated the absolute and relative MSE reductions, as shown below. Foreseer achieves an average relative MSE reduction of 7.67% across six datasets.

Regarding GBA, we note that relatively small absolute MSE differences between attention mechanisms are also observed in existing studies, such as the NormLin comparisons in OLinear [5] (Table 8). Such differences alone do not necessarily indicate a limited contribution. Our Table 4 shows that GBA achieves strong performance across the evaluated datasets, while the additional comparison with Plain Cosine Attention (see A2) further supports the effectiveness of its geometric formulation.

WeatherElectricitySolar-EnergyTrafficPEMSD3PEMSD4Avg
Absolute MSE Reduction0.0050.0040.0110.0060.0150.0210.0103
Relative MSE Reduction (%)2.07%2.42%5.42%1.35%14.15%20.59%7.67%

(ii) Clarification of the 15.05% MSE Reduction. The reported 15.05% MSE reduction refers to the specific experimental setting in Figure 1(b), where using GBA reduces MSE from 0.186 with standard Full Attention to 0.158 on Solar-Energy (T=τ=96)(T=\tau=96). The original statement does not present this result as an average improvement across datasets. More importantly, the effectiveness of GBA is not limited to this particular case. As shown in Table 4, GBA outperforms standard Full Attention across all four evaluated datasets, achieving an average relative MSE reduction of 3.66%. We will replace the original statement in the Introduction with this average improvement to provide a more representative assessment of GBA's effectiveness.

W4. Interpretability of Predictability Basis

A4: We have added a new subsection, “Appendix D.3 Spectral Analysis of Predictability Basis,” to provide a more interpretable view of the temporal structures captured by the basis. Specifically, we apply FFT to each orthonormal basis direction and visualize its normalized one-sided power spectrum.

Figure 8 presents the spectral distributions of all retained basis directions across four datasets, revealing distinct frequency characteristics and varying degrees of spectral concentration. Figure 9 further examines the first six directions, showing that some exhibit pronounced narrowband peaks, while others contain broader or multiple frequency components. These additional visualizations reveal the frequency structures represented by the Predictability Basis, going beyond the coefficient-level differences shown in Figures 1(a) and 7.

W5. Formatting Compliance.

A5: We have corrected the formatting in the revised manuscript.

Q1. Effectiveness of Full-Rank Predictability Basis

A6:

(i) Structural Clarification. We agree that, when r=Tr=T, the fixed orthogonal projection in Prospector can be algebraically absorbed into the subsequent unconstrained linear layer. However, Foreseer is not merely an orthogonal transformation. It also introduces two learnable channel-specific bias terms, Δ1\Delta_{1} and Δ2\Delta_{2}, in Prospector and Reconstructor, respectively. These bias terms cannot, in general, be absorbed by the shared biases of standard linear layers. Therefore, the full Foreseer module is not equivalent to a simple orthogonal change of basis.

(ii) Evidence from Basis Comparisons. To further examine the role of the full-rank Predictability Basis, we compared it with Random Orthogonal Basis and PCA Basis under the same architecture and training settings (see A1 (iii)). Although all three bases preserve the full temporal dimension, Predictability Basis achieves better forecasting performance on Electricity and Traffic datasets. These results suggest that the data-derived choice of temporal coordinates can be beneficial even without dimensionality reduction.

(iii) Algebraic Absorbability vs. Empirical Effectiveness. Algebraic absorbability does not, by itself, guarantee identical empirical performance under independent model training. A related example is the Fourier transform, which is also an invertible transformation and can be absorbed into an appropriate unconstrained linear mapping, yet serves as the foundation of effective frequency-domain forecasting methods[2-4]. Similarly, Navigator incorporates a data-derived orthogonal transformation together with channel-specific adaptations, whose combined effectiveness is supported by the ablation results in Table 2.

AuthorsOpenReview ↗

Response to Reviewer EM36 (Part 3)

Q2. Choice of rr and ϵ\epsilon

A7:

(i) Choice of rr. For long-term forecasting (τ∈{96,192,336,720}\tau \in\{96,192,336,720\}), we set r=T/2=48r=T/2=48 for datasets with relatively few channels (the four ETT datasets, Weather, and Solar-Energy), and r=T=96r=T=96 for Electricity and Traffic, which have substantially more channels. For short-term forecasting (τ∈{12,24,48,96}\tau \in\{12,24,48,96\}), we set r=T/4=24r=T/4=24 for PEMSD3, PEMSD4, and PEMSD8, and r=T/8=12r=T/8=12 for PEMSD7.

(ii) Choice of ϵ\epsilon. The ridge regularization coefficient ϵ\epsilon is fixed at 10−310^{-3} across all experiments.

We have included these settings in Appendix B.3 to improve reproducibility.

References:

[1] Relations between two sets of variates. Hotelling, Harold (1992).

[2] Frequency-domain MLPs are More Effective Learners in Time Series Forecasting (NeurIPS 2023).

[3] FITS: MODELING TIME SERIES WITH 10k PARAMETERS (ICLR 2024).

[4] FilterNet: Harnessing Frequency Filters for Time Series Forecasting (NeurIPS 2024).

[5] OLinear: A Linear Model for Time Series Forecasting in Orthogonally Transformed Domain (NeurIPS 2025).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.