acceptodds
Under review as a conference paper at ICLR 2027

Not All History Matters: Query-Conditioned Evidence Retrieval for Time Series Forecasting

Abstract

Large language models have shown promise for transferable time-series forecasting, yet most LLM-based forecasters still process the entire historical context, making long-horizon prediction vulnerable to redundant tokens, irrelevant temporal noise, and high computational cost. We aim to address the core gap of how an LLM forecaster can decide which parts of a long history are truly useful for a specific forecasting query and horizon. We propose Forecast-Conditioned Evidence Seeking (ForeSeek), a query-conditioned evidence retrieval framework that reformulates LLM-based forecasting as a retrieve-then-predict process. ForeSeek introduces Horizon-Aware Evidence Routing to select forecast-relevant historical segments, an Attention Conservation objective to preserve the predictive information flow of full-context reasoning, and a Dual-Path Forecasting design that combines evidence-based LLM reasoning with a lightweight temporal path for stable prediction. Experiments on seven benchmarks show that ForeSeek achieves strong long-term, few-shot, and zero-shot forecasting performance, while reducing LLM backbone FLOPs by 70.99% and total FLOPs by 52.56% on Time-LLM under the default Top-K ratio of 0.3. These results suggest that forecast-conditioned evidence selection offers an effective and efficient alternative to uniform full-context inference for LLM-based time-series forecasting.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Models, Evidence Retrieval, Efficient Inference, Time Series Forecasting

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_KVJJOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes ForeSeek, a framework for reducing the computational cost of LLM-based time-series forecasting through query-conditioned evidence selection. It selects segments from a mixed sequence of textual prompts and temporal patches using a horizon-aware router. During training, a full-context teacher provides attention-alignment and prediction-distillation supervision. At inference, the teacher is removed, and predictions from the selected-evidence LLM branch are combined with those from a lightweight temporal branch. The framework is instantiated on Time-LLM and S2IP-LLM and evaluated on seven forecasting benchmarks, including long-term, few-shot, and cross-dataset transfer settings. Additional experiments examine selector alternatives, component ablations, random-seed stability, and computational costs. The reported results show competitive forecasting accuracy, improvements across the evaluated ETT transfer tasks, and substantial inference FLOPs reductions, although accuracy and wall-clock benefits vary across datasets and backbones.

Critical strengths

A useful accuracy–efficiency trade-off. On Time-LLM, ForeSeek reduces total inference FLOPs by 52.56% and measured end-to-end latency by approximately 28.5% at the default retention ratio. It also improves average MSE on ETTh2 and ETTm2 by approximately 5% and 8%, respectively. These results provide concrete evidence that selective context processing can reduce deployment costs while maintaining or improving forecasting accuracy on several datasets, supporting the method’s practical value.

Consistent gains in the evaluated transfer tasks. Across the eight ETT cross-dataset transfer tasks, both ForeSeek variants improve average MSE over their corresponding original backbones, and a ForeSeek variant achieves the best average MSE in every task among the evaluated methods. This consistency across Time-LLM and S2IP-LLM strengthens the empirical case for the framework’s usefulness under distribution shift within the ETT benchmark family.

Critical weaknesses

The core method is specified inconsistently across the paper. Equation (4) and Algorithm 1 construct the routing query from both the recent state and the forecast horizon, whereas the backbone-specific implementations in Equations (24) and (29) use only future temporal markers, omitting the recent-state component. Similarly, Equation (20) introduces a supervised teacher loss that is absent from Equation (10) and Algorithm 1. These discrepancies concern central components of the proposed method and make it difficult to determine which formulation produced the reported results. Clarifying the implemented query, teacher training procedure, and complete optimization objective is necessary to assess reproducibility and whether the experiments validate the stated contribution.

The ablations do not sufficiently isolate the contribution of forecast-conditioned routing. Table 16 provides useful matched-budget comparisons, but only ForeSeek’s selector is marked as trainable; the horizon-only and cosine-similarity selectors are not. Consequently, the comparison does not isolate the benefit of combining recent-state and horizon information from the benefit of learning the selector. Moreover, the paper states that gradients do not pass through discrete Top-K selection and that the router is trained through attention alignment. Removing alignment therefore leaves no specified learning signal for the router-specific parameters, complicating the interpretation of Tables 5 and 17. Comparisons using equally trainable recent-only, horizon-only, and full-query routers, together with clarification of how the no-alignment variants are trained, would substantially strengthen the evidence for the paper’s central mechanism.

Questions for authors

Clarification: Which query formulation and training objective produced the reported results, given the inconsistencies between Eqs. (4), (24), and (29), and between Eqs. (10) and (20)?

Clarification: When attention alignment is removed, what learning signal updates the router parameters, given that Top-K selection is non-differentiable?

Targeted validation: Does ForeSeek still outperform equally trainable recent-only and horizon-only routers under the same supervision and token budget?

4 comments

We are grateful for your time and insightful feedback on the routing mechanism and its training. Below, we clarify the gradient flow and use matched controls to examine the contribution of each query component.

W1 / Q1. Which query and optimization objective define the method?

Response.\color{green}{\mathbf{Response.}} Thank you for pointing out the inconsistency between the main text and the appendix. Both retained implementations construct the routing query as q=Wq(eq+rH)q=W_q(e_q+r_H), where eqe_q summarizes recent patch tokens and rHr_H encodes known time markers for the forecast interval. We corrected Eqs. (24) and (29) to include eqe_q.

Equation (10) combines fused-forecast MSE, attention alignment, and prediction consistency,

L=Ltask+λalignLalign+λconsLcons.\mathcal L=\mathcal L_{\mathrm{task}}+\lambda_{\mathrm{align}}\mathcal L_{\mathrm{align}}+\lambda_{\mathrm{cons}}\mathcal L_{\mathrm{cons}}.

The auxiliary teacher MSE had weight zero in the reported configuration. Revised Eq. (20) omits that term. The alignment and consistency terms apply stop-gradient to the teacher attention and forecast, respectively. Revised Eq. (22) now uses the same stop-gradient convention as Eq. (9) and Algorithm 1.

We further clarify the two auxiliary objectives in Eq. (10) and their default coefficients. Beyond the task loss, both auxiliary terms are used in all reported experiments with λalign=1\lambda_{\mathrm{align}}=1 and λcons=0.5\lambda_{\mathrm{cons}}=0.5.

The Attention Conservation term is

Lalign=KL(sg(pA)∥pR).\mathcal{L}_{\mathrm{align}} = \mathrm{KL}(\mathrm{sg}(p_A) \Vert p_R).

This term aligns the routing distribution pRp_R with the stop-gradient teacher attention pAp_A and is the only direct learning signal for the score-only router parameters, which is why removing alignment removes their gradient, as measured by the no-alignment ablation.

The Prediction Consistency term is

Lcons=SmoothL1(Y^ev,sg(Y^T)).\mathcal{L}_{\mathrm{cons}} = \mathrm{SmoothL1}(\hat{Y}_{\mathrm{ev}}, \mathrm{sg}(\hat{Y}_T)).

This term distills the teacher forecast Y^T\hat{Y}_T into the evidence path Y^ev\hat{Y}_{\mathrm{ev}} to stabilize prediction under compressed context.

The auxiliary teacher MSE in the original appendix was intended to give the teacher branch direct prediction supervision so that its forecast would be better calibrated early in training and thus a more stable target for consistency distillation. The term was assigned weight zero (λaux=0\lambda_{\mathrm{aux}}=0). It therefore contributed zero to the training objective. The full-context branch provided targets through the two stop-gradient objectives above. We have removed the term from Appendix A.1, and the revised Eqs. (20)–(22) now match the three-term objective and stop-gradient convention in the main text and Algorithm 1. Both auxiliary terms act only during training and add no inference cost.

In the saved ForeSeek-Time implementation, the full-context and evidence paths share a pretrained, frozen GPT-2 and have separate prediction heads. The full-context GPT-2 pass runs without gradient tracking. With auxiliary MSE disabled and the teacher forecast detached, the separate full-context head receives no gradient from the ForeSeek objective.

In the saved ForeSeek-S2IP implementation, the two paths share the pretrained GPT-2 and the forecasting head. The full-context GPT-2 pass runs without gradient tracking. The shared head, GPT-2 layer norms, and positional embeddings can receive gradients through the evidence path.


W2 / Q2. What trains router parameters when attention alignment is removed?

Response.\color{green}{\mathbf{Response.}} For parameters used only to compute routing scores, hard Top-K provides no gradient from the task or consistency loss. With pR=softmax⁡(s)p_R=\operatorname{softmax}(s) and detached pAp_A, Eq. (7) gives

=\lambda_{\mathrm{align}}\bigl(p_R(i)-p_A(i)\bigr).$$ Removing alignment therefore removes the specified learning signal for score-only parameters. Prediction heads and shared representations can still receive gradients through their forecasting losses. This ablation measures the effect of removing routing supervision. To compare trainable routers directly, the following controls use real, shuffled, and mean-attention targets. Each has a trainable KL objective, the same full query, and a 57-token budget. Entries are final MSE. | Alignment target | ETTh2 / 96 | ETTh2 / 720 | ETTm2 / 96 | ETTm2 / 720 | | --- | --- | --- | --- | --- | | Sample-specific attention | 0.281 | 0.392 | 0.163 | 0.358 | | Cross-sample shuffled attention | 0.282 | 0.363 | 0.166 | 0.361 | | Training-average attention | 0.288 | 0.367 | 0.164 | 0.365 | | No alignment | 0.288 | 0.387 | 0.179 | 0.371 | Real attention gives the lowest MSE in three settings. Shuffled attention performs best on ETTh2/720. The first three rows compare the information supplied by different training targets. The no-alignment row measures the effect of removing that supervision entirely. --- > #### **W2 / Q3. Does ForeSeek outperform equally trainable recent-only and horizon-only routers?** $\color{green}{\mathbf{Response.}}$ Yes, in all four evaluated settings. The comparison you request uses the same alignment target, downstream architecture, and 57-token budget, varying only the query inputs. Entries are final MSE for ForeSeek-Time. | Trainable query | ETTh2 / 96 | ETTh2 / 720 | ETTm2 / 96 | ETTm2 / 720 | | --- | --- | --- | --- | --- | | Recent only | 0.288 | 0.404 | 0.167 | 0.363 | | Horizon only | 0.299 | 0.410 | 0.170 | 0.367 | | **Recent + horizon** | 0.281 | 0.392 | 0.163 | 0.358 | Relative to the better single-input query, the joint query reduces MSE by 1.4–3.0%. With trainability, supervision, and token budget held fixed, this comparison supports the complementary value of recent state and horizon information in the tested settings.

Thank you for the clarifications and additional experiments. The revised formulation and the matched trainable query controls have addressed some of my earlier concerns.

I still have two concerns:

  1. It is unclear how the separate full-context prediction head in ForeSeek-Time was initialized. Although it receives no gradient, was it loaded from a pretrained forecasting model? If so, what data was used to train it?

  2. The new attention controls do not fully convince me of the benefit of sample-specific attention supervision. The gains over the best shuffled or average targets are small in three settings. More notably, on ETTh2/720, shuffled attention achieves 0.363 MSE, compared to 0.392 with sample-specific attention. These results make it difficult to attribute the improvements specifically to the proposed attention-conservation mechanism.

I appreciate the authors' efforts during the rebuttal, yet I decided to maintain current score.

Thank you for identifying these two remaining issues.Full-context head initialization. In the reported ForeSeek-Time runs, we initialized the separate full-context prediction head using PyTorch’s default linear-layer initialization. We used no forecasting checkpoint or separate training dataset for this head. The two paths share the pretrained GPT-2 backbone. The full-context forecast is detached in the prediction consistency loss, so that loss does not update the separate head. We will make the initialization explicit and clarify the status of this forecast as a consistency target.Sample-specific attention supervision. We agree that the results support a narrower claim. Sample-specific attention yielded modestly lower MSE than the better of the two controls in three of the four settings. On ETTh2 at horizon 720, cross-sample shuffling achieved 0.363 MSE versus 0.392 for sample-specific attention. These results provide limited evidence for an incremental benefit from matching the attention target to each sample. We will revise the manuscript to reflect that limitation when discussing which components account for ForeSeek’s overall forecasting gains.

Reviewer_HxHNOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses redundant historical context in LLM-based time-series forecasting and proposes ForeSeek, a retrieve-then-predict framework. It partitions a mixed sequence of textual prompts and time-series patches into candidate segments, selects Top-K evidence using a forecast-conditioned query, and trains the router and evidence pathway using attention and prediction supervision from a full-context teacher. At inference, the teacher is removed, and the evidence-based LLM prediction is fused with a direct temporal prediction. The framework is mainly instantiated on GPT-2-based Time-LLM and S2IP-LLM. Evaluation includes seven long-term forecasting benchmarks, few-shot learning on ETT datasets, and eight cross-dataset zero-shot transfer tasks. The appendix adds target-trained evaluation on two GIFT-Eval datasets, matched-token-budget selector comparisons, teacher-supervision ablations, random-seed analysis, and efficiency measurements. At the default retention ratio of 0.3, the paper reports reductions of 70.99% in LLM FLOPs and 52.56% in total FLOPs for Time-LLM.

Critical strengths

Meaningful inference-efficiency gains coexist with competitive forecasting results. Table 14 (p. 25) supports the Time-LLM FLOPs reductions in the abstract: total FLOPs decrease from 319.71G to 151.68G. At the default ratio, end-to-end latency decreases from 26.95 ms to 19.27 ms, a reduction of approximately 28.5%. For S2IP-LLM, total FLOPs decrease from 84.56G to 26.41G, although end-to-end latency decreases only from 23.09 ms to 22.20 ms, approximately 3.9%. Computational savings and measured speedup should therefore be distinguished. Across the eight transfer tasks in Table 4, a ForeSeek variant achieves the best average MSE in every task. The coexistence of compressed context and strong transfer performance is the paper's most practically valuable result.

Critical weaknesses

W1. The core query mechanism and teacher-training procedure lack a consistent, reproducible specification.Equation (4) in the main text (p. 5) and Algorithm 1, steps 10-13 (p. 17), use q = Wq(eq + rH), explicitly including recent-state information. The concrete implementations in Appendix A.2, Eq. (24), and A.3, Eq. (29) (p. 15), instead define q = Wq Wh mean(My), without eq. Tables 6 and 16 nevertheless describe recent-state information as a query component or routing input. In addition, Eqs. (9)-(10) and Algorithm 1 use three losses and stop the gradient through the teacher prediction. Appendix Eqs. (20)-(22) (p. 14) add a label-supervised teacher term, lambda_aux L_aux, and omit the stop-gradient notation from prediction consistency. The teacher's pretraining, parameter sharing, and update rules are not clearly specified. W2. Several baseline averages disagree with the listed per-horizon results, the author should rigorously verify the experimental data. • In Table 8 (p. 20), original S2IP-LLM on ETTm2 has MSE values 0.168, 0.229, 0.285, and 0.371. Their arithmetic mean is 0.26325, but both the Avg row and main Table 1 report 0.266. • In Table 11 (p. 22), GPT4TS on ETTh2→ETTh1 has MSE values 0.732, 0.578, 0.759, and 0.781. Their mean is 0.7125, but the Avg row and main Table 4 report 0.757. • In the same table, DLinear on ETTh2→ETTm2 has MSE values 0.240, 0.395, 0.345, and 0.432. Their mean is 0.353, but the Avg row and main Table 4 report 0.328.

Questions for authors

Q1. Does the query used in the experiments include eq? Please reconcile Eq. (4), Eqs. (24)/(29), and Recent State=Yes in Table 16. The horizon-only comparator in Table 16 is not trainable, while ForeSeek is trainable. Are there existing results that keep training and supervision fixed while removing only the recent-state or horizon input, to support their specific complementary roles? Q2. Is the teacher pretrained and frozen, parameter-shared with the evidence pathway, or jointly trained? Is lambda_aux L_aux used for the reported results, and is the teacher prediction detached in the consistency loss? Please provide the loss coefficients, recent-query-window length, and default segment configuration. Q3. What explains the discrepancies between the per-horizon results and Avg rows above? Q4. Why does the full-context row of Table 16 differ from original Time-LLM in Table 1, for example 0.358 versus 0.361 on ETTh2 and 0.266 versus 0.275 on ETTm2? Were these separate runs or configurations? Do all selectors retain the same prediction heads, dual-path architecture, and teacher-supervision settings?

2 comments

Thank you for the time and care you devoted to reviewing the formulation and experimental results. Your detailed comments are valuable, and we address them below with definitions, controlled comparisons, and numerical checks.

W1 / Q1. Which query is evaluated, and do recent state and horizon contribute independently?

Response.\color{green}{\mathbf{Response.}} Your requested controlled comparison explicitly evaluates the full query in Eq. (4) and Algorithm 1:

q=Wq(eq+rH),eq=MeanPool⁡(Eq),rH=ψ(My).q=W_q(e_q+r_H),\qquad e_q=\operatorname{MeanPool}(E_q),\qquad r_H=\psi(M_y).

Here MyM_y contains calendar markers known for the prediction interval. The recent-only, horizon-only, and joint routers are all trainable, use the same alignment target, and retain 57 tokens. Entries are final MSE for ForeSeek-Time.

Trainable queryETTh2 / 96ETTh2 / 720ETTm2 / 96ETTm2 / 720
Recent only0.2880.4040.1670.363
Horizon only0.2990.4100.1700.367
Recent + horizon0.2810.3920.1630.358

The joint query improves all four comparisons under matched trainability and supervision. The horizon-only router accesses history through candidate keys, while the joint query also uses recent state to score those keys. This comparison isolates the incremental value of including recent state in the query.

The saved implementations for both backbones use the joint query in Eq. (4). The revised Eqs. (24) and (29) include the recent-state term eqe_q. The saved ForeSeek-Time code confirms the joint-query implementation.


W1 / Q2. How is the teacher trained, and how do auxiliary supervision and stop-gradient interact?

Response.\color{green}{\mathbf{Response.}} The full-context pass provides the attention target and, when prediction consistency is enabled, the forecast target. We apply stop-gradient to pAp_A in the alignment term and to Y^T\widehat Y_T in the consistency term, as specified in Eqs. (7) and (9) and Algorithm 1. The auxiliary teacher MSE shown in the original appendix had weight zero in the reported configuration (λaux=0\lambda_{\mathrm{aux}}=0). We removed this term from Appendix A.1 and made stop-gradient explicit in Eq. (22).

In the saved ForeSeek-Time implementation, the full-context and evidence paths share a pretrained, frozen GPT-2 and have separate prediction heads. The full-context GPT-2 pass runs without gradient tracking. With auxiliary MSE disabled and the teacher forecast detached, the separate full-context head receives no gradient from the ForeSeek objective.

We further clarify the two auxiliary objectives in Eq. (10) and their default coefficients. Beyond the task loss, both auxiliary terms are used in all reported experiments with λalign=1\lambda_{\mathrm{align}}=1 and λcons=0.5\lambda_{\mathrm{cons}}=0.5.

The Attention Conservation term is

Lalign=KL(sg(pA)∥pR).\mathcal{L}_{\mathrm{align}} = \mathrm{KL}(\mathrm{sg}(p_A) \Vert p_R).

This term aligns the routing distribution pRp_R with the stop-gradient teacher attention pAp_A and is the only direct learning signal for the score-only router parameters, which is why removing alignment removes their gradient, as measured by the no-alignment ablation.

The Prediction Consistency term is

Lcons=SmoothL1(Y^ev,sg(Y^T)).\mathcal{L}_{\mathrm{cons}} = \mathrm{SmoothL1}(\hat{Y}_{\mathrm{ev}}, \mathrm{sg}(\hat{Y}_T)).

This term distills the teacher forecast Y^T\hat{Y}_T into the evidence path Y^ev\hat{Y}_{\mathrm{ev}} to stabilize prediction under compressed context.

The auxiliary teacher MSE in the original appendix was intended to give the teacher branch direct prediction supervision so that its forecast would be better calibrated early in training and thus a more stable target for consistency distillation. The term was assigned weight zero (λaux=0\lambda_{\mathrm{aux}}=0). It therefore contributed zero to the training objective. The full-context branch provided targets through the two stop-gradient objectives above. We have removed the term from Appendix A.1, and the revised Eqs. (20)–(22) now match the three-term objective and stop-gradient convention in the main text and Algorithm 1. Both auxiliary terms act only during training and add no inference cost.

The two paths share the pretrained GPT-2 and the forecasting head. The full-context GPT-2 pass runs without gradient tracking. The shared head, GPT-2 layer norms, and positional embeddings can receive gradients through the evidence path.


W2 / Q3. Why do several averages differ from the listed horizon results?

Response.\color{green}{\mathbf{Response.}} Thank you for identifying these discrepancies. The four displayed MSEs for the original S2IP-LLM baseline on ETTm2 average to 0.26325. We corrected the reported average from 0.266 to 0.263 in Tables 1 and 8.

In Table 11, we corrected the H=192H=192 MSE from 0.578 to 0.758 for GPT4TS on ETTh2 → ETTh1 and from 0.395 to 0.295 for DLinear on ETTh2 → ETTm2. The original baseline table reports averages of 0.757 and 0.328, respectively. Hu et al., ICLR 2025, Table 14


Q4. Why does Table 16's full-context row differ from the original backbone in Table 1?

Response.\color{green}{\mathbf{Response.}} The new experiment distinguishes the original Time-LLM backbone from ForeSeek with full context. This lets you assess the contribution of selection within the ForeSeek architecture:

ModelTokensETTh2 / 96ETTh2 / 720ETTm2 / 96ETTm2 / 720
Original Time-LLM1910.2930.4210.1750.389
ForeSeek architecture, full context1910.2840.3980.1690.372
ForeSeek, selected context570.2810.3920.1630.358

Entries are final MSE. The middle row retains the ForeSeek architecture and complete context. The first row uses the original backbone. The 57-token selector comparisons share the same downstream architecture. ForeSeek improves over both full-context controls in all four settings, supporting a selection benefit within the same forecasting architecture.


Reviewer_JSenOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how LLM based time series forecasters can avoid processing the entire historical context when only part of the history is useful for a given forecasting target. The proposed method, ForeSeek, treats forecasting as a retrieve then predict problem. It selects historical segments conditioned on the recent query window and forecast horizon, then feeds the selected evidence to the LLM based forecasting path. The method also uses an attention conservation objective to keep the compressed evidence path close to full context reasoning, and combines it with a lightweight direct temporal path for more stable prediction.

Critical strengths

The paper addresses a real limitation of current LLM based time series forecasting methods. Many recent methods focus on how to adapt LLMs to time series, but still send a large constructed context to the language backbone. The idea that the useful historical evidence should depend on the query window and the prediction horizon is reasonable for time series, and the paper makes this point clearly.

The empirical results are fairly convincing in terms of efficiency. The method is tested on multiple datasets and under different settings, and the reported FLOPs and token reductions are substantial. I also found the ablations on evidence source, training losses, and the direct path helpful, since they show that the gain is not only from using a shorter context.

Critical weaknesses

The comparison to non LLM forecasting models could be more complete. The paper mainly argues that ForeSeek improves LLM based forecasting by making the LLM path more selective and efficient, which is a useful contribution. However, for practical time series forecasting, the key question is also whether the added LLM and retrieval machinery is justified compared with strong specialized models. The paper includes some forecasting baselines, but the discussion could more clearly position the method against efficient non LLM models in terms of both accuracy and total cost.

Some parts of the evidence routing design still feel somewhat heuristic. The method uses segment pooling, top K evidence selection, and a fixed retention ratio in the main setting. The ablations show that this works better than random or recent evidence, but I would like to see more analysis of when the selected evidence is actually meaningful, and whether the same settings remain robust across different datasets, horizons, and context lengths. This matters because the central claim of the paper is not only compression, but forecast conditioned evidence retrieval.

Questions for authors

How sensitive is ForeSeek to the choice of Top K ratio in practical use? The appendix includes some sensitivity results, but it would be helpful to know whether the default value of 0.3 was tuned on validation sets and whether one should expect to retune it for each new dataset or horizon.

Can the authors clarify how much extra training cost is introduced by the full context teacher and attention conservation objective? The inference cost is clearly reduced, but the training time and memory overhead are also important for judging the practical benefit.

In the zero shot setting, is the evidence router trained only on the source dataset and then transferred directly, or are any statistics or normalization parameters adapted on the target dataset? This clarification would help assess how strong the transfer result is.

Are there failure cases where ForeSeek selects misleading historical segments and performs worse than the full context model? A short analysis of such cases would make the evidence retrieval claim more convincing.

3 comments

We appreciate the time you spent reviewing our work and your constructive suggestions on efficiency and robustness. We address your questions below with cost comparisons, sensitivity results, and a failure-case analysis.

W1. Is the LLM and retrieval machinery justified against efficient specialized forecasters?

Response.\color{green}{\mathbf{Response.}} The comparisons below address your question about the practical value of LLM-based forecasting. They cover accuracy on ETTm1 and ETTm2, followed by computational cost. Each accuracy cell is MSE / MAE.

MethodETTm1 / 96ETTm1 / 720ETTm2 / 96ETTm2 / 720
DLinear0.299 / 0.3430.425 / 0.4210.167 / 0.2690.397 / 0.421
PatchTST0.302 / 0.3540.446 / 0.4390.172 / 0.2630.366 / 0.394
iTransformer0.312 / 0.3660.441 / 0.4420.179 / 0.2710.378 / 0.397
GTR+MLP0.293 / 0.3450.428 / 0.4230.168 / 0.2570.372 / 0.393
Time-LLM0.294 / 0.3450.427 / 0.4310.175 / 0.2650.389 / 0.410
ForeSeek-Time0.309 / 0.3650.432 / 0.4420.163 / 0.2540.358 / 0.388
S2IP-LLM0.293 / 0.3480.406 / 0.4170.168 / 0.2590.371 / 0.400
ForeSeek-S2IP0.301 / 0.3510.423 / 0.4160.169 / 0.2570.365 / 0.392

The ETTm2/96 profiling figures are:

MethodTotal FLOPs (G)Latency (ms)Inference / training memory (GB)
DLinear0.00145.40.02 / 0.03
PatchTST0.4093.50.04 / 0.94
iTransformer0.00392.30.02 / 0.03
GTR+MLP0.0162.30.04 / 0.08
Time-LLM319.7126.951.09 / 2.66
ForeSeek-Time151.6819.270.80 / 1.91
S2IP-LLM84.5623.090.62 / 1.32
ForeSeek-S2IP26.4122.200.55 / 1.36

The LLM profiling entries reproduce Table 14's measurements. The sensitivity experiments below are independent runs. Specialized models have substantially lower costs, with GTR+MLP and DLinear offering competitive accuracy on ETTm1. ForeSeek-Time achieves the lowest MSE in both ETTm2 columns.

For existing LLM forecasters, ForeSeek improves the computation–accuracy trade-off. Time-LLM's total FLOPs fall by 52.56% and measured latency by 28.5%. S2IP's measured latency falls by 3.9%. Reporting both FLOPs and latency makes the deployment benefit explicit for each backbone.


W2 / Q1. Are pooling, Top-K, and the default retention ratio robust choices?

Response.\color{green}{\mathbf{Response.}} Pooling constructs compact retrieval indices, and the selected original tokens enter the LLM. Top-K sets the computation budget. The following experiments examine how the retention ratio, context length, and segment configuration affect accuracy.

The ForeSeek-Time retention sweep below uses T=512T=512. Cells are MSE / MAE.

Retention ratioTokensETTm2 / 96ETTm2 / 720
0.1190.167 / 0.2580.429 / 0.425
0.2390.164 / 0.2550.380 / 0.393
0.3570.163 / 0.2540.358 / 0.388
0.5960.164 / 0.2560.411 / 0.415
0.71350.175 / 0.2670.367 / 0.391

At fixed ρ=0.3\rho=0.3, a separate context-length comparison gives the following MSE:

Look-back TETTm2 / 96: ForeSeek / full contextETTm2 / 720: ForeSeek / full context
2560.172 / 0.1780.407 / 0.435
5120.163 / 0.1650.358 / 0.413
10240.186 / 0.1990.409 / 0.373

At ρ=0.3\rho=0.3, ForeSeek improves over full context in five of these six comparisons. The results also show that the longer horizon is more sensitive to context length and retention ratio. Independent validation-selection runs give the following comparison:

Dataset / HValidation-selected ratioFixed 0.3 MSESelected-ratio MSEFixed-ratio excess (%)
ETTm2 / 960.30.1630.1630.00%
ETTm2 / 7200.70.3580.3453.77%
ETTh2 / 960.70.2810.2790.72%
ETTh2 / 7200.70.3920.3890.77%

Excess (%) = 100 × (fixed-ratio MSE / validation-selected MSE − 1). These validation-selection comparisons are independent of the preceding sweep. For ETTm2/720 at ρ=0.7\rho=0.7, the sweep reports 0.367 and the selection run reports 0.345. The fixed ratio stays within 3.77% of the validation-selected result across these runs. Validation selection provides a practical way to choose the budget for a new dataset or horizon.

The segment/pooling comparison below reports MSE:

Length / stridePoolingETTh2 / 96ETTh2 / 720ETTm2 / 96ETTm2 / 720
1 / 1Identity0.2810.3920.1630.358
2 / 2Attention0.2790.3930.1610.354
4 / 4Attention0.2800.3980.1620.368
4 / 4Mean0.2790.3950.1650.367

Length-2 attention pooling achieves the lowest MSE in both ETTm2 columns. Mean pooling is also competitive, improving the length-4 result in three columns. These results show that ForeSeek accommodates several indexing choices. Table 19 evaluates configurations with 56, 58, and 112 tokens, so its overlap comparison also reflects a larger token budget.


Q2. What extra training cost comes from the teacher and attention objective?

Response.\color{green}{\mathbf{Response.}} Table 15 quantifies the teacher's incremental cost and the total training cost relative to the original backbone:

BackboneTraining variantStep time (ms)Peak memory (MB)
Time-LLMOriginal55.69 ± 1.692719.6
Time-LLMForeSeek, no teacher58.69 ± 1.531596.2
Time-LLMForeSeek, full training69.06 ± 0.971953.0
S2IP-LLMOriginal33.70 ± 7.401350.7
S2IP-LLMForeSeek, no teacher26.85 ± 2.441384.3
S2IP-LLMForeSeek, full training48.70 ± 3.811395.0

Relative to the originals, full training increases step time by 24.0% / 44.5%, while memory changes by −28.2% / +3.3% for Time-LLM / S2IP-LLM. Relative to no-teacher ForeSeek, the teacher increases step time by 17.7% / 81.4%.

The additional ETTm2 training-time records separately list teacher preparation and reported total training:

MethodTeacher / total training, H=96 (h)Teacher / total training, H=720 (h)
Time-LLM— / 0.689— / 0.677
ForeSeek-Time0.128 / 0.8550.126 / 0.839
S2IP-LLM— / 0.020— / 0.015
ForeSeek-S2IP0.013 / 0.0290.009 / 0.021

The teacher is used during training only. Deployment uses the router and two prediction paths, retaining the inference savings reported above. The step-time, memory, and total-hour measurements quantify the training cost of obtaining those savings.

Q3. Is the router transferred from the source, and are target statistics adapted?

Response.\color{green}{\mathbf{Response.}} Yes. The router and forecasting parameters are trained on the source and remain fixed on the target. Normalization statistics are fitted on the target training split. For ETTh1 → ETTh2, the protocol is:

StageDataOperation
Source trainingETTh1 training splitFit the source scaler. Train ForeSeek on standardized source data.
Source validationETTh1 validation splitApply the source scaler. Evaluate source validation loss.
Target normalizationETTh2 training splitFit a separate variable-wise scaler. Keep model/router parameters fixed.
Target evaluationETTh2 test splitApply the fixed target scaler. Evaluate in target-standardized space.

Scaler fitting uses only the target training split. The source-trained router and forecasting parameters are transferred directly, with target-history standardization and no target-domain fine-tuning. This specifies both parts of the adaptation protocol you asked about.


Q4. Can misleading selections make ForeSeek worse than full context?

Response.\color{green}{\mathbf{Response.}} Yes. Appendix F.4 (Figure 11) shows a failure on the ETTh2 OT variable at H=192H=192.

Model in the illustrated OT windowStandardized MSE
Time-LLM0.246
Full context0.399
ForeSeek0.653

The highlighted routing intervals cover a nearly flat period and the recent decline, with limited coverage of the earlier oscillations. ForeSeek underestimates the amplitude of the subsequent oscillations and predicts a higher level than observed. Appendix F.4 discusses this example in light of the segment-wise Top-K rule and the fixed scalar fusion gate.


Reviewer_BKyNOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents ForeSeek, a query-conditioned evidence retrieval framework for efficient LLM-based time-series forecasting. It introduces Horizon-Aware Evidence Routing to select relevant historical segments, Attention Conservation to align retrieval with a full-context teacher, and dual-path forecasting to combine LLM-based predictions with a lightweight temporal predictor. Experiments on seven datasets using Time-LLM and S2IP-LLM show improvements in long-term, few-shot, and zero-shot forecasting. At a retention ratio of 0.3, ForeSeek reduces LLM backbone FLOPs by approximately 71% and total FLOPs by 52.56% on Time-LLM. Additional experiments include component ablations, retrieval comparisons, and efficiency analyses.

Critical strengths
  1. The paper addresses a relevant problem in efficient LLM-based time-series forecasting. Selecting potentially useful historical segments rather than processing the entire context is a reasonable approach to reducing computational costs and irrelevant information.
  2. The proposed framework can be integrated with different LLM-based forecasting architectures. The combination of evidence retrieval and a lightweight temporal path offers a practical approach to balancing prediction accuracy and computational efficiency.
  3. The experiments cover seven datasets and two forecasting backbones, including long-term, few-shot, and zero-shot settings. The authors provide comparisons with random, recent, uniform, similarity-based, and teacher-attention selection strategies, along with component ablations, efficiency measurements, and sensitivity analyses.
Critical weaknesses
  1. The methodological novelty is somewhat limited. Historical retrieval and context compression have been widely explored in time-series forecasting. ForeSeek combines query-conditioned retrieval, attention alignment, and dual-path prediction, but its distinction from existing methods is not sufficiently established.
  2. The relationship between retrieval relevance and forecasting utility remains unclear. Query-relevant historical segments do not necessarily improve prediction accuracy and may introduce redundant or misleading information. Although the retrieval comparisons show improvements, they do not directly establish whether the selected segments provide greater forecasting utility.
  3. The motivation for Attention Conservation is insufficiently justified. The method assumes that teacher attention can identify useful forecasting evidence, but high attention does not necessarily imply greater predictive contribution. Although the ablations demonstrate the benefit of attention alignment, they do not validate this assumption.
Questions for authors

What evidence supports the assumption that query-relevant historical segments provide greater forecasting utility? Have the authors measured how retaining or removing selected segments affects prediction error?

Other concerns are discussed in the Weaknesses section.

4 comments

Thank you for your time and thoughtful comments on the novelty and forecasting value of ForeSeek. Below, we clarify the motivation and present the evidence addressing your concerns.

W1. What distinguishes ForeSeek from existing retrieval and compression methods?

Response.\color{green}{\mathbf{Response.}} ForeSeek's central insight is that the history worth processing depends on the current state and the prediction interval. This makes context selection part of the forecasting problem. ForeSeek directs LLM computation toward evidence relevant to the forecast, while a lightweight temporal path retains the complete numerical history.

RAFT [1] augments the input with training windows and their continuations. GTR [2] retrieves learned global-cycle information. LongLLMLingua [3] compresses text conditioned on a question. ForeSeek learns a forecasting-specific selection rule within the existing mixed context. Its query combines recent state and future time markers to determine which numerical patches and prompt tokens receive LLM processing.

The following comparison tests the value of combining the two query inputs. All three routers are trainable, use the same alignment target, and retain 57 tokens. Entries are final MSE for ForeSeek-Time.

Trainable queryETTh2 / 96ETTh2 / 720ETTm2 / 96ETTm2 / 720
Recent only0.2880.4040.1670.363
Horizon only0.2990.4100.1700.367
Recent + horizon0.2810.3920.1630.358

The joint query achieves lower MSE in all four settings. With trainability and supervision held fixed, these results support the complementary value of recent state and forecast timing.

[1] Retrieval Augmented Time Series Forecasting.

[2] Enhancing Multivariate Time Series Forecasting with Global Temporal Retrieval.

[3] LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression.


W2 / Q1. Does query-relevant evidence improve forecasting accuracy?

Response.\color{green}{\mathbf{Response.}} To address your utility question, the comparison below measures forecasting error from different retained evidence sets within the same ForeSeek architecture. Each compressed variant retains 57 tokens. Cells report evidence-path MSE / final MSE.

SelectorTokensETTh2 / 96ETTh2 / 720ETTm2 / 96ETTm2 / 720
ForeSeek, full context1910.357 / 0.2840.401 / 0.3980.277 / 0.1690.423 / 0.372
Random570.378 / 0.2900.458 / 0.4440.282 / 0.1660.424 / 0.372
Recent570.381 / 0.2920.437 / 0.4190.290 / 0.1770.424 / 0.369
Uniform570.372 / 0.2940.441 / 0.4220.265 / 0.1710.423 / 0.363
ForeSeek570.345 / 0.2810.397 / 0.3920.250 / 0.1630.417 / 0.358

ForeSeek lowers both errors relative to random, recent, and uniform selection in all four settings. Its selected evidence improves prediction within the LLM path itself, and this advantage carries through to the fused forecast. These retention comparisons provide direct predictive evidence for the selection strategy under the evaluated training protocol.

The OT example in Appendix F.4 (Figure 11) illustrates a failure case. The highlighted routing focuses on a flat interval and a recent decline, with limited coverage of the repeated oscillations. ForeSeek produces a low-amplitude forecast with an upward level bias.

Model in the illustrated OT windowStandardized MSE
Time-LLM0.246
Full context0.399
ForeSeek0.653

This pattern points to the importance of covering the signal's dynamic structure within the selected context.


W3. Does Attention Conservation incorrectly equate attention with predictive contribution?

Response.\color{green}{\mathbf{Response.}} Attention Conservation uses the teacher's attention as a training target for the router, as explained in Appendix E.3. It transfers the teacher's allocation of attention to a differentiable routing distribution. MiniLM [1] provides an established example of using attention distributions for distillation. In ForeSeek, forecasting errors from retained evidence provide the separate test of predictive utility.

The controls below keep the query and 57-token budget fixed. The first three routers all learn through KL alignment, with only the target changed. Entries are final MSE.

Alignment targetETTh2 / 96ETTh2 / 720ETTm2 / 96ETTm2 / 720
Sample-specific attention0.2810.3920.1630.358
Cross-sample shuffled attention0.2820.3630.1660.361
Training-average attention0.2880.3670.1640.365
No alignment0.2880.3870.1790.371

Sample-specific attention gives the lowest MSE in three settings. On ETTh2/720, shuffled attention yields 0.363 versus 0.392. These results show that the value of sample-specific supervision varies with the task. Together with the retention comparisons, they support attention alignment as a useful way to train forecast-conditioned selection.

[1] MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers.

Reviewer_BKyNOpenReview ↗

Response to Rebuttal

I appreciate the author providing additional experiments and clarifications. However, the ablation studies do not address the concerns regarding the reliability of the attention-based supervision mechanism. Therefore, I will maintain my original rating.

Thank you for following up. We agree that the ablations provide a limited test of the reliability of teacher attention. At a fixed token budget, the retention experiments show that ForeSeek’s learned selector lowers forecasting error relative to random, recent, and uniform selection in the four tested settings. Whether teacher attention reliably identifies which individual tokens contribute to a forecast remains open. In the matched supervision controls, sample-specific attention had the lowest MSE in three of four settings, but shuffled attention performed better on ETTh2 at horizon 720. We will revise the manuscript to describe teacher attention as a training signal for the router whose benefit depends on the forecasting setting.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.