acceptodds
Under review as a conference paper at ICLR 2027

Information Anchors: Understanding and Using Forecast-Relevant Information in Time-Series Foundation Models

Abstract

Time-series foundation models (TSFMs) achieve strong forecasting performance, but output accuracy alone reveals little about what internal information their forecasts rely on, how that information is organized, or whether it is stable and functionally meaningful. We introduce INFORMATION ANCHORS, a framework for identifying hidden representations that carry strong information about the future. Using mutual information as a common measure, our framework reveals where forecast-relevant information concentrates, what forecast properties it represents, and how strongly model predictions respond to it. Across six heterogeneous TSFMs and seven data regimes, we find that forecast-relevant information is concentrated in a limited set of temporal regions rather than uniformly distributed across model representations. Compared with representations carrying less future information, information anchors expose more forecast properties, are associated with distinctive patterns in the observed history, and produce stronger prediction responses under controlled intervention. We further demonstrate the practical value of this information through ANCHOR-RAG, which uses information-anchor signals to guide the retrieval and integration of historical evidence. Overall, INFORMATION ANCHORS makes forecast-relevant information in TSFMs identifiable, interpretable, and practically useful, providing a general framework that connects model understanding with improved forecasting.

Primary area:
interpretability and explainable AI
Keywords:
time-series foundation models, mutual information, representation probing, activation intervention, information localization, retrieval-augmented forecasting

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_qA2HOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper introduces "Information Anchors" to identify high-mutual-information (MI) temporal states inside frozen Time-Series Foundation Models. It studies a series of six different model architectures including encoder, decoder, and encoder-decoder architectures, covering many state-of-the art or popular models in the time series community.

The authors use their MI tool to evaluate where the information relevant to the model’s forecasts resides within the internal states of these models, establishing clearly that temporal patches do not carry future information equally. They validate these “anchors” by trying to recover future properties, and replace “important” patches to see how forecasting performance degrades if you do so. The authors then attempt to link high-MI patched to “motifs” in the historical time series, such as rising/falling trends, peaks, etc.

The work then implements Anchor-RAG, a rag-style pipeline, to try to improve model’s performance.

Critical strengths
  1. Architectural Breadth: Evaluating across a number of different architectures proves the anchor results aren’t just an artifact of a single architecture or model class, and demonstrates it occurs for shallow and deep networks.

  2. Volume of Empirical Results: The paper presents a large breadth of experiments, running evaluations across many model-regime combinations, multiple standard time series datasets, and studying a variety of probes.

  3. Ablations: The ablation study effectively isolates the performance contributions of the retrieval metric versus the post-processing mechanics that the authors add. I also thank the authors for providing code.

Critical weaknesses
  1. The major weakness of this paper is the narrative structure, reading "experiment log" structure masked by AI polish. The manuscript lacks a cohesive scientific narrative, reading instead like an assortment of experimental sweeps. The heavy use of AI-generated phrasing serves to gloss over a fundamental lack of intuitive explanation for the phenomena being observed. I found the claims hard to follow and verify in the results, and I find it very hard to translate from the results to the what does this mean.

  2. Core methodological details like introductions and definitions for the history motifs are buried in the appendix, or not covered, as well of a variety of other information needed to understand the overall work. Sections like 3.3 introduce a dense sequence of equations f without grounding them in physical or data-driven intuition, and focus on complexity disproportionate to the core idea: The actual theoretical contribution of the paper - using MI to select historical anchors - is confined to a simple re-ranking step.

  3. Aggregating motif enrichments across 36 atlases completely washes out the physical data generating processes. A "local peak" means something fundamentally different in periodic electricity data versus noisy weather data. While there are some heat maps of different motif/dataset combinations in the appendix, they are hard to interpret or pull any information out of. The paper fails to explain how anchors adapt to these different domain dynamics.

  4. Over-Engineered Post-Processing: The ANCHOR-RAG pipeline relies heavily on heuristics. Ablations show the pure MI signal only drives a marginal 0.846% MSE improvement. While some other components (BCSA and disagreement gate act) seem to act as emergency shock absorbers to prevent catastrophic tail-errors (like the Exchange Rate MSE spiking to 38.3487 without the gate). While these heuristics helped on these simplistic timeseries datasets, the level of bespoke customization needed to see the marginal improvements leads me to believe that they will not have a net positive effect on real world data, limiting the generalization of the technique

  5. Missing Zero-Shot Baseline: The downstream evaluation only compares against the TS-RAG baseline. There is no direct comparison to the raw, unaugmented foundation model to prove the entire retrieval apparatus is strictly necessary.

Questions for authors
  1. Can you provide a dataset-disaggregated breakdown of the motif enrichments to show how High-MI structures differ between strongly periodic datasets and intermittent ones?
  2. Why was a direct comparison between ANCHOR-RAG and the raw, zero-shot foundation model omitted from the downstream evaluation? Are they other baselines you can compare to?
  3. Given the ablation results in Table 2, how do you justify the complexity of the BCSA and disagreement gate when they appear to function primarily to correct out-of-distribution candidate mismatches rather than amplifying the MI signal itself?
6 comments
AuthorsOpenReview ↗

Response to Reviewer qA2H [Part 1]

Dear Reviewer qA2H,

Thank you for reading our paper and for your constructive feedback. Below, we explain the main findings and point to the corresponding revisions, which are marked in blue in the revised PDF.

1. Connecting the experiments

Your Concern (W1): The major weakness of this paper is the narrative structure, reading "experiment log" structure masked by AI polish. The manuscript lacks a cohesive scientific narrative, reading instead like an assortment of experimental sweeps. The heavy use of AI-generated phrasing serves to gloss over a fundamental lack of intuitive explanation for the phenomena being observed. I found the claims hard to follow and verify in the results, and I find it very hard to translate from the results to the what does this mean.

Thank you for your feedback on the presentation of our work. The revised text centers on one question: if some historical patches are more informative about the future than others, what do those patches tell us, and can we use that information to improve forecasting?

  • What the model retains about the future. The MI maps locate informative patches. The probes show what can be read out from these patches, and the replacement tests assess how they affect the forecast.
  • What those patches look like in the input. The motif analysis connects high-MI states to observed historical patterns, with interpretations discussed by dataset.
  • How the finding informs retrieval. The uneven information distribution motivates giving informative patches more weight when comparing histories. ANCHOR-RAG tests this idea through retrieval and forecast fusion.

Sections 3.3, 4.3, 4.4, and the Conclusion now make these connections explicit. Revisions are marked in blue.


AuthorsOpenReview ↗

Response to Reviewer qA2H [Part 2]

2. MI-weighted retrieval and forecast fusion

Your Concern (W2): Core methodological details like introductions and definitions for the history motifs are buried in the appendix, or not covered, as well of a variety of other information needed to understand the overall work. Sections like 3.3 introduce a dense sequence of equations f without grounding them in physical or data-driven intuition, and focus on complexity disproportionate to the core idea: The actual theoretical contribution of the paper - using MI to select historical anchors - is confined to a simple re-ranking step.

Your Concern (W4): Over-Engineered Post-Processing: The ANCHOR-RAG pipeline relies heavily on heuristics. Ablations show the pure MI signal only drives a marginal 0.846% MSE improvement. While some other components (BCSA and disagreement gate act) seem to act as emergency shock absorbers to prevent catastrophic tail-errors (like the Exchange Rate MSE spiking to 38.3487 without the gate). While these heuristics helped on these simplistic timeseries datasets, the level of bespoke customization needed to see the marginal improvements leads me to believe that they will not have a net positive effect on real world data, limiting the generalization of the technique

Your Concern (Q3): Given the ablation results in Table 2, how do you justify the complexity of the BCSA and disagreement gate when they appear to function primarily to correct out-of-distribution candidate mismatches rather than amplifying the MI signal itself?

Thank you for pointing out that we could have explained the role of MI in Anchor-RAG and the scope of the ablation more clearly.

2.1 Our design

Anchor-RAG uses MI-based scores to give more weight to historical tokens that are more informative about the future. We use these scores to weight the cosine distances between corresponding query and candidate tokens, yielding one distance per candidate. This distance is used at three points:

  1. Select histories. Rank the shared Top-20 candidate pool and select ten histories.
  2. Combine retrieved futures. After BCSA aligns the retrieved continuations, use the distance to assign more weight to closer candidates.
  3. Correct the forecast. Reuse the candidate weights from (2) to measure disagreement among the continuations. The gate reduces the strength of the correction when disagreement is high.

AuthorsOpenReview ↗

Response to Reviewer qA2H [Part 3]

2. MI-weighted retrieval and forecast fusion (continued)

2.2 Original ablation

In w/o MI metric, we used Euclidean distance to rank candidates in (1) and compute their weights in (2). Step (3) then used those Euclidean-based weights to compute disagreement. BCSA, the disagreement gate, and residual correction remained active. Relative to this control, the full model reduced macro MSE by 0.846% (95% CI [0.460%, 1.228%]). This comparison measures the contribution of the MI metric within the same retrieval and correction framework.

2.3 Revised ablation

We now also report a joint control, w/o MI metric, BCSA, and gating, to evaluate how the complete design performs against a simpler retrieval-and-correction pipeline. Starting from w/o MI metric, we additionally remove BCSA and set the gate to one, retaining residual correction at the full model’s validation-selected gain. Relative to this joint control, the full model reduces macro MSE by 21.11%. Table R1 maps each ablation to steps (1)–(3) and specifies exactly what changes.

Table R1. Mean relative error reductions from each ablation to full Anchor-RAG across seven equally weighted datasets. Positive values favor Full.

AblationPipeline step and experimental changeMSE effect [95% CI]MAE effect [95% CI]
Euclidean fusion weights(2)–(3). Keep Full's frozen retrieval forecast and MI-selected candidates fixed. Use Euclidean distances to weight fusion and disagreement. Retain BCSA and the gate.+0.308% [0.140, 0.483]+0.238% [0.129, 0.344]
w/o MI metric(1)–(3). Use Euclidean distance for ranking and fusion weighting, and reuse those weights to compute disagreement. Retain alignment, gating, and residual correction.+0.846% [0.460, 1.228]+0.743% [0.545, 0.953]
w/o continuation alignment (BCSA)(2) Combine retrieved futures. Remove endpoint and scale alignment before fusion.+13.674% [12.739, 14.532]+9.712% [8.755, 10.573]
w/o disagreement gating(3) Correct the forecast. Set the gate to one. Retain MI-based ranking and fusion weights.+41.338% [39.676, 42.951]+22.876% [21.068, 23.993]
w/o MI metric, BCSA, and gatingJoint ablation across (1)–(3). Use Euclidean ranking and fusion weights, remove alignment, and set the gate to one.+21.11% [20.18, 22.07]+21.15% [20.61, 21.69]

Disabling only the gate applies the correction at its full validation-selected gain even when candidates disagree. On ETTm2 and Exchange Rate, large errors at a small fraction of forecast origins drive up MSE. The joint control changes which continuations are selected, how they are weighted, and how they are aligned, so the two controls apply different corrections even though both set the gate to one. Section 4.4 and Table 2 report the per-dataset results. Section 4.3 introduces the history-pattern labels, with their operational definitions in Appendix H.1 (Table 26).


Reviewer_5Y75OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper mainly studies which positions in a given time series foundation model are relevant to the model's predictions. The authors locate the key positions that influence prediction by defining the correlation between the statistical information of the forecast and the model's intermediate layers, and then weight these key positions to better retrieve historically similar samples and reduce prediction error.

Critical strengths

The problem the authors study is practically meaningful, and by applying the located key information to similar-sample retrieval, they expand the application scenarios of key information localization. The experiments are fairly thorough and cover multiple classical foundation models.

Critical weaknesses

1, Hidden state replacement still produces OOD information, and the replacement itself is related not only to the content of the information but also to the position. 2, The downstream gains are not obvious: the full Anchor-RAG reduces MSE by about 3.02% compared with TS-RAG, but after keeping alignment, gating, and correction while removing only MI, the increment is only about 0.846%.

Questions for authors

1, Is layer selection better than a simple strategy, such as directly choosing the last layer or the second-to-last layer? 2, Is the algorithm's performance affected by the forecasting horizon? Does the accuracy of the selected positions decrease for long-horizon forecasting? 3, How high is the additional cost of the algorithm?

5 comments
AuthorsOpenReview ↗

Response to Reviewer 5Y75 [Part 1]

Dear Reviewer 5Y75,

Thank you for reading our paper and for your constructive feedback. We address your concerns below. All revisions are marked in blue in the revised PDF.

1. Distribution shift and positional effects in hidden-state replacement

Your Concern (W1): Hidden state replacement still produces OOD information, and the replacement itself is related not only to the content of the information but also to the position.

  • Goal. We tested whether replacing High-MI patches still affects forecasts more than replacing Low-MI patches when the inserted vector is identical and the recipient positions are adjacent.
  • Paired control. We paired neighboring High-MI and Low-MI patches within the same two-patch time bin. In one forward pass, we replaced only the High-MI patches. In a separate pass, we replaced only their paired Low-MI patches. For each pair, both passes used the same hidden-state vector from an observed series, with the layer and replacement budget held fixed.
  • Result. At the 25% budget, the two conditions had comparable perturbation magnitudes. High-MI replacement changed forecasts more in 35 of 42 model–dataset pairs and produced a larger average increase in MSE against ground truth (Table R1).
  • Interpretation. Under these controls, MI identifies positions whose replacement is more disruptive to forecasts. The resulting activations may still be out of distribution, and exact position may still contribute to the difference.

Table R1. Shared-donor replacement within two-patch time bins at the 25% budget.

MeasureResult
Perturbation RMS ratio (High / Low), 95% CI1.005 [0.994, 1.022]
High–Low state-distance contrast, 95% CI−0.014 [−0.127, 0.076]
High–Low forecast-change MSE, crossed-bootstrap 95% CI+0.168 [0.048, 0.327]
Positive model–dataset cells35/42
Cell-level paired intervals excluding zero35/42
High–Low difference in ground-truth ΔMSE, 95% CI+0.152 [0.044, 0.310]

Section 4.3 and Appendix G.1 report the test (Figure 11 and Tables 23–25).


AuthorsOpenReview ↗

Response to Reviewer 5Y75 [Part 2]

2. MI-guided retrieval and forecast fusion

Your Concern (W2): The downstream gains are not obvious: the full Anchor-RAG reduces MSE by about 3.02% compared with TS-RAG, but after keeping alignment, gating, and correction while removing only MI, the increment is only about 0.846%.

Thank you for pointing out that we could have explained the role of MI in Anchor-RAG and the scope of the ablation more clearly.

2.1 Our design

Anchor-RAG uses MI-based scores to give more weight to historical tokens that are more informative about the future. We use these scores to weight the cosine distances between corresponding query and candidate tokens, yielding one distance per candidate. This distance is used at three points:

  1. Select histories. Rank the shared Top-20 candidate pool and select ten histories.
  2. Combine retrieved futures. After BCSA aligns the retrieved continuations, use the distance to assign more weight to closer candidates.
  3. Correct the forecast. Reuse the candidate weights from (2) to measure disagreement among the continuations. The gate reduces the strength of the correction when disagreement is high.

2.2 Original ablation

In w/o MI metric, we used Euclidean distance to rank candidates in (1) and compute their weights in (2). Step (3) then used those Euclidean-based weights to compute disagreement. BCSA, the disagreement gate, and residual correction remained active. Relative to this control, the full model reduced macro MSE by 0.846% (95% CI [0.460%, 1.228%]). This comparison measures the contribution of the MI metric within the same retrieval and correction framework.

2.3 Revised ablation

We now also report a joint control, w/o MI metric, BCSA, and gating, to evaluate how the complete design performs against a simpler retrieval-and-correction pipeline. Starting from w/o MI metric, we additionally remove BCSA and set the gate to one, retaining residual correction at the full model’s validation-selected gain. Relative to this joint control, the full model reduces macro MSE by 21.11%. Table R2 maps each ablation to steps (1)–(3) and specifies exactly what changes.

Table R2. Mean relative error reductions from each ablation to full Anchor-RAG across seven equally weighted datasets. Positive values favor Full.

AblationPipeline step and experimental changeMSE effect [95% CI]MAE effect [95% CI]
Euclidean fusion weights(2)–(3). Keep Full's frozen retrieval forecast and MI-selected candidates fixed. Use Euclidean distances to weight fusion and disagreement. Retain BCSA and the gate.+0.308% [0.140, 0.483]+0.238% [0.129, 0.344]
w/o MI metric(1)–(3). Use Euclidean distance for ranking and fusion weighting, and reuse those weights to compute disagreement. Retain alignment, gating, and residual correction.+0.846% [0.460, 1.228]+0.743% [0.545, 0.953]
w/o continuation alignment (BCSA)(2) Combine retrieved futures. Remove endpoint and scale alignment before fusion.+13.674% [12.739, 14.532]+9.712% [8.755, 10.573]
w/o disagreement gating(3) Correct the forecast. Set the gate to one. Retain MI-based ranking and fusion weights.+41.338% [39.676, 42.951]+22.876% [21.068, 23.993]
w/o MI metric, BCSA, and gatingJoint ablation across (1)–(3). Use Euclidean ranking and fusion weights, remove alignment, and set the gate to one.+21.11% [20.18, 22.07]+21.15% [20.61, 21.69]

Disabling only the gate applies the correction at its full validation-selected gain even when candidates disagree. On ETTm2 and Exchange Rate, large errors at a small fraction of forecast origins drive up MSE. The joint control changes which continuations are selected, how they are weighted, and how they are aligned, so the two controls apply different corrections even though both set the gate to one. Section 4.4 and Table 2 report the per-dataset results.


AuthorsOpenReview ↗

Response to Reviewer 5Y75 [Part 3]

3. Layer selection for analysis and a fixed final layer for forecasting

Your Concern (Q1): Is layer selection better than a simple strategy, such as directly choosing the last layer or the second-to-last layer?

Thank you for raising this question. The representation study and ANCHOR-RAG use layers for different purposes.

  • Representation study. Our goal is to understand how the relevance of hidden tokens to future observations varies across layers and temporal positions. We therefore compute the MI map for every layer in each backbone. The map then guides where we run the follow-up probes and replacement tests.
  • ANCHOR-RAG. Our goal is to evaluate whether the proposed MI distance improves retrieval-augmented forecasting. To keep the representation fixed when evaluating the distance, we use tokens from a single, predefined layer, Chronos-Bolt’s final encoder layer, for all queries and candidates. Section 3.3 describes this setup.

4. Forecasting at longer horizons

Your Concern (Q2): Is the algorithm's performance affected by the forecasting horizon? Does the accuracy of the selected positions decrease for long-horizon forecasting?

Thank you for asking about longer forecasts and patch stability.

  • Forecast accuracy. We evaluated ANCHOR-RAG, TS-RAG, Cross-RAG, and Chronos-Bolt Base on seven datasets at three longer horizons (192, 256, and 384 steps), using the same 512-origin rollout cohort for all comparisons. ANCHOR-RAG achieved lower mean MSE and MAE than both TS-RAG and Cross-RAG at each horizon. Against TS-RAG, it reduced MSE in all 21 dataset–horizon combinations, with mean relative reductions of 3.97%, 4.39%, and 4.76%, respectively (Table R3). The separate full 64-step test showed a 3.02% mean MSE reduction versus TS-RAG (Table 1).
  • Stability of selected positions. In a separate analysis of the MI atlas, the highest-scoring patch at 64 steps remained the top patch on 5/7 datasets at 128 steps and 5/7 datasets at 192 steps. These counts measure whether the selected position stays the same when the horizon changes. In the forecasting rollouts, ANCHOR-RAG re-encodes the updated history every 64 steps and recomputes candidate distances with the MI relevance model fitted for the remaining horizon (Appendix I).

Table R3. Mean relative error reductions versus TS-RAG on the shared rollout cohort.

HorizonMSE reduction [95% CI]MAE reduction [95% CI]MSE winsMAE wins
192 steps3.971% [3.234, 4.639]0.662% [0.336, 0.945]7/75/7
256 steps4.389% [3.466, 5.181]0.762% [0.361, 1.133]7/75/7
384 steps4.756% [3.495, 5.867]0.671% [0.125, 1.162]7/75/7

Tables 3–5 in the revised manuscript report the per-dataset results for all four methods.


Reviewer_6jAVOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper investigates how information relevant to future forecasting is distributed, represented, and utilized within Time Series Foundation Models. The authors introduce the concept of Information Anchors, which are identified by calculating the mutual information (MI) between hidden representations at different layers and temporal patches and the statistical properties of the ground-truth future. This allows them to locate hidden states that contain more forecasting-relevant information. Subsequently, the authors conduct a series of experiments, including Linear Probing, Hidden-State Intervention, and historical pattern analysis, to investigate what future-related properties these information anchors encode, whether they influence the model's forecasting behavior, and which historical time-series patterns they typically correspond to. Finally, the authors propose Anchor-RAG, which uses information anchors to guide historical sample retrieval and candidate future fusion. By keeping the original TSFM frozen, the proposed method improves forecasting performance, demonstrating the practical value of the identified internal information.

Critical strengths

The research question is relatively novel and has research value. From the perspective of forecasting-relevant information, the paper investigates how information is distributed across different temporal positions and hidden layers within TSFMs. It further analyzes the relationship between these representations and forecasting behavior through representation probing and hidden-state interventions. Compared with studies that focus primarily on forecasting accuracy, this work provides a valuable interpretability perspective.The extensive experiments across multiple TSFM architectures and datasets, together with representation probing and controlled interventions, provide relatively comprehensive empirical support for the proposed framework.

Critical weaknesses
  1. The necessity of mutual information as the core value function is insufficiently established. The central methodological choice is to use MI to identify forecast-relevant representations. However, the experiments do not systematically compare MI against alternative value functions for identifying informative states or selecting retrieval candidates.

  2. The paper reports relatively low reselection probabilities under block resampling. Across the 42 information maps, the average reselection probabilities are 0.484 for the selected patch, 0.333 for the selected layer, and only 0.188 for the exact layer–patch pair. These results raise questions about the reliability of anchors as identifiable representation-level objects.

Questions for authors
  1. Have the authors considered alternative criteria for ranking hidden states or retrieval candidates, such as predictive utility, linear probing scores, gradient-based attribution, or representation similarity? How do their selected anchors and downstream forecasting performances compare with MI-based selection under identical conditions?

  2. The full Anchor-RAG improves macro MSE by approximately 0.85% relative to the variant without MI. Can the authors further isolate the effects of MI-based candidate ranking and MI-based weighting, ideally with alternative selection criteria and independently validated configurations?

  3. Have the authors investigated the correlation between MI scores and intervention effects across all patches? For example, could the authors report the Spearman rank correlation between MI-based rankings and intervention-effect rankings? A weak correlation would raise concerns about whether MI is a reliable proxy for the functional importance of hidden representations in forecasting.

6 comments
AuthorsOpenReview ↗

Response to Reviewer 6jAV [Part 1]

Dear Reviewer 6jAV,

Thank you for reading our paper and for your constructive feedback. All revisions are marked in blue in the revised PDF.

1. Alternative scores for informative patches

Your Concern (W1): The necessity of mutual information as the core value function is insufficiently established. The central methodological choice is to use MI to identify forecast-relevant representations. However, the experiments do not systematically compare MI against alternative value functions for identifying informative states or selecting retrieval candidates.

Your Concern (Q1): Have the authors considered alternative criteria for ranking hidden states or retrieval candidates, such as predictive utility, linear probing scores, gradient-based attribution, or representation similarity? How do their selected anchors and downstream forecasting performances compare with MI-based selection under identical conditions?

Thank you for suggesting a direct comparison. We evaluated three final-layer patch-weight profiles: MI, forecast-gradient sensitivity, and uniform cosine weights. MI and gradient profiles were computed from discovery histories.

In the matched RAG experiment, we replaced only the patch-weight profile inside the weighted cosine distance. The resulting candidate distances determine the Top-10 ranking and the candidate weights used for aligned-future fusion and disagreement estimation. We kept the 64-step test origins, TS-RAG Top-20 pool, frozen checkpoint, BCSA, correction gate, residual branch, and MI-selected correction settings fixed. In an additional sensitivity check, each profile instead selected its own correction gain and disagreement scale on validation data before evaluation on the same test origins. Table R1 reports the fixed-setting comparison.

Table R1. Matched 64-step forecasts across seven datasets. Absolute errors and dataset-level relative reductions from TS-RAG both give equal weight to each dataset. Positive reductions indicate lower error than TS-RAG. Brackets give paired 95% moving-block confidence intervals over test origins.

MethodMSEMAEMSE reduction vs TS-RAG (%) [95% CI]MAE reduction vs TS-RAG (%) [95% CI]
TS-RAG0.19390.2491——
ANCHOR-RAG (MI)0.18840.2468+3.024 [2.405, 3.657]+1.132 [0.782, 1.484]
Gradient sensitivity0.18880.2472+2.664 [2.069, 3.267]+0.861 [0.511, 1.210]
Uniform cosine0.18870.2470+2.895 [2.240, 3.566]+1.060 [0.701, 1.417]

Selected positions. The criteria emphasize different temporal patches across datasets. Across the seven datasets, the average per-query overlap between MI's four highest-weight patches and the corresponding top four from gradient sensitivity is 0.36/4. Uniform cosine assigns equal weights to all patches. RAG uses soft weights over all 32 patches.


AuthorsOpenReview ↗

Response to Reviewer 6jAV [Part 2]

2. Anchor locations under resampling

Your Concern (W2): The paper reports relatively low reselection probabilities under block resampling. Across the 42 information maps, the average reselection probabilities are 0.484 for the selected patch, 0.333 for the selected layer, and only 0.188 for the exact layer–patch pair. These results raise questions about the reliability of anchors as identifiable representation-level objects.

Thank you for asking how the resampling numbers relate to the anchor claim. The reported 0.188 is the mean exact layer–patch reselection rate over 42 model–dataset maps, with 50 block-resampling replicates per map. An exact-pair match requires both its layer and its patch to match, so its recovery rate is expected to be lower than either marginal rate. The size of the gap depends on the model and dataset; Table R2 shows the full mean and three dataset means.

Table R2. Reselection rates under block resampling. Dataset rows average six models; the first row averages all 42 maps.

ScopeSelected patchSelected layerExact layer–patch pair
All 42 maps0.4840.3330.188
ETTh20.6270.4870.340
ETTm10.7930.3100.267
Weather0.6630.3530.237

The audit also measures the broader high-score region: the top-quarter patch-set Jaccard is 0.415 across the 42 maps. We report exact-coordinate recovery alongside this set-level measure so readers can see both resolutions of the MI map. The subsequent experiments ask what the selected states contain and how the frozen forecaster uses them: held-out probes quantify decodable future properties, matched-donor interventions find a larger forecast change from high- than nearby low-MI replacement in 35/42 model–dataset pairs, and dataset-specific motif analyses describe the observed histories at high-score positions. Together, these measurements support interpreting anchors as forecast-relevant temporal regions within each model and dataset (Sections 4.2–4.3 and the appendix).


AuthorsOpenReview ↗

Response to Reviewer 6jAV [Part 3]

3. MI-guided retrieval and forecast fusion

Your Concern (Q2): The full Anchor-RAG improves macro MSE by approximately 0.85% relative to the variant without MI. Can the authors further isolate the effects of MI-based candidate ranking and MI-based weighting, ideally with alternative selection criteria and independently validated configurations?

Thank you for pointing out that we could have explained the role of MI in ANCHOR-RAG and the scope of the ablation more clearly.

3.1 Our design

Historical patches carry different amounts of information about the future. We therefore use MI scores to weight the cosine distances between corresponding query and candidate tokens, yielding one distance per candidate. This MI-weighted distance serves as the relevance measure across retrieval and fusion at three points:

  1. Select histories. Rank the shared TS-RAG Top-20 candidate pool and select ten histories.
  2. Combine retrieved futures. After BCSA aligns the retrieved continuations, use the distance to weight them.
  3. Correct the forecast. Reuse the candidate weights from (2) to measure disagreement. The gate reduces the correction when disagreement is high.

3.2 Original ablation

In w/o MI metric, we use TS-RAG's Euclidean distance to rank candidates in (1) and compute their fusion weights in (2). Step (3) then uses those Euclidean-based weights to compute disagreement. BCSA, the gate, and residual correction remain active. Relative to this control, Full reduces macro MSE by 0.846% (95% CI [0.460%, 1.228%]) and MAE by 0.743% [0.545%, 0.953%] across seven equally weighted datasets. This comparison measures the MI distance within the same retrieval and correction framework.

3.3 Separate ranking and weighting ablations

We further isolate the two decisions asked about: MI-based candidate ranking and MI-based weighting. Both controls keep Full's frozen retrieval forecast and validation-selected correction settings fixed. Table R3 changes only the correction candidates or the distance used to weight fusion and disagreement.

Table R3. Mean relative error reductions from each correction-branch ablation to Full across seven equally weighted datasets (%). Positive values favor Full; brackets are paired 95% moving-block intervals.

AblationPipeline step and experimental changeMSE effect [95% CI]MAE effect [95% CI]
w/o MI rank (correction branch)(1), correction candidates only. Keep Full's frozen retrieval forecast; use the first ten candidates in TS-RAG's Euclidean order for correction, then compute their fusion and disagreement weights from MI distances. BCSA and the gate stay active.+0.055 [−0.160, 0.257]+0.067 [−0.036, 0.171]
Euclidean fusion weights(2)–(3). Keep Full's frozen retrieval forecast and MI-selected candidates fixed. Use TS-RAG's Euclidean distances to weight fusion and disagreement; retain BCSA and the gate.+0.308% [0.140, 0.483]+0.238% [0.129, 0.344]

Under these fixed settings, Full has 0.055% lower MSE than the Euclidean-ranked correction and 0.308% lower MSE than Euclidean-weighted fusion. Thus, the two rows distinguish the contribution of candidate selection from that of weighting, while the original 0.846% comparison evaluates their use together across the retrieval and correction framework. Part 1 reports the matched alternative-score comparison and the separate validation-setting check; manuscript Table 2 and the appendix give the corresponding ablations.


Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal Summary

Dear Reviewers, AC, SAC, and Program Chairs,

Thank you for your time and thoughtful feedback. Information Anchors studies where frozen time-series foundation models encode information about the future, what properties can be recovered from those states, and how forecasts respond to their replacement. ANCHOR-RAG uses this information to guide historical retrieval and forecast fusion. Below, we summarize the strengths recognized by reviewers and our responses to their concerns.


Key strengths noted by the reviewers

  • A novel interpretability perspective. The work examines future-related information inside frozen forecasting models and connects representation analysis with forecasting behavior (6jAV Strengths).
  • Practical value for retrieval. The identified information guides the retrieval of historical examples and the use of their continuations (5Y75 Strengths).
  • Broad empirical evidence. Reviewers recognized the coverage of multiple architectures, datasets, probes, and interventions (6jAV Strengths, 5Y75 Strengths, qA2H S1–S2).
  • Ablations clarifying component contributions. Reviewer qA2H acknowledged the experiments separating the retrieval metric’s contribution from the additional processing components (qA2H S3).

Key revisions in response to the concerns

  • Anchor validation (6jAV W1–W2, 5Y75 W1). We compared alternative scoring profiles under matched settings, with MI yielding the lowest mean MSE among the three reported profiles, and clarified the interpretation of anchor reselection. A further control used identical donor vectors at nearby High- and Low-MI positions with comparable perturbation magnitudes. High-MI replacement changed forecasts more in 35/42 model–dataset pairs.

  • MI metric and forecast correction (6jAV Q2, 5Y75 W2, qA2H W2/W4/Q3). We clarified how the same MI-weighted distance guides candidate selection, continuation weighting, and weighted disagreement. The revised ablations distinguish the metric’s contribution from the combined effects of the MI metric, alignment, and gating, and explain the large errors observed when gating is disabled (Section 4.4, Table 2).

  • Scientific narrative and dataset-specific patterns (qA2H W1–W3/Q1). We revised the explanations connecting information localization, probing, intervention, and retrieval. We introduced the history-pattern labels in the main text and provided dataset-specific contrasts showing how the structures associated with high MI vary across datasets (Section 4.3, Figure 14).

  • Baselines, longer horizons, and cost (5Y75 Q1–Q3, qA2H W5/Q2). We clarified that representation analysis covers all layers, while ANCHOR-RAG uses the final encoder layer. We added Chronos-Bolt Base and matched Cross-RAG comparisons. ANCHOR-RAG achieved lower MSE than both on all seven datasets at each evaluated horizon—64, 192, 256, and 384 steps. Additional amortized online inference time relative to TS-RAG was 0.13% at batch size 256 (Tables 1, 3–6).


Review assessments and discussion status

Although 5Y75 and qA2H recommended weak rejection, their detailed assessments also recognized the paper’s merits. 5Y75 reported no substantive soundness concerns (3/4) and rated the presentation clear and easy to follow (3/4). qA2H described the soundness concerns as minor or fixable (2/4). All three reviewers indicated that they learned something interesting or useful.

After reading our responses, Reviewer 6jAV reported no further questions and maintained the positive rating. We have also posted point-by-point replies and follow-up comments to Reviewers 5Y75 and qA2H, whose feedback on the additional evidence is still pending.

All revisions are marked in blue. We would appreciate your consideration of the clarified contribution and additional evidence alongside the initial reviews, and welcome further discussion.

Sincerely,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.