- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper introduces "Information Anchors" to identify high-mutual-information (MI) temporal states inside frozen Time-Series Foundation Models. It studies a series of six different model architectures including encoder, decoder, and encoder-decoder architectures, covering many state-of-the art or popular models in the time series community.
The authors use their MI tool to evaluate where the information relevant to the model’s forecasts resides within the internal states of these models, establishing clearly that temporal patches do not carry future information equally. They validate these “anchors” by trying to recover future properties, and replace “important” patches to see how forecasting performance degrades if you do so. The authors then attempt to link high-MI patched to “motifs” in the historical time series, such as rising/falling trends, peaks, etc.
The work then implements Anchor-RAG, a rag-style pipeline, to try to improve model’s performance.
Critical strengths
-
Architectural Breadth: Evaluating across a number of different architectures proves the anchor results aren’t just an artifact of a single architecture or model class, and demonstrates it occurs for shallow and deep networks.
-
Volume of Empirical Results: The paper presents a large breadth of experiments, running evaluations across many model-regime combinations, multiple standard time series datasets, and studying a variety of probes.
-
Ablations: The ablation study effectively isolates the performance contributions of the retrieval metric versus the post-processing mechanics that the authors add. I also thank the authors for providing code.
Critical weaknesses
-
The major weakness of this paper is the narrative structure, reading "experiment log" structure masked by AI polish. The manuscript lacks a cohesive scientific narrative, reading instead like an assortment of experimental sweeps. The heavy use of AI-generated phrasing serves to gloss over a fundamental lack of intuitive explanation for the phenomena being observed. I found the claims hard to follow and verify in the results, and I find it very hard to translate from the results to the what does this mean.
-
Core methodological details like introductions and definitions for the history motifs are buried in the appendix, or not covered, as well of a variety of other information needed to understand the overall work. Sections like 3.3 introduce a dense sequence of equations f without grounding them in physical or data-driven intuition, and focus on complexity disproportionate to the core idea: The actual theoretical contribution of the paper - using MI to select historical anchors - is confined to a simple re-ranking step.
-
Aggregating motif enrichments across 36 atlases completely washes out the physical data generating processes. A "local peak" means something fundamentally different in periodic electricity data versus noisy weather data. While there are some heat maps of different motif/dataset combinations in the appendix, they are hard to interpret or pull any information out of. The paper fails to explain how anchors adapt to these different domain dynamics.
-
Over-Engineered Post-Processing: The ANCHOR-RAG pipeline relies heavily on heuristics. Ablations show the pure MI signal only drives a marginal 0.846% MSE improvement. While some other components (BCSA and disagreement gate act) seem to act as emergency shock absorbers to prevent catastrophic tail-errors (like the Exchange Rate MSE spiking to 38.3487 without the gate). While these heuristics helped on these simplistic timeseries datasets, the level of bespoke customization needed to see the marginal improvements leads me to believe that they will not have a net positive effect on real world data, limiting the generalization of the technique
-
Missing Zero-Shot Baseline: The downstream evaluation only compares against the TS-RAG baseline. There is no direct comparison to the raw, unaugmented foundation model to prove the entire retrieval apparatus is strictly necessary.
Questions for authors
- Can you provide a dataset-disaggregated breakdown of the motif enrichments to show how High-MI structures differ between strongly periodic datasets and intermittent ones?
- Why was a direct comparison between ANCHOR-RAG and the raw, zero-shot foundation model omitted from the downstream evaluation? Are they other baselines you can compare to?
- Given the ablation results in Table 2, how do you justify the complexity of the BCSA and disagreement gate when they appear to function primarily to correct out-of-distribution candidate mismatches rather than amplifying the MI signal itself?
6 comments
Response to Reviewer qA2H [Part 2]
2. MI-weighted retrieval and forecast fusion
Your Concern (W2): Core methodological details like introductions and definitions for the history motifs are buried in the appendix, or not covered, as well of a variety of other information needed to understand the overall work. Sections like 3.3 introduce a dense sequence of equations f without grounding them in physical or data-driven intuition, and focus on complexity disproportionate to the core idea: The actual theoretical contribution of the paper - using MI to select historical anchors - is confined to a simple re-ranking step.
Your Concern (W4): Over-Engineered Post-Processing: The ANCHOR-RAG pipeline relies heavily on heuristics. Ablations show the pure MI signal only drives a marginal 0.846% MSE improvement. While some other components (BCSA and disagreement gate act) seem to act as emergency shock absorbers to prevent catastrophic tail-errors (like the Exchange Rate MSE spiking to 38.3487 without the gate). While these heuristics helped on these simplistic timeseries datasets, the level of bespoke customization needed to see the marginal improvements leads me to believe that they will not have a net positive effect on real world data, limiting the generalization of the technique
Your Concern (Q3): Given the ablation results in Table 2, how do you justify the complexity of the BCSA and disagreement gate when they appear to function primarily to correct out-of-distribution candidate mismatches rather than amplifying the MI signal itself?
Thank you for pointing out that we could have explained the role of MI in Anchor-RAG and the scope of the ablation more clearly.
2.1 Our design
Anchor-RAG uses MI-based scores to give more weight to historical tokens that are more informative about the future. We use these scores to weight the cosine distances between corresponding query and candidate tokens, yielding one distance per candidate. This distance is used at three points:
- Select histories. Rank the shared Top-20 candidate pool and select ten histories.
- Combine retrieved futures. After BCSA aligns the retrieved continuations, use the distance to assign more weight to closer candidates.
- Correct the forecast. Reuse the candidate weights from (2) to measure disagreement among the continuations. The gate reduces the strength of the correction when disagreement is high.
Response to Reviewer qA2H [Part 3]
2. MI-weighted retrieval and forecast fusion (continued)
2.2 Original ablation
In w/o MI metric, we used Euclidean distance to rank candidates in (1) and compute their weights in (2). Step (3) then used those Euclidean-based weights to compute disagreement. BCSA, the disagreement gate, and residual correction remained active. Relative to this control, the full model reduced macro MSE by 0.846% (95% CI [0.460%, 1.228%]). This comparison measures the contribution of the MI metric within the same retrieval and correction framework.
2.3 Revised ablation
We now also report a joint control, w/o MI metric, BCSA, and gating, to evaluate how the complete design performs against a simpler retrieval-and-correction pipeline. Starting from w/o MI metric, we additionally remove BCSA and set the gate to one, retaining residual correction at the full model’s validation-selected gain. Relative to this joint control, the full model reduces macro MSE by 21.11%. Table R1 maps each ablation to steps (1)–(3) and specifies exactly what changes.
Table R1. Mean relative error reductions from each ablation to full Anchor-RAG across seven equally weighted datasets. Positive values favor Full.
| Ablation | Pipeline step and experimental change | MSE effect [95% CI] | MAE effect [95% CI] |
|---|---|---|---|
| Euclidean fusion weights | (2)–(3). Keep Full's frozen retrieval forecast and MI-selected candidates fixed. Use Euclidean distances to weight fusion and disagreement. Retain BCSA and the gate. | +0.308% [0.140, 0.483] | +0.238% [0.129, 0.344] |
| w/o MI metric | (1)–(3). Use Euclidean distance for ranking and fusion weighting, and reuse those weights to compute disagreement. Retain alignment, gating, and residual correction. | +0.846% [0.460, 1.228] | +0.743% [0.545, 0.953] |
| w/o continuation alignment (BCSA) | (2) Combine retrieved futures. Remove endpoint and scale alignment before fusion. | +13.674% [12.739, 14.532] | +9.712% [8.755, 10.573] |
| w/o disagreement gating | (3) Correct the forecast. Set the gate to one. Retain MI-based ranking and fusion weights. | +41.338% [39.676, 42.951] | +22.876% [21.068, 23.993] |
| w/o MI metric, BCSA, and gating | Joint ablation across (1)–(3). Use Euclidean ranking and fusion weights, remove alignment, and set the gate to one. | +21.11% [20.18, 22.07] | +21.15% [20.61, 21.69] |
Disabling only the gate applies the correction at its full validation-selected gain even when candidates disagree. On ETTm2 and Exchange Rate, large errors at a small fraction of forecast origins drive up MSE. The joint control changes which continuations are selected, how they are weighted, and how they are aligned, so the two controls apply different corrections even though both set the gate to one. Section 4.4 and Table 2 report the per-dataset results. Section 4.3 introduces the history-pattern labels, with their operational definitions in Appendix H.1 (Table 26).
Response to Reviewer qA2H [Part 1]
Dear Reviewer qA2H,
Thank you for reading our paper and for your constructive feedback. Below, we explain the main findings and point to the corresponding revisions, which are marked in blue in the revised PDF.
1. Connecting the experiments
Thank you for your feedback on the presentation of our work. The revised text centers on one question: if some historical patches are more informative about the future than others, what do those patches tell us, and can we use that information to improve forecasting?
Sections 3.3, 4.3, 4.4, and the Conclusion now make these connections explicit. Revisions are marked in blue.