- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper proposes CAVIR, a retrieval augmentation framework for frozen time-series foundation models. It combines retrieval intended to be invariant to scale and translation, alignment of retrieved future trajectories to the query’s coordinates, aggregation through one-dimensional Wasserstein quantile barycenters, an adaptive controller that can abstain from retrieval, and conformal interval calibration. The submission develops nearest-neighbor bias–variance arguments and claims covering-number and distributional error guarantees. Evaluation includes four ETT datasets, additional forecasting domains, experiments with three foundation-model backbones, and ablation and deduplication analyses. The reported results include consistent but mostly modest ETT improvements, larger error reductions on Weather and Traffic, and approximately nominal coverage in selected shock experiments.
Critical strengths
The combination addresses a practically meaningful problem in using external memory with frozen forecasters. The paper recognizes that retrieval usefulness varies across queries and that similarity of past contexts does not necessarily imply agreement among future trajectories. Combining distribution-level forecast aggregation with an explicit option to decline retrieval is a useful design direction. This perspective is informative and could provide a practical way to improve frozen models without retraining their backbone. I find the attempt to connect neighborhood quality, forecast uncertainty, and retrieval decisions valuable, even though the proposed guarantees require substantial revision.
There is encouraging empirical evidence, together with useful attention to retrieval-specific failure modes. Table 1 reports improvements in both MSE and CRPS on all four ETT datasets; the ETTm1 reduction from in MSE is appreciable. The cumulative ablation and experiments examining target-memory exclusion and near-duplicate removal address relevant questions about where retrieval gains originate. The paper also reports failures when absolute scale carries predictive information, which is an informative limitation of the proposed invariance. These observations provide some support for the practical direction, although the reporting and protocol concerns below limit the conclusions that can currently be drawn.
Critical weaknesses
The central geometric and distributional guarantees are not established for the method as defined. The bounded affine transformation family in Eq. (1) is generally not a group: bounding scale and translation parameters does not preserve closure under composition or inversion. For example, allowing scales in permits multiplication by two, but composing this transformation with itself produces multiplication by four. Consequently, the claimed orbit quotient and compact-group argument underlying Proposition 4 do not follow from the stated construction. Moreover, the alignment residual in Eq. (2) need not be symmetric because scaling is not a Euclidean isometry. It therefore cannot simply be treated as the claimed quotient metric. These issues undermine the advertised covering-number contraction, rather than only its exposition.
There is a separate gap in Proposition 5. Section 4.4 describes as an empirical quantile function derived from a retrieved trajectory, whereas Appendix F.6 assumes , the true conditional quantile function. The Jensen argument is valid for those oracle distributions, but does not account for estimation error in the empirical quantities supplied by the method. In particular, the stated bound becomes zero when all retrieved contexts coincide with the query, even though finitely many observed futures cannot generally determine the true conditional distribution exactly.
These problems affect headline theoretical contributions and require a corrected formulation and analysis. The insufficient specification of how each trajectory yields horizon-specific quantiles also makes it difficult to determine precisely which estimator the theory should analyze.
The calibration and evaluation evidence does not substantiate the claimed reliability of the complete system. The conformal construction is insufficiently specified, and its guarantees are overstated. The score and endpoint correction in Section 4.4 follow conformalized quantile regression, which should be attributed to Romano, Patterson, and Candès (2019). The cited covariate-shift result requires an unchanged conditional outcome distribution and appropriate likelihood-ratio weighting; the cited result beyond exchangeability instead permits a nonzero coverage gap. Neither directly establishes nominal coverage under arbitrary abrupt regime changes. The submission needs to identify the applicable assumptions and the actual weighted calibration procedure.
The empirical evidence also needs reconciliation. In Table 1, Cov@80 worsens from 78.83% to 75.38% on ETTh2 and from 79.58% to 76.38% on ETTm2. Table 6 reports substantially different coverage from the main tables. For example, 37.66% versus 80.12% on ETTh1. A consistent affine change of output units alone cannot change interval coverage; if the target channels, evaluation windows, or forecasting configurations differ, those differences must be stated. Furthermore, removing the selective gate leaves every reported Table 6 metric unchanged across all four datasets, so that ablation does not demonstrate the gate’s contribution in that setting.
Finally, the approximately 80% shock coverage is reported without interval-width or interval-score comparisons and without a backbone baseline using the identical conformal calibration procedure. These comparisons are necessary to establish useful uncertainty improvement and attribute it to the proposed retrieval framework. Taken together, these issues prevent the current experiments from establishing the paper’s central calibration and safety claims.
Minor: The presentation makes the method unnecessarily difficult to follow. The manuscript would benefit from careful proofreading: for example, the abstract contains a sentence fragment beginning “Because existing methods…” and the typo “uncertainity.” Beyond language, the ordering of the exposition needs improvement. Section 4.5 discusses Theorem 1 and Proposition 2 before their statements while introducing several controller quantities, making it difficult to distinguish the implemented procedure from its theoretical motivation. The relationship between the embedding distance in Eq. (4) and the proposed quotient distance in Eq. (2) also needs to be explained explicitly. Figure 1 identifies the main modules, but does not sufficiently clarify how retrieved trajectories become quantile forecasts or how the controller determines retrieval and fusion decisions. A self-contained algorithm description, consistent notation, and a clearer account of the data flowing between modules would substantially improve readability.
Questions for authors
- How can the transformation family in Eq. (1) satisfy the group axioms, and how can Eq. (2) satisfy symmetry? Please provide a corrected construction and identify which theoretical conclusions, especially Proposition 4, remain valid.
- How is constructed from a single retrieved future trajectory, separately for each forecast horizon? Is it based on additional model forecasts, values pooled across time, or an empirical distribution across neighbors? How does Proposition 5 account for the estimation error of the quantities actually used?
- Which precise coverage theorem applies to the implemented conformal procedure? Please specify the likelihood-ratio estimation and weighted quantile rule, the assumptions about temporal dependence and distribution shift, and which residuals are available when an interval is issued. In the “strict zero-shot” experiments, are target-domain labels used for controller training or calibration?
- What differences in target channels, evaluation windows, preprocessing, or model configuration explain Tables 1/4 versus Table 6? How do the raw ETTh1 MSE values cited in the main text relate to Table 6, and why are the full and ungated configurations identical throughout that table?
- On matched evaluation windows, how does CAVIR compare with the backbone using the same conformal calibrator, and with a straightforward normalized-neighbor baseline using the same fusion and calibration? Please include interval width or interval score alongside coverage. These comparisons would materially affect my assessment of the additional value of the proposed framework.