- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper proposes S2G-HAMC, a context calibration method that keeps the backbone TSFM frozen, constructs multiple canonical input contexts, and combines the TSFM's forecasts on these contexts with fixed weights to improve forecast robustness. The method consists of two components. The S2G Adapter adaptively smooths the input window according to observation density to build the canonical context for the frozen TSFM. HAMC combines the forecasts from multiple contexts with different bandwidth scales using fixed, horizon-dependent weights. Throughout this process, the backbone parameters are left entirely unchanged, and only two S2G scalars are trained. With this approach, the method improves the WQL of TSFMs by 7.62–8.60% when 80% of the observations are missing, and S2G alone achieves improvements of 7.39–8.24%. These results show that the proposed method can improve TSFM performance through context calibration alone, without modifying the TSFM weights.
Critical strengths
- The method trains only two scalars, which represent the base bandwidth and the sensitivity to observation density, and uses a fixed-weight combination scheme. It thereby improves TSFM forecasting performance without any additional trainable parameters.
- Instead of fitting to the real benchmark datasets used for evaluation, the scalars are calibrated on synthetic data containing a trend, seasonal components, a chirp, and heteroskedastic noise. This has the advantage of not fitting directly to the evaluation datasets.
- Experiments on diverse datasets from various domains show that the proposed method can indeed improve TSFM performance.
Critical weaknesses
- W1. Poor Readability
- Many key terms are used without being defined. For example, "canonical context" is mentioned repeatedly throughout the paper, starting from its first appearance in the Abstract. However, apart from an indirect description in Sec. 3.2, it is never explained, and even Sec. 3.2 does not define it precisely.
- Many notations are used without definition. For example, and in Eq. (4) are not defined in the main text, and their units are not specified.
- Terms are used inconsistently. The term "query" appears in several forms, such as "model queries", "canonical query", "query", and "query-local", so readers must infer what each one refers to.
- W2. Lacks Details
- The proposed architecture involves many hyperparameters, such as the support score clip, bandwidth clip, and kernel function. Table 7 reports sensitivity for only some of them (temperature, bandwidth clip, and ridge), and only for Chronos-2 on four datasets. No rationale or impact analysis is provided for the others, including the support score clip, the kernel, and the fallback constants.
- In particular, for HAMC, a main component, the paper does not explain why the forecasts are combined with the stated ratios, including 0.95 / 0.05 in Eq. (6).
- The paper states that and are trained on synthetic data. However, the synthetic data generation process is described only as an equation, and it is not specified what data is used or how it is generated.
- The candidate grid, selection criteria, and results corresponding to "Benchmark validation selects…" in Sec. 3.4 and "as described in the appendix" in Sec. 4.1 are not provided in the appendix.
- W3. Limited Reproducibility
- The source code is not released, so it cannot be verified whether the implementation matches the description.
- Context length appears to be a key hyperparameter, but only the candidate lengths are given (Table 9); the actually selected values are not reported.
- W4. Lacks Diverse Experimental Results
- No experimental results are provided for different forecast horizons.
- The paper states that confidence intervals are provided, but the full raw results are not reported; only averages are presented.
- Table 3 presents various TSFMs as baselines, but the proposed method is applied only to Chronos-2. Its performance when applied to the other models is not reported.
- The method has a large number of hyperparameters, but no ablation is provided for each of them.
Questions for authors
- Q1. As mentioned in W3, could you release the source code through the supplementary material or an anonymous GitHub repository?
- Q2. Including the points raised in W2 and W4, could you provide the design choice, final selected value, ablation results, and other details for every hyperparameter? In particular, please include why clipping is applied and what effect it has, as well as an ablation on the combination ratios in HAMC.
- Q3. As noted in W1, could you provide a consolidated list of all terminology and notation used in the paper?
- Q4. Could you also report results under a fixed setting in which the context length and HAMC constants are not selected using benchmark validation data?