- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposed a method CARE-TSFM to improve the time series forecasting by selectively modifying the historical context fed to a frozen time series foundation model. It uses features from the history, the model's initial forecast, and task metadata to predict the benefits of 24 candidate modifications. A gate decides whether to apply the selected modification or keep the initial forecast. The controller is trained offline using intervention outcomes on pre-test data. Experiments on 97 GIFT-Eval configs show improvements in average MASE and CRPS across six forecasters, with large gains for Kairos but smaller gains for the other models.
Critical strengths
- The paper addresses a well-motivated question and it makes a useful distinction between identifying a possible context issue, selecting a potentially helpful modification, and deciding whether the intervention is worthwhile. Ablations show that always applying the selected modification can degrade the performance.
- The paper evaluates six forecasting models and multiple domains.
Critical weaknesses
- Most of the improvements are small for 5/6 of the models and their robustness is also not clear. Specifically, except for Kairos, MASE improves from 0.21% to 1.29% and CRPS improves from 0.09% to 1.06%. It is difficult to assess whether these improves are reliable without demonstrating results across training seeds. This is my major concern that the proposed method may have limited practical value.
- It is not quite clear how much the seven evidence categories contribute beyond learning which preprocessing operation to apply. The evidence removal ablations retain much of the improvement. There is a lack of evidence to show the benefit of grouping them into the proposed categories.
Questions for authors
- What is the computational cost of generating the offline training labels and how much inference latency does the proposed method add for each model?
- How large are the harmful interventions when they occur?
Evidence contribution, computational cost, and intervention risk
Thank you for the careful review and constructive suggestions.
W1 — Response to shared concerns
Your W1 shares the reliability concern raised in vG1K Q1 and Zann W1/Q1. Five-seed results and paired intervals are in
Response to shared concerns, C1. Reviewer vG1K (Critical Strengths, first paragraph) and reviewer Zann (Critical Strengths, first item) also cite TimesFM-3 CRPS changing from +0.66% to −0.10% without gating as evidence for the execution decision, inSection 6.2, Table 3.W2 — Contribution of the seven evidence scores. What do the scores and their grouping add beyond learning a preprocessing policy?
Using only the seven scores beats the other two single-source controls in all 12 gain-point-estimate comparisons. Each controller is refitted and its validation gate reselected. The feature controls give:
The uniform remove-seven comparison gives:
Remove-seven retains positive gains on both metrics. The results support standalone predictive value and candidate-ranking utility. See
Appendix E.10, Table 16 (new experiment/analysis).Q1–Q2 — Computational cost and intervention risk. What are the labeling cost and per-backbone online latency, and how severe are harmful interventions?
Your Q1 is also raised in vG1K Q2 and Zann Q4; your Q2 overlaps with Zann Q1 on intervention risk. Per-model labeling and online timings, harm magnitudes, and domain denominators are answered in
Response to shared concerns, C2.