- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper audits context-corruption sensitivity across eight time-series foundation models and nine benchmark targets. It proposes a training-free pipeline combining robust clipping, gated level correction, and distributional hedging. It also studies why recent level changes affect forecasts and formalizes the ambiguity between transient artifacts and persistent regime changes. The strongest contribution is the empirical diagnosis, including explicit accounting for aggregation choices and damage to clean-data forecasts.
Critical strengths
-
The audit is broad and reports consequential failure cases transparently.
- The paper evaluates several model families, reports dataset-normalized aggregates alongside disaggregated results, and measures both corruption recovery and clean-data costs. Its poor results on drifting series are particularly informative: they show why an apparently successful sanitizer can be harmful when genuine changes resemble defects. The model-free detector comparisons also appropriately limit claims about detector novelty.
-
The artifact–regime distinction provides a useful deployment perspective.
- Proposition 1 correctly establishes an idealized impossibility result for identical contexts with two different futures under full CRPS. The empirical evaluation then explores correction, preservation, and hedging. The paper explicitly acknowledges that its covariate-routing experiment supplies distinguishing information by construction and that independent natural-incident validation remains future work.
Critical weaknesses
-
The experiments establish behavioral recency sensitivity more convincingly than they identify its architectural cause. Section 4.3 argues that normalization by window statistics predicts approximately four times greater sensitivity to a body shift than a tail shift. This implication requires assumptions about the normalized predictor. For example, a normalized forecaster with reproduces persistence: it responds to the tail and ignores the body despite using normalization. The location intervention therefore distinguishes recent-level dependence from a pure window-statistics explanation, but does not generally exclude normalization-mediated mechanisms. The direct TimeMoE intervention is informative; broader causal conclusions across families require additional interventions or narrower wording.
-
The interpretation of the scaling result is confounded by the clean-error denominator. Table 9 reports increasing degradation ratios across the Chronos size ladder, but degradation is corrupted MAE divided by clean MAE. Better clean accuracy can increase this ratio even when absolute corrupted error or sensitivity to the injected offset remains unchanged. The observed increase in relative degradation is valid. The stronger explanation that larger models are inherently more faithful extrapolators of corrupted levels requires scale-specific anchoring coefficients or added error normalized by a common dataset scale. Uncertainty estimates are also important for interpreting the small base-to-large difference.
Questions for authors
- Can the mechanism claim be supported by interventions on normalization statistics in additional model families, or narrowed to behavioral recent-level dependence?
- Does the scaling ordering persist for corruption-induced forecast displacement and added error normalized by context scale?
- Proposition 1 concerns an oracle mixture of two deterministic outcomes under full CRPS. What optimality or calibration claims remain for the implemented mixture of imperfect model distributions under the censored CRPS actually evaluated?
- Please clarify whether Table 3 reports electricity-specific coefficients and how its conclusions relate to the per-dataset exceptions in Table 13.
Response to Reviewer tH6D
Response to Reviewer tH6D:
Thank you for a precise review. Two of your points exposed errors in our own claims, which we corrected. The revised PDF marks new and changed text in blue, and App. O lists the changes.
Q4. Was Table 3 electricity-specific? Yes, in effect. It fitted one regression through the origin across all datasets in raw units. Electricity's offsets are two orders of magnitude larger than any other dataset's, so they set every coefficient; the printed values equal electricity's own. Table 13 and the figure now weight datasets equally. The conclusion survives, with new numbers: alpha_tail = 0.60–0.83 against alpha_body = 0.09–0.35. The tail dominates in 43 of 45 model-dataset cells, and both exceptions are on zero-inflated solar (TimesFM 0.37 vs 0.39; TimeMoE 0.36 vs 0.68). The same pooling had also set the TimeMoE intervention numbers in the text. With equal weights, alpha_tail falls from 0.60 to 0.41 and alpha_body rises from 0.35 to 0.61 under body-robust statistics, in the same direction as before.
Q1. Mechanism. We agree with your example: a normalized forecaster with g(z) = z_L behaves like persistence, so the location test cannot rule out normalization in general. We did both things you suggested.
Q2. Scaling. You were right that the ratio could mislead. We re-ran the ladder on all nine datasets at 100 origins (the submission used three datasets and 40 origins), with paired bootstrap intervals (Table 27).
"Scale" is each dataset's mean context standard deviation, which does not depend on the model. The clean error barely moves, so the rise from tiny to base is not a denominator effect: displacement and added error both grow. From base to large, displacement does not change (difference [-0.01, +0.02], 95% interval), and added error rises only a little ([+0.01, +0.09]). We withdrew "larger models are more faithful extrapolators" and the prediction that better forecasters will be more sensitive. The new claim is narrower: fragility grows from tiny to base and then levels off, so a larger model is not a more robust one.
Q3. What does Proposition 1 say about the implemented mixture? Proposition 1 is about an oracle that knows both futures, under full CRPS. For a pool of two imperfect forecasts we claim no optimality. Under full CRPS an exact identity carries over, now Proposition 2: CRPS(F1/2 + F2/2, y) = CRPS(F1, y)/2 + CRPS(F2, y)/2 - (1/4) times the integral of (F1 - F2)^2. So the exact pool scores no worse than the average of keep and de-shift, whatever the models' quality. Two things separate this from what we evaluate:
The equal weight is minimax only in the idealized case. For real models the best weight on keep, on a 0.1 grid chosen on the evaluated windows, is 0.6–0.7 (App. O.4), because de-shift fails harder in its bad world. That is an in-sample choice, not an estimated minimax weight. Neither result is a calibration claim; coverage is measured, not implied (Sec. 4.4).