- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper asks whether text-assisted forecasters on Time-MMD gain from the events or the dates in paired documents. It applies three test-time interventions to trained models, namely deleting the text, replacing event content and masking only dates, and finds that most react to dates but hardly to events. It then proposes SPRA, an adapter that predicts a frozen backbone's errors from document rewrites with explicit dates removed, gated by statistics of the recent series. Experiments cover nine domains, five backbones and eight multimodal baselines, and SPRA ranks first in 79 of 90 domain-backbone-metric comparisons. On three domains, replacing event content changes the mean absolute error (MAE) of every baseline by under 1%, while masking dates reproduces a median 79% of the text-removal penalty for the two fusion methods.
Critical strengths
The three-intervention design identifies which part of the text a forecaster uses. Standard modality ablation shows only that text helps. Applying text deletion, event-content replacement and date-only masking to the same trained model separates calendar cues from event content without retraining, and the authors apply it to eight existing methods.
Critical weaknesses
Date-masking penalties do not show that the textual gain comes from calendar cues. For TaTS on Climate with PatchTST, text lowers MAE by about 5% versus the text-free backbone (Table 1), yet deleting it costs 24% and masking dates 19% (Table 7), both far above the gain. A baseline retrained on date-masked text would test whether models infer their position in the cycle from dates.
Questions for authors
None.
Thank you for identifying the gap between the test-time penalty and the gain over the numerical backbone. We ran the requested date-masked retraining.
W1. Penalty vs. gain on Climate. TaTS's MAE (0.392) is 5.1% below that of a separately trained PatchTST (0.413); removing text or masking dates at test time raises TaTS's MAE by 23.8% and 19.2% relative to its own predictions. A model can depend strongly on an input even when its advantage over another model is modest. §4.3 and Table 3 now present the interventions as model sensitivity, with ratios relative to the text-removal penalty.
Retraining (§4.3, Table R1; App. D.6). We retrain TaTS with calendar expressions replaced by [DATE], or with only those kept, in training, calibration, and testing, with all else fixed.
Table R1. MSE with PatchTST; new rows average three seeds. Best values are bold.
These results support the predictive value of dates under the tested conditions; the percentages compare configurations against the separately trained PatchTST.
Seasonal phase (App. D.5). At test time, we shift the baselines' explicit calendar features and SPRA's report dates.
Table R3. Test-time MAE change (%), ordered Economy/Energy/Traffic.
The larger six-month effect is consistent with sensitivity to seasonal phase. Date shifts change SPRA's MAE by at most 0.32%; the in-place direction flip raises it by 14.25–19.39%.
External retraining (App. J.2). Beyond Time-MMD, we retrain each model on the whole text, its calendar part, or its news part.
Table R5. MSE; ¹From News to Forecast (three seeds), ²TimesX (one seed). Best values are bold.
For TaTS and MM-TSFLib, calendar-only MSE is within about 3.3% of whole-text MSE, and news-only MSE is 0.9–9.1% higher than calendar-only. SPRA with news alone beats its backbone in all six settings and stays within 1.5% of whole-text MSE.