- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper studies an interesting problem in text-conditioned time-series forecasting models, when language context announces events that are not reflected in the numerical data. The paper claims that the selection-based approaches have limitations when they cannot recover events that are not in the candidate pool, and the required sampling budget can grow exponentially with the event’s surprise. Therefore, the paper applies structured, context-aware operators like scaling, shifting, adding, or setting values over specified time intervals, to the model’s sampled trajectories, with an LLM extracting the operator parameters directly from text. Extensive experiments on event-announcement tasks show that these operators outperform even oracle-assisted selection, while zero-shot LLM-generated edits outperform selection on most in-scope cases and transfer across different forecasting backbones. However, the paper and rebuttal shows that the proposed approach works mainly for clearly specified, localized events; they do not determine when editing will actually improve the forecast or make it worse.
Critical strengths
-
I think the paper identifies a fundamental limitation of context-blind proposal-and-selection methods where a selector cannot recover a context-induced event that does not appear in the sampled candidate pool. The action-space perspective is an interesting insight.
-
Extensive experiment results provide credible evidence that structured, context-aware modifications can substantially outperform selection in the targeted event-announcement regime.
Critical weaknesses
-
One major limitation is that the proposed operators are designed for CiK tasks that explicitly specify event timing, direction, and magnitude, while they perform poorly on out-of-scope families and Time-MMD. Since the operator schema closely matches the benchmark’s event structure, it remains unclear whether the reported advantage generalizes to independently designed text-conditioned forecasting tasks.
-
It is not clear when a proposed operator should be applied. The strongest comparisons rely on oracle-fitted operators, while the zero-shot extractor can be harmful outside the intended regime. Moreover, the proposed gating strategies fail to reliably distinguish cases where editing improves the forecast from those where it degrades the original prediction. This substantially limits the practical applicability and general significance of the approach.
Questions for authors
-
How to determine the operator schema, and is the schema fixed before examining the CiK tasks?
-
How should a deployed system determine whether to apply an operator or preserve the original forecast?
-
How sensitive are the conclusions to the choice of selection baseline?