- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Context2Action, an agentic framework for real-world-context-aware time-series forecasting. The method constructs local and global context memories based on historical forecasting residuals and employs two retrieval-augmented LLM experts to generate structured adjustments to the predictions of a frozen numerical forecasting model. The framework further incorporates a hierarchical decision mechanism and uses supervised fine-tuning and GRPO to optimize forecasting actions. Experiments on four datasets demonstrate improvements over numerical and context-aware forecasting baselines.
Critical strengths
- The paper presents a practical approach to incorporating real-world contextual information into time-series forecasting through explicit and interpretable forecast adjustments, rather than directly generating predictions.
- The proposed framework achieves improvements over numerical and context-aware baselines on four datasets. The ablation studies and transferability experiments further demonstrate the potential effectiveness and adaptability of the framework.
Critical weaknesses
- The framework mainly combines existing techniques, including residual correction, retrieval-augmented LLMs, and SFT/GRPO. The methodological novelty beyond their integration is not sufficiently established.
- Historical forecasting residuals do not necessarily reflect the effects of real-world contexts, as they may also arise from model errors or unobserved factors. The reliability of using residuals as context-action supervision remains unclear.
- The proposed local adjustments and global level/trend corrections rely on relatively simple predefined operations. It is unclear whether these actions can adequately capture complex, nonlinear, or interacting contextual effects.
- The framework assumes that relevant contextual information is available at prediction time. However, the availability of future weather, news, or unexpected events is not sufficiently clarified, raising concerns about practical applicability and evaluation validity.
- The largest improvement comes from M5, where contextual events and their effects are synthetically constructed. Furthermore, the same event mechanisms appear in both training and testing, making the claimed generalization to open-world contexts less convincing.
- The experiments lack strong lightweight baselines, such as context-conditioned residual regression or simpler retrieval-based correction methods. It is therefore difficult to determine whether the complex agentic framework is necessary.
- Although several ablations are provided, the individual contributions of the No-Action mechanism, contextual information quality, and structured action design are not sufficiently isolated. Additional controlled experiments are needed to establish the effectiveness of the core components.
Questions for authors
- How does the method distinguish context-induced deviations from ordinary forecasting errors or unobserved factors? What evidence supports the reliability of residuals as supervision for context-action learning?
- How is the availability of contextual information ensured at prediction time, particularly for future weather, news, and unexpected events? Are all contextual inputs strictly restricted to information available at the forecasting origin?
- Given that M5 uses synthetically constructed events and shares event mechanisms between training and testing, how well does the framework generalize to unseen event types or real-world contextual disruptions?
- What is the key methodological novelty beyond combining residual correction, retrieval-augmented LLMs, and SFT/GRPO? Why is an agentic framework necessary compared with simpler context-aware correction methods?
3 comments
Response to Reviewer QQcd (2/3)
3. Generalization Beyond M5 Event Mechanisms
Your Concern (W5 & Q3): M5 uses synthetic events and shares event mechanisms between training and testing, making open-world generalization unclear.
M5 is designed as a controlled evaluation in which the underlying numerical series remains realistic while event effects can be precisely examined. Each event mechanism appears through different textual descriptions and independently sampled effects across training and testing. Therefore, the test instances cannot be solved by memorizing a fixed description or correction magnitude.
However, we agree that this setting evaluates generalization across new event occurrences, descriptions, and effect realizations, rather than entirely unseen event categories. We will narrow the corresponding claim accordingly.
Generalization beyond the controlled M5 setting is evaluated on three real-world datasets:
- Electricity contains weather, calendar, and public-news contexts;
- NYC Taxi contains real venue events and cancellations;
- Bitcoin contains timestamped real-world financial news.
Context2Action improves the strongest baselines on all three datasets. These results provide complementary real-world evidence without relying on the M5 event-generation process.
4. Core Novelty and the Need for an Agentic Framework
Your Concern (W1, W6 & Q4): The framework combines residual correction, retrieval-augmented LLMs, and SFT/GRPO. What is the methodological novelty, and why is an agentic framework needed instead of a simpler correction method?
The novelty does not lie in introducing residual modeling, retrieval, or GRPO individually. It lies in a new context-to-action forecast-revision paradigm.
Traditional forecasting models primarily extrapolate numerical patterns, while real-world context is heterogeneous, open-form, and does not directly specify a numerical correction. Using it requires several coupled decisions:
- determine whether a context is relevant;
- retrieve comparable historical evidence;
- infer the affected interval, direction, and magnitude;
- coordinate local effects with overall level/trend calibration;
- reject an unsupported revision.
Context2Action formulates these decisions as explicit, inspectable actions instead of directly generating a new forecast. The numerical predictor remains frozen, while the experts translate contextual evidence into revisions that can transfer across forecasting backbones. The agentic formulation is therefore used for evidence comparison, contextual reasoning, structured action generation, and acceptance decisions, rather than merely as an alternative implementation of residual regression.
4.1 Direct Residual-Transfer Control
We additionally compare with lightweight retrieval-based corrections using the same base forecasts, contexts, retrieval pool, and test windows.
| Method | MAE (×10³) | MSE (×10⁶) | RMSE (×10³) | WMAPE (%) | MAE Reduction |
|---|---|---|---|---|---|
| Base Predictor | 1.499 | 5.826 | 2.414 | 11.09 | – |
| Top-1 Direct Transfer | 1.691 | 7.136 | 2.671 | 12.51 | −12.82% |
| Top-3 Direct Transfer | 1.489 | 5.563 | 2.359 | 11.02 | 0.66% |
| Top-5 Direct Transfer | 1.443 | 5.361 | 2.315 | 10.67 | 3.76% |
| Context2Action | 1.055 | 3.133 | 1.770 | 7.81 | 29.59% |
Directly transferring the nearest residual is harmful, while averaging multiple residuals yields only limited improvement. This result indicates that the gain primarily comes from adapting historical evidence to the current context and forecast state, rather than simply reusing numerical corrections.
Response to Reviewer QQcd (3/3)
5. Expressiveness of Structured Actions
Your Concern (W3): The predefined local and global operations may not capture complex, nonlinear, or interacting contextual effects.
The constrained action space is intentional: it preserves the temporal structure learned by the numerical predictor and makes every revision inspectable.
- Local actions independently specify the start time, duration, and signed magnitude of each contextual effect.
- Multiple overlapping contexts are considered jointly, allowing their interactions to affect the final timing and magnitude.
- The Global Expert further adjusts systematic level and trend deviations after local revision.
Therefore, a forecasting window can contain multiple localized actions together with a global calibration, producing a piecewise and context-dependent revision rather than a single uniform correction.
We nevertheless agree that the current action space is not intended to represent every possible nonlinear transformation. This is a deliberate interpretability–expressiveness trade-off, and richer nonlinear action bases are a valuable direction for future work. We will clarify this scope rather than claiming unrestricted expressiveness.
6. Isolating No-Action and Component Contributions
Your Concern (W7): The individual contributions of No-Action, contextual-information quality, and structured action design are not sufficiently isolated.
6.1 No-Action Ablation
We remove only the acceptance decision while always applying the same valid candidate actions.
| Stage | Always Apply | With No-Action | MAE Reduction | Rejection Rate |
|---|---|---|---|---|
| Local Expert | 1.192 | 1.137 | 4.61% | 6.25% |
| Global Expert | 1.077 | 1.055 | 2.04% | 9.38% |
This confirms that No-Action rejects a small but consequential set of harmful revisions.
For example, one candidate proposed a constant +27.1% adjustment over five days, although the retrieved evidence declined from +28.0% and +29.4% to +6.1%, +4.2%, and −1.8%. The expert selected NO_ACTION. Evaluation performed afterward showed that this decision reduced MAE by 12.12% relative to applying the candidate.
6.2 Context Quality and Structured Revision
- Random Retrieval vs. Top-1 vs. Context2Action isolates the importance of contextual relevance and multi-case evidence selection.
- Direct residual transfer vs. Context2Action tests whether retrieved numerical effects alone explain the improvement.
- Always Apply vs. No-Action isolates the acceptance mechanism.
- Local Only, Global Only, and the complete framework distinguish localized contextual revision from level/trend calibration.
Together, these controls show that the improvement arises from the complete context-to-action process—relevant evidence retrieval, structured revision, and utility-aware acceptance—rather than from residual correction or output-format constraints alone.
Response to Reviewer QQcd (1/3)
Dear Reviewer QQcd,
Thank you for your careful review and constructive feedback. We appreciate your recognition of the practical, interpretable forecast-revision formulation and the consistent improvements across four datasets. We address all weaknesses and questions below with clarifications and additional controlled experiments.
1. Reliability of Residual-Grounded Supervision
We agree that an individual residual should not be interpreted as a causal estimate of a contextual effect. Context2Action instead treats residuals as noisy empirical evidence and controls their use through the following design:
The ablations provide supporting evidence. Random retrieval consistently underperforms relevant retrieval and even harms Bitcoin, while directly using the nearest case remains inferior to multi-candidate reasoning.
Thus, the method does not assume that every residual is context-induced. It learns which historical evidence is useful for forecast revision. We will clarify this distinction and avoid causal wording in the revision.
2. Availability of Contextual Information
Our protocol strictly restricts contextual inputs to information that would reasonably be available when the forecast is issued. No realized future context or post-origin information is used.
Unexpected events are included only when an announcement or alert is already available; otherwise, they are not provided to the model. Weather inputs are forecasts or alerts rather than realized observations. In Figure 3(d), all Bitcoin news items were published before the forecast origin, and the model estimates their possible carry-over effects.
We will add the following explicit temporal-cutoff statement:
This protocol is consistent with established context-aware forecasting settings, such as EventTSF [1] and the M5 [2] forecasting task, which use scheduled events and other covariates available when predictions are issued.
References
[1] Y. Ge, M. Jin, Y. Zhao, H. Li, B. Du, C. Xu, and S. Pan. “EventTSF: Event-Aware Non-Stationary Time Series Forecasting.” Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence, pp. 4313–4321, 2026.
[2] S. Makridakis, E. Spiliotis, and V. Assimakopoulos. “M5 Accuracy Competition: Results, Findings, and Conclusions.” International Journal of Forecasting, vol. 38, no. 4, pp. 1346–1364, 2022.