- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes a framework for news-augmented time-series forecasting that uses causal, commonsense, and logic-consistent reflection to identify useful news. The main component is an Explainable Process Reward Model (EPRM), which selects news based on both its impact on forecasting accuracy and the quality of the associated reasoning. The selected news is used to improve an LLM-based forecasting model, and the refined filtering logic is applied during inference.
The authors evaluate their method on four datasets (electricity, Bitcoin, traffic, and exchange rates) and report improvements over existing baselines. They also provide ablation studies, human and LLM-based evaluations of explanation quality, robustness tests against spurious news, and convergence analyses.
Critical strengths
- S1. The paper presents an interesting approach to news-augmented time series forecasting by considering causal, commonsense, and logical consistency alongside forecasting accuracy. Incorporating reasoning quality into news selection is a potentially useful way to reduce the influence of irrelevant or misleading news.
- S2. The proposed method improves forecasting accuracy across four real-world datasets. The authors also provide ablation studies and robustness experiments, which help support the effectiveness of the proposed framework.
Critical weaknesses
- W1. It remains unclear whether the performance improvements come from the proposed causal, commonsense, and logic-consistent reasoning (as the authors intended) or simply from filtering out noisy and irrelevant news. Although the authors provide ablation studies, they do not compare against simpler selection strategies, such as semantic similarity-based filtering or LLM-based relevance scoring.
- W2. The performance improvements may partly result from reducing the amount of noisy information rather than improving reasoning quality. For example, the authors do not report or control the number of selected news articles or input tokens across different variants.
- W3. The ablations are conducted only on the Bitcoin dataset.
- W4. The method uses fixed thresholds for rational-quality labeling and news selection, as well as a fixed number of candidate articles and refinement rounds. The authors do not provide sufficient sensitivity analyses for these choices.
- W5. It remains unclear whether the benefits of reasoning-based news selection generalize to other forecasting architectures, including time-series foundation models such as Chronos, TimesFM, and Moirai.
- W6. The paper does not provide a detailed comparison of computational costs with the iterative baseline, such as training time, LLM calls, and inference overhead.
- W7. There could exist potential overfitting to the validation set, as the method repeatedly evaluates forecasting performance on historical validation data to refine news selection and update the filtering logic.
Questions for authors
- Q1. Have the authors considered comparing the proposed method with simpler news selection strategies, such as semantic similarity-based filtering or LLM-based relevance scoring?
- Q2. How many news articles and input tokens are selected by each variant in the ablation studies? Could the performance degradation of the variant without rational-quality labels be explained by differences in the amount of selected news?
- Q3. Do the contributions of causal, commonsense, and logic-consistent reasoning remain consistent across the other three datasets? The current ablation studies are limited to Bitcoin.
- Q4. How sensitive is the proposed method to the rational-quality thresholds, the number of candidate articles, and the number of refinement rounds?
- Q5. Is the proposed news selection framework applicable to other forecasting backbones, such as time-series foundation models such as Chronos, TimesFM, and Moirai?
- Q6. What is the computational overhead of the proposed method compared with the iterative baseline, including LLM calls, EPRM training, and repeated forecasting model fine-tuning?
- Q7. Since the filtering logic is repeatedly updated based on validation RMSE, how do the authors ensure that it does not overfit to the validation set? Have the authors evaluated the stability of the results across different data splits or random seeds?
19 comments
Response to Reviewer qxQF (Part2, W2 & Q2)
2. Controlling Selected News Volume and Input Tokens across Variants (W2 & Q2)
To directly answer Q2, we report the average number of selected news articles and average input news tokens per forecasting window across all four domains for Full (Ours) and the ablated variants:
| Variant | Electricity (Articles / Tokens) | Bitcoin (Articles / Tokens) | Traffic (Articles / Tokens) | Exchange (Articles / Tokens) |
|---|---|---|---|---|
| Full (Ours) | 0.99 / 278.10 | 3.96 / 838.22 | 1.27 / 307.23 | 4.00 / 918.14 |
– causal | 1.04 / 243.89 | 3.99 / 792.93 | 1.32 / 284.17 | 4.00 / 779.47 |
– commonsense | 1.03 / 228.29 | 3.98 / 724.92 | 1.36 / 266.07 | 4.00 / 710.95 |
– logic | 1.05 / 227.78 | 3.98 / 754.32 | 1.35 / 265.63 | 4.00 / 762.00 |
– rational label | 1.06 / 283.56 | 4.00 / 903.55 | 1.36 / 327.25 | 4.00 / 935.45 |
These statistics confirm that our gains come from reasoning and selection quality rather than feeding fewer articles or shorter prompts into the forecaster:
- Consistent Per-Window Budgets and Virtually Identical Article Counts: To fit within the 2,048-token context window alongside numerical time series and rationales (Section 3.5, Appendix A), all variants (including
IB) share the same per-article truncation (1,500 chars) and cap selection at a maximum of 4 articles per window. In event-dense financial domains (Bitcoin, Exchange), nearly every window reaches this 4-article cap (3.96–4.00), whereas in infrastructure domains (Electricity, Traffic), routine windows naturally contain only 1–2 relevant events passing initial filtering (0.99–1.36). Across all domains, average article counts differ by at most 0.00 to 0.09 articles betweenFulland the ablations (where in a few noisy windows, fewer than 4 candidates passFull's 3-dimension gate —while at least one passes, so empty-gate fallback is not needed—leaving those slots unfilled). FullUses More Tokens than– causal / – commonsense / – logic, Yet Achieves Much Lower Error: Because ablating one reasoning dimension omits that rationale field from the prompt (Appendix G.5),– causal,– commonsense, and– logic(as well asIB, which omits all three rationale fields) actually feed fewer news tokens into the forecaster (e.g.,724.92–792.93vs.838.22on Bitcoin;227.78–243.89vs.278.10on Electricity), yet suffer substantially worse RMSE (1057.66–1192.60vs.834.73on Bitcoin). This directly rules out the hypothesis thatFullwins simply by reducing input token length.- Why
– rational labelDegrades (+53.0% RMSE on Bitcoin) Under the Same Budget:– rational labelandFullretain the exact same 3-part rationale format and have nearly identical article counts (4.00vs.3.96) and token counts (903.55vs.838.22on Bitcoin;935.45vs.918.14on Exchange). Without the rationality constraint (), the EPRM overfits to coincidental historical error reductions (, reward hacking) and selects spurious headlines that hurt test generalization. - Token-Length-Invariant Verification: In our
Chronos-bolt-minitransfer experiment (Table R2 in Point 5), all selected articles in a window are pooled into a single fixed-dimensional vector (all-mpnet-base-v2), eliminating token count entirely; our EPRM-selected news still consistently outperformsIBnews.
Response to Reviewer qxQF (Part3, W3 & Q3)
3. Ablation Studies Across All Four Datasets (W3 & Q3)
We expanded the ablation study in Table 2 across all four benchmark datasets (reporting test RMSE below; full metrics will be added to Section 4.3):
| Variant | Electricity | Bitcoin | Traffic | Exchange |
|---|---|---|---|---|
| Full (Ours) | 272.07 | 834.73 | 20.42 | 5.95 |
– causal | 348.55 | 1192.60 | 21.08 | 5.99 |
– commonsense | 340.99 | 1057.66 | 20.59 | 6.04 |
– logic | 354.04 | 1081.93 | 21.77 | 5.97 |
– rational label | 345.84 | 1277.13 | 20.91 | 5.98 |
– all (IB) | 286.05 | 944.62 | 21.35 | 7.54 |
- Consistent Necessity Across All Domains: Across all four datasets, Full (Ours) consistently achieves the lowest RMSE (
272.07,834.73,20.42, and5.95), outperforming every ablated variant and confirming that all three rational constraints and the joint label are necessary. On Exchange (7.54for IB) and Traffic (21.35for IB), partial variants already improve over IB (5.97–6.04on Exchange; three of four variants at20.59–21.08on Traffic), while combining all components in Full yields the strongest performance. - Component Coupling in High-Noise Regimes (Bitcoin & Electricity): Unlike
IB(which retrieves via simple category keywords without rationale injection or reward ranking), our framework actively injects generated rationales into the forecasting prompt (Appendix G.5) and searches over candidates via the EPRM. In domains with intense speculative noise or abrupt spikes (Bitcoin and Electricity), generative expansion and reward search act as a double-edged sword unless fully regularized: droppingrational labelcauses reward hacking on coincidental ; droppingcommonsenseorlogicallows hallucinated multi-hop narratives or cross-round contradictions; and droppingcausalselects broad background news lacking a directionalcause -> mediator -> effectpathway.
Response to Reviewer qxQF (Part1, W1 & Q1)
We sincerely thank Reviewer qxQF for the thorough reading and constructive comments. Below, we address each of your concerns (
W1–W7) and questions (Q1–Q7) with new baselines, token/article volume statistics, cross-domain ablations, backbone transfers, runtime breakdowns, and multi-split/multi-seed stability analyses.1. Comparison with Simpler News Selection Strategies (W1 & Q1)
To verify that our improvements stem from causal, commonsense, and logic-consistent reasoning rather than simple noise removal, we evaluated two standard non-EPRM selection baselines under the exact same Qwen3-8B (LoRA) backbone, candidate pool, 4-article per-window cap, and 1,500-character truncation budget:
Sim-Filter(Semantic Similarity / Dense RAG): Selects news articles by dense embedding cosine similarity (all-mpnet-base-v2) to the forecasting task query.LLM-Rel(Single-Pass LLM Relevance Scoring): Prompts the same LLM agent (DeepSeek-V4-Flash) to filter relevant articles directly in a single pass, without iterative explainable reflection or EPRM scoring.Table R1 (Top): Comparison with simpler context-selection baselines on Qwen3-8B (Test RMSE; lower is better).
No-News(Numerical Only)Sim-Filter(Dense RAG)LLM-Rel(Single-Pass LLM)IB(Wang et al., 2024)Oursconsistently achieves the lowest RMSE across all four domains, outperformingSim-Filterby 13.5% on average,LLM-Relby 9.0%, andIBby 10.5%.LLM-Rel(21.13,6.12) outperforms multi-roundIB(21.35,7.54). This confirms that unguided iterative retrieval can inject spurious articles that degrade forecasts over rounds, whereas our structured reflection and joint EPRM gate effectively prevent this degradation (20.42,5.95).