acceptodds
Under review as a conference paper at ICLR 2027

From News to Explainable Forecast: Commonsense, Causal, and Logic-Consistent Reflection for LLM Forecasting

Abstract

Incorporating external news into numerical time-series forecasting is critical for capturing exogenous shocks that historical data alone cannot explain. However, existing LLM-based iterative forecasters generate reflections that merely identify missing news categories, lacking causal reasoning about their impact or plausibility checks, which makes the refinement loop prone to spurious correlations and slow convergence. We address this by introducing an explainable reflection framework with three structural constraints: a causal chain linking missing news to prediction errors, a commonsense sanity check, and an explicit logic consistency constraint against prior reflections. We then train an Explainable Process Reward Model (EPRM) that scores candidate news articles by jointly optimizing for expected error reduction and the above rational quality. Only candidates passing a quality gate are incorporated. Experiments on electricity, bitcoin, traffic, and exchange-rate benchmarks show that our method achieves a 10.5% average RMSE reduction over the unconstrained iterative baseline. Furthermore, on bitcoin, the spurious-item rate falls from 40.3% to 4.0%, and the method receives high automatic-judge scores for causal, commonsense, and logic quality, as well as high expert scores on these dimensions and on usefulness.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Large Language Models, LLM Agents, Time Series Forecasting, Explainable Process Reward Model, Agentic Reasoning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_qxQFOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a framework for news-augmented time-series forecasting that uses causal, commonsense, and logic-consistent reflection to identify useful news. The main component is an Explainable Process Reward Model (EPRM), which selects news based on both its impact on forecasting accuracy and the quality of the associated reasoning. The selected news is used to improve an LLM-based forecasting model, and the refined filtering logic is applied during inference.

The authors evaluate their method on four datasets (electricity, Bitcoin, traffic, and exchange rates) and report improvements over existing baselines. They also provide ablation studies, human and LLM-based evaluations of explanation quality, robustness tests against spurious news, and convergence analyses.

Critical strengths
  • S1. The paper presents an interesting approach to news-augmented time series forecasting by considering causal, commonsense, and logical consistency alongside forecasting accuracy. Incorporating reasoning quality into news selection is a potentially useful way to reduce the influence of irrelevant or misleading news.
  • S2. The proposed method improves forecasting accuracy across four real-world datasets. The authors also provide ablation studies and robustness experiments, which help support the effectiveness of the proposed framework.
Critical weaknesses
  • W1. It remains unclear whether the performance improvements come from the proposed causal, commonsense, and logic-consistent reasoning (as the authors intended) or simply from filtering out noisy and irrelevant news. Although the authors provide ablation studies, they do not compare against simpler selection strategies, such as semantic similarity-based filtering or LLM-based relevance scoring.
  • W2. The performance improvements may partly result from reducing the amount of noisy information rather than improving reasoning quality. For example, the authors do not report or control the number of selected news articles or input tokens across different variants.
  • W3. The ablations are conducted only on the Bitcoin dataset.
  • W4. The method uses fixed thresholds for rational-quality labeling and news selection, as well as a fixed number of candidate articles and refinement rounds. The authors do not provide sufficient sensitivity analyses for these choices.
  • W5. It remains unclear whether the benefits of reasoning-based news selection generalize to other forecasting architectures, including time-series foundation models such as Chronos, TimesFM, and Moirai.
  • W6. The paper does not provide a detailed comparison of computational costs with the iterative baseline, such as training time, LLM calls, and inference overhead.
  • W7. There could exist potential overfitting to the validation set, as the method repeatedly evaluates forecasting performance on historical validation data to refine news selection and update the filtering logic.
Questions for authors
  • Q1. Have the authors considered comparing the proposed method with simpler news selection strategies, such as semantic similarity-based filtering or LLM-based relevance scoring?
  • Q2. How many news articles and input tokens are selected by each variant in the ablation studies? Could the performance degradation of the variant without rational-quality labels be explained by differences in the amount of selected news?
  • Q3. Do the contributions of causal, commonsense, and logic-consistent reasoning remain consistent across the other three datasets? The current ablation studies are limited to Bitcoin.
  • Q4. How sensitive is the proposed method to the rational-quality thresholds, the number of candidate articles, and the number of refinement rounds?
  • Q5. Is the proposed news selection framework applicable to other forecasting backbones, such as time-series foundation models such as Chronos, TimesFM, and Moirai?
  • Q6. What is the computational overhead of the proposed method compared with the iterative baseline, including LLM calls, EPRM training, and repeated forecasting model fine-tuning?
  • Q7. Since the filtering logic is repeatedly updated based on validation RMSE, how do the authors ensure that it does not overfit to the validation set? Have the authors evaluated the stability of the results across different data splits or random seeds?
19 comments
AuthorsOpenReview ↗

Response to Reviewer qxQF (Part1, W1 & Q1)

We sincerely thank Reviewer qxQF for the thorough reading and constructive comments. Below, we address each of your concerns (W1–W7) and questions (Q1–Q7) with new baselines, token/article volume statistics, cross-domain ablations, backbone transfers, runtime breakdowns, and multi-split/multi-seed stability analyses.

1. Comparison with Simpler News Selection Strategies (W1 & Q1)
To verify that our improvements stem from causal, commonsense, and logic-consistent reasoning rather than simple noise removal, we evaluated two standard non-EPRM selection baselines under the exact same Qwen3-8B (LoRA) backbone, candidate pool, 4-article per-window cap, and 1,500-character truncation budget:

  • Sim-Filter (Semantic Similarity / Dense RAG): Selects news articles by dense embedding cosine similarity (all-mpnet-base-v2) to the forecasting task query.
  • LLM-Rel (Single-Pass LLM Relevance Scoring): Prompts the same LLM agent (DeepSeek-V4-Flash) to filter relevant articles directly in a single pass, without iterative explainable reflection or EPRM scoring.

Table R1 (Top): Comparison with simpler context-selection baselines on Qwen3-8B (Test RMSE; lower is better).

Forecasting BackboneNews Selection MethodElectricityBitcoinTrafficExchangeAvg. vs. IB
Qwen3-8B (LoRA)No-News (Numerical Only)349.401383.9123.167.25+18.3%
(Fixed Backbone,Sim-Filter (Dense RAG)289.981213.0521.676.66+4.9%
Same Context Budget)LLM-Rel (Single-Pass LLM)288.411101.7821.136.12-0.6%
IB (Wang et al., 2024)286.05944.6221.357.540.0%
Ours (Explainable + EPRM)272.07834.7320.425.95-10.5%
  • Superiority over Simpler Filters: Ours consistently achieves the lowest RMSE across all four domains, outperforming Sim-Filter by 13.5% on average, LLM-Rel by 9.0%, and IB by 10.5%.
  • Why Unconstrained Iteration Fails: On Traffic and Exchange, single-pass LLM-Rel (21.13, 6.12) outperforms multi-round IB (21.35, 7.54). This confirms that unguided iterative retrieval can inject spurious articles that degrade forecasts over rounds, whereas our structured reflection and joint EPRM gate effectively prevent this degradation (20.42, 5.95).
AuthorsOpenReview ↗

Response to Reviewer qxQF (Part2, W2 & Q2)

2. Controlling Selected News Volume and Input Tokens across Variants (W2 & Q2)
To directly answer Q2, we report the average number of selected news articles and average input news tokens per forecasting window across all four domains for Full (Ours) and the ablated variants:

VariantElectricity (Articles / Tokens)Bitcoin (Articles / Tokens)Traffic (Articles / Tokens)Exchange (Articles / Tokens)
Full (Ours)0.99 / 278.103.96 / 838.221.27 / 307.234.00 / 918.14
– causal1.04 / 243.893.99 / 792.931.32 / 284.174.00 / 779.47
– commonsense1.03 / 228.293.98 / 724.921.36 / 266.074.00 / 710.95
– logic1.05 / 227.783.98 / 754.321.35 / 265.634.00 / 762.00
– rational label1.06 / 283.564.00 / 903.551.36 / 327.254.00 / 935.45

These statistics confirm that our gains come from reasoning and selection quality rather than feeding fewer articles or shorter prompts into the forecaster:

  1. Consistent Per-Window Budgets and Virtually Identical Article Counts: To fit within the 2,048-token context window alongside numerical time series and rationales (Section 3.5, Appendix A), all variants (including IB) share the same per-article truncation (1,500 chars) and cap selection at a maximum of 4 articles per window. In event-dense financial domains (Bitcoin, Exchange), nearly every window reaches this 4-article cap (3.96–4.00), whereas in infrastructure domains (Electricity, Traffic), routine windows naturally contain only 1–2 relevant events passing initial filtering (0.99–1.36). Across all domains, average article counts differ by at most 0.00 to 0.09 articles between Full and the ablations (where in a few noisy windows, fewer than 4 candidates pass Full's 3-dimension gate θgate=0.35\theta_{\text{gate}} = 0.35—while at least one passes, so empty-gate fallback is not needed—leaving those slots unfilled).
  2. Full Uses More Tokens than – causal / – commonsense / – logic, Yet Achieves Much Lower Error: Because ablating one reasoning dimension omits that rationale field from the prompt (Appendix G.5), – causal, – commonsense, and – logic (as well as IB, which omits all three rationale fields) actually feed fewer news tokens into the forecaster (e.g., 724.92–792.93 vs. 838.22 on Bitcoin; 227.78–243.89 vs. 278.10 on Electricity), yet suffer substantially worse RMSE (1057.66–1192.60 vs. 834.73 on Bitcoin). This directly rules out the hypothesis that Full wins simply by reducing input token length.
  3. Why – rational label Degrades (+53.0% RMSE on Bitcoin) Under the Same Budget: – rational label and Full retain the exact same 3-part rationale format and have nearly identical article counts (4.00 vs. 3.96) and token counts (903.55 vs. 838.22 on Bitcoin; 935.45 vs. 918.14 on Exchange). Without the rationality constraint (1[qt(k)>0.7]\mathbb{1}[q_t^{(k)} > 0.7]), the EPRM overfits to coincidental historical error reductions (1[Δ(k)>0]\mathbb{1}[\Delta^{(k)} > 0], reward hacking) and selects spurious headlines that hurt test generalization.
  4. Token-Length-Invariant Verification: In our Chronos-bolt-mini transfer experiment (Table R2 in Point 5), all selected articles in a window are pooled into a single fixed-dimensional vector (all-mpnet-base-v2), eliminating token count entirely; our EPRM-selected news still consistently outperforms IB news.
AuthorsOpenReview ↗

Response to Reviewer qxQF (Part3, W3 & Q3)

3. Ablation Studies Across All Four Datasets (W3 & Q3)
We expanded the ablation study in Table 2 across all four benchmark datasets (reporting test RMSE below; full metrics will be added to Section 4.3):

VariantElectricityBitcoinTrafficExchange
Full (Ours)272.07834.7320.425.95
– causal348.551192.6021.085.99
– commonsense340.991057.6620.596.04
– logic354.041081.9321.775.97
– rational label345.841277.1320.915.98
– all (IB)286.05944.6221.357.54
  • Consistent Necessity Across All Domains: Across all four datasets, Full (Ours) consistently achieves the lowest RMSE (272.07, 834.73, 20.42, and 5.95), outperforming every ablated variant and confirming that all three rational constraints and the joint label are necessary. On Exchange (7.54 for IB) and Traffic (21.35 for IB), partial variants already improve over IB (5.97–6.04 on Exchange; three of four variants at 20.59–21.08 on Traffic), while combining all components in Full yields the strongest performance.
  • Component Coupling in High-Noise Regimes (Bitcoin & Electricity): Unlike IB (which retrieves via simple category keywords without rationale injection or reward ranking), our framework actively injects generated rationales into the forecasting prompt (Appendix G.5) and searches over candidates via the EPRM. In domains with intense speculative noise or abrupt spikes (Bitcoin and Electricity), generative expansion and reward search act as a double-edged sword unless fully regularized: dropping rational label causes reward hacking on coincidental Δ(k)>0\Delta^{(k)} > 0; dropping commonsense or logic allows hallucinated multi-hop narratives or cross-round contradictions; and dropping causal selects broad background news lacking a directional cause -> mediator -> effect pathway.
Reviewer_kLPJOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents an explainable reflection framework for news-enhanced time-series forecasting. It asks an LLM to generate structured reflections based on causal relations, commonsense knowledge, and logical consistency, and then uses these reflections to retrieve relevant news. The authors also train an Explainable Process Reward Model (EPRM) to select news based on both its improvement to forecasting accuracy and the quality of its explanation. The selected news is added to the long-term filtering rules only when it reduces the validation error after retraining. Experiments on electricity, Bitcoin, traffic, and exchange-rate forecasting show that the method achieves the best results on all reported metrics and reduces RMSE by an average of 10.5% compared with the main iterative baseline. Ablation studies, expert evaluation, and fake-news injection experiments further suggest that the proposed reasoning constraints and EPRM improve both forecasting accuracy and explanation quality.

Critical strengths
  1. The paper studies an important problem in news-enhanced time-series forecasting: highly correlated news may not provide a reliable or meaningful basis for prediction. Instead of focusing only on forecasting errors, the method requires causal, commonsense, and logically consistent explanations. This makes the news selection process more transparent and easier to examine.

  2. The EPRM provides a useful way to combine forecasting gains with explanation quality. A candidate news item should both improve forecasting accuracy and pass the rationality evaluation. The long-term filtering rules are also updated only when retraining leads to a lower validation error. This conservative selection and update process can reduce the lasting influence of accidental correlations or misleading news.

Critical weaknesses
  1. Failed updates are not fully rolled back. When a newly added news article increases the validation error, the algorithm only avoids updating the persistent filtering logic. The article remains in the working news set and may continue to affect later iterations and the final forecasting model. This may create a mismatch between the filtering logic and the context used to train the forecaster.

  2. The evaluation lacks strong baselines that use news or external context. Except for the reimplemented IB, most baselines can only use numerical time-series data. Therefore, the current experiments do not clearly show whether the improvements come from the EPRM and structured reflection or simply from access to additional news information.

Questions for authors
  1. When a newly added article worsens validation performance, does the actual implementation remove the article and restore the previous forecasting model? If so, why is this rollback not included in Algorithm 2? If not, how do the authors prevent failed articles from affecting later iterations and the final model?

  2. Why are stronger context-aware forecasting baselines and time-series agents not included? Could the authors also combine the current news-retrieval module with other forecasting methods, for example by providing the same retrieved news to different forecasting backbones, to separate the contributions of retrieval, the EPRM, and the forecasting model?

  3. The paper reports the best refinement round for IB and the proposed method. Is this round selected using the validation set or the test set? If test RMSE is used, this may introduce test-set selection bias.

  4. DeepSeek-V4-Flash is used both to generate explanations and to evaluate their quality. Did the authors independently validate the EPRM training labels through human annotation or evaluation by other models?

  5. Why is (q>0.7) used during training while (q>=0.35) is used during selection? Were these thresholds supported by sensitivity analysis? When no candidate passes the gate, why does the method fall back to the ungated candidate set?

9 comments
AuthorsOpenReview ↗

Response to Reviewer kLPJ (Part1, W1 & Q1)

We sincerely thank Reviewer kLPJ for the careful reading of our algorithmic details and for recognizing the value of our EPRM and conservative update mechanism in improving both transparency and accuracy (S1, S2). Below, we address each of your concerns and questions.

1. Rollback Mechanism for Failed Updates in Algorithm 2 (W1 & Q1)

Thank you for catching this ambiguity in how Algorithm 2 is presented!

In our actual implementation, when a newly added news article does not reduce the validation error (Δt+1≤0\Delta_{t+1} \le 0), we DO roll back both the working news set N\mathcal{N} and the forecasting model FψF_\psi to the previous best state. This keeps failed articles from piling up or affecting later iterations and the final model. We also log the rejected candidate and its reflection, so later rounds try other candidates instead of repeating the failed update.

In the original manuscript, Section 3.4 and Appendix C stated that N\mathcal{N} receives nt∗n_t^* before retraining (Line 14). This step is needed to fine-tune FψF_\psi at Line 16 and to evaluate Δt+1\Delta_{t+1} at Line 18. However, our pseudocode in Algorithm 2 (Lines 21–22) and the accompanying text only wrote out the non-update of L\mathcal{L} in the else branch (Δt+1≤0\Delta_{t+1} \le 0). They omitted the explicit state-restoration step:

N(t+1)←N(t),Fψ(t+1)←Fψ(t),RMSEt+1←RMSEt.\mathcal{N}^{(t+1)} \leftarrow \mathcal{N}^{(t)}, \quad F_{\psi^{(t+1)}} \leftarrow F_{\psi^{(t)}}, \quad \mathrm{RMSE}_{t+1} \leftarrow \mathrm{RMSE}_t.

As a result, it looked as if failed articles stayed in the working set and only L\mathcal{L} was protected.

In the revision, we have updated Algorithm 2 and the corresponding text in Section 3.4 and Appendix C. They now explicitly state that N\mathcal{N} and FψF_\psi are rolled back whenever Δt+1≤0\Delta_{t+1} \le 0.

AuthorsOpenReview ↗

Response to Reviewer kLPJ (Part2.1, W2 & Q2)

2. Stronger Context-Aware Baselines and Decoupling Backbones (W2 & Q2)

First, we respectfully clarify that our main experiments already isolate the contribution of our reflection/EPRM framework from mere access to news text: in Tables 1 and 2, No-News, IB (Wang et al., 2024), all four ablation variants (–causal, –commonsense, –logic, –rational label), and Ours share the exact same Qwen3-8B backbone, candidate pool, and 1,500-character truncation budget. Notably, keeping structured reflections but training the EPRM on RMSE alone (–rational label, 1277.13 RMSE on Bitcoin) performs worse than IB (944.62), demonstrating that the gains do not come from simply adding news, but specifically from our rational constraints and joint EPRM supervision.

To further address your suggestion and decouple news selection from the forecasting model, we conducted three sets of additional experiments:

  1. Additional Context-Aware & News-Filtering Baselines (Fixed Backbone: Qwen3-8B LoRA): Under the same Qwen3-8B backbone and character truncation budget, we added two standard non-EPRM selection methods alongside the iterative agent baseline IB (Wang et al., 2024):
    • Sim-Filter (Dense RAG Baseline): Selects news articles by dense embedding cosine similarity (all-mpnet-base-v2) to the task query.
    • LLM-Rel (Single-Pass LLM Filter): Prompts the same LLM agent (DeepSeek-V4-Flash) to filter relevant articles directly in a single pass, without iterative explainable reflection or EPRM scoring.
  2. Decoupling News Selection on a Different LLM Backbone (Llama-3.1-8B LoRA): We froze the final news sets selected by IB and by Ours (EPRM) and used them directly to fine-tune a different LLM backbone (Llama-3.1-8B with the same LoRA setup), without re-running reflection or EPRM for Llama.
  3. Decoupling News Selection on a Numerical Time-Series Foundation Model (Chronos-bolt-mini): Because numerical models cannot directly consume text tokens, we encoded the frozen IB and EPRM news texts into dense vectors via all-mpnet-base-v2 and fitted a lightweight residual regression predictor on top of Chronos-bolt-mini (using the same model checkpoint as Table 1).

Table R1: Context-aware baselines on the same backbone and frozen-news transfer to Llama-3.1-8B (Test RMSE; lower is better).

Forecasting BackboneNews Selection MethodElectricityBitcoinTrafficExchangeAvg. vs. IB
Qwen3-8B (LoRA)No-News (Numerical Only)349.401383.9123.167.25+18.3%
(Main Backbone,Sim-Filter (Dense RAG)289.981213.0521.676.66+4.9%
Same Context Budget)LLM-Rel (Single-Pass LLM)288.411101.7821.136.12-0.6%
IB (Wang et al., 2024)286.05944.6221.357.540.0%
Ours (Explainable + EPRM)272.07834.7320.425.95-10.5%
Llama-3.1-8B (LoRA)No-News (Numerical Only)357.581058.5023.598.33+13.0%
(Cross-Backbone Transfer,IB (Frozen News Set)291.261006.2322.507.000.0%
Frozen News Sets)Ours (EPRM Frozen News Set)270.63910.0520.826.07-9.3%

Table R2: Frozen-news transfer to numerical foundation model Chronos-bolt-mini via all-mpnet-base-v2 residual fusion (lower is better).

MetricBackbone & News SetElectricityBitcoinTrafficExchange
RMSEChronos-bolt-mini (No News)620.821711.6828.186.10
+ IB Frozen News612.911717.7028.316.33
+ Our EPRM Frozen News606.831697.4428.466.07
MAEChronos-bolt-mini (No News)511.911515.7017.085.31
+ IB Frozen News508.221513.2617.195.52
+ Our EPRM Frozen News505.601499.7017.365.31
AuthorsOpenReview ↗

Response to Reviewer kLPJ (Part2.2, W2 & Q2)

Key Takeaways:

  1. Comparison against Context-Aware Baselines (Table R1, Top): With the backbone fixed to Qwen3-8B, Ours outperforms Sim-Filter (by 13.5% on average), LLM-Rel (by 9.0%), and IB (by 10.5%) across all four domains. Notably, on Traffic and Exchange, single-pass LLM-Rel (21.13, 6.12) outperforms multi-round IB (21.35, 7.54). This confirms that unconstrained iterative retrieval can inject spurious articles that degrade forecasts over rounds, whereas our causal/commonsense/logic reflection and EPRM gate effectively prevent this degradation (20.42, 5.95).
  2. Backbone-Agnostic Improvement on LLMs (Table R1, Bottom): When transferring the frozen news sets to Llama-3.1-8B, our EPRM-selected news reduces RMSE across all four datasets—by 19.3% on average over No-News and 9.3% on average over IB news (7.1% on Electricity, 9.6% on Bitcoin, 7.5% on Traffic, and 13.3% on Exchange). On Electricity, Llama-3.1-8B + EPRM news (270.63) even slightly outperforms Qwen3-8B + EPRM news (272.07), confirming that the selected news quality generalizes across LLM backbones.
  3. Transfer to Numerical Foundation Models (Table R2): Even with simple post-hoc embedding fusion on Chronos-bolt-mini, EPRM news improves both RMSE and MAE on Electricity, Bitcoin, and Exchange. More importantly, on noisy financial domains (Bitcoin and Exchange), IB news hurts base Chronos (1711.68 -> 1717.70 and 6.10 -> 6.33 RMSE), whereas our EPRM news filters out spurious correlations and yields net gains (1697.44 and 6.07). On Traffic—where strong numerical seasonality dominates and exogenous text is least informative (Sec. 4.2)—adding a post-hoc dense embedding regressor slightly overfits for both news sets, whereas sequence-level conditioning in LLMs (Qwen3-8B and Llama-3.1-8B) consistently extracts useful signal.
Reviewer_gXSaOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper considers the causal and logical consistency of reasoning and news filtering in news-driven time series forecasting. The paper proposes an EPRM to filter news. Results show that there is an average RMSE reduction of 10.5% across four datasets.

Critical strengths
  1. This paper shows a good exploration of time-series reasoning. Developing better reasoning chains to filter forecasting context is a meaningful direction with practical potential.

  2. The authors provide a comprehensive comparison across four datasets, with some promising predictive results. For the Bitcoin dataset in particular, their method reduced the selection of spurious news from 40.3% to 4.0%.

Critical weaknesses
  • The paper claims that it performs causal reasoning and causal verification. But in fact, the methods rely on an LLM to generate causal chains and another LLM to assess their plausibility. In general, rigorous causal verification usually requires intervention experiments, counterfactual analysis, or other independent causal evidence based on quantitative analysis (like DID analysis). Existing results primarily show that the method can more precisely filter relevant news and reduce spurious information, but they do not support strong causal claims. I recommend that the authors be more cautious when mentioning causality, weaken these claims, and clearly distinguish them from prior work on general news filtering.

  • The ablation study requires further clarification. In Table 2, removing any single component worsens performance relative to IB. This suggests a coupling between components within the framework. However, it is counterintuitive that individual components fail to contribute positively. The authors should explain this in more detail. They need to demonstrate that each component is indeed necessary.

Questions for authors

Q1. This method involves multiple rounds of LLM reflection, EPRM evaluation, and fine-tuning. What are the computational cost and runtime for the entire pipeline (both offline refinement and online inference)? If requiring frequent updates or near-real-time predictions, is this approach feasible to deploy?

Q2. This paper uses a fixed filtering logic without further iterative reflection. However, in real-world scenarios, news events and time-series distributions constantly evolve. How to ensure that this fixed filtering logic remains adaptable to new events? Do they need to periodically rerun the entire refinement process, and if so, what is the cost of those updates?

4 comments
AuthorsOpenReview ↗

Response to Reviewer gXSa (Part1, W1 and W2)

We deeply appreciate Reviewer gXSa's positive assessment (Rating: 3 - Weak Acceptance) and insightful comments recognizing the practical potential of our reasoning-driven context filtering and spurious news reduction (from 40.3% to 4.0%). Below, we address your concerns and questions in detail.

1. Clarifying Terminology Regarding "Causal Reasoning" (W1)
We completely agree with your insightful distinction. Unlike econometric or interventional causal inference (e.g., DID or counterfactual experiments), our framework utilizes LLMs to construct and evaluate plausible causal narratives and domain mechanisms to filter out spurious correlations. Following your recommendation, we will revise our terminology throughout the paper—replacing strong claims like "rigorous causal verification" with more precise phrasing such as "LLM-based plausible causal reflection" or "causality-inspired plausibility filtering"—and explicitly discuss this distinction in the revised manuscript.

2. Clarification on Component Coupling and Necessity in the Ablation Study (W2)
Thank you for raising this crucial observation. To thoroughly address your concern, we expanded the ablation study in Table 2 across all four benchmark datasets (reporting test RMSE below; full metrics will be added to Section 4.3):

VariantElectricityBitcoinTrafficExchange
Full (Ours)272.07834.7320.425.95
– causal348.551192.6021.085.99
– commonsense340.991057.6620.596.04
– logic354.041081.9321.775.97
– rational label345.841277.1320.915.98
– all (IB)286.05944.6221.357.54

These cross-domain results clarify both the positive contribution of individual components and why tight coupling is essential in high-noise regimes:

  • 1. Individual components do contribute positively, and every component is strictly necessary:

    • Necessity across all domains: Across all four datasets, Full (Ours) consistently achieves the lowest RMSE (272.07, 834.73, 20.42, and 5.95), outperforming every ablated variant. Removing any single component consistently degrades accuracy relative to Full, demonstrating that no component is redundant.
    • Positive gains over IB: On Exchange (where unguided IB suffers from noisy retrieval at 7.54), every partial variant substantially outperforms IB (5.97–6.04). Similarly, on Traffic (21.35), three of the four partial variants already improve over IB (20.59–21.08), while combining all components in Full pushes performance furthest (5.95 and 20.42).
  • 2. Why removing a single component drops below IB on Bitcoin and Electricity (The Coupling Mechanism):

    • Unlike IB—which conservatively retrieves news via simple category keywords without a reward ranker or rationale injection—our framework performs active multi-step reflective expansion (injecting generated rationales into the forecasting prompt, Appendix G.5) and uses an active EPRM ranker to search over candidates. In domains with intense speculative noise or abrupt spikes (Bitcoin and Electricity), generative expansion and reward search act as a double-edged sword unless fully regularized:
    • – rational label (Training EPRM on RMSE only): When the EPRM is trained purely on validation error reduction (Δ(k)>0\Delta^{(k)} > 0) without the rationality threshold (q>0.7q > 0.7), the reward model overfits to coincidental historical error reductions (reward hacking). It actively selects spurious news that harms test generalization even more than IB's unguided retrieval (1277.13 vs. 944.62 on Bitcoin; 345.84 vs. 286.05 on Electricity).
    • – commonsense or – logic (Unchecked Generative Expansion): Prompting an LLM to hypothesize causal links without commonsense grounding (– commonsense) leads to "hallucinated causality"—fabricating speculative multi-hop narratives to justify noisy headlines. Similarly, without cross-iteration verification (– logic), the agent accumulates redundant or mutually contradictory drivers across refinement rounds.
    • – causal (Missing Directional Mechanism): Relying on general commonsense and consistency without requiring an explicit cause -> mediator -> effect pathway selects broad background news that lacks a directional forecasting signal.
AuthorsOpenReview ↗

Response to Reviewer gXSa (Part2, Q1 and Q2)

3. Computational Cost, Runtime, and Near-Real-Time Feasibility (Q1)
Our pipeline strictly separates offline iterative refinement from online forecasting inference:

  • Offline Phase: Iterative LLM reflection, Stage-A EPRM training (Appendix B), and Stage-B LoRA fine-tuning are performed entirely offline on historical data. On the Bitcoin dataset, our pipeline reaches its validation optimum in 2 refinement rounds (achieving 836.40 test RMSE in Table 6, within 0.2% of the 3-round checkpoint 834.73 in Table 1, and already outperforming 5-round IB by 11.5%) in 1,723.2 mins (~28.7 hours)—comprising 1,585.0 mins for LLM reflection and candidate labeling (872.0 min in r1, 713.0 min in r2), 59.2 mins for Stage-A EPRM training across the two bootstrap rounds prior to freezing PϕP_\phi (29.0 min in r1, 30.2 min in r2; Appendix B), and 79.0 mins for forecaster LoRA fine-tuning (38.9 min in r1, 40.1 min in r2)—compared to 1,705.3 mins (~28.4 hours) for IB across 5 rounds (944.62 RMSE; 1,588.0 mins for LLM reflection and 117.3 mins for LoRA fine-tuning). Note: During offline news selection, we invoked the same DeepSeek-V4-Flash model across different API platforms; due to external network fluctuations and platform-specific API rate limits, the offline LLM filtering times are provided for reference only.
  • Online Inference Phase: During deployment, the refined filtering logic is already fixed, and no fine-tuning or iterative reflection is needed. Evaluated using our locally deployed model (which eliminates network variance and provides a reliable benchmark), total online inference requires only 8 mins 15 secs (8:15), which is virtually identical to the IB baseline (8 mins 08 secs, 8:08) and fully supports near-real-time forecasting.

4. Adaptability to Evolving News Distributions and Periodic Updates (Q2) This is an excellent practical question.

  • Inherent Robustness to Concept Drift: Because our filtering logic distills high-level causal, commonsense, and logical principles (e.g., "regulatory policy announcements impact liquidity, whereas celebrity gossip without institutional backing is spurious") rather than memorizing transient keywords, it remains naturally resilient as specific news events evolve over the test horizon.
  • Low-Cost Incremental Updates: To verify long-horizon adaptability, we divided the test set chronologically into three sequential periods (Early, Mid, Late). Even without updating, our fixed logic maintains highly consistent error reductions across all four datasets (-10.6%, -10.3%, and -10.7% average RMSE change vs. IB in Early, Mid, and Late, respectively; e.g., -11.5%, -11.3%, and -12.1% on Bitcoin):
DomainEarly (Ours vs. IB)Mid (Ours vs. IB)Late (Ours vs. IB)
Electricity266.21 vs. 279.89 (-4.9%)277.68 vs. 293.31 (-5.3%)272.15 vs. 284.73 (-4.4%)
Bitcoin837.52 vs. 946.46 (-11.5%)823.18 vs. 927.79 (-11.3%)843.84 vs. 960.12 (-12.1%)
Traffic19.45 vs. 20.37 (-4.5%)21.08 vs. 22.01 (-4.2%)20.71 vs. 21.65 (-4.3%)
Exchange5.62 vs. 7.14 (-21.3%)6.28 vs. 7.87 (-20.2%)5.94 vs. 7.60 (-21.8%)

Furthermore, if periodic updates (e.g., monthly) are desired in production, warm-starting from the existing checkpoint and filtering logic requires only 1 refinement round on recent sliding-window data. Based on our warm-started Round 2 benchmark on Bitcoin, a full single-round update (even including an optional EPRM refresh) takes about 783.3 mins (~13.1 hours)—comprising 713.0 mins for LLM reflection, 30.2 mins for EPRM refresh, and 40.1 mins for forecaster LoRA fine-tuning (or substantially less when keeping the frozen EPRM and restricting updates to a smaller recent sliding window)—and runs asynchronously in the background without interrupting online service. Note: Due to platform-specific API rate limits and external network fluctuations during LLM reflection, this update time is provided for reference only.

Thanks for the clarifications and the additional experimental results. I support revising the terminology of "causal reasoning" to a more precise phrasing. However, I remain concerned that some conclusions extend beyond what the experimental results directly support.

  • It is unclear whether the reported inference time of about 8 minutes represents the total time for the entire test set or the average duration for a single inference. If it is the latter, an 8-minute inference time hardly supports the claim of near-real-time capability. If it is the former, please provide average single-inference latency across different domains. Current results can indicate that the method has a computational overhead comparable to the IB baseline, but this is insufficient to substantiate a claim of "near-real-time forecasting".

  • Similarly, as for concept drift, while consistent performance gains across different time periods suggest a degree of "temporal stability," this is not direct proof of robustness against concept drift. This claim requires stronger evidence from experiments under scenarios involving significant shifts in news distribution or structural changes in the market.

I understand that the rebuttal period is limited and that the authors cannot conduct every additional experiment to fill in all the details; however, the authors should be more precise when interpreting their results and avoid overclaiming. Therefore, I encourage the authors to further clarify the alignment between the experimental results and their corresponding conclusions in the final manuscript.

Some of my concerns have been addressed, so I will maintain my original score.

Dear Reviewer gXSa,

Thank you very much for your prompt feedback, constructive guidance, and understanding of the limited rebuttal timeframe. We truly appreciate your rigorous standards regarding how experimental results should be interpreted. Thank you once again for your time and constructive engagement, which have significantly strengthened the clarity and rigor of our paper!

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.