- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper provides a framework to improve LLM-based future-event forecasting through historical process guidance and adaptive investigation. It extracts reusable investigation structures from historical forecasting trajectories and introduces evidence-driven targeted continuation. Then a forecast synthesis agent evaluates the collected evidence and produces the final prediction and probability. Experiments of three different LLM backbones were conducted on FutureX-past and ForeWeave-Live benchmark. Results shows improvements in overall forecasting scores over the baselines.
Critical strengths
-
The paper addresses the challenge of identifying unresolved conditions in LLM-based future-event forecasting. This differs from conventional approaches that primarily focus on evidence retrieval and forecast aggregation.
-
The framework combines multiple investigation perspectives, targeted follow-up investigations, and evidence-based synthesis to improve forecasting. Its ability to revisit unresolved issues from existing investigation trajectories is particularly interesting.
-
The paper evaluates ForeWeave using three LLM backbones on both retrospective and prospective forecasting tasks. The results show consistent improvements in overall score compared with the baselines.
Critical weaknesses
- Although ForeWeave consistently achieves higher overall scores, the results in Table 3 show that its Brier score is usually higher. Similar inconsistencies are observed on ForeWeave-Live as well. This creates concerns about ForeWeave's performance consistency.
- The paper provided comparisons involve base agent, self-consistency, and a textual experience-guided baseline. However, it lacks direct comparisons with closely related forecasting frameworks, such as Agentic Time Machine and ForecastCompass, making it difficult to assess its advantages over existing methods.
- The individual effectiveness of stable pivots is not isolated from the anchor process graphs, and the proposed prefix-based targeted continuation is not directly compared against alternative follow-up strategies, such as starting a new investigation targeting the same unresolved issue.
Questions for authors
- The paper lacks direct experimental comparisons with closely related forecasting frameworks. Is it possible for the authors to provide the results of the frameworks mentioned in the Related Work section? For example, Agentic Time Machine and ForecastCompass.
- The paper states that ForeWeave-Live questions were collected on September 23–24, 2026. But the case study in Section 4.4 shows a match held on September 22 in the Netherlands. Could the authors please clarify the time zone used for collection timestamps and provide the exact forecast timestamp for this case to verify its prospective validity?
- Why does ForeWeave worsen the Brier score in many cases on FutureX-Past despite improving the overall score?
- Can the authors provide the ablation study of mechanisms like stable pivots and prefix-based continuation to justify their necessity?
Author Response (1/3)
Dear Reviewer SsHS,
We sincerely thank you for your thoughtful review and helpful suggestions, and for recognizing the value of investigating unresolved conditions. Below are our detailed responses to your concerns:
W2&Q1: Can the authors provide comparisons with Agentic Time Machine (ATM) and ForecastCompass?
Under our evaluation setting, ForeWeave outperforms our adapted ForecastCompass (FoCo) reproduction by 4.20–9.21 Overall Score points across Qwen, DeepSeek, and GPT on the same FutureX-Past questions.
Scores are official weighted Overall Scores. Base Agent and ForeWeave scores follow the main-results table.
Data and reproduction setting. To enable a fair comparison, we construct FoCo’s factor and reasoning memories using the same independently collected set of resolved historical questions used for ForeWeave’s process library. This keeps the historical question source and dataset size consistent across the two methods.
Implementation details. We follow the original FoCo procedure to construct its factor and reasoning memories. We use our Tavily retrieval, temporal auditing, and answer-format requirements. The forecasting stage allows up to 60 turns without a separate search-call cap, matching the setting used for Base Agent.
Findings and methodological distinction. Under our evaluation setting, ForeWeave outperforms the adapted FoCo baseline across all three backbones, extending the empirical comparison to a related forecasting-memory framework. FoCo organizes historical experience into factor and reasoning memories, while ForeWeave uses historical investigation structures to guide evidence collection and targeted continuation. These methodological distinctions are discussed in Sec5.
ATM's official implementation is not publicly available, limiting faithful reproduction of its forecasting system. We therefore focus the additional empirical comparison on FoCo.