acceptodds
Under review as a conference paper at ICLR 2027

ForeWeave: Learning What to Investigate for Event Forecasting

Abstract

Forecasting future events requires agents to acquire evidence before decisive outcomes are known. Yet an apparently plausible forecast can still depend on consequential conditions that remain unresolved. We introduce ForeWeave, a framework for process-guided adaptive investigation that connects reusable structures from historical investigations with evidence gaps revealed during a current investigation. ForeWeave induces historical process structures to guide three view-specific investigations, then performs targeted continuation from a selected trajectory prefix when an unresolved issue could change the forecast. A Forecast Synthesis Agent audits the initial and continued records, performs targeted verification when blocking evidence gaps remain, and returns the final answer and probability. We evaluate ForeWeave on FutureX-Past with Qwen, DeepSeek, and GPT backbones, and on ForeWeave-Live with DeepSeek and GPT. Across both benchmarks, ForeWeave achieves the highest Overall Score, outperforming the strongest baseline by 1.96–6.45 points. It achieves Overall Scores of 49.33, 56.66, and 59.51 on FutureX-Past with Qwen, DeepSeek, and GPT, respectively, and 70.59 and 67.65 on ForeWeave-Live with DeepSeek and GPT.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Event Forecasting, LLM Agents, Adaptive Investigation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_SsHSOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper provides a framework to improve LLM-based future-event forecasting through historical process guidance and adaptive investigation. It extracts reusable investigation structures from historical forecasting trajectories and introduces evidence-driven targeted continuation. Then a forecast synthesis agent evaluates the collected evidence and produces the final prediction and probability. Experiments of three different LLM backbones were conducted on FutureX-past and ForeWeave-Live benchmark. Results shows improvements in overall forecasting scores over the baselines.

Critical strengths
  1. The paper addresses the challenge of identifying unresolved conditions in LLM-based future-event forecasting. This differs from conventional approaches that primarily focus on evidence retrieval and forecast aggregation.

  2. The framework combines multiple investigation perspectives, targeted follow-up investigations, and evidence-based synthesis to improve forecasting. Its ability to revisit unresolved issues from existing investigation trajectories is particularly interesting.

  3. The paper evaluates ForeWeave using three LLM backbones on both retrospective and prospective forecasting tasks. The results show consistent improvements in overall score compared with the baselines.

Critical weaknesses
  1. Although ForeWeave consistently achieves higher overall scores, the results in Table 3 show that its Brier score is usually higher. Similar inconsistencies are observed on ForeWeave-Live as well. This creates concerns about ForeWeave's performance consistency.
  2. The paper provided comparisons involve base agent, self-consistency, and a textual experience-guided baseline. However, it lacks direct comparisons with closely related forecasting frameworks, such as Agentic Time Machine and ForecastCompass, making it difficult to assess its advantages over existing methods.
  3. The individual effectiveness of stable pivots is not isolated from the anchor process graphs, and the proposed prefix-based targeted continuation is not directly compared against alternative follow-up strategies, such as starting a new investigation targeting the same unresolved issue.
Questions for authors
  1. The paper lacks direct experimental comparisons with closely related forecasting frameworks. Is it possible for the authors to provide the results of the frameworks mentioned in the Related Work section? For example, Agentic Time Machine and ForecastCompass.
  2. The paper states that ForeWeave-Live questions were collected on September 23–24, 2026. But the case study in Section 4.4 shows a match held on September 22 in the Netherlands. Could the authors please clarify the time zone used for collection timestamps and provide the exact forecast timestamp for this case to verify its prospective validity?
  3. Why does ForeWeave worsen the Brier score in many cases on FutureX-Past despite improving the overall score?
  4. Can the authors provide the ablation study of mechanisms like stable pivots and prefix-based continuation to justify their necessity?
1 comment
AuthorsOpenReview ↗

Author Response (1/3)

Dear Reviewer SsHS,

We sincerely thank you for your thoughtful review and helpful suggestions, and for recognizing the value of investigating unresolved conditions. Below are our detailed responses to your concerns:

W2&Q1: Can the authors provide comparisons with Agentic Time Machine (ATM) and ForecastCompass?

Under our evaluation setting, ForeWeave outperforms our adapted ForecastCompass (FoCo) reproduction by 4.20–9.21 Overall Score points across Qwen, DeepSeek, and GPT on the same FutureX-Past questions.

BackboneBase AgentForecastCompass (reproduced)ForeWeaveGain over FoCo
Qwen43.7142.4449.33+6.89
DeepSeek51.8552.4656.66+4.20
GPT51.6250.3059.51+9.21

Scores are official weighted Overall Scores. Base Agent and ForeWeave scores follow the main-results table.

  • Data and reproduction setting. To enable a fair comparison, we construct FoCo’s factor and reasoning memories using the same independently collected set of resolved historical questions used for ForeWeave’s process library. This keeps the historical question source and dataset size consistent across the two methods.

  • Implementation details. We follow the original FoCo procedure to construct its factor and reasoning memories. We use our Tavily retrieval, temporal auditing, and answer-format requirements. The forecasting stage allows up to 60 turns without a separate search-call cap, matching the setting used for Base Agent.

  • Findings and methodological distinction. Under our evaluation setting, ForeWeave outperforms the adapted FoCo baseline across all three backbones, extending the empirical comparison to a related forecasting-memory framework. FoCo organizes historical experience into factor and reasoning memories, while ForeWeave uses historical investigation structures to guide evidence collection and targeted continuation. These methodological distinctions are discussed in Sec5.

ATM's official implementation is not publicly available, limiting faithful reproduction of its forecasting system. We therefore focus the additional empirical comparison on FoCo.

Reviewer_2HpoOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes an event prediction framework ForeWeave. The framework learns investigation structures from historical trajectories, guides three types of investigations tailored to specific perspectives, and extends selected trajectories to fill gaps in the evidence. An integrated agent is responsible for generating the final predictions. Experiments on the FutureX-Past and ForeWeave-Live datasets demonstrate that the framework improves task scores across a variety of foundation models.

Critical strengths
  1. Reusing historical investigation processes and targeting unresolved issues are promising ideas.
  2. ForeWeave achieves higher scores across the evaluated settings. The mechanism analyses provide some evidence for the benefits of multi-view investigation and targeted continuation.
Critical weaknesses
  1. ForeWeave improved task scores, its Brill scores were lower than those of Base Agent on all three FutureX-Past foundation models. The evaluation sample size was small, with only 6 questions each in L1 and L3. The evaluation lacked confidence intervals and paired significance tests. These issues undermine the credibility of the empirical conclusions.
  2. There is a lack of sufficient isolated analysis of the proposed contributions. ForeWeave employs more exploration and synthesis steps than the baseline models. There is a lack of powerful multi-view baseline models with comparable computational budget. The experiments failed to distinguish between prefix-based continuation and entirely new, targeted exploration. It remains unclear which components account for the performance improvements.
Questions for authors
  1. The live case in Section 4.4 involves an incident that occurred on September 22, but the real-time dataset in the report was collected on September 23-24. Does this case fall within the scope of the evaluation? When were the prediction results recorded? Was the historical process repo constructed using only information available before the relevant evaluation cutoff date?
  2. Under matched token and tool budget conditions, how does ForeWeave perform compared to a robust multi-perspective baseline? How does the prefix-based continuation method perform compared to a novel directed search approach?
  3. Can the authors provide paired confidence intervals, significance tests, and the number of valid counts for the Brill score? Why did the improvement in task scores not translate into a higher Brill score?
Reviewer_w9rTOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

Foreweave

When a forecast is made, there are certain assumptions that the forecast relies on. Classic forecasting harnesses may make mistakes. In this paper, the authors mine historic forecasts and their trajectories to create a structure over prior forecasts. They then develop a forecasting harness that consists of three different prompts: current state, temporal dynamics, and disconfirmations. Afterwards, they are able to use a graph representation of these past runs to conduct local edits where mistakes are made. Finally, the forecast is synthesized together.

Weaknesses:

  • This paper is awful to read. I could barely understand what they are trying to do. It uses overcomplex language to define, what appears to be something fairly simple: “creating a corpora of past forecasts

  • It is really unclear what motivated this harness or even the problem the paper is trying to solve.

“A revenue forecast, for example, may rely on positive performance signals without establishing whether they concern the target business segment and reporting period” (pdf) I don't know what this means.

It is unclear to me how “current state, temporal dynamics, and disconfigurations” were created.

This paper seems like it’s doing a few things:

  1. Proposing an architecture for a forecasting agent.

  2. Processing historic transcripts

  3. Targeted continuation descriptions

Each of these ideas could be really interesting, but it feels like no one area is explored in sufficient depth.

Questions:

  • How are “usable investigation trajectories” available for prediction markets? Do you have historic forecasts from the APIs?

  • You state the in the second sentence of the abstract: “Yet an apparently plausible forecast can still depend on consequential conditions that remain unresolved”. Do you think you ran any tests that actually show that the harness you built addresses this?

  • Are you worried about the lack of cost controlled comparisons in the evals?

Writing:

Overall, the writing is impossible to parse. I spent WAY too long just trying to understand the overly complex language. Something that could be addressed by writing simply.

“ForeWeave induces historical process structures to guide three viewspecific investigations, then performs targeted continuation from a selected trajectory prefix when an unresolved issue could change the forecast.” (pdf)

  • Even after reading the paper, coming back to this I have no idea what it says.

“ForeWeave distills reusable checking relationships from historical trajectories into answer-free process guidance” (pdf)

  • Again, unnecessarily complex sentence. What are “checking relationships”?
Critical strengths

Look at summary

Critical weaknesses

Look at summary

Questions for authors

Look at summary

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.