acceptodds
Under review as a conference paper at ICLR 2027

Belief-State Management Meets Rubric Rewards for Agentic Forecasting

Abstract

Current large language model (LLM) agents still struggle to forecast real-world events. This stems from two primary limitations: i) standard ReAct harness accumulates reasoning and evidence within one growing context, and ii) outcome-based rewards in forecasting lead to a slow and noisy signal that cannot distinguish sound reasoning from lucky guesses. To address these, we introduce a forecasting training framework including harness design and post-training. For the harness side, we propose Proactive Belief-State Management (PBM), equipping agents with a fold tool to compress lengthy context into compact belief states and reset the context for subsequent exploration. For the training side, we develop SEIPH, a five-dimensional, outcome-blind rubric, providing process-quality rewards. Using the rubric-based reward, we post-train a 35B model with PPO and GRPO, yielding DeepForecaster-PPO and DeepForecaster-GRPO. Theoretically, we prove that idealized fold-based stopping converges and calibrated local rewards reduce segment-level gradient variance. Empirically, DeepForecaster-PPO (35B) achieves the best Brier score on both ForecastBench and Polymarket-Sim, outperforming frontier agents based on DeepSeek-V4-Pro (1.6T), Kimi-K2.6 (1T), and GPT-5.5. DeepForecaster-GRPO ranks second on both FutureX metrics. Ablations reveal that PBM also improves the untrained base model without any training. Calibration analysis shows that DeepForecaster-PPO makes more accurate high-confidence predictions than Qwen3.6 and GPT-5.5. Trajectory analysis further shows that forecasts generally improve from the first to the final belief state as agents gather evidence and revise their judgments.

Primary area:
foundation or frontier models, including LLMs
Keywords:
forecasting, LLM agent, harness, reinforcement learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_H6DjOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes DeepForecaster, a forecasting framework combining (i) Proactive Belief-State Management (PBM), which periodically compresses search trajectories into structured belief states and resets the context, and (ii) SEIPH, an outcome-blind five-dimensional rubric that provides segment-level process rewards for RL. The authors train 35B Qwen3.6 models with SFT followed by PPO/GRPO and evaluate them on FutureX, ForecastBench, and Polymarket-Sim. DeepForecaster-PPO achieves the best Brier scores on ForecastBench and Polymarket-Sim and outperforms several much larger frontier models, while ablations suggest that PBM itself improves the base model.

Critical strengths
  1. Strong empirical results with useful ablations.
  2. A coherent combination of harness design and training signal.
Critical weaknesses
  1. Dependence on an LLM-based reward judge. SEIPH is scored by GLM-5.2, but there is limited validation of judge reliability. Since the central RL contribution relies on these process rewards, this makes it difficult to determine how much of the improvement comes from genuinely better forecasting/reasoning versus optimizing for the judge's preferences.
  2. Limited evaluation scale and disentanglement. The main Polymarket-Sim result uses only 126 resolved questions from a short time period, making it unclear whether the reported gains generalize across different forecasting periods and event distributions. Also, the reported gains from PBM, SFT, and RL are somewhat intertwined. Stronger multi-seed and broader temporal evaluation would make the main claims more robust.
Questions for authors
  1. How reliable is the GLM-5.2 SEIPH judge? Do you have human agreement measurements or an independent evaluation showing that SEIPH scores correlate with human assessments of forecasting-process quality?
  2. How robust are the reported improvements across seeds and evaluation periods? In particular, do the PBM and PPO gains persist when evaluating on a larger set of questions spanning substantially longer or different time periods?
1 comment

Thank you for your review. We provide additional evidence on judge reliability and robustness below.

Response to Q1

We report agreement with human judgments and the relationship between rubric rewards and forecasting performance.

(1) Agreement with human judgments. On 50 manually labeled segments, GLM-5.2’s overall process score achieves Pearson r = 0.732, supporting alignment with human assessments of forecasting-process quality.

Table R1: Agreement with human judgments. MAE is measured on the 0–1 scale.

Pearson r [95% CI]Spearman ρ [95% CI]MAE ↓
0.732 [0.511, 0.877]0.612 [0.328, 0.808]0.1115

(2) Relationship with forecasting performance. We evaluate eight model configurations on the same set of questions. Mean rubric reward correlates positively with 1-Brier, with Pearson r = 0.798, 95% CI [0.445, 0.910]. This provides additional evidence that the process quality measured by SEIPH is associated with forecasting performance at the model level.

Table R2: Rubric scores and forecasting performance.

Model+HarnessRubric reward↑1−Brier↑
Qwen3.6+ReAct0.8110.766
Qwen3.6 PBM-SFT0.8790.797
DeepForecaster-PPO+PBM0.9320.816
DeepForecaster-GRPO+PBM0.9390.809
DeepSeek-V4-Flash+ReAct0.8570.771
DeepSeek-V4-Pro-Preview+ReAct0.8810.815
GLM-5.2+ReAct0.9080.791
Kimi-K2.6+ReAct0.8960.806

We have added the human agreement results to Appendix D.1 (p. 26), and the relationship with forecasting performance to Appendix D.3 (p. 26).

Response to Q2

We added 179 Polymarket-Sim questions from November 8 to 13, 2026, with three inference runs. Together with the original 126 questions from August 31 to September 5, this gives 305 questions across two evaluation periods more than two months apart.

1. PBM and PPO improvements persist across periods and repeated runs.

Table R3: Forecasting performance across two evaluation periods and the combined dataset.

Model + HarnessOriginal 126: 1−Brier ↑New 179: 1−Brier ↑Combined 305: 1−Brier ↑Combined 305: Accuracy (%) ↑
Kimi-K2.6 + ReAct0.82810.8745 ± 0.00360.855378.14
DeepSeek-V4-Flash + ReAct0.81920.8816 ± 0.00130.855880.33
GLM-5.2 + ReAct0.82570.8776 ± 0.00310.856178.58
Qwen3.6 + ReAct0.77850.8490 ± 0.01370.819975.08
Qwen3.6 + PBM0.80650.8731 ± 0.00680.845677.81
DeepForecaster-PPO + PBM0.85340.8918 ± 0.00230.875981.97
DeepForecaster-GRPO + PBM0.83980.8806 ± 0.00780.863881.09

(1) PBM improves Qwen3.6 in both periods, increasing (1-Brier) by 0.0280 and 0.0241, respectively. PPO achieves the highest (1-Brier) in both periods. (2) On the new questions, all three PPO runs outperform all three runs of the strongest ReAct baseline, DeepSeek-V4-Flash.

2. Our evaluation covers different forecast horizons and prediction dates.

Our original evaluation also includes longer-horizon questions from FutureX and ForecastBench, alongside short-horizon questions from Polymarket. FutureX was evaluated over three weekly rounds, and the original Polymarket-Sim evaluation covered six daily rounds. The new November evaluation adds a separate period, providing further evidence that the gains persist across prediction dates.

We have added the November evaluation and repeated-run results to Appendix C.3 (p. 24). The forecast horizons and original evaluation rounds are described in Appendix B.2 (p. 20).

Reviewer_W1J8OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes DeepForecaster, an LLM agent framework for forecasting real-world events. It introduces Proactive Belief-State Management, which integrates search and fold tools to manage forecasting trajectories through structured belief states. It also designs SEIPH, a five-dimensional rubric reward, and train a 35B-A3B model using SFT followed by PPO or GRPO. Experiments on multiple forecasting benchmarks compare the trained models with frontier models.

Critical strengths
  1. Meaningful problem. The paper studies problem in real-world forecasting, the difficulty of managing long agent trajectories and the delayed and noisy nature of outcome-based feedback. These are important issues for developing forecasting agents.

  2. Promising empirical results. DeepForecaster demonstrates practical potential through its SFT and RL training pipeline. DeepForecaster achieves Brier scores of 0.151 on ForecastBench and 0.147 on Polymarket-Sim, outperforming several frontier models.

Critical weaknesses
  1. Limited methodological novelty and unclear research contribution.

The proposed framework shares substantial similarities with existing approaches. AgentFold [1] has already explored proactive context folding, while Context Folding [2] further combines context management with RL and process rewards. Rubrics as Rewards [3] and CaRR [4] have also investigated rubric-based supervision for RL. Although the proposed PBM and SEIPH are tailored to forecasting, the overall approach appears closer to an engineering integration of existing techniques than a clearly novel methodological contribution. The paper does not sufficiently distinguish its key innovations from these related works or demonstrate why existing approaches cannot be directly adapted to the same task.

  1. Insufficient validation of the proposed rubric rewards.

SEIPH is a central contribution, but the five dimensions and their equal weighting are manually specified, and the scores rely on a single LLM judge. Although the framework includes validity checks, these do not establish the correctness or reliability of the rubric-based rewards. Previous work, such as Rubrics as Rewards [3], has investigated rubric quality through reference-guided generation, judge-alignment evaluations, and ablations of different rubric designs. In contrast, this paper provides limited validation of whether SEIPH scores consistently reflect forecasting quality or whether its individual dimensions are necessary. This is important because optimizing the apparent quality of reasoning does not necessarily improve the accuracy or calibration of forecasting probabilities.

References

[1] Ye, Rui, et al. "Agentfold: Long-horizon web agents with proactive context management."

[2] Sun, Weiwei, et al. "Scaling Long-Horizon Agent via Context Folding."

[3] Gunjal, Anisha, et al. "Rubrics as rewards: Reinforcement learning beyond verifiable domains."

[4] Zhang, Jiajie, et al. "Chaining the evidence: Robust reinforcement learning for deep search agents with citation-aware rubric rewards."

Questions for authors
  1. What is the primary methodological contribution beyond integrating existing context-folding and rubric-based RL techniques? There are currently many harnesses designed for frontier models. How does PBM differ from existing approaches, and what prevents these approaches from being directly applied to forecasting?
  2. How are the correctness and reliability of SEIPH rewards validated? Have the authors evaluated human–judge agreement, judge consistency, or the relationship between rubric scores and actual forecasting quality? What motivates the five equally weighted dimensions?
  3. Although the author has explained that the training data was filtered by time, potential data leakage still exists in the training process; for example, when training involves using search engines, this may introduce additional trend information into the model. How are such issues avoided? Furthermore, as predicting future events is inherently uncertain, could testing over a single time period introduce bias?
3 comments

Thank you for constructive feedback. We address the concerns below.

Response to Q1

Our contribution is to organize forecasting around evolving beliefs that guide further research and provide a clear basis for process supervision. We make this clearer in Section 2.1 in the revised paper.

1. Existing harnesses do not fully meet the needs of forecasting. Forecasting is a long-horizon reasoning and search task. It requires maintaining evidence from multiple sources, the reasons behind current probabilities, and useful directions for further research. (1) General context management is not well suited to forecasting tasks. For example, AgentFold requires compression and summarization at each intermediate step. Codex and Claude Code automatically compact context at a token threshold. Therefore, general compression alone does not ensure that these forecasting needs are met. (2) Existing forecasting harnesses provide limited context management. BLF updates beliefs at every tool call, potentially adding low-value information, while retaining the full interaction history.

PBM encourages the agent to complete a research stage before summarizing, preserving accumulated evidence while updating its belief. We also conduct comparison with other harnesses in Table R1. Our PBM achieves the highest average accuracy.

Table R1: Forecasting accuracy(%)↑ across harnesses.

ModelReActCodexAgentFoldBLFPBM
Kimi-K2.681.7579.8978.7782.6882.12
DeepSeek-V4-Flash83.8083.2482.6883.8084.36
GLM-5.283.0582.1281.5681.5683.80
Average82.8781.7581.0082.6883.43

2. Delayed and noisy outcomes motivate rubric rewards. Forecasting outcomes can take weeks or longer to resolve. Moreover, a lucky guess can succeed while a well-supported forecast can fail. These properties motivate timely supervision of forecasting behavior. (1) Process rewards. SEIPH evaluates evidence use, reasoning, and probability judgments using information available during forecasting. (2) Stage-level supervision. PBM naturally provides meaningful research boundaries, allowing SEIPH to assign multiple rewards within one prediction task and give feedback on each stage’s work.

Response to Q2

We evaluate human–judge agreement, consistency across judges, and relationship between rubric rewards and forecasting performance. We have added these results into Appendix D (pp. 26-27).

(1) Human–judge agreement. On 50 manually labeled segments, GLM-5.2 agrees with human judgments at Pearson r=0.732 in Table R2.

Table R2: Consistency with human judgments and LLM judges.

Compared with GLM-5.2Pearson rSpearman ρMAE↓
Human0.7320.6120.1115
DeepSeek-V4.1-Flash0.6770.6720.167
GLM-5.30.6740.6480.202

(2) Judge consistency. GLM-5.2 agrees with both LLM judges (Table R2) and has a mean reward SD of 0.071 across three evaluations (Table R3). During RL, we average three evaluations per segment to reduce noise.

Table R3: Stability of different judges.

JudgeMean reward SD↓
GLM-5.20.071
DeepSeek-V4.1-Flash0.155
GLM-5.30.073

(3) Relationship with forecasting performance. Across eight agents, rubric reward correlates with (1-Brier) at Pearson r=0.798, 95% CI [0.445, 0.910]. This indicates a positive relationship between rubric scores and forecasting performance.

Table R4: Rubric rewards and forecasting performance.

Model+HarnessRubric reward↑1−Brier↑
Qwen3.6+ReAct0.8110.766
Qwen3.6 PBM-SFT0.8790.797
DeepForecaster-PPO+PBM0.9320.816
DeepForecaster-GRPO+PBM0.9390.809
DeepSeek-V4-Flash+ReAct0.8570.771
DeepSeek-V4-Pro-Preview+ReAct0.8810.815
GLM-5.2+ReAct0.9080.791
Kimi-K2.6+ReAct0.8960.806

(4) Choice of dimensions and weights. We chose the five dimensions based on common errors found when manually inspecting trajectories. We also observed similar average scores across the five dimensions. Therefore, we choose the simple and reasonable equal weighting for our paper.

Response to Q3

We train and evaluate on live questions before outcomes are known. RL questions resolved after training, so agents could not retrieve realized outcomes through web search.

  1. Current trends are valid inputs for forecasting. Actually, we allow agents to search and read publicly available web information. Like human forecasters, they are expected to use current events and trends to predict future.

  2. We evaluate across different time periods and forecast horizons. FutureX covers three weekly rounds in September 2026, predicting events within the following week. Polymarket-Sim covers six daily rounds from August 31 to September 5, predicting next-day events. These cover relatively longer and shorter forecast horizons, respectively. We generate new predictions in each round and report averages across the three weeks for FutureX and the six daily rounds for Polymarket-Sim.

Thank you for the response and the additional experiments. The results address most of my concerns, though I still have concerns about the method's novelty. I will raise my rating to 3 (weak acceptance).

Thanks for your time and feedback. Your suggestions indeed help us improve our work.

Our novelty lies in a forecasting training recipe that tightly couples PBM with SEIPH. PBM organizes the forecasting process into meaningful stages that enable finer-grained supervision by SEIPH, while SEIPH evaluates each stage’s quality to improve belief management without relying on the final outcome.

We are happy to address any further questions you may have. Thanks again!

Reviewer_33XsOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a training framework that combines PBM with segmented process rewards to enhance agentic forecasting. It proactively invokes a fold action for organizing the current prediction, evidence, search history, and pending tasks into a structured state, and uses this to reset the context. It also designs the SEIPH scoring criterion to provide process rewards for each research segment without relying on the event’s final outcome. It trains DeepForecaster using PPO and GRPO based on SFT, and evaluates forecasting performance on FutureX, ForecastBench, and Polymarket-Sim.

Critical strengths
  • The agent maintains the current prediction, evidence, search history, and to-do list in a structured state and resumes its investigation from that state after a fold, which helps reduce logs of raw interactions while preserving the progress.

    • Training using process rewards also provides feedback even before events are revealed, aligning with the reality of delayed outcomes in prediction tasks.
  • The PPO version achieved higher Brier scores on ForecastBench and Polymarket-Sim benchmarks than other LLM baselines it was compared against. Even when using the same base model weights, the introduction of PBM alone improved results on Polymarket-Sim, providing preliminary evidence of the effectiveness of state management itself.

Critical weaknesses
  • Experiments primarily compare the method described in this paper with the ReAct framework; there is a lack of direct comparisons with existing active context management methods, or belief-state forecasting methods.

  • SEIPH scoring relies on a single LLM judge, but the paper does not sufficiently examine its consistency with human judgments or its stability across different judges.

  • Polymarket-Sim contains only 126 questions, and PPO has a Brier score advantage of 0.006 over the strongest LLM baseline; however, the main results table does not report paired confidence intervals for differences between models or the variance across multiple runs.

  • This one is minor: The SFT and RL experiments in this paper were conducted exclusively using Qwen3.6-35B-A3B; other models were primarily included in the comparison as inference baselines. It remains unclear whether the benefits of PBM and segmented process rewards can be generalized to different model families or scales. Have the authors obtained results from training with PBM and segmented process rewards on other backbone models?

Questions for authors

See weaknesses above.

3 comments

Thank you for your constructive feedback. We address your concerns below with additional experiments and analyses.

Response to Q1

Thank you for this suggestion. We have added comparisons with Codex, AgentFold[1], and the belief-state forecasting method BLF[2] on 179 new Polymarket-Sim questions from November 8 to 13, 2026.

Table R1: Forecasting accuracy (%) ↑ across different harnesses.

ModelReActCodexAgentFoldBLFPBM
Kimi-K2.681.7579.8978.7782.6882.12
DeepSeek-V4-Flash83.8083.2482.6883.8084.36
GLM-5.283.0582.1281.5681.5683.80
Average82.8781.7581.0082.6883.43

Shown in Table R1, our proposed PBM achieves the highest average forecasting accuracy across three backbones. We have added these harness comparisons to Section 5.3 and Figure 5 (p. 8) of the revised paper.

References:

[1] Ye R, Zhang Z, Li K, et al. "Agentfold: Long-horizon web agents with proactive context management"

[2] Kevin Murphy. "Agentic Forecasting with Structured Linguistic Beliefs"

Response to Q2

We manually labeled 50 segments to assess agreement with human judgments and evaluated consistency across judges. The results are shown in Table R2. (1) GLM-5.2’s scores correlate with human at Pearson r = 0.732, supporting alignment with human assessments of process quality. (2) Its scores also correlate with those from DeepSeek-V4.1-Flash and GLM-5.3, with Pearson r = 0.677 and 0.674, respectively.

Table R2: Consistency results with human judgements and LLM judges.

Compared with GLM-5.2Pearson r [95% CI]Spearman ρ [95% CI]MAE ↓
Human judgments0.732 [0.511, 0.877]0.612 [0.328, 0.808]0.1115
DeepSeek-V4.1-Flash0.677 [0.353, 0.858]0.672 [0.535, 0.772]0.167
GLM-5.30.674 [0.226, 0.895]0.648 [0.470, 0.783]0.202

We have added these human agreement and cross-judge consistency results to Appendix D.1 (p. 26) of the revised paper.

Response to Q3

We compute a paired confidence interval for the original results and add 179 Polymarket-Sim questions from November 8 to 13, 2026, with three independent runs.

1. Paired confidence intervals clarify the strength of the evidence.

On the original 126 questions, PPO improves Brier over DeepSeek-V4-Pro by 0.0059 with a 95% paired confidence interval of [−0.0081, 0.0213]. On the 179 new questions, PPO improves Brier over the strongest ReAct baseline in this evaluation, DeepSeek-V4-Flash, by 0.0101, with a 95% paired confidence interval of [0.0042, 0.0172]. Its accuracy also increases by 1.68 percentage points, with an interval of [0.19, 3.54].

Table R3: Paired improvements on the 179 new questions.

ComparisonΔ(1−Brier) [95% CI]ΔAccuracy [95% CI]
PPO + PBM vs. DeepSeek-V4-Flash + ReAct+0.0101 [0.0042, 0.0172]+1.68 [0.19, 3.54]
Qwen3.6 + PBM vs. Qwen3.6 + ReAct+0.0241 [0.0074, 0.0413]+3.54 [0.19, 7.08]
PPO + PBM vs. Qwen3.6 + PBM+0.0187 [0.0064, 0.0322]+3.17 [0.56, 5.96]

2. Repeated runs show consistent performance.

PPO achieves the highest mean performance on both metrics across three inference runs. PBM also significantly improves the untrained Qwen3.6 model over ReAct.

Table R4. Performance on the 179 new questions.

ModelHarness1−Brier ↑Accuracy (%) ↑
Kimi-K2.6ReAct0.8745 ± 0.003681.75 ± 1.96
DeepSeek-V4-FlashReAct0.8816 ± 0.001383.80 ± 0.00
GLM-5.2ReAct0.8776 ± 0.003183.05 ± 1.71
Qwen3.6-35B-A3BReAct0.8490 ± 0.013778.77 ± 2.01
Qwen3.6-35B-A3BPBM0.8731 ± 0.006882.31 ± 1.29
DeepForecaster-PPOPBM0.8918 ± 0.002385.47 ± 0.56
DeepForecaster-GRPOPBM0.8806 ± 0.007883.99 ± 0.32

Across the combined 305 questions, PPO achieves a Brier score of 0.1241, improving over the strongest ReAct baseline evaluated on all 305 questions, GLM-5.2 (0.1439), by 0.0198. These additional results strengthen the evidence for the improvement.

We have added the repeated-run results and the paired confidence interval improvement results to Appendix C.3 (p. 24).

Response to Q4

PBM and segmented process rewards are designed to work across backbone models. We chose Qwen3.6-35B-A3B because it was a strong open-weight model available at the time and offered a suitable balance between capability and training cost within our budget. Since RL training and forecasting evaluation require substantial compute and time, we have not conducted SFT or RL on other backbones. We appreciate the reviewer’s understanding.

I appreciate the authors’ efforts in providing updated results and addressing my comments. And I will raise the score.

Thanks for the feedback and increasing the score. Your suggestions indeed help us improve our work. Thanks!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.