acceptodds
Under review as a conference paper at ICLR 2027

Lazy Belief Expansion: Deciding What to Observe for World-Model Planning

Abstract

World models predict action consequences, but prediction alone does not determine which evidence the next decision needs. Exposing all available information adds processing cost and can distract reasoning, while a fixed compressed view can omit a decision-critical detail. We introduce *Lazy Belief Expansion* (*LBE*), a mechanism that lets the agent decide which evidence should condition world-model predictions and the next action. *LBE* retains source access while maintaining a compact working belief that can expand as evidence needs emerge. A controller chooses whether to *READ* from the current source, *ACT* to obtain missing evidence through interaction, or *COMMIT* the belief to task-directed planning; both environmental routes use the same planner. In a training-free web instantiation using *WMA* as the base planner, calibrated success rates for the 812-task benchmark are 19.33% for *LBE* and 18.72% for *WMA*. Among successful *LBE* tasks, final planning evidence is reduced by 54.6% relative to the corresponding untruncated source observations and by 52.8% relative to the context-capped full-observation inputs supplied to *WMA*; for tasks successfully completed in at most 5, 6–10, and 11–15 steps, *LBE*'s mean input-plus-output token usage is 22.4%, 16.0%, and 4.7% lower than *WMA*'s, respectively. Together, these results show that *LBE* integrates deciding what to know into deciding what to do without modifying the underlying predictive world model.

Primary area:
applications to robotics, autonomy, planning
Keywords:
world models, LLM agents, web navigation, active information acquisition, belief state, inference-time planning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_1pKeOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes the Lazy Belief Expansion (LBE) framework, which uses three decision operations: READ, ACT, and COMMIT. READ extracts additional information from the current observation, ACT acquires new information through interaction with the environment, and COMMIT passes the current belief to the planner for task-directed action selection.

Critical strengths

The proposed framework is simple and can be integrated into existing planners without additional training, which makes it practical and easy to adopt. The paper demonstrates improved efficiency through a meaningful reduction in token usage.

Critical weaknesses
  1. The proposed belief update appears to augment the existing context with retrieved information using fairly standard techniques, which seems conceptually close to Agentic RAG and adaptive retrieval methods. Similarly, the READ–ACT–COMMIT structure resembles existing agentic workflows that alternate between information retrieval, tool use, and task execution. It would be helpful if the authors could clarify the distinct algorithmic contribution of LBE relative to these retrieval-based approaches, for example through a more explicit discussion or a direct empirical comparison.

  2. Although the paper is framed around world-model-based planning, the READ–ACT–COMMIT decision does not appear to make use of the world model’s predictions or uncertainty estimates. The world model is subsequently used to plan both information-seeking and task-directed actions. Could the authors clarify whether LBE is specifically designed for world-model-based planning, or whether it is intended as a general mechanism applicable to any LLM-based agent? If the latter, it may be worth adjusting the framing accordingly. If the former, further explanation of how LBE benefits from the world model would strengthen the paper.

  3. The original WMA paper evaluates on two benchmarks, WebArena and Mind2Web, whereas this paper reports results only on WebArena. This leaves some uncertainty about how well the approach transfers to environments with different observation structures or task distributions. Additional evaluation on Mind2Web would help demonstrate the generalizability of the proposed approach.

Questions for authors

The main questions addressed in the Critical Weaknesses above.

Reviewer_Fc5WOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

LBE inserts a training-free controller between a web agent's observation and the WMA world-model planner. It keeps the current accessibility tree and exposes an 800-token BM25 overview of it. A GPT-4o-mini controller then chooses to READ more of the page, ACT to gather missing evidence, or COMMIT to a task action; ACT and COMMIT both call the unchanged WMA planner.

On the 812-task WebArena benchmark, success matches WMA: 19.33% vs. 18.72%, and the authors claim no gain. Among successful tasks, the final evidence the planner sees is 54.6% smaller than the page source. Token use is 22.4%, 16.0%, and 4.7% lower for successes in ≤5, 6–10, and 11–15 steps. Ablations show that READ and the ACT/COMMIT split matter for success, and that forcing three reads per step costs 83–86% more tokens.

Critical strengths

The ablations isolate the mechanism. With the planner and models fixed:

  • removing READ drops success from 19.33% to 13.42%;
  • merging ACT and COMMIT drops it to 16.63%; neither range overlaps LBE's;
  • forcing three reads keeps success but costs 83–86% more tokens.

→ This is the only evidence that deciding when to read matters beyond compression, and it is what keeps the paper from a clear reject.

The distinction between unexposed evidence (read it) and unavailable evidence (act for it) is a clean design principle showing progressive disclosure which is increasingly becoming an industry standard. The paper also declines to overclaim → a reusable idea even if the evaluation falls short.

Critical weaknesses

Critical 1: the efficiency claim, the paper's main contribution, isn't established.

  • Token savings are computed only over successful tasks (about 19% of the benchmark), this is a clear flaw.
  • They shrink to 4.7% on longer tasks.
  • They mix GPT-4o-mini and Llama-8B tokens.
  • FocusAgent is cheaper in every bin. → With success at parity, nothing shows a practical end-to-end saving.

Critical 2: the numbers don't reconcile.

Table 12's per-range success counts cannot come from the runs in Table 10. Full-Init's counts sum to 163 successes against a best run of 158; No-Intent's sum to 125 against a worst run of 128. The FocusAgent minimum of 15.40% is unattainable on 812 tasks. The run count is never stated. The 0.6-point gap (about 5 tasks) is far smaller than known WebArena evaluator error: AgentOccam corrected evaluators on 82 tasks. → The small effects can't be trusted without an audit.

Secondary: "training-free" overstates portability, and the key baseline is missing. The WMA world model was fine-tuned on 14K transitions collected from the WebArena sites themselves. AgentOccam's observation interface (43.1% on WebArena, zero-shot, GPT-4-Turbo) is cited but not run → the comparison conflates the interface with an in-domain model, far below the frontier.

Secondary: the organizing principle is already published. "Selective exposure with recoverable access" means compact summaries in front of the planner, with raw content retrievable on demand, motivated by raw HTML hurting web agents. That is the published design of ATLAS (Cheng et al., 2025, arXiv:2510.22732), which the paper does not cite. ATLAS's ablation shows a raw-HTML memory lowering WebArena-Lite success from 47.9% to 44.8%, while summaries raise it to 57.4% → the novelty reduces to the programmatic overview, within-step READ, and ACT/COMMIT intent; the paper should cite and discuss ATLAS.

Questions for authors

[Clarification] Runs and reconciliation. State the run count and seeds. Reconcile Tables 10 and 12, add LBE's row to Table 12, and fix the 15.40% minimum - this decides whether the soundness problems are transcription errors.

[Clarification] Full-benchmark, paired cost. Report total tokens, dollar cost, and wall-clock time over all 812 tasks including failures, broken down by component, plus a paired comparison on tasks both systems solve.

[Clarification] Baseline setup. WMA reports 13.5% with GPT-4o-mini at a 5-action budget, while your WMA reaches 18.72% at 15 steps. What else differs? Do your results hold under corrected evaluators?

[Clarification] World-model input and intent. How is the compact belief serialized for WMA's adapters, which were trained on WMA's own format? What exactly changes in the planner input between ACT and COMMIT?

[Suggestion] Run AgentOccam's interface with the same backbone. Add a second planner, model, and benchmark. Audit how many ACT actions change site state.

[Suggestion] Beyond ATLAS, cite the closest published instances of the idea:

Agent-E (Abuelsaad et al., 2024), whose web agent chooses among three DOM views per task; SWE-agent (Yang et al., 2024), where a 100-line windowed file viewer beats both a 30-line window and the whole file; MemGPT (Packer et al., 2023), which pages context in on demand. Also run AgentOccam's interface with the same backbone, add a second planner, model, and benchmark, and audit how many ACT actions change site state.

Reviewer_a84tOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper argues that world-model planning involves two decisions: predicting action consequences, and choosing which evidence to condition the predictions on. It proposes Lazy Belief Expansion (LBE), a method to have more control over an existing predictive layer (i.e., WMA).

At each step, a LLM controller based on GPT-4o mini would choose to either expand the belief (READ), hand the belief to a planner to output an evidence-seeking environment action (ACT), or have the same planner perform a task-directed action (COMMIT).

The authors evaluate the method on WebArena, where success improves slightly over WMA, but within noise. Among successful LBE tasks, the mean input+output tokens are moderately lower than WMA, especially for shorter tasks. The paper also includes a case study where reading depth follows the decision rather than the page length. Ablations are provided in Appendix C for different components of the method.

Critical strengths
  1. The paper provides a useful decomposition of the real problem. Active belief inference is an important and under-formalized part of LLM-based world-model agents. The identified distinction between unexposed evidence (covered by READ) and unavailable evidence (covered by ACT) is clean and general. It is possible to transfer this idea beyond web navigation and plug into an existing LLM-WM-based planner without modifying the WM.
  2. The proposed method lowers token cost at comparable success, with supporting analyses. Specifically, LBE matches WMA's success while using fewer tokens on short and medium-length successful tasks. Ablation studies show Fixed-READ reaches similar success but uses much more tokens, which supports adaptive stopping as the main source of the savings.
Critical weaknesses
  1. Novelty is overstate relative to prior work on LLM-based belief inference with world models. Section 2 says existing methods "do not model whether or when the scope of conditioning evidence should be expanded," which is too strong.
  • In [1], a LLM encoder writes a natural language belief state token by token, so the model already decides how much of the observation to encode and when to stop. [1] also plans with an LLM-based WM over these beliefs on web tasks, including WebArena
  • [2] and [3] already let the model decide when to retrieve more information, while [4] plans with an LLM used as a world model.
  • [5] also anticipates LBE's motivation at a conceptual and theoretical level with its Theorem 1 about the capacity of discrete, language-based states scales by lengthening the token sequence.
  • The paper would be stronger if it stated its distinctive contribution more precisely and explained what each element adds over a summary-based encoder whose length the LLM already controls.
  1. The empirical evidence is thin for the paper's claims.
  • The 0.61-point improvement over WMA is within noise, the number of repeated runs is not stated, and the brackets are min/max rather than confidence intervals.
  • The token saving results are only on successful tasks and unpaired, so the systems are compared on different task subsets
  • Only one benchmark (WebArena) and one controller/policy model (GPT-4o mini) are tested
  • There is no basic baseline -- what if we just used the same LLM to do that with full observation and free-form reasoning, without the world model?
  • The ablations are not thoroughly explained and can affect the main paper's claims. No-READ and No-Intent are both below plain WMA, so compact belief appears to hurt unless both reading and intent separation are present.

[1] https://arxiv.org/abs/2507.23773 [2] https://arxiv.org/abs/2305.06983 [3] https://arxiv.org/abs/2310.11511 [4] https://arxiv.org/abs/2305.14992 [5] https://arxiv.org/abs/2507.05169

Questions for authors
  1. How many repeated runs are used for each number? Can you provide 95% CI instead of the min/max?
  2. What is the total input+output token cost for all WebArena tasks, including flavors, for LBE and WMA?
  3. How does LBE compare with a direct LLM baseline (e.g., GPT-4o-mini with free-form CoT)? What if we change the LLM?
  4. Please provide more detailed explanation for the ablation study results
  5. Please explain what LBE adds beyond a summary-based LLM encoder (where the model already decides how much to write)

It will also be good to move a condensed ablation table into the main text, and expand the related-work discussion to cover the missing references

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.