acceptodds
Under review as a conference paper at ICLR 2027

EvoBelief: Falsifiable World Models for Self-Improving LLM Agents

Abstract

Self-evolving LLM agents typically improve how they act through accumulated experience, reflection, or skills. Yet even procedurally capable agents can fail when their beliefs about environment dynamics are incomplete, incorrect, or outdated. We study a complementary axis of self-evolution: evolving those beliefs during deployment. Such beliefs can support multiple tasks, but incorrect ones can also propagate across them. We introduce EvoBelief, a self-evolving world model that represents environment knowledge as falsifiable hypotheses. Unexpected outcomes and decision-relevant unknowns trigger hypothesis formation. Later observations and live controlled experiments provide independent validation before a rule can guide predictions or plans within its supported scope. Counterexamples narrow or retire rules as the environment changes. The language model remains frozen, and each rule remains linked to its evidence and the decisions it changes. Across four benchmarks spanning static dynamics, hidden drift, and scientific discovery, EvoBelief improves task performance and reuses learned rules across tasks. On ALFWorld without actor-side transition memory, it improves success over the matched Frozen WM control by 23.9 and 31.3 percentage points with Claude Haiku 4.5 and GPT-5.4, respectively. On MiniGrid, it revises stale rules and recovers after unannounced dynamics changes. In DiscoveryWorld's Reactor Lab, it discovers and validates quantitative laws, completing 83.3% of Normal-tier and 58.3% of Challenge-tier instances, versus 0% for each evaluated baseline. These results demonstrate self-improvement through testable, revisable environment knowledge rather than model-parameter updates.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Recursive Self-Improvement, World Model, Self-evolving Agents

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_5XzKOpenReview ↗
Rating
3
Confidence
3
Informativeness
3(+1)
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how agents can improve incomplete, incorrect, or outdated beliefs about their environment without updating the underlying LLM. EvoBelief maintains testable hypotheses, validates them through subsequent observations or active experiments, and allows supported rules to influence prediction and planning while revising or withdrawing them when counterexamples arise. Experiments across four benchmarks demonstrate improved task performance, cross-task rule reuse, recovery from hidden dynamics changes, and law discovery in simulated scientific environments.

Critical strengths
  1. The paper clearly separates storing an experience from allowing it to guide later decisions, which makes the contribution more substantial than a conventional memory mechanism.
  2. The paper provides a reasonably complete evidence chain beyond final success rates.
  3. This paper is well-written.
Critical weaknesses
  1. The generality of the framework remains limited by how it is instantiated. This makes it difficult to separate transferable methodological contributions from engineering tailored to these environments.
  2. Independent validation is central to the paper, but its additional benefit is not clearly isolated. See the questions below.
Questions for authors
  1. How much of the post-drift recovery in MiniGrid depends on rule withdrawal, rather than renewed exploration or subsequent planning?
  2. Removing the commitment gate gives results close to the full method in Table 12. Under what conditions does independent validation actually improve decisions?
  3. Which hypothesis types, conditions, functional forms, and experiment steps are predefined? Which components must change for a new environment?
  4. Beyond rule-use counts, what evidence shows that retaining rules across tasks actually improves success on later tasks?
5 comments
AuthorsOpenReview ↗

[1/3] Response to Reviewer 5XzK

We thank the reviewer for the thoughtful review, and for recognizing that EvoBelief separates storing an experience from letting it guide decisions!

[W1, Q3] Generality, and what is fixed in advance

EvoBelief is designed to learn how a particular environment works. For example, a MiniGrid agent can be told that it may pick up a key and toggle a door, and that the door is locked. It still has to discover which key opens the door, and whether that stays true later. The knowledge the agent learns should differ from one environment to the next. We never gave it the rules. Our question was whether an agent given a basic description of an environment can discover the hidden rules of how it works.

Across the submitted implementations, a hypothesis specifies a condition, an action or procedure, and a predicted outcome (C, A, Y). Appendix B describes action effects, properties and measurements, multi-step procedures, and conditional constraints. For cross-episode reuse, evidence must span at least two episodes and two bindings. Each benchmark also had a predefined vocabulary for conditions and outcomes, for example four state conditions on MiniGrid, and Reactor Lab considered four law families (constant, proportional, linear, and quadratic). Experiment steps are not written by hand but compiled from each hypothesis, as steps that set up its condition, take its action, and read its outcome. In the submitted versions, this vocabulary, a few task helpers, and the ways learned rules are used (Table 4, Appendix C) change with the environment, while the lifecycle stays the same.

To test generality more directly, we evaluated a version of EvoBelief in which the environment supplies only the goal, the observation text, and the list of available actions. The language model proposes candidate hypotheses directly from observations, without relying on benchmark-specific rules for hypothesis generation. The same procedure tests each rule's predictions and narrows or retires a rule when it is contradicted. The same code and prompts run unchanged on MiniGrid and ALFWorld. All methods were re-run in one shared setup with the same inputs, model (Claude Haiku 4.5), and budget, where each actor writes a short thought and sees the full history of the current episode up to that step, without retrieving transitions from earlier episodes (as in standard ReAct).

Claude Haiku 4.5, no transition memoryMiniGrid, 560 test episodesALFWorld, 134 unseen tasks
ReAct0.7%64.2%
Frozen WM (same actor, no learned rules)2.5%70.1%
WorldEvolver0.9%56.0%
ITP-I1.1%73.1%
EvoBelief, general version70.9%86.6%

On MiniGrid, where the environment departs from what the language model expects, the general version is ahead of all four baselines in this table. On ALFWorld, where the model's expectations are mostly right, it is still ahead of them. Our reply to Reviewer 8uWN also compares it with AutoManual and WALL-E 2.0. These results show that an environment can supply the actions while leaving the relevant conditions and outcomes for the agent to discover. We will add the general version and these results to the revision.

AuthorsOpenReview ↗

[2/3] Response to Reviewer 5XzK

[W2, Q2] When independent validation improves decisions

On deterministic MiniGrid, the submitted implementation induced rules in a small, predefined language. Table 12 showed little difference between full EvoBelief (69.6%) and allowing a rule to act on its first supported instance (68.3%). Those rules were rarely wrong, so the gate had little to filter. We revisited this question with a language model proposing the hypotheses. We therefore ran the gate ablation again on MiniGrid and ALFWorld. Shadow retains hypothesis formation and testing while disabling direct use of learned rules. In the no-gate arm, a proposed rule is immediately eligible to guide behavior and loses its authority after two consecutive counterexamples.

Success, general version, Haiku 4.5ShadowNo gateFull
MiniGrid29.8%40.2%70.9%
ALFWorld68.7%84.3%86.6%

On MiniGrid, the full method outperforms both variants, and the gate already leads before the hidden change, while the dynamics are still unchanged (78.9% vs. 68.6%). On ALFWorld, where the language model already knows most of the dynamics, the two versions solve a similar number of tasks, but the gate makes rule interventions far more reliable. With the gate, 12% of vetoes blocked an executable action, against 44% without it. A language model proposing rules freely is itself a noisy source of hypotheses, and the gate keeps that noise from reaching behavior. Validation therefore improves decisions when acting on a wrong rule is costly, as when the environment departs from what the language model expects or changes over time.

AuthorsOpenReview ↗

[3/3] Response to Reviewer 5XzK

[Q1] What drives recovery after the change in MiniGrid

Starting from the same state at the switch, disabling rule narrowing and retirement reduced post-switch success from 62.9% to 11.1%. We compared two continuations of each of the 14 test streams. The first is the full method, whose numbers are the test run's own episodes after the change. The second keeps everything, including experiments and new hypotheses, but never retires or narrows a confirmed rule.

MiniGrid, general version, Haiku 4.5, 280 episodes after the changeFull methodWithout narrowing or retirement
Success62.9%11.1%
Actions driven by outdated rules22849
Outdated rules still in force at the end1 of 3939 of 39

Every stream had confirmed rules that became wrong at the change. With retirement, 38 of these 39 rules lost their authority. Without it, they kept acting and drove 849 actions, most of them based on wrong predictions. Exploration and new hypotheses continued in both versions, so they do not bring recovery back on their own. The shadow version, which learns and explores in the same way but never lets rules act, reaches 22.1% after the change.

[Q4] Whether keeping rules improves later success

Yes. To isolate the effect of keeping rules, we ran a diagnostic test on ALFWorld with no learning during the test. This is not how EvoBelief is deployed, since it normally keeps learning. The agent first learned on the 140 development tasks. We then froze its rules and ran the 134 unseen tasks twice, once with these rules and once without them. Both runs use EvoBelief with the rule store frozen and experiments disabled, not the Frozen WM baseline of the main table. So the only difference between the two runs is the rules kept from earlier tasks.

ALFWorld, general version, Haiku 4.5, 134 unseen tasks, rules frozenSuccess
Without rules64.9% (87/134)
With the rules kept from the development tasks73.1% (98/134)

The kept rules let the agent solve 21 tasks that it misses without them, while it misses 10 that it solves without them. They vetoed 421 drafted actions in 119 tasks, and 22 of these vetoes were wrong. In the main results, the agent starts with no rules and learns during the test, and success reaches 86.6%.

We thank the reviewer again for the constructive feedback and questions. They helped make our work clearer and stronger, and we will add these results and analyses to the revision.

Reviewer_mnXcOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper describes EvoBelief, a pipeline for LLM agents to update a knowledge base of falsifiable hypotheses (encoded as a triplet of condition, action/procedure and consequence). A key idea is to turn surprising outcomes into testable hypotheses, which can be confirmed either through passive of active observation.

The idea is instantiated in 4 distinct text environment settings (ALFWorld, ScienceWorld, MiniGrid, DiscoveryWorld), with somewhat bespoke constructions for each of them. Across these settings, they show meaningful improvement in the performance metrics over relevant baselines. They perform various analyses, and give numerous examples, showing effects like transferring to new settings and updating beliefs under drift.

Critical strengths

The EvoBelief setup is quite intuitive and has been implemented in a way that seems to be effective on these tasks. The detailed analysis is comprehensive and suggestive, with fairly compelling examples.

Critical weaknesses

This currently seems to require a fairly customized approach for each type of environment, lending it less relevant for general usage where in most real-world settings there is a complicated, ill-defined set of actions, conditions, and outcomes.

Questions for authors

N/A

3 comments
AuthorsOpenReview ↗

Response to Reviewer mnXc

We thank the reviewer for the encouraging review, and for recognizing that EvoBelief is intuitive and effective. We agree that generality to new environments is central for a self-improving method like ours.

EvoBelief is designed to learn how a particular environment works. For example, a MiniGrid agent can be told that it may pick up a key and toggle a door, and that the door is locked. It still has to discover which key opens the door, and whether that stays true later. The knowledge the agent learns should differ from one environment to the next. We never gave it the rules. Our question was whether an agent given a basic description of an environment can discover the hidden rules of how it works.

To test generality more directly, we evaluated a version of EvoBelief in which the environment supplies only the goal, the observation text, and the list of available actions. The language model proposes candidate hypotheses directly from observations, without relying on benchmark-specific rules for hypothesis generation. The same procedure tests each rule's predictions and narrows or retires a rule when it is contradicted. The same code and prompts run unchanged on MiniGrid and ALFWorld. All methods were re-run in one shared setup with the same inputs, model (Claude Haiku 4.5), and budget, where each actor writes a short thought and sees the full history of the current episode up to that step, without retrieving transitions from earlier episodes (as in standard ReAct).

Claude Haiku 4.5, no transition memoryMiniGrid, 560 test episodesALFWorld, 134 unseen tasks
ReAct0.7%64.2%
Frozen WM (same actor, no learned rules)2.5%70.1%
WorldEvolver0.9%56.0%
ITP-I1.1%73.1%
EvoBelief, general version70.9%86.6%

On MiniGrid, where the environment departs from what the language model expects, the general version is ahead of all four baselines in this table. On ALFWorld, where the model's expectations are mostly right, it is still ahead of them. Our reply to Reviewer 8uWN also compares it with AutoManual and WALL-E 2.0. These results show that an environment can supply the actions while leaving the relevant conditions and outcomes for the agent to discover. We will add the general version and these results to the revision. Thank you again for this helpful comment.

Thank you for the responses, they are appreciated. My overall assessment of the work stays the same.

AuthorsOpenReview ↗

Thank you for the feedback and support!

Thank you so much for your constructive feedback and for your continued support of our work. We sincerely appreciate your recognition of EvoBelief as intuitive and effective. We will remain active until the end of the discussion period and would be glad to address any further questions.

Reviewer_8uWNOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

EvoBelief keeps a frozen LLM and adds a persistent, external store W_t of typed rules (C, A, Y) describing environment dynamics. Prediction mismatches or decision-critical unknowns trigger rival hypotheses. These gain “behavioral authority” only after meeting count thresholds across episodes, bindings, and independent tests, using passive transitions or live experiments, and they lose it under counterexamples. Validated rules revise predictions, veto candidates, or are chained into plans. Rule induction, testing, retrieval, and planning are performed by benchmark-specific code; the LLM drafts actions, gives anchor predictions, and parses observations.

The method is evaluated on ALFWorld and ScienceWorld, a custom MiniGrid suite with a silent key switch, and DiscoveryWorld (Reactor Lab, Plant Nutrients). It reports large gains over a Frozen WM control and over ReAct, ITP-I, WorldEvolver, and the Hypothesizer, including 83%/58% Reactor Lab completion versus 0% for all baselines.

Critical strengths
  1. The paper implemented a principled “belief evolution” lifecycle for LLM agents, explicitly separating situational state (z_t) from a persistent, falsifiable, and evidence-gated world model (W_t).
  2. The hypothesis structure (C, A, Y), evidence ledger, admissibility criteria, and commitment thresholds are described with care; design choices are motivated and linked to failure modes (e.g., confirmation bias).
  3. Evaluates on multiple regimes and shows gains over baselines (ReAct, WorldEvolver, ITP-I). teh authors also report efficiency metrics (decisions, model calls) and documents adaptation over time, including rule retirement and revalidation after drift.
Critical weaknesses
  1. Limited novelty: The core update loop is already well established and studied in detail in previous papers: a frozen LLM runs experiments, distills interaction into persistent external knowledge, revises it under contradicting evidence, and reuses it across tasks. AutoManual builds and updates environment rules from interaction on ALFWorld. WALL-E and WALL-E 2.0 induce and prune environment rules to align an LLM world model. WorldLLM performs curiosity-driven theory-making. Other works missing from related works like HExA most generally does experiment-driven in-context self-improvement with a persistent bank that is refined or replaced when contradicted, using domain-independent LLM distillation across many more domains, including physics-law discovery. No rule-learning baseline (AutoManual, WALL-E 2.0) is run under the same harness. It would be better to cite and refer these in the paper and distinguish how EvoBelife differ and improve over them [1,2,3,4,5].

  2. The one distinguishing component is unsupported: EvoBelief’s central claim is that validation before authority is what leads to improvements. Its only gate ablation is null: 68.3 without the gate versus 69.6 with it (Table 12). Without the gate, the contribution reduces to experimentation plus a persistent rule store, which prior work already provides.

  3. Gains may come mainly from exploration and confounding factirs and not from the belief lifecycle: On MiniGrid, the shadow variant (rules learned but never executed, experiments still run) reaches 57.1, versus 69.6 for full EvoBelief and about 1% for Frozen WM. On ScienceWorld, validated rules changed only 14 decisions across 90 tasks (Table 6), yet the gain over Frozen WM is +17.8 points, which points to the 424 experiment steps and hand-written procedural safeguards. Frozen WM lacks the experiments, the plan executor (which can act outside A_t), and possibly the safeguards, so it is not a matched control.

  4. The method is engineering for benchmark specific improvements and the baselines are not tool-matched: Rule induction, typed state schemas, retrieval, and applicability are heuristics and hardcoded per benchmark (Table 4, App. C). DiscoveryWorld laws are fitted by code regression, and the Plant Nutrients instantiation was redesigned after failing its pilot. The 0% Reactor Lab baselines plausibly reflect only the absence of a regression tool. The comparison is therefore between a domain-engineered symbolic rule learner and prompt-only agents, which does not support a general claim about belief evolution for LLM agents.

[1] Chandra, Abhranil, et al. “Hierarchical Experimentalist Agents.” arXiv, 2026, arxiv.org/abs/2606.29315. [2] Chen, Minghao, et al. “AutoManual: Constructing Instruction Manuals by LLM Agents via Interactive Environmental Learning.” Advances in Neural Information Processing Systems, vol. 37, 2024, pp. 589–631. [3] Levy, Guillaume, et al. “WorldLLM: Improving LLMs’ World Modeling Using Curiosity-Driven Theory-Making.” arXiv, 2025, arxiv.org/abs/2506.06725. [4] Zhou, Siyu, et al. “WALL-E: World Alignment by Rule Learning Improves World Model-Based LLM Agents.” arXiv, 2024, arxiv.org/abs/2410.07484. [5] Zhou, Siyu, et al. “WALL-E 2.0: World Alignment by NeuroSymbolic Learning Improves World Model-Based LLM Agents.” arXiv, 2025, arxiv.org/abs/2504.15785.

Questions for authors
  1. What can EvoBelief do that AutoManual, WALL-E 2.0, and HExA cannot, and what evidence shows that difference matters? Please compare empirically against AutoManual or WALL-E 2.0 on ALFWorld under the same single-attempt harness.
  2. The gate ablation is null on MiniGrid. Please ablate it on ALFWorld and ScienceWorld and under injected observation or transition noise. If it is null there too, what evidence supports the central claim?
  3. With only 14 rule-changed decisions, where does the ScienceWorld gain come from? Please report Frozen WM with the same experiments, safeguards, and plan executor.
  4. How does EvoBelief perform with LLM-driven hypothesis induction instead of per-benchmark code over typed schemas?
  5. Does ReAct with a Python or regression tool solve Reactor Lab? Which functional families make up the hypothesis space, and how were they chosen?
9 comments
AuthorsOpenReview ↗

[1/5] Response to Reviewer 8uWN

We thank the reviewer for the constructive and thorough review, and for recognizing our belief lifecycle and evaluation. The questions are specific, and they pointed us to the right experiments.

[W1, Q1] How EvoBelief differs from the cited works

EvoBelief differs from them in 3 ways. First, it learns a task-agnostic world model as rules about the environment, and these rules transfer to other tasks in the same environment. Second, it continually learns the environment dynamics during deployment without assuming a training stage, and this is a practical real-world setting. Third, because it learns while acting, a rule gains credibility only by correctly predicting outcomes it had not yet seen, and counterexamples narrow or retire it. This lets the agent withdraw stale rules and validate new ones when the environment changes. In its published protocol, AutoManual learns rules on training tasks, uses the resulting manual unchanged at test time, and is not tested in an environment that changes. WALL-E, WALL-E 2.0 and WorldLLM do learn how actions change the environment, but none of them is tested under changing dynamics during deployment. HExA is a valuable concurrent work that we will cite, and it does not address how an agent keeps adapting when the environment changes during deployment.

As suggested by the reviewer, we compared EvoBelief with AutoManual and WALL-E 2.0 on ALFWorld and MiniGrid using their released code, and with AutoManual on DiscoveryWorld (as it outperforms WALL-E 2.0 on ALFWorld). Within each benchmark, the methods received the same tasks, observations, environment-step budget, and one attempt per task. The ALFWorld and MiniGrid comparisons use the general version described in [W4, Q4]. The DiscoveryWorld comparison uses the DiscoveryWorld version described in [Q5, W4].

ALFWorld, 134 unseen tasks, Claude Haiku 4.5, no transition memorySuccess
ReAct64.2%
Frozen WM70.1%
WALL-E 2.076.1%
AutoManual83.6%
EvoBelief, general version86.6%

On MiniGrid we use the two task families with 7 new test seeds each, giving 14 streams of 40 episodes with a silent key switch after episode 20.

MiniGrid, 560 test episodes, Claude Haiku 4.5, no transition memorySuccess
WALL-E 2.015.0%
AutoManual, given the same inputs as EvoBelief53.8%
EvoBelief, general version70.9%

On DiscoveryWorld, every method had the same Python tool (NumPy and polyfit for regression), and AutoManual's prompt limit was raised from 13k to 60k tokens.

GPT-5.4AutoManualEvoBelief, DiscoveryWorld version
DiscoveryWorld, 48 test instances4.2%89.6%

In the DiscoveryWorld comparison above, AutoManual updates its rules after a task ends. Each instance has its own hidden law, so a rule learned at the end of an instance cannot guide actions earlier in that instance. Overall, EvoBelief is on par with AutoManual where the language model's prior matches the environment, as on ALFWorld, and clearly ahead where it does not, as on MiniGrid and DiscoveryWorld.

AuthorsOpenReview ↗

[2/5] Response to Reviewer 8uWN

[W2, Q2] When does the commitment gate help?

On MiniGrid, the submitted implementation induced rules in a small, predefined language. Table 12 showed little difference between full EvoBelief (69.6%) and allowing a rule to act on its first supported instance (68.3%). Those rules were rarely wrong, so the gate had little to filter. We therefore ran the gate ablation again with the general version on MiniGrid and ALFWorld. On ScienceWorld we ran it with the submitted version, together with the controls of [W3, Q3]. Shadow retains hypothesis formation and testing while disabling direct use of learned rules. In the general version, the no-gate arm allows a proposed rule to guide behavior immediately and withdraws it after two consecutive counterexamples. The ScienceWorld row uses the submitted implementation, with the commitment requirement relaxed.

SuccessShadowNo gateFull
MiniGrid, general version, Haiku 4.529.8%40.2%70.9%
ALFWorld, general version, Haiku 4.568.7%84.3%86.6%
ScienceWorld, submitted version, GPT-5.474.4%67.8%75.6%

On MiniGrid, the full method outperforms both variants, and the gate already leads before the hidden change, while the dynamics are still unchanged (78.9% vs. 68.6%). On ScienceWorld, removing the gate drops success below the shadow version, so rules acting without validation hurt. On ALFWorld, where the language model already knows most of the dynamics, the two versions solve a similar number of tasks, but the gate makes rule interventions far more reliable. 12% of its vetoes blocked an executable action, against 44% without it. A language model proposing rules freely is itself a noisy source of hypotheses, and the gate keeps that noise from reaching behavior. Validation therefore improves decisions when acting on a wrong rule is costly, as when the environment departs from what the language model expects or changes over time.

AuthorsOpenReview ↗

[3/5] Response to Reviewer 8uWN

[W3, Q3] Where does the ScienceWorld gain come from?

Active experiments are an integral part of EvoBelief’s belief lifecycle. An unresolved hypothesis can guide the agent’s next action by prompting a test, and the outcome provides evidence for revising that hypothesis. Once validated, a rule can also change a prediction or plan directly. The 14 decisions in Table 6 count this latter route. The same ScienceWorld runs also recorded 424 environment steps spent testing open hypotheses. We ran the requested control on the same 90 ScienceWorld tasks with GPT-5.4. The shadow agent retains EvoBelief’s task safeguards, active experiments, and goal-step executor, while disabling the use of validated rules to change predictions or plans. Experiments retain their ability to select a legal action outside the initial candidate set AtA_t. The goal-step executor is separate from the rule-based planner in Section 3.5. We reran both Frozen WM and full EvoBelief in the same batch, so their scores differ slightly from Table 1.

ScienceWorld, submitted version, GPT-5.4, 90 tasksSuccess
Frozen WM50.0% (45/90)
Frozen WM with the same safeguards60.0% (54/90)
Shadow (Frozen WM with safeguards, experiments, and executor)74.4% (67/90)
EvoBelief without the gate67.8% (61/90)
EvoBelief75.6% (68/90)

The safeguards help, but they do not explain the gap. With the same safeguards, EvoBelief solves 16 tasks that Frozen WM misses and misses only 2 that it solves. Most of the remaining gain comes from hypothesis-driven experiments, and validated rules add to it only with the commitment gate in place. Without the gate, rules acting on thin evidence pull success below the shadow version. On ScienceWorld the lifecycle therefore acts mainly by deciding what to test, as Section 4.5 reports, while on MiniGrid and ALFWorld letting validated rules act carries most of the gain over the shadow version ([W2, Q2]). We will add the analysis to the revision.

Comments on the paper

3 comments
AuthorsOpenReview ↗

Summary of Revisions

We sincerely thank the reviewers for their thoughtful and constructive feedback. We have updated the manuscript, with changes marked in blue.

  • Novelty and comparison with prior work. A capable agent can still choose the wrong action when its beliefs about the world are incomplete or inaccurate. Conceptually, EvoBelief continually updates a predictive world model during deployment. A hypothesis can guide action only after independent evidence from subsequent interactions confirms its predictions, and later counterevidence can narrow or retire the rule. Section 2 explains how this distinguishes EvoBelief from AutoManual, WALL-E 2.0, WorldLLM, and HExA, none of which is tested under dynamics that change during deployment. Empirically, EvoBelief performs on par with AutoManual on ALFWorld, whose dynamics largely match the language model’s prior, and clearly outperforms both AutoManual and WALL-E 2.0 on MiniGrid, where the agent must adapt to a changing environment (Table R2).
  • Generality. To test EvoBelief in a more general setting, we let a language model propose hypotheses directly from observations, in place of the benchmark-specific vocabulary and code induction of the submitted implementations. The belief lifecycle is unchanged, so every hypothesis is still tested and narrowed or retired when contradicted. On MiniGrid, where the dynamics change during deployment, EvoBelief outperforms every baseline. On ALFWorld, whose dynamics largely match the language model’s prior, it is on par with the strongest baseline, AutoManual, and ahead of all others. (Section 4.6, Table R2). These results mirror those of the submitted implementations, so the gains do not rely on benchmark-specific hypothesis generation.
  • Ablations and adaptation. The commitment gate and rule retirement are integral to EvoBelief, since together they keep unverified or outdated beliefs from steering behavior. On MiniGrid, removing the gate lowers success from 70.9% to 40.2%, and disabling narrowing and retirement lowers post-change success from 62.9% to 11.1%. On ScienceWorld, giving Frozen WM the same task safeguards does not close the gap, most of the remaining gain comes from hypothesis-driven experiments, and removing the gate pulls success below the shadow variant. On ALFWorld, retaining rules improves success on unseen tasks (Tables R2 to R5).
  • Scientific discovery with matched tools. EvoBelief tests candidate scientific laws by predicting the results of new measurements. With access to the same Python regression tool, it completes 19 of 24 Reactor Lab instances, compared with 5 for ReAct and 10 for the Hypothesizer (Section 4.4, Table R1).

Together, these results show that explicit, testable beliefs provide an effective substrate for continued learning after deployment, especially when an environment changes or behaves differently from what the language model expects. We believe EvoBelief opens a valuable direction for the community, and the reviewers’ questions made the paper clearer and its evidence substantially stronger. We will remain active until the end of the discussion period and look forward to further questions and feedback.

Dear Reviewers,

Please review the authors' rebuttal and participate in the discussion to address any remaining questions before the period closes on Nov 18 AoE.

Thank you!

AC

Hi 8uWN, thanks for your review. Could you please read the authors’ rebuttal and leave a brief response in the discussion thread? It would help to know which of your main concerns the response addresses and which remain. If your recommendation changes, please briefly explain what led to the change. A few sentences are enough. Thanks for your help!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.