acceptodds
Under review as a conference paper at ICLR 2027

Agent-Editing World Model: Rethinking World Modeling for LLM Agents

Abstract

Recent advances in large language models (LLMs) have enabled agents to tackle long-horizon tasks across diverse environments. To further improve agent performance, existing language world models typically predict environment observations, yet reconstructing high-entropy, execution-dependent tool responses offers limited value when real feedback is available. Meanwhile, agents suffer from task-state contamination, where unsupported assumptions and outdated plans persist in history and distort subsequent decisions. We propose the Agent-Editing World Model (AEWM), which models how reasoning and actions shape future task progress rather than simulating tool responses. AEWM combines Action Judge to distinguish Critical, Exploratory, and Noisy decisions with State Revision to edit noisy reasoning–action continuations from the same observed history. EditAct integrates these capabilities with real execution, directly changing the state underlying subsequent decisions rather than merely providing critiques. We train AEWM across Search, Terminal, and Software Engineering through mid-training and supervised fine-tuning. AEWM achieves 70.5% macro-F1 on our Action Judge benchmark, exceeding the strongest frontier baseline by 10.6 points. Across six benchmarks and three agent backbones, EditAct improves average scores by 3.2–6.7 points over the strongest baseline. Furthermore, rejection sampling fine-tuning on verified EditAct trajectories, termed AEWM-RFT, improves over Self-RFT by 2.2–2.6 points across three domains without online AEWM guidance. The code is available at https://anonymous.4open.science/r/AEWM_EditAct-58F9.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
World Models, LLM Agents

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_AES4OpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how to improve long-horizon LLM agents when unsupported assumptions or outdated plans persist in their interaction histories. It proposes Agent-Editing World Model (AEWM), which judges the expected contribution of a proposed reasoning–action pair and replaces proposals classified as noisy before execution. The EditAct framework combines these interventions with real environment feedback while preserving the preceding history. A unified model learns judgment and revision across Search, Terminal, and Software Engineering through mid-training and supervised fine-tuning. Experiments on six benchmarks and three agent backbones show average-score gains of 3.2–6.7 points over the strongest reported baseline. Fine-tuning agents on verified EditAct trajectories also improves performance without online AEWM guidance.

Critical strengths
  1. A valuable motivation and a concrete intervention mechanism. Real tool feedback does not necessarily correct an agent's mistaken interpretation or outdated plan. The paper addresses this problem by judging proposed decisions and jointly revising their reasoning and actions before execution. I find this a useful direction for world modeling in tool-using agents. The distinction between productive exploration and noisy decisions is also well motivated.

  2. Broad evaluation with informative ablations. The results cover three domains, six benchmarks, and three agent backbones. The ablations examine random intervention, resampling, hint-based guidance, separate reasoning and action revision, and model substitution. These comparisons support the value of learned intervention selection and joint revision. The AEWM-RFT results further show that the resulting trajectories provide useful supervision for agents that subsequently run without AEWM guidance.

Critical weaknesses
  1. Failure modes of online intervention remain underexplored. The aggregate results support a positive overall effect, and the paper provides successful intervention cases and search-behavior statistics. However, Section 5.2 explicitly focuses on selected successful interventions, leaving limited analysis of ineffective revisions or useful exploration mistakenly classified as noisy. This makes it difficult to assess when intervention may be unnecessary or counterproductive. (Lines 424–431; Figure 4; Appendix D, lines 1244–1249.)

  2. The stability of smaller gains and the actual inference costs are unclear. The paper reports means over three independent runs without measures of variability. Some differences are modest: on BrowseComp, Full AEWM scores 48.1 versus 47.4 for DeepSeek-V4-Pro WM. The appendix specifies evaluation budgets and generation limits, but does not report average total token usage, model calls, or wall-clock time. These omissions limit the interpretation of smaller improvements and the performance–cost trade-off. (Lines 305–307; Table 3, lines 334–340; lines 1130–1133; Table 13, lines 1134–1143.)

Questions for authors
  1. Failure and ineffective-intervention analysis: Could you provide examples or a small audit of failed or ineffective online interventions, particularly cases where useful exploration is incorrectly classified as NOISY? Please describe the observed consequences of these revisions for subsequent task progress and how the cases were selected. This would help clarify the method's reliability and failure boundaries. A small analysis of existing trajectories would help make these limitations more concrete. (Lines 424–431; Appendix D, lines 1244–1249.)

  2. Stability and inference cost: How consistent are the reported gains across the three evaluation runs, and what inference cost is required to obtain them? Please report standard deviations or individual-run scores for the main comparisons, together with average per-task token usage, model-call counts, and wall-clock time for EditAct, ReAct, and Best@3 under the reported agent and task settings. Costs should include judgment, revision, and candidate-selection calls where applicable. This would help assess both smaller gains, such as the 0.7-point BrowseComp improvement over DeepSeek-V4-Pro WM, and the performance–cost trade-off. (Lines 305–307; Table 3, lines 334–340; lines 1130–1133; Appendix C.2–C.3, lines 1148–1224.)

5 comments

We sincerely thank the reviewer for the thoughtful feedback and constructive suggestions. We appreciate the recognition of our motivation and experimental results. Below, we provide additional analysis of ineffective interventions and report further results on performance stability and inference costs.

Q1. Failed and ineffective interventions

Table 11 reports 72.5% overall prediction accuracy on our Action Judge benchmark, corresponding to 27.5% classification errors. We identify two representative failure patterns in which useful actions are incorrectly classified as NOISY:

1. Incorrectly skipping a necessary file inspection. In the Terminal task fix-git, the agent needed to recover a lost commit and integrate its changes into the main branch. Applying the commit produced a merge conflict. The agent proposed reading the conflicted file, a useful step for understanding the current conflict before editing it. Action Judge classified this inspection as NOISY because earlier versions of the file had already been viewed and the intended incoming changes were known. State Revision replaced reading the file with listing the names of conflicted files. This revealed no conflict content, so the agent read the file in the next step anyway. The task ultimately succeeded, but the intervention introduced an unnecessary detour. The underlying error was treating earlier knowledge of the file as sufficient after its state had changed.

2. Incorrectly assuming an untested function works correctly. In Doc2Repo's doc2mark task, the agent was reconstructing a document-conversion library. A first test of its public convenience functions failed because an API key was missing. The agent proposed rerunning those functions with an explicit API-key argument, which would check the public wrappers' argument handling and execution. Action Judge classified this as NOISY, reasoning that the wrappers were already correct because their underlying loader had been tested. State Revision instead supplied the key through an environment variable and generated a malformed shell command. The command failed before performing the test, requiring another verification step. The revision therefore delayed useful validation and left the proposed explicit-argument path unchecked at that point. Here, confidence in one component prematurely displaced evidence about another interface.

Q2. Stability and inference cost

We additionally report the standard deviations across three independent runs, together with token usage, for Qwen3.5-35B-A3B. Tokens represent total model tokens per task, in millions.

MethodBrowseCompTerminal-Bench 2Doc2Repo
Tokens (M)Score (%)Tokens (M)Score (%)Tokens (M)Score (%)
ReAct1.3740.9 ± 1.024.6837.1 ± 4.103.9442.8 ± 0.06
Step-level Best@33.0944.2 ± 1.343.1544.9 ± 1.124.6943.7 ± 2.16
Trajectory-level Best@34.1146.0 ± 2.0214.1543.8 ± 5.629.9343.7 ± 0.16
EditAct (Ours)3.4548.1 ± 2.726.2948.3 ± 3.537.5248.9 ± 1.68

The reported standard deviations indicate reasonably consistent performance across runs, despite some task-dependent variation. The additional token cost relative to ReAct mainly arises from Action Judge and State Revision processing the interaction history again, increasing input-token usage. Compared with the Best@3 baselines, EditAct achieves better performance across all three tasks, using fewer tokens than trajectory-level Best@3 and remaining within the same order of magnitude as step-level Best@3, albeit with higher token usage. These results highlight the effectiveness of targeted decision correction while making the associated performance–cost trade-off explicit.

Thank you to the authors for your thoughtful responses and the additional results provided. Your previous answers have addressed some of the questions I initially raised. I would like to discuss a few further points:

  1. What is the proportion of these two types of "classic" errors relative to the overall distribution of error types? Could you provide a statistical distribution of common errors?

  2. Could you provide statistics on the transitions between ReAct and EditAct? Specifically, I am interested in the breakdown of task outcomes: success→failure, failure→success, success→success, and failure→failure. In my view, only a sufficiently granular evaluation and analysis can truly clarify the effectiveness of EditAct.

We sincerely thank the reviewer for the helpful follow-up questions. We further analyzed the common errors made by Action Judge and examined how EditAct changes task outcomes compared with ReAct, including both successful corrections and negative interventions.

1. Distribution of False-NOISY Errors

Our previous response focused on two specific failure cases. We have now extended this analysis to examine a broader range of errors and identify common failure patterns across three domains. Specifically, we analyze cases where Action Judge incorrectly classifies useful actions as NOISY. The table below shows the distribution of error types in each domain.

Error TypeSearchTerminalSWE
Necessary inspection mistaken for repetition30%45%15%
Untested behavior assumed correct0%10%35%
Useful exploration dismissed as low-value65%15%10%
Feasible tool operations incorrectly expected to fail5%20%10%
Necessary implementation or repair dismissed as unnecessary0%10%30%

Search: Useful exploration is most frequently undervalued (65%). Action Judge sometimes rejects promising search directions because they appear indirect or unlikely to yield immediate answers. Necessary source checks are also mistaken for repetitive operations (30%), overlooking their value in verifying information.

Terminal: Necessary inspection is most frequently mistaken for repetition (45%). The previously discussed fix-git case illustrates this error: Action Judge rejected reading a conflicted file because an earlier version had already been inspected, overlooking the file's changed state. Another common error is incorrectly predicting that feasible tool operations will fail (20%).

SWE: Assuming untested behavior is correct (35%) and dismissing necessary implementation (30%) are the most common errors. The previously discussed doc2mark case illustrates the former: Action Judge assumed the public wrappers worked because the underlying loader had been tested, incorrectly rejecting a necessary verification step. It also sometimes dismisses necessary implementation or repair as unnecessary.

These findings also highlight directions for further improving AEWM in future work, particularly in recognizing the value of exploration, identifying when additional verification is necessary, and avoiding unsupported assumptions about untested functionality.

2. Paired ReAct → EditAct Outcomes

To better understand the effectiveness of EditAct, we analyze how task outcomes change from ReAct to EditAct, considering both improvements and regressions. For BrowseComp and Terminal-Bench 2.0, we examine transitions between task success and failure. For Doc2Repo, we analyze changes in test pass rates.

The results are summarized below.

BenchmarkSuccess→FailureFailure→SuccessSuccess→SuccessFailure→Failure
BrowseComp7.27%14.51%33.60%44.63%
Terminal-Bench 2.04.12%15.36%32.96%47.57%

EditAct converts substantially more failed tasks into successful ones than the reverse. On BrowseComp, 14.51% of comparisons change from failure to success, approximately twice the 7.27% that change from success to failure. On Terminal-Bench 2.0, this difference is even larger: 15.36% improve from failure to success, compared with only 4.12% that regress.

The success-to-failure transitions also reflect the risks of incorrect interventions discussed in Question 1: useful actions may be mistakenly classified as NOISY and replaced with less effective decisions. Nevertheless, EditAct corrects substantially more failures than the regressions it introduces, demonstrating that AEWM-guided decision correction provides a clear overall benefit to task-solving performance.

For Doc2Repo, we analyze whether EditAct increases or decreases the test pass rate:

Test Pass-Rate Change (ReAct → EditAct)Percentage
Increases48%
Decreases29%
Unchanged23%

EditAct also improves Doc2Repo performance in both the frequency and magnitude of positive changes. Test pass rates increase in 48% of comparisons, compared with 29% showing decreases. Moreover, the average improvement is larger than the average decline (20.88 vs. 13.35 percentage points), contributing to the overall performance increase from 42.8% to 48.9%.

Overall, the results consistently show that EditAct produces more positive changes than negative ones across all three benchmarks. In particular, its ability to turn failed tasks into successful ones demonstrates the effectiveness of AEWM-guided decision correction in improving agent task-solving performance.

We sincerely thank the reviewer again for the valuable suggestions. We will incorporate these additional analyses and findings into the revised version of our paper.

Reviewer_pkeVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes the Agent-Editing World Model (AEWM), which shifts language world modeling from predicting environment observations to predicting the downstream effects of agent decisions (CRITICAL, EXPLORATORY, NOISY) and directly editing noisy reasoning-action continuations before execution. The integrated inference method, EditAct, is evaluated across Search, Terminal, and SWE domains on six benchmarks using three Qwen3.5 backbones. The authors also introduce AEWM-RFT, using verified EditAct trajectories to fine-tune agents offline. Results show consistent improvements over ReAct and Best@3 baselines, supported by detailed inference and training ablations.

Critical strengths
  1. The motivation is novel, shifting the prediction focus from tool responses to modeling and editing agent state effects. It effectively addresses a realistic failure mode—task state pollution—in long-horizon agents.
  2. The ablation studies on reasoning are exhaustive, rigorously isolating the respective contributions of learned gating vs. random intervention, direct replacement vs. prompting/resampling, and joint editing vs. partial editing.
  3. The evaluation covers three distinct domains and three model scales, with improvements remaining valid on OOD (Out-of-Distribution) data, indicating a certain degree of generalization capability.
Critical weaknesses
  1. The paper lacks a direct comparison with traditional world model baselines (e.g., models trained to predict tool responses) under the same setting and data scale.
Questions for authors
  1. Could you provide an ablation study where the same Qwen3.5-35B-A3B backbone is fine-tuned on equivalent data to predict environmental observations (i.e., a traditional world model)? This would help disentangle the benefits derived from the agent editing objective versus those from standard supervised training.
4 comments

We sincerely thank the reviewer for the constructive suggestion. We agree that comparing AEWM with traditional world models is important for understanding the benefits of our modeling objective. We therefore conducted additional experiments using Qwen-AgentWorld, evaluating two approaches to applying traditional world models to agent tasks.

1. Selection of a Traditional World Model

We select Qwen-AgentWorld-35B-A3B [1] as a representative traditional world model. Due to computational resource constraints, we did not train another world model from scratch. Instead, we chose Qwen-AgentWorld for three reasons:

  • Large-scale training: It is trained on over 10 million environment interaction trajectories through continual pre-training, supervised fine-tuning, and reinforcement learning.
  • Broad domain coverage: Its training covers seven domains, including Search, Terminal, and SWE, which align with our evaluation domains.
  • Same backbone as AEWM: Qwen-AgentWorld-35B-A3B uses the same Qwen3.5-35B-A3B backbone as our AEWM and is specifically trained to predict environment observations.

2. Comparison Methods

We evaluate two approaches to using Qwen-AgentWorld. We do not directly integrate it into EditAct because the two models serve different purposes: AEWM evaluates the contribution of agent decisions and revises unproductive reasoning–action pairs, whereas Qwen-AgentWorld predicts environment feedback. EditAct is specifically designed around the former capability.

Method 1. Direct agent execution. We directly evaluate Qwen-AgentWorld-35B-A3B as an inference agent on BrowseComp, Terminal-Bench 2.0, and Doc2Repo, examining whether environment-modeling capabilities improve task-solving performance.

Method 2. World-model-assisted inference (Imagine-then-Plan). We adapt Imagine-then-Plan [2], which uses predicted environment feedback to guide agent decisions. Specifically, we implement a one-step lookahead (K=1):

  • Qwen-AgentWorld predicts a hypothetical action and its expected environment feedback from the current task history.
  • Qwen3.5-35B-A3B uses this prediction to reconsider its decision and execute an action in the real environment.
  • The actual tool response updates the interaction history, while predicted feedback serves as temporary guidance.

This allows us to evaluate how traditional environment prediction helps an inference agent.

3. Experimental Results

MethodBrowseCompTerminal-Bench 2.0Doc2Repo
ReAct (Qwen3.5-35B-A3B)40.937.142.8
Qwen-AgentWorld (Direct Agent)28.630.345.2
Qwen-AgentWorld + Imagine-then-Plan (K=1)42.733.746.0
EditAct (Ours)48.148.348.9

Directly using Qwen-AgentWorld as an inference agent improves Doc2Repo performance but reduces performance on BrowseComp and Terminal-Bench 2.0. This is understandable because Qwen-AgentWorld is primarily trained to predict environment feedback rather than directly solve tasks.

Using predicted environment feedback to guide inference improves performance on BrowseComp and Doc2Repo, but reduces performance on Terminal-Bench 2.0. One possible reason is that complex terminal environments are difficult to predict accurately, and discrepancies between predicted and actual feedback can mislead the agent. Additional prediction calls also consume inference time without necessarily improving decisions.

EditAct consistently achieves the best performance, outperforming world-model-assisted inference by 5.4, 14.6, and 2.9 points across the three benchmarks.

4. Conclusion

These results support our central motivation: modeling the effects of agent decisions can be more effective than predicting environment observations.

For tool-using agents, real environment feedback is already available through tool execution. Predicting this feedback introduces additional uncertainty, particularly in complex environments.

AEWM instead focuses on whether a proposed decision advances task progress and how to correct it when necessary. By directly improving reasoning and actions while relying on real tool feedback, EditAct avoids the need to simulate complex environment observations.

The consistent improvements across all three benchmarks further support the effectiveness of our agent-editing objective for long-horizon task solving.

References

[1] Qwen-AgentWorld: Language World Models for General Agents.

[2] Imagine-then-Plan: Agent Learning from Adaptive Lookahead with World Models.

Thanks to the authors for their detailed rebuttal and for conducting additional experiments with Qwen-AgentWorld-35B-A3B. I appreciate the effort to compare AEWM against a traditional observation-predicting world model using both direct execution and Imagine-then-Plan (K=1) inference.

The results showing EditAct outperforming Qwen-AgentWorld by 5.4, 14.6, and 2.9 points across the three benchmarks provide meaningful empirical support for your central motivation: modeling decision effects is more effective than simulating high-entropy tool responses when real feedback is available. The finding that predicted feedback can mislead agents in complex terminal environments aligns well with the paper's claims.

However, my original question specifically requested fine-tuning the same backbone on equivalent data to isolate the benefit of the agent-editing objective from standard supervised training. The authors acknowledge that computational constraints prevented this exact controlled ablation, substituting an existing large-scale checkpoint instead. Because Qwen-AgentWorld was trained on over 10 million trajectories with a different multi-stage recipe, differences in data distribution and training scale remain potential confounders. Therefore, while the rebuttal offers valuable evidence, it does not fully satisfy the strict controlled comparison requested.

So I recommend maintaining the rating at 2. Although the new comparison with Qwen-AgentWorld partially addresses my concern and supports the paper's motivation, the inability to perform the strictly controlled ablation on equivalent data means my primary methodological concern remains only partially resolved. The presentation issues noted in my original review also persist.

We sincerely thank you for your thoughtful follow-up and for recognizing the value of our additional experiments. We would like to further clarify why we believe our comparison provides strong evidence for AEWM.

First, AEWM and traditional world models have fundamentally different modeling objectives and supervision targets, making strictly equivalent training data difficult to establish, even when using the same source trajectories.

More importantly, our comparison uses a traditional world model trained on a substantially larger scale than AEWM. Qwen-AgentWorld-35B-A3B shares the same backbone as AEWM and was trained on over 10 million interaction trajectories across multiple domains. Despite using less training data, AEWM achieves better performance across all three benchmarks.

We believe this provides particularly compelling practical evidence for our modeling objective, beyond what a matched-scale comparison alone could demonstrate. Our results show that AEWM outperforms a traditional world model even when the latter benefits from substantially more training data.

We appreciate your acknowledgment that these findings meaningfully support our central motivation. We will also improve the clarity of our presentation in the revised version.

Thank you again for your valuable feedback and constructive discussion!

AuthorsOpenReview ↗

Further Clarification Regarding the Additional Experiments

We sincerely thank you for your thoughtful follow-up. We would like to further clarify why we believe our additional experiments have addressed the underlying concern.

Your original request was to compare AEWM with a traditional world model trained on the same data scale. Instead, we conducted a more demanding practical comparison. We evaluated Qwen-AgentWorld, which shares the same backbone as AEWM but was trained on over 10 million interaction trajectories, compared with only hundreds of thousands of trajectories used for AEWM. Thus, Qwen-AgentWorld was trained on a substantially larger dataset than AEWM.

Despite using significantly less training data, AEWM outperforms this extensively trained traditional world model across all three benchmarks. We believe this provides particularly strong evidence for our modeling objective, beyond what a matched-scale comparison alone would demonstrate.

We understand that a matched-scale experiment could offer a more controlled comparison. However, the fundamental purpose of such experiments is to establish the effectiveness of the proposed modeling objective, and we believe our additional results have already provided compelling evidence toward this goal. We also appreciate your explicit acknowledgment that these findings meaningfully support our central motivation.

We respectfully hope you will reconsider the significance of these results when evaluating our work. We will also improve the clarity of our presentation in the revised version.

Thank you again for your valuable feedback and constructive discussion!

Reviewer_tSuuOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes the Agent-Editing World Model (AEWM), which evaluates and revises an agent’s proposed reasoning–action pair before execution. Action Judge classifies decisions as CRITICAL, EXPLORATORY, or NOISY, while State Revision replaces proposals judged NOISY. EditAct integrates these capabilities with real environment execution. AEWM is trained across Search, Terminal, and Software Engineering through mid-training and supervised fine-tuning. The paper evaluates decision judgment and downstream task performance, and further uses AEWM-guided trajectories for rejection sampling fine-tuning.

Critical strengths

Consistent downstream improvements across tasks and model sizes. EditAct improves performance across six benchmarks and three Qwen3.5 inference-agent sizes, including out-of-distribution evaluations. AEWM-RFT also improves over Self-RFT without requiring online AEWM guidance, demonstrating that the resulting trajectories provide useful supervision.

Several relevant alternatives are evaluated. The ablations include Random Gate, Agent Resampling, AEWM Hint, Action-only Revision, Reasoning-only Revision, Self-WM, and DeepSeek-V4-Pro WM. These comparisons provide evidence that intervention selection and the revision mechanism matter, rather than attributing all improvements to additional generation opportunities.

Critical weaknesses

1.The experiments do not sufficiently establish the advantage of a separate AEWM over the inference agent’s own self-correction. The motivating argument is that unsupported assumptions and outdated plans persist in the agent’s history and contaminate subsequent decisions. This motivates correction, but does not by itself explain why correction should be assigned to a separate model. A direct test would hold model weights and available evidence fixed, and compare ordinary inference, explicit self-checking within the agent’s ongoing interaction, and the same model operating in a separate Action Judge/State Revision role.

The Self-WM ablation is relevant and partially addresses this question: it uses Qwen3.5-35B-A3B for both inference and revision. However, it does not directly compare this role separation against explicit self-correction within the ongoing interaction. Moreover, Self-WM underperforms ReAct on BrowseComp and Terminal-Bench 2.0, whereas the strongest results require the separately trained AEWM.

Consequently, the current evidence establishes the usefulness of the trained auxiliary model more clearly than it establishes the benefit of the proposed separation of roles. This distinction is particularly important for the smaller inference agents, where AEWM also has greater model capacity. A controlled same-model comparison would help clarify whether EditAct provides an advantage beyond access to a more capable revision model.

2.The allocation of substantial training resources to AEWM is insufficiently justified against directly improving the inference agent. AEWM receives approximately 52B tokens of mid-training followed by SFT on 120K examples. In contrast, Self-WM uses the inference model without this additional specialized training. Thus, the comparison changes both the organization of inference and the training received by the model responsible for correction. The SFT-only and mid-training-only ablations clarify how to train AEWM, but do not address whether comparable resources would be better spent training the inference agent to act or self-correct directly. Similarly, AEWM-RFT demonstrates the value of AEWM-guided trajectories after AEWM has been constructed, rather than controlling for that upstream investment.

A smaller-scale, single-domain comparison under comparable training budgets would already be informative. For example, the authors could compare training AEWM for Action Judge/State Revision against training the inference agent on corresponding task-solving or self-correction supervision. Evaluating whether the trained AEWM can serve directly as an inference agent, with appropriate interface adaptation, would provide another useful reference. The paper also lacks a consolidated account of data-generation costs, training compute hours, and actual inference overhead, making the practical tradeoff difficult to assess.

Questions for authors

Under the same model and available evidence, how does the Self-WM procedure compare with explicit self-checking and correction within the inference agent’s ongoing interaction? Can the authors isolate the benefit of assigning Action Judge and State Revision to a separate role?

Under comparable training budgets, how does training AEWM compare with directly training the inference agent for task execution or self-correction? Some experiments would help clarify the rationale for this allocation of resources.

Can the authors report data-generation costs, training compute hours, and actual inference overhead, including generated tokens and latency? How do these costs compare with the strongest baselines?

5 comments
AuthorsOpenReview ↗

Response to Question 1&2

We sincerely thank the reviewer for the thoughtful questions and constructive feedback.

Our goal is to learn a reusable model of how agent decisions affect task progress. AEWM evaluates and revises proposed reasoning–action pairs based on the interaction history. We aim to show that this learned capability improves online decision-making and provides better supervision for subsequent agent training.

Question 1: Why train a separate AEWM instead of relying on agent self-correction?

Inference agents can already self-correct, but this ability becomes less reliable over long interactions. A search agent may change search directions, and a terminal agent may revise code after a failed test. However, unsupported assumptions and outdated plans can persist in the interaction history, leading subsequent decisions along unproductive paths. We therefore train Action Judge to identify such decisions and State Revision to jointly correct their reasoning and actions.

Our ablations compare several ways of correcting agent decisions. With Qwen3.5-35B-A3B fixed as the inference agent, Table 3 evaluates:

  • ReAct: Standard inference, where the agent performs self-correction naturally during task execution.
  • AEWM Hint: Action Judge detects noisy decisions, and the agent performs self-correction using AEWM-provided reasoning hints.
  • Agent Resampling: Action Judge detects noisy decisions, and the agent self-corrects by regenerating a reasoning–action pair from the same history.
  • Self-WM: The agent uses its own model weights to judge and revise decisions through separate EditAct roles, enabling self-correction without a separately trained model.
MethodBrowseCompTerminal-Bench 2Doc2Repo
ReAct40.937.142.8
AEWM Hint41.839.343.2
Agent Resampling36.240.442.8
Self-WM39.330.744.4
EditAct48.148.348.9

Trained AEWM consistently outperforms these alternative correction methods across all three tasks. The results demonstrate the benefits of learned decision evaluation and direct joint revision compared with hint-guided self-correction, resampling, and using the inference agent itself for judgment and revision.

Question 2: Why invest in AEWM rather than only training the inference agent?

AEWM not only improves online decisions but also helps train stronger inference agents. Through online guidance, AEWM corrects unproductive decisions and helps generate more successful task trajectories. We verify these trajectories using real task outcomes and use them for rejection sampling fine-tuning, termed AEWM-RFT. Self-RFT and AEWM-RFT use the same task sources, dataset sizes, filtering strategy, and fine-tuning configuration, differing only in how their training trajectories are generated.

MethodBrowseCompTerminal-Bench 2Doc2Repo
Base agent40.937.142.8
Self-RFT43.240.846.1
AEWM-RFT45.443.448.6

AEWM-guided correction can also improve the inference agent itself through fine-tuning. All three methods are evaluated using ReAct without online AEWM guidance. AEWM-RFT outperforms Self-RFT by 2.2, 2.6, and 2.5 points, respectively, showing that AEWM-guided trajectories provide more effective training supervision. Furthermore, AEWM can be used with different inference agents: the same trained AEWM supports 4B, 9B, and 35B-A3B agents across six benchmarks in Search, Terminal, and SWE.

Comparison with a terminal agent trained on the same tasks. Our terminal world-model training data are constructed from the same open-source synthetic tasks provided by CalibForge [1]. While CalibForge uses these tasks to directly fine-tune Qwen3.5-35B-A3B for terminal task solving, we use them to train AEWM for decision evaluation and revision. This provides a relevant comparison between the two training approaches using the same task sources and model backbone.

MethodBackboneTerminal-Bench 2.0
CalibForgeQwen3.5-35B-A3B47.57
EditAct (Ours)Qwen3.5-35B-A3B48.3

These results show that EditAct achieves competitive performance compared with directly fine-tuning the inference agent, providing additional evidence for the effectiveness of AEWM-guided decision correction.

[1] CalibForge: Adversarial Solver Calibration for Scaling Learnable Terminal Tasks

AuthorsOpenReview ↗

Response to Question 3

Question 3: What are the training and inference costs?

Our training data and training configuration are detailed in Appendix A. AEWM uses 52.16B mid-training tokens and 120K curated SFT examples, including 60K Action Judge and 60K State Revision examples, with 40K examples per domain. Appendix A further provides the data composition, optimization settings, and training schedules.

We additionally report the generated token usage of different inference methods. The following table presents the main evaluation scores alongside recorded output-token usage per task, measured in thousands of tokens. The statistics include recorded generations from the inference agent, Action Judge, State Revision, and verifiers.

MethodBrowseCompTerminal-Bench 2Doc2Repo
Generated tokens (K)Score (%)Generated tokens (K)Score (%)Generated tokens (K)Score (%)
ReAct19.3040.934.0437.140.8742.8
Step-level Best@3174.5644.2176.5044.9198.2343.7
Trajectory-level Best@357.9046.0102.1243.881.7343.7
EditAct (Ours)59.0248.166.2948.3101.9048.9

The additional generation cost of EditAct mainly comes from Action Judge and selectively invoked State Revision. Compared with ReAct, these additional model calls increase output-token usage, while the revised decisions can also change subsequent inference trajectories.

Despite this additional cost, EditAct achieves better performance with substantially fewer generated tokens than Step-level Best@3 across all three tasks. Compared with Trajectory-level Best@3, EditAct uses fewer tokens on Terminal-Bench 2, comparable tokens on BrowseComp. Importantly, EditAct achieves the highest performance on all three benchmarks, demonstrating that targeted decision correction can improve task-solving performance while maintaining competitive generation costs.

AuthorsOpenReview ↗

Kind Follow-up on Our Rebuttal and Request for Further Discussion

Thank you again for your thoughtful review and constructive feedback. We have provided detailed responses to your questions and concerns in our previous comments. As the rebuttal period is approaching its end, we would greatly appreciate the opportunity to hear your thoughts on our responses and discuss any remaining concerns. Thank you very much for your time and consideration!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.