acceptodds
Withdrawn from ICLR 2027

World State Generator

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Language agents solve complex tasks through plans and actions. A single step the world refuses puts the goal out of reach, and what the agent does next decides the task. Prompted planners fail at exactly this point, rewriting the refused step in new words, meeting the same refusal, and burning the attempt budget without moving. They fail because the plan was never tied to the world, so a refusal has nothing in the plan to attach to. A world is where a task runs, and it has its own rules, its own admissible actions, and its own constraints. We build synthetic worlds across 7 domains and extract training data from them. A program enforces each world's rules and grades its goal, and every world is admitted only if its goal is reachable from its initial state. Agents run inside and leave verified failures paired with repairs that carried the run to a state the world certified, a record of about 226K trajectories. On this record we train the *World State Generator*, a model that writes a plan as checkable states of the world and keeps that plan aligned with the world it runs in. That alignment is what a plan written in language lacks, since the world it runs in has physical limits, logical dependencies, and required orders the language never states, and the plan encounters these rules only when a state fails. *WSG* takes that failure as the rule the world has stated and rewrites the remaining states to obey it, so the plan bends to the world as the run goes on. Across 7 public benchmarks, *WSG* raises end-to-end success for two open models near 30B parameters over prompting and brings to the level of proprietary model.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Language agents, plan repair, replanning, synthetic environments, synthetic corpus, supervised fine-tuning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_NtT7OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents World State Generator (WSG), a fine-tuned replanning component built upon the existing State-Centric Decision Process (SDP) framework. The authors construct synthetic environments across seven domains and collect failure–repair trajectories through agent interactions. Approximately 239K repair examples are retained from 226K episodes and used to fine-tune two language models with supervised learning.

WSG is activated when an intermediate target state fails, generating a revised sequence of states based on environmental feedback. Evaluations on seven agent benchmarks show improvements over the prompted SDP baseline, with additional analyses of recovery rates and execution efficiency.

Critical strengths
  1. Practical synthetic data construction. The paper presents a systematic pipeline for collecting failure–repair examples from executable synthetic environments across multiple domains. This provides a potentially useful source of supervision for training agent recovery behavior.

  2. Consistent empirical improvements. WSG improves the reported metrics across seven benchmarks and two open-weight backbones. The comparison with prompted SDP is reasonably controlled, as the underlying runtime and other agent components remain unchanged. The recovery analysis provides some supporting evidence for improved replanning behavior.

Critical weaknesses
  1. Limited methodological novelty and insufficient experimental evidence supporting the core design.

The proposed method relies heavily on the existing SDP framework, while the actual training procedure is standard supervised fine-tuning with LoRA. The primary contribution therefore lies in the construction and use of failure–repair supervision. However, the paper lacks important controlled comparisons, including successful-trajectory SFT, training without refutation information, and different levels of repair verification.

Although Section 5.1 discusses training on initial plans, no quantitative comparison is provided. Consequently, it remains unclear whether the improvements specifically result from the proposed failure-conditioned supervision or from more general adaptation to the SDP planning format. This limits the strength of the paper's central methodological claim.

  1. Insufficiently justified verification claims and unclear presentation of the proposed approach.

The paper repeatedly emphasizes verified failure–repair pairs, yet the filtering criterion only requires at least one state in a repaired continuation to be certified. Only 48.9% of retained repairs come from episodes that eventually reach the goal, and intermediate states are validated by an LLM in most domains. Since the entire repaired continuation is used for training, the actual reliability of the supervision remains insufficiently established.

The presentation further weakens the clarity of the contribution. Figure 1 is particularly difficult to follow: its decorative visual style, numerous small labels, icons, and dense connections make the overall pipeline hard to understand. As the main methodological illustration, it fails to clearly distinguish the existing SDP components from the proposed contributions. A simpler, more structured diagram would substantially improve readability.

The writing also relies on repetitive and metaphorical descriptions rather than consistently providing precise technical explanations. Important distinctions, such as programmatic verification versus LLM-based state validation, are not communicated clearly enough. Overall, the paper requires considerable effort to identify what is technically new and how the proposed training mechanism differs from the underlying framework.

Several reporting inconsistencies further reduce clarity and confidence in the experimental presentation. The main text reports 238,563 retained repair examples, whereas Appendix Table 7 refers to 207,660. The reported average repair length also differs (5.5 versus 4.9), and the DeepPlanning evaluation set contains 150 tasks in Table 8 but only 120 in the breakdown of Table 18. Additionally, some claims about end-to-end success improvements are imprecise because several benchmarks report partial task scores rather than strict success rates.

Questions for authors

Q1. Can the authors provide controlled ablations comparing the proposed repair SFT against successful-trajectory SFT and repair training without the refutation signal? These comparisons are important for establishing the necessity of the proposed supervision.

Q2. How reliable are the intermediate-state certifications produced by the LLM validator? What percentage of retained repair sequences are fully verified, and how does training exclusively on goal-reaching repairs affect performance?

Q3. Could the authors clarify the inconsistencies in dataset size, average repair length, and DeepPlanning evaluation sample count? Are these statistics derived from different filtering stages or experimental configurations?

Q4. Can the authors more explicitly distinguish the contributions of WSG from the existing SDP framework and explain which components are newly introduced? A clearer methodological diagram and a more precise description of the verification process would also help assess the contribution.

Reviewer_uaTbOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper trains a LoRA adapter, the World State Generator (WSG), for the replanning call of the State-Centric Decision Process (SDP, Jeong et al., 2026). SDP is an agent runtime with three roles. A proposer writes the plan as a chain of natural-language state predicates, a realizer issues tool calls toward each state, and a validator checks whether the state holds. When a state is refuted, the proposer is called again with the goal, the certified prefix and the refutation.

The authors build about 48K synthetic worlds over 7 domains: physical, multi-hop QA, web search, tool use, constraint, reasoning and math. A world is admitted only if a program shows its goal is reachable. They run Qwen3-32B, Qwen3.6-27B and Gemma4-31B inside SDP for 226K episodes, keep repairs after which at least one state was certified (about 239K pairs), and train on them with SFT. At test time the prompted backbone still writes the first plan and fills the realizer and validator roles; only repair calls go to WSG.

On 7 benchmarks (τ²-bench, ScienceWorld, ToolSandbox, PlanBench-XL, REAL, DeepPlanning-Shopping, AgentBoard-PDDL), WSG improves over prompted SDP by 3.4–31.9 points on both backbones. The paper claims this brings the models into the range of published proprietary results.

Critical strengths

The proposed refiner did improve upon its own SDP baseline after training.

Critical weaknesses

The SDP baseline could be too weak. There is no result for the same backbones used the standard way, i.e. native function calling or each benchmark’s official harness, with thinking enabled; Appendix B.3 disables thinking for every call. There is evidence that the SDP baseline sits far below what these backbones normally achieve. The Gemma 4 31B model card reports 76.9% on τ²-bench (“average over 3”), while the paper reports 42.5 for SDP and 56.3 for WSG with the same model. https://huggingface.co/google/gemma-4-31B-it The reference points are often older systems: GPT-4o and GPT-4 on ToolSandbox and AgentBoard, Claude-3.7-Sonnet on REAL, and Claude-2 or Haiku-3 inside the “range”. 2026 open 30B models plausibly match these without WSG.

It's hard to understand why we need to go through these complex procedures as well.

The paper writing needs to be improved. It's hard to understand this paper.

Questions for authors
  • Could you please provide more results w/ standard baselines such as the backbones themselves and reject sampling instead of refining?
  • Could you please explain why we need this method if the performance is worse than the baselines?
Reviewer_Lq3ROpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Language agents often fail on long-horizon tasks when a plan step is refused by the environment. Specifically, naive planners tend to rephrase the failed step and waste trial budgets, because plans are not grounded to world constraints. To solve this, this paper introduces World State Generator (WSG), a LoRA adapted planner model trained to revise an agent’s plan after execution feedback reveals a failed state. WSG writes plans as checkable world states that it keeps the initial plan from the base LLM but takes over replanning upon environment rejection. Building on the State-Centric Decision Process (SDP), it constructs synthetic worlds across 7 domains and extracts approximately 239K failure–repair pairs from the 226K episodes. Extensive experiments across 7 benchmarks show consistent improvements over prompted SDP.

Critical strengths
  1. Unlike most existing datasets focus primarily on successful trajectories, this paper systematically targets the failure-and-repair distribution by constructing 47,946 synthetic worlds with provably reachable goals across 7 domains. Thereby this paper builds a reliable workflow for generating high-utility repair data.
  2. WSB demonstrates zero shot transferability across 7 heterogeneous public agent benchmarks. Its improvements across all benchmarks suggest that it learns a useful ability to revise plans from failure feedback, rather than simply memorizing fixes for its training environments.
Critical weaknesses
  1. The baselines are too narrow to support the central claims. The only comparison is prompted SDP vs. WSG on the same backbone. Several baselines are needed to establish that refutation-conditioned repair supervision is what matters, e.g., SFT on the successful trajectories, and standard agents, including ReAct and Reflexion.
  2. The main novelty lies in the data collection and filtering pipeline. Methodologically, WSG applies rejection-sampling SFT to the replanning call of the existing SDP runtime, inheriting its state representation, execution, and validation. This is useful in engineering, but not methodological contribution. Without matched-data comparisons against simpler fine-tuning approaches, it remains unclear whether the proposed pipeline contributes more than additional task-specific supervision.
Questions for authors
  1. Under the same training data and compute budgets, how does WSG compare with training on unfiltered repairs? How do downstream results change when training only on repairs from successful trajectories?
  2. What is the performance of a prompted SDP baseline with thinking mode enabled for proposer calls, and with a stronger model as replanner? How does WSG compare in cost and accuracy?
  3. At evaluation, does the prompted SDP replanner receive the same information used during extraction (A.3)?
  4. Which training domains drive which benchmark gains? Do you conduct ablation study to explain which domains contribute most?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.