acceptodds
Under review as a conference paper at ICLR 2027

CWME: Continual World Model Evolution via Interaction Experience Learning

Abstract

World models provide interactive agents with environmental knowledge to support reasoning and decision making. However, continual interaction introduces new states and outcomes that may reveal limitations in previously acquired knowledge. Two key challenges arise in continual world modeling: (1) Interaction feedback is insufficiently integrated to validate and refine existing world knowledge, limiting its continual evolution. (2) Task-relevant historical experience fails to adapt to evolving environmental states, making direct reuse unreliable and repeated reasoning costly. To address these challenges, we propose CWME, a framework for Continual World Model Evolution via Interaction Experience Learning. CWME maintains Situational Knowledge, Action Trajectory Experience, and Failure-Aware Knowledge. These forms of knowledge provide environmental constraints, behavioral priors, and reuse boundaries for future decisions. Knowledge-Informed Trajectory Generation adapts historical behavioral structure to the current interaction instead of replaying past solutions. Experience-Guided Decision Routing distinguishes retrieval relevance from current-state applicability and selects direct reuse, lightweight verification, or fresh reasoning accordingly. Evolving Interaction Experience Updating incorporates interaction outcomes to refine accumulated knowledge and the conditions under which it can be reused. Extensive experiments on the public interactive benchmarks demonstrate that CWME outperforms all evaluated baselines in task performance while improving reasoning efficiency, further demonstrating the benefits of continual knowledge refinement and state-aware experience reuse.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
World Model, Interactive Agent

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_E24NOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes CWME, which maintains Situational Knowledge, Action Trajectory Experience, and Failure-Aware Knowledge to support interactive agents. It retrieves and adapts historical trajectories, assesses their applicability to the current state, and selects among direct reuse, verification by a small model, and full reasoning. The system continually updates structured knowledge from execution feedback while model parameters remain frozen during evaluation. The authors report improvements in task performance and reasoning efficiency on ScienceWorld and ALFWorld.

Critical strengths
  1. Historical experience can be relevant to a task yet inapplicable to the current state, which is a practical problem for interactive agents. The paper explicitly distinguishes retrieval relevance from applicability to the current decision and combines execution feedback with reasoning-route selection into a reasonably complete experience-reuse pipeline.
  2. The experiments demonstrate the system's practical value. CWME achieves 52.02 on ScienceWorld, compared with 42.63 for the strongest baseline, and 68.17% success on ALFWorld Unseen, compared with 56.97% for the strongest baseline. Relative to the Always Fresh variant, it improves the ScienceWorld score while reducing both token usage and wall-clock time by approximately 40%.
Critical weaknesses
  1. Limited methodological novelty. The paper primarily combines structured memory, trajectory reuse, applicability assessment, and adaptive reasoning. Component ablations show that these mechanisms are useful, but targeted comparisons with the closest experience-memory and continual-evolution methods are missing, leaving the independent technical advance unclear.
  2. The world-model framing lacks targeted validation. The experiments mainly evaluate experience reuse and task performance, without directly testing predictions of action effects or state transitions. The distinction from structured agent memory remains unclear. The continual-evolution experiments also contain only 25 episodes per seed, providing limited support for long-term adaptation.
  3. The definition of Verify supervision is questionable. Appendix A.3 states that Verify can only approve the original action and that success following rejection is attributed to Fresh. However, Eq. (26) requires examples where Direct fails and Verify succeeds. Under the same environment state and deterministic transitions, the same action should produce the same outcome, so the construction of this label needs clarification.
Questions for authors
  1. What is the core technical advance over the closest memory and continual-evolution methods? Can it be validated through comparisons with the same backbone, initial experience, and interaction budget?
  2. What environment-modeling capabilities does the world model add beyond structured agent memory? Could you provide evaluations of predictive capabilities and longer interaction streams?
  3. Please provide concrete examples where Direct fails and Verify succeeds, report the number of Verify-class samples, and clarify whether three-way matching guarantees identical full environment states.
5 comments

We thank the reviewer for the careful evaluation and helpful suggestions for improving our work.

W1/Q1. Methodological Novelty and Baseline Comparisons

1. Methodological Novelty.

The core innovation of CWME is not a simple combination of memory and reasoning components, but addressing two interconnected challenges in continual world-model evolution: insufficient knowledge refinement and unreliable experience reuse.

First, existing methods can accumulate interaction experience, but newly observed outcomes may reveal that previously acquired knowledge is incomplete or outdated. To address this, Experience-Driven Knowledge Evolution uses execution feedback to reinforce, revise, or suppress existing knowledge and its validity conditions, rather than simply appending new memories.

Second, historically successful experience may remain task-relevant but become inapplicable when environmental conditions change. To address this, Knowledge-Informed Trajectory Generation aligns and grounds historical behaviors in the current state, while Reliability-Guided Decision Routing assesses their applicability and dynamically allocates reasoning effort, avoiding unreliable reuse and unnecessary fresh reasoning.

The key technical advance is the closed-loop coupling of knowledge evolution and reliable experience reuse: evolving knowledge guides current decisions, while execution outcomes continuously refine the knowledge and applicability conditions used in subsequent interactions. This enables CWME to jointly address knowledge obsolescence and state-dependent experience validity, rather than treating memory updating and experience reuse as independent processes.

2. Advantages over Existing Memory and Continual-Learning Methods.

Compared with existing approaches, CWME provides three distinctive advantages. First, unlike PatchWorld, which emphasizes constructing executable structured world models, CWME further integrates interaction feedback to continually refine world knowledge and its reuse conditions. Second, compared with ExpeL, CLIN, and MemP, which emphasize transferable experience, continual knowledge acquisition, or procedural memory evolution, CWME explicitly adapts historical trajectories to current environmental states rather than relying primarily on retrieved knowledge or reusable procedures. Third, unlike MemRL, which learns memory utility from historical feedback, CWME additionally assesses current-state trajectory applicability and dynamically determines whether experience should be reused or fresh reasoning is required.

To evaluate these advantages, we compare CWME with these representative methods using Qwen3.5-9B as the full-reasoning backbone, under matched benchmark settings and interaction budgets. The following table reports mean ± standard deviation over three seeds.

MethodScienceWorld Score ↑ALFWorld Seen SR (%) ↑ALFWorld Unseen SR (%) ↑
PatchWorld16.81 ± 0.5645.03 ± 1.9644.25 ± 0.80
ExpeL11.75 ± 0.3310.68 ± 1.2816.87 ± 0.00
CLIN35.77 ± 0.6448.07 ± 1.7454.90 ± 1.33
MemP13.63 ± 2.5357.75 ± 1.9044.11 ± 2.52
MemRL42.63 ± 2.9259.07 ± 2.2251.03 ± 1.06
CWME (ours)52.02 ± 0.9060.40 ± 1.9668.17 ± 1.70

CWME achieves the highest performance across all three settings. On ScienceWorld, it outperforms MemRL, the strongest baseline, by 9.39 score points. On ALFWorld, CWME also surpasses MemP and MemRL on Seen tasks and achieves a substantial advantage over CLIN on Unseen tasks. These results indicate that continually refining knowledge while explicitly validating and adapting historical experience offers advantages over knowledge accumulation, procedural memory evolution, and feedback-based memory utility estimation alone.

W2/Q2. Environmental Modeling and Long-Term World-Model Evolution

1. Environmental Modeling Beyond Structured Memory.

CWME goes beyond storing interaction histories by maintaining environmental states, action preconditions, expected effects, and validity constraints. Situational Knowledge captures environmental relations and observed transitions, Action Trajectory Experience records reusable behaviors and their expected effects, and Failure-Aware Knowledge identifies conditions under which prior experience becomes unreliable. Together, these representations support action-conditioned state prediction and state-aware planning, while execution feedback continuously refines outdated knowledge.

To directly evaluate this capability, we introduce an action-conditioned state-transition prediction experiment, comparing CWME with Persistence, Action-only prediction, Trajectory Retrieval, and Frozen Structured World Model.

MethodExact Acc. ↑Changed Acc. ↑Mix ↑Pour ↑Wait ↑Coverage ↑
Persistence61.430.000.0039.8288.19—
Action-only16.0739.8165.2225.660.69—
Trajectory Retrieval74.6456.4891.3056.6486.1177.50
Frozen Structured WM79.2963.8986.9671.6884.0384.29
Evolving CWME86.4380.5695.6579.6590.2893.57

Exact Acc. measures overall transition-prediction accuracy; Changed Acc. evaluates cases involving actual state changes.

CWME achieves the highest accuracy and coverage across all evaluated metrics. Its advantage over Trajectory Retrieval indicates that environmental modeling extends beyond recalling historical transitions, while its improvement over Frozen Structured WM demonstrates the benefit of continually refining action-effect knowledge through interaction feedback. In particular, stronger Changed Acc. supports its ability to predict actual environmental changes rather than merely assume state persistence.

2. Long-Term World-Model Evolution.

To address the limited horizon of the original five-cycle experiment, we extend continual interaction to 20 cycles (100 episodes per run), comparing CWME against Frozen World Model and Append-only Evolution.

MethodCycle 5 ↑Cycle 10 ↑Cycle 15 ↑Cycle 20 ↑
Frozen World Model50.3351.2050.1349.60
Append-only Evolution51.2749.8750.3351.47
CWME55.0055.6053.4755.13

CWME consistently outperforms both baselines across all evaluation checkpoints, maintaining its advantage through Cycle 20. Unlike Frozen World Model, which cannot incorporate new experience, or Append-only Evolution, which accumulates knowledge without revising outdated information, CWME selectively updates existing knowledge and its validity conditions based on execution feedback. Its sustained performance over the extended interaction stream supports the effectiveness of continual knowledge refinement beyond the original short-horizon evaluation.

W3/Q3. Verified Reuse Supervision

1. The definition of Verify supervision. Under identical complete environment states and deterministic transitions, identical actions must produce identical outcomes. We agree with this observation. The distinction is that Verified Reuse can refine action arguments rather than merely approve or reject the original action, which was not sufficiently clarified in Appendix A.3.

Specifically, an action is represented as a=(f,R,G)a=(f,R,G), where ff is the action family, RR denotes ordered semantic roles, and GG contains concrete arguments. Direct Reuse executes the grounded historical action, while Verified Reuse may substitute or disambiguate its arguments while preserving the action family, role structure, and argument arity. The refined action must also satisfy current-state admissibility constraints. Changes requiring a different action structure are handled by Fresh Reasoning. Therefore, Direct and Verify may execute different concrete actions from the same state, without contradicting deterministic transitions.

2. Example (ScienceWorld, use-thermometer).

RouteExecuted ActionInterpretation
Directgo to hallwayHistorical destination
Verifygo to bathroomState-grounded destination refinement

Both actions preserve the go action structure but use different destination arguments. The task requires reaching the bathroom, Direct fail to advance the task while Verify can succeed.

3. Three-Route Supervision Statistics.

We analyze 3,000 decision samples using Direct (D), Verify (V), and Fresh (F) counterfactual evaluations. The supervision label selects the lowest-cost successful route, excluding cases where all routes fail.

Success (D,V,F)SamplesRatioLabel
(1,1,1)1,32744.23%Direct
(0,1,1)50316.77%Verify
(0,0,1)77625.87%Fresh
(0,0,0)1876.23%Excluded
(1,1,0)712.37%Direct
(1,0,1)581.93%Direct
(1,0,0)321.07%Direct
(0,1,0)461.53%Verify

Ratios for supervised classes are calculated over 2,813 valid samples.

The 549 Verify-class samples comprise 503 (0,1,1) and 46 (0,1,0) cases, where Direct fails but Verify succeeds. Verify success requires observed progress from the independently executed refined action, not merely verifier approval. If verification rejects the candidate and execution falls back to Fresh, any subsequent success is attributed to Fresh.

4. Counterfactual State Matching.

The three routes are evaluated under strictly matched complete environment states. For each counterfactual sample, Direct, Verify, and Fresh are evaluated from the same reconstructed decision state, with identical task configurations, random seeds, action histories, candidate trajectories, and initial world knowledge. Environment states are reconstructed through deterministic replay, and state equivalence is verified before executing each route. Knowledge updates are disabled during comparison to prevent cross-route interference. Thus, the only difference between the three branches is the selected reasoning route, ensuring that supervision labels reflect genuine differences in route outcomes rather than inconsistencies in environmental states.

Reviewer_yaMROpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes CWME, which addresses two problems in world models, continual learning and evolution. CWME achieves the best performance against the baselines on ScienceWorld and ALFWorld, and in particular improves over the strongest baseline by 11%p on the Unseen split of ALFWorld.S1. The problem definition is clear. The paper precisely identifies two critical issues that can arise during the continual learning of a world model.

S2. The problem definition and the methodology are consistent. Each component of the method directly targets and resolves a problem the paper identifies.

Critical strengths

S1. The problem definition is clear. The paper precisely identifies two critical issues that can arise during the continual learning of a world model.

S2. The problem definition and the methodology are consistent. Each component of the method directly targets and resolves a problem the paper identifies.

Critical weaknesses

W1: Insufficient justification for individual components of the method. The justification for each component of the proposed method is unclear. The ablation study (Table 3) justifies each component at a high level, but the design intent behind the finer-grained elements within each component remains unclear. For example, no justification could be found for defining ht,i\mathbf h_{t, i} as ht,i=[ets;et,iτ;∣ets−et,iτ∣;ets⊙et,iτ]\mathbf h_{t, i} = [\mathbf e_t^s; \mathbf e_{t, i}^\tau; | \mathbf e_t^s - \mathbf e_{t, i}^\tau |; \mathbf e_t^s \odot \mathbf e_{t, i}^\tau]. In my view, the third term could just as well have been an L2 distance or a cosine similarity instead of an L1 distance.

W2: The environments in the main experiments are too weak to demonstrate the strength of the method. The goal of this paper is a world model that keeps evolving, but ScienceWorld and ALFWorld are weak testbeds for showing the advantage of such a method. ALFWorld is split into seen and unseen tasks, but the environment itself does not change continuously (actions such as picking up or heating objects do change the environment state, but in my opinion this degree of change is insufficient). It would be informative to see performance in a continuously changing environment such as HAZARD [1].

[1] Zhou, Qinhong, et al. “Hazard challenge: Embodied decision making in dynamically changing environments.” International Conference on Learning Representations. Vol. 2024. 2024.

Questions for authors

Q1. In Table 1, CWME improves from Seen to Unseen by 7.77%p, whereas Reflexion improves by as much as 17.25%p. How should this be interpreted?

Q2. Over how many trials or random seeds was the performance in each experiment measured?

4 comments

We appreciate the reviewer's thoughtful comments and insightful recommendations.

W1. Justification and Ablation of Individual Method Components

The Candidate Assessment module aims to determine whether a retrieved trajectory is applicable to the current state, rather than merely measuring semantic similarity. Given the state embedding ese^s and candidate-trajectory embedding eτe^\tau, we construct:

h=[es;eτ;∣es−eτ∣;es⊙eτ].h=[e^s;e^\tau;|e^s-e^\tau|;e^s\odot e^\tau].

The raw embeddings preserve the semantic content of the state and trajectory. The element-wise absolute difference captures dimension-specific mismatches, allowing the assessment network to learn which discrepancies matter for applicability. The Hadamard product captures complementary dimension-wise interactions.

Importantly, the third term is a vector of absolute differences, not a scalar L1 distance. An L1/L2 distance or cosine similarity compresses the relationship between two embeddings into a single value. In contrast, the difference vector preserves mismatch patterns across dimensions, allowing the assessment network to learn their relative importance rather than relying on a fixed global similarity measure.

To evaluate these design choices, we conduct fine-grained feature ablations comparing the full representation with raw embeddings alone, removal of individual pairwise terms, scalar L1/L2 and cosine alternatives, and squared differences. The following table reports mean ± standard deviation (higher is better).

Feature VariantMacro-F1 ↑AUROC ↑AUPRC ↑
Full Pairwise (CWME)0.8566 ± 0.01420.9596 ± 0.00340.8717 ± 0.0151
Raw Only0.8418 ± 0.01430.9533 ± 0.00480.8495 ± 0.0208
w/o Difference0.8366 ± 0.01650.9481 ± 0.00660.8346 ± 0.0257
w/o Hadamard0.8532 ± 0.00910.9565 ± 0.00240.8595 ± 0.0135
L1 Scalar0.8047 ± 0.03290.9275 ± 0.02360.7677 ± 0.0831
L2 Scalar0.8355 ± 0.00770.9490 ± 0.00370.8351 ± 0.0141
Cosine Scalar0.8393 ± 0.01120.9504 ± 0.00210.8420 ± 0.0108
Squared Difference0.8411 ± 0.01920.9523 ± 0.00530.8456 ± 0.0231

The full representation achieves the highest mean Macro-F1, AUROC, and AUPRC among the evaluated variants. Removing the element-wise difference degrades performance, supporting the importance of explicitly modeling dimension-level state–trajectory mismatches. Replacing it with scalar L1/L2 distances or cosine similarity also reduces performance, suggesting that global similarity measures lose useful information for state-conditioned applicability assessment.

Meanwhile, removing the Hadamard product produces a smaller reduction in mean performance, indicating that dimension-wise interactions provide complementary information. The squared-difference variant also underperforms the full representation, supporting the use of element-wise absolute differences in the evaluated setting. These findings provide empirical justification for the individual pairwise features and their combination.

W2. Validating World-Model Evolution in Continually Changing Environments

ScienceWorld and ALFWorld primarily evaluate how agents acquire and update world knowledge through multi-step interaction. However, they do not fully capture environments that change independently of agent actions. To address this limitation, we extend our evaluation to HAZARD, a physically simulated 3D embodied benchmark featuring continuously changing environmental dynamics.

HAZARD includes three disaster scenarios: Fire, Flood, and Wind. Fire propagation, changing water levels, and wind-driven object displacement can alter environmental constraints independently of agent actions. Consequently, previously successful trajectories may become invalid as object states and spatial relations change. We evaluate Reflexion, MemRL, and CWME under matched scenarios, observation conditions, environment configurations, and interaction budgets. The following table reports rescued-object Value (higher is better), Steps (lower is better), and Damage (lower is better).

ScenarioMethodValue (%) ↑Steps ↓Damage (%) ↓
FireReflexion41.07349.1040.91
FireMemRL52.97310.3043.48
FireCWME61.13208.6738.15
FloodReflexion34.72306.4257.89
FloodMemRL37.82337.8452.63
FloodCWME43.24310.0047.62
WindReflexion2.611106.00—
WindMemRL8.191097.60—
WindCWME19.62645.50—

Value denotes the percentage of rescued object value; Steps denotes the mean number of rescue steps; Damage denotes the percentage of rescued objects damaged. Following the HAZARD evaluation protocol, Damage is not reported for Wind.

CWME achieves the highest rescued-object value across all three scenarios, with fewer steps than MemRL and lower damage in both Fire and Flood. Although CWME requires slightly more steps than Reflexion in Flood, it achieves higher rescued value and lower damage. These results demonstrate CWME's effectiveness in physically simulated environments with continuously changing dynamics, extending its applicability beyond ScienceWorld and ALFWorld and supporting the ability of its evolving world model to adapt historical experience to changing environmental conditions.

Q1. Interpreting Seen–Unseen Performance Differences on ALFWorld

Reflexion's larger Seen–Unseen improvement is primarily due to its relatively poor performance on Seen tasks. This is because Reflexion relies on retrospective verbal reflection rather than explicit state-conditioned action validation. When tasks involve complex action preconditions, object-state changes, or multi-step dependencies, its reflections do not directly enforce action validity, limiting its effectiveness on Seen tasks. On Unseen tasks, feedback-driven self-correction helps Reflexion recover from unsuccessful actions and achieve better performance. Consequently, its lower Seen success rate produces a larger Seen–Unseen difference.

The relevant results from Table 1 are summarized below:

MethodSeen SR (%) ↑Unseen SR (%) ↑Unseen − Seen
Reflexion39.72 ± 1.1356.97 ± 3.37+17.25
CWME60.40 ± 1.9668.17 ± 1.70+7.77

In contrast, CWME explicitly models environmental constraints and validates historical experience against the current state, enabling reliable decisions across both splits. CWME outperforms Reflexion on both Seen and Unseen tasks, by 20.68 and 11.20 percentage points, respectively. Because CWME already achieves substantially higher performance on Seen tasks, its Seen–Unseen increase is naturally smaller. Therefore, Reflexion's larger increase does not indicate stronger generalization; CWME demonstrates higher absolute performance across both evaluation splits.

Q2. Independent Trials and Random Seeds Across Experiments

Unless otherwise specified, all experiments in the original manuscript are conducted over three independent runs with different random seeds. Results are aggregated across these runs, with mean ± standard deviation reported where applicable.

The evaluation protocols for the individual experiments are summarized below:

ExperimentSeedsEvaluation ScopeMetrics
ScienceWorld Overall (Table 1)330 task familiesTask Score
ALFWorld Overall (Table 1)36 task families, Seen/UnseenSuccess Rate
Continual Evolution (Fig. 2)35 cycles × 5 episodes per seedScore, Knowledge Updates
Candidate Assessment (Table 2)3Episode-disjoint test setMacro-F1, AUROC, AUPRC
Decision Routing (Fig. 3)35 controlled task familiesScore, Routing Ratios, Fresh Calls
Component Ablations (Table 3)3Controlled ScienceWorld/ALFWorld settingsScore, Success Rate
Reasoning Efficiency (Fig. 4)3Full ScienceWorld benchmarkCalls, Tokens, Time
Router Prediction (App. Table 13)3Held-out decision statesPrecision, Recall, F1
Model Scale (App. Table 19)3ScienceWorldScore, Calls, Tokens, Time
Sensitivity (App. Table 20)35 controlled task familiesScore, Time

I maintain my rating of weak acceptance, as the response confirms my initial positive assessment.

Reviewer_Qbe5OpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a framework that uses an LLM to organize accumulated knowledge into an editable memory and to reuse it for planning in text-based environments. The goal is to improve the continual learning performance of world models. The main contribution is a method that lets the LLM integrate past experience by reading and editing this memory. It is evaluated on two text-based environments, where it outperforms the baselines.

Critical strengths
  • The framework leverages the reasoning capabilities of LLMs to integrate past experience more effectively, which improves continual learning over the baselines.
  • The approach outperforms the baselines on both studied tasks.
Critical weaknesses
  • The evaluation is limited to text-based environments, whose state and action spaces are discrete. It is unclear how the approach would transfer beyond this setting (e.g., continuous control or visual observations).
  • The use of the term "world model" is loose. The method is an LLM equipped with an editable memory. It has no learned transition model and does no predictive control. This is quite far from what is usually meant by world models, i.e., action-conditioned transition models used to roll out trajectories during planning. Here, the LLM simply chooses actions at planning time based on a memory of past trajectories. I would encourage the authors to either reframe the contribution or justify the terminology more carefully.
  • Frontier LLMs such as Astra or Opus have shown strong off-the-shelf planning capabilities, yet none are evaluated. Their cost may explain this, but it leaves open whether the proposed framework still brings gains in that regime. A cost-matched comparison (in dollars or tokens) against such models would be ideal.
Questions for authors
  • Could the approach be extended beyond text-based environments? If so, how would the memory and planning components need to change?
  • Would the framework remain relevant with models that already plan well off the shelf? Or do its gains shrink as base model capability increases?
5 comments

We sincerely thank the reviewer for the constructive feedback and valuable suggestions.

W1/Q1. Generalization Beyond Text-Based Environments

1. Multimodal Adaptation.

CWME is not inherently limited to text-based environments. Its core mechanisms focus on continual world-knowledge evolution, state-conditioned experience reuse, and adaptive decision-making, which can operate on structured environmental information derived from different observation modalities.

To validate CWME's adaptability to multimodal environments, we extend our evaluation to HAZARD, a physically simulated 3D embodied benchmark featuring Fire, Flood, and Wind scenarios. Unlike ScienceWorld and ALFWorld, HAZARD provides visual observations and dynamically changing physical environments, where fire propagation, rising water levels, and wind-driven object displacement can invalidate previously successful behaviors.

Our current implementation adopts a visual-to-text adaptation strategy: visual observations are converted into textual descriptions of objects, spatial relations, and environmental states. These descriptions are then incorporated into CWME's existing world-model representation, allowing trajectory generation, applicability assessment, decision routing, and knowledge evolution to operate without modifying their core mechanisms. This enables CWME to process information originating from visual observations while retaining its language-based memory and planning framework.

We compare Reflexion, MemRL, and CWME under matched scenarios, observation conditions, environment configurations, and interaction budgets.

ScenarioMethodValue (%) ↑Steps ↓Damage (%) ↓
FireReflexion41.07349.1040.91
FireMemRL52.97310.3043.48
FireCWME61.13208.6738.15
FloodReflexion34.72306.4257.89
FloodMemRL37.82337.8452.63
FloodCWME43.24310.0047.62
WindReflexion2.611106.00—
WindMemRL8.191097.60—
WindCWME19.62645.50—

Value denotes the percentage of rescued object value; Steps denotes the mean number of rescue steps; Damage denotes the percentage of rescued objects damaged. Following the HAZARD evaluation protocol, Damage is not reported for Wind.

CWME achieves the highest rescued-object value across all three scenarios, with fewer steps than MemRL throughout and lower damage in Fire and Flood. These results demonstrate that CWME can effectively adapt to visually observed, dynamically changing 3D environments through a visual-to-text interface, extending its applicability beyond purely text-based benchmarks.

2. Extension of Memory and Planning Components.

For memory, a direct multimodal extension would incorporate visual features and spatial representations alongside semantic knowledge, enabling more precise modeling of object states, environmental changes, and uncertainty.

For planning, trajectory grounding and applicability assessment could incorporate visual-spatial consistency and perceptual uncertainty. Continuous actions could be represented as parameterized skills, with CWME handling high-level planning and a low-level controller executing motor commands.

These extensions would enhance perceptual grounding and action precision while preserving CWME's core knowledge-evolution and decision-routing mechanisms.

W2. World-Model Transition Prediction and Planning

CWME is more than an LLM with editable memory. Its world model Mt=(st,Kt)M_t=(s_t,K_t) represents the current environmental state together with action preconditions, observed state transitions, expected effects, and validity constraints. Unlike conventional agent memory that primarily stores historical experiences, CWME uses action-conditioned knowledge to anticipate environmental changes, guide decisions, and revise outdated knowledge through execution feedback.

Expected Action Effects and Short-Horizon Planning. CWME uses historical action-effect knowledge to construct short-horizon trajectory candidates. Knowledge-Informed Trajectory Generation aligns historical trajectories with the current state and grounds their actions using environmental constraints and expected state transitions. Candidate Assessment and Decision Routing then determine whether the candidate is applicable and how much additional reasoning is required. Only the selected next action is executed, followed by observation and replanning. Thus, CWME supports action-effect-informed, closed-loop planning.

Action-Conditioned State-Transition Prediction. To directly validate the predictive capabilities of CWME's world model, we introduce an action-conditioned state-transition prediction experiment, comparing CWME with Persistence, Action-only prediction, Trajectory Retrieval, and Frozen Structured World Model.

MethodExact Acc. ↑Changed Acc. ↑Mix ↑Pour ↑Wait ↑Coverage ↑
Persistence61.430.000.0039.8288.19—
Action-only16.0739.8165.2225.660.69—
Trajectory Retrieval74.6456.4891.3056.6486.1177.50
Frozen Structured WM79.2963.8986.9671.6884.0384.29
Evolving CWME86.4380.5695.6579.6590.2893.57

Exact Acc. measures overall transition-prediction accuracy; Changed Acc. evaluates cases involving actual state changes, preventing persistence predictions from dominating the evaluation.

CWME achieves the highest prediction accuracy and coverage across all evaluated metrics. Its improvement over Trajectory Retrieval demonstrates that the world model captures action-conditioned state changes beyond historical trajectory recall. Moreover, CWME outperforms Frozen Structured WM, showing that continually revising action-effect knowledge improves environmental prediction rather than merely increasing stored experience. The substantial improvement in Changed Acc. further confirms its ability to predict actual state changes instead of simply assuming state persistence.

Experience-Driven World-Model Evolution. Unlike static structured memory, CWME continuously updates environmental relations, expected action effects, and trajectory validity conditions based on observed outcomes. These updates directly influence subsequent trajectory grounding, applicability assessment, and action selection. The world model therefore serves as an evolving representation of environmental dynamics that supports prediction and planning, rather than merely a repository of past interactions.

W3/Q2. Effectiveness with Frontier LLMs and Performance–Cost Trade-offs

CWME complements strong backbone models by improving experience reuse and reducing unnecessary fresh reasoning. Even when an LLM has strong planning capabilities, previously successful experience may become inapplicable as environmental states change. CWME addresses this through evolving world knowledge, state-conditioned applicability assessment, and adaptive reasoning allocation.

To examine whether CWME remains effective with stronger models, we conduct additional experiments using Astra and Qwen3.5-9B on five representative ScienceWorld tasks. For each backbone, we compare Always Fresh with CWME under matched task settings and interaction budgets. The following table reports task scores, average Fresh Calls, average token usage, and recorded API costs.

BackboneMethodScore ↑Fresh Calls ↓Tokens (K) ↓Cost ($) ↓
Qwen3.5-9BAlways Fresh56.6032.1245.18—
Qwen3.5-9BCWME61.7828.0038.20—
AstraAlways Fresh69.6027.6044.153.41
AstraCWME73.4024.2036.972.78

Fresh Calls, Tokens, and Cost are averaged over the evaluated tasks; Cost represents the mean recorded API expenditure per task.

**CWME improves task performance with both backbones. Importantly, Astra + CWME achieves a higher score (73.40 vs. 69.60) while reducing token usage by 16.3% and API cost by 18.5% compared with Astra Always Fresh. This demonstrates that even with a strong planning model, CWME provides a favorable performance–cost trade-off by reusing applicable experience and reserving expensive reasoning for decisions where it is necessary.

These results indicate that CWME remains complementary to frontier LLMs, improving both task effectiveness and reasoning efficiency rather than relying on limitations of weaker backbone models.

Comments on the paper

2 comments

Dear Reviewers,

Thank you for your efforts so far. As we are at the end of the discussion period, I would like to kindly encourage you to review the authors' rebuttals and engage in further discussions, particularly if the authors have addressed your questions or concerns.

Thank you for your time and contributions!

AC

Author Summary: Key Clarifications and Additional Experimental Evidence

We sincerely thank all reviewers for their constructive feedback and valuable suggestions. The discussion has helped us clarify CWME's technical contributions and substantially strengthen its experimental validation. Beyond addressing individual comments, we have added several targeted experiments that directly examine the main concerns raised during the review. We would like to highlight the key advances made during the rebuttal.

Methodological Novelty and Comparison with Prior Methods.

We clarified that CWME is not simply a combination of memory and reasoning components, but addresses two interconnected challenges: outdated world knowledge and unreliable reuse of historical experience under changing environmental conditions. Its central contribution is a closed loop between continual knowledge evolution and state-conditioned experience reuse, where execution feedback refines both environmental knowledge and the conditions governing subsequent decisions.

We also clarified the distinctions from PatchWorld, ExpeL, CLIN, MemP, and MemRL. Under a common full-reasoning backbone and matched evaluation budgets, CWME achieves 52.02 on ScienceWorld and 68.17% success on ALFWorld Unseen, outperforming the strongest evaluated baselines. These comparisons demonstrate the practical advantage of jointly refining world knowledge and validating historical experience before reuse.

Direct Validation of World-Model Capabilities.

To address the concern that CWME resembles structured memory rather than a world model, we introduced a new action-conditioned state-transition prediction experiment. CWME achieves 86.43% overall prediction accuracy and 80.56% accuracy on actual state changes, outperforming both trajectory retrieval and a frozen structured world model.

These results provide direct evidence that CWME captures action-conditioned environmental changes beyond historical experience retrieval. We further clarified how expected action effects support short-horizon, closed-loop planning, while execution feedback continually refines the underlying world knowledge.

Long-Term Evolution and Generalization Beyond Text-Based Environments.

We extended the original continual-evolution evaluation from 5 to 20 interaction cycles (100 episodes per run). CWME consistently outperforms Frozen World Model and Append-only Evolution across all reported checkpoints, demonstrating sustained benefits from selective knowledge refinement over longer interaction streams.

We additionally evaluated CWME on HAZARD, a physically simulated 3D benchmark with Fire, Flood, and Wind scenarios. Using a visual-to-text observation interface, CWME achieves the highest rescued-object value across all three scenarios compared with Reflexion and MemRL. These experiments extend the evidence beyond purely text-based benchmarks and demonstrate the framework's effectiveness under continuously changing physical dynamics.

Effectiveness with Stronger Foundation Models.

To examine whether CWME remains useful when backbone planning capabilities improve, we added experiments using Astra. CWME improves Astra's task score from 69.60 to 73.40, while reducing token usage by 16.3% and average API cost by 18.5%.

This demonstrates that CWME remains beneficial with a stronger planning backbone, improving task outcomes while reducing reasoning costs rather than relying on weaknesses of smaller models.

Additional Methodological Clarifications.

We also addressed the concerns regarding Verified Reuse supervision by clarifying its constrained action-argument refinement, strict counterfactual state matching, and success-based label construction. The additional analysis reports 549 Verify-class samples and explains how Direct failure and Verify success can arise without contradicting deterministic environment transitions. Fine-grained feature ablations further support the design of the applicability representation.

Taken together, the rebuttal substantially strengthens the paper beyond its original submission. The additional experiments directly validate predictive world-model capabilities, longer-term adaptation, cross-environment applicability, and effectiveness with stronger foundation models. We believe these results, together with the original benchmark comparisons, provide stronger and more comprehensive evidence for CWME's technical contribution and practical significance.

We respectfully hope the reviewers and Area Chair will consider these substantive additions and clarifications in their final assessment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.