- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes CWME, which maintains Situational Knowledge, Action Trajectory Experience, and Failure-Aware Knowledge to support interactive agents. It retrieves and adapts historical trajectories, assesses their applicability to the current state, and selects among direct reuse, verification by a small model, and full reasoning. The system continually updates structured knowledge from execution feedback while model parameters remain frozen during evaluation. The authors report improvements in task performance and reasoning efficiency on ScienceWorld and ALFWorld.
Critical strengths
- Historical experience can be relevant to a task yet inapplicable to the current state, which is a practical problem for interactive agents. The paper explicitly distinguishes retrieval relevance from applicability to the current decision and combines execution feedback with reasoning-route selection into a reasonably complete experience-reuse pipeline.
- The experiments demonstrate the system's practical value. CWME achieves 52.02 on ScienceWorld, compared with 42.63 for the strongest baseline, and 68.17% success on ALFWorld Unseen, compared with 56.97% for the strongest baseline. Relative to the Always Fresh variant, it improves the ScienceWorld score while reducing both token usage and wall-clock time by approximately 40%.
Critical weaknesses
- Limited methodological novelty. The paper primarily combines structured memory, trajectory reuse, applicability assessment, and adaptive reasoning. Component ablations show that these mechanisms are useful, but targeted comparisons with the closest experience-memory and continual-evolution methods are missing, leaving the independent technical advance unclear.
- The world-model framing lacks targeted validation. The experiments mainly evaluate experience reuse and task performance, without directly testing predictions of action effects or state transitions. The distinction from structured agent memory remains unclear. The continual-evolution experiments also contain only 25 episodes per seed, providing limited support for long-term adaptation.
- The definition of Verify supervision is questionable. Appendix A.3 states that Verify can only approve the original action and that success following rejection is attributed to Fresh. However, Eq. (26) requires examples where Direct fails and Verify succeeds. Under the same environment state and deterministic transitions, the same action should produce the same outcome, so the construction of this label needs clarification.
Questions for authors
- What is the core technical advance over the closest memory and continual-evolution methods? Can it be validated through comparisons with the same backbone, initial experience, and interaction budget?
- What environment-modeling capabilities does the world model add beyond structured agent memory? Could you provide evaluations of predictive capabilities and longer interaction streams?
- Please provide concrete examples where Direct fails and Verify succeeds, report the number of Verify-class samples, and clarify whether three-way matching guarantees identical full environment states.
5 comments
W2/Q2. Environmental Modeling and Long-Term World-Model Evolution
1. Environmental Modeling Beyond Structured Memory.
CWME goes beyond storing interaction histories by maintaining environmental states, action preconditions, expected effects, and validity constraints. Situational Knowledge captures environmental relations and observed transitions, Action Trajectory Experience records reusable behaviors and their expected effects, and Failure-Aware Knowledge identifies conditions under which prior experience becomes unreliable. Together, these representations support action-conditioned state prediction and state-aware planning, while execution feedback continuously refines outdated knowledge.
To directly evaluate this capability, we introduce an action-conditioned state-transition prediction experiment, comparing CWME with Persistence, Action-only prediction, Trajectory Retrieval, and Frozen Structured World Model.
| Method | Exact Acc. ↑ | Changed Acc. ↑ | Mix ↑ | Pour ↑ | Wait ↑ | Coverage ↑ |
|---|---|---|---|---|---|---|
| Persistence | 61.43 | 0.00 | 0.00 | 39.82 | 88.19 | — |
| Action-only | 16.07 | 39.81 | 65.22 | 25.66 | 0.69 | — |
| Trajectory Retrieval | 74.64 | 56.48 | 91.30 | 56.64 | 86.11 | 77.50 |
| Frozen Structured WM | 79.29 | 63.89 | 86.96 | 71.68 | 84.03 | 84.29 |
| Evolving CWME | 86.43 | 80.56 | 95.65 | 79.65 | 90.28 | 93.57 |
Exact Acc. measures overall transition-prediction accuracy; Changed Acc. evaluates cases involving actual state changes.
CWME achieves the highest accuracy and coverage across all evaluated metrics. Its advantage over Trajectory Retrieval indicates that environmental modeling extends beyond recalling historical transitions, while its improvement over Frozen Structured WM demonstrates the benefit of continually refining action-effect knowledge through interaction feedback. In particular, stronger Changed Acc. supports its ability to predict actual environmental changes rather than merely assume state persistence.
2. Long-Term World-Model Evolution.
To address the limited horizon of the original five-cycle experiment, we extend continual interaction to 20 cycles (100 episodes per run), comparing CWME against Frozen World Model and Append-only Evolution.
| Method | Cycle 5 ↑ | Cycle 10 ↑ | Cycle 15 ↑ | Cycle 20 ↑ |
|---|---|---|---|---|
| Frozen World Model | 50.33 | 51.20 | 50.13 | 49.60 |
| Append-only Evolution | 51.27 | 49.87 | 50.33 | 51.47 |
| CWME | 55.00 | 55.60 | 53.47 | 55.13 |
CWME consistently outperforms both baselines across all evaluation checkpoints, maintaining its advantage through Cycle 20. Unlike Frozen World Model, which cannot incorporate new experience, or Append-only Evolution, which accumulates knowledge without revising outdated information, CWME selectively updates existing knowledge and its validity conditions based on execution feedback. Its sustained performance over the extended interaction stream supports the effectiveness of continual knowledge refinement beyond the original short-horizon evaluation.
W3/Q3. Verified Reuse Supervision
1. The definition of Verify supervision. Under identical complete environment states and deterministic transitions, identical actions must produce identical outcomes. We agree with this observation. The distinction is that Verified Reuse can refine action arguments rather than merely approve or reject the original action, which was not sufficiently clarified in Appendix A.3.
Specifically, an action is represented as , where is the action family, denotes ordered semantic roles, and contains concrete arguments. Direct Reuse executes the grounded historical action, while Verified Reuse may substitute or disambiguate its arguments while preserving the action family, role structure, and argument arity. The refined action must also satisfy current-state admissibility constraints. Changes requiring a different action structure are handled by Fresh Reasoning. Therefore, Direct and Verify may execute different concrete actions from the same state, without contradicting deterministic transitions.
2. Example (ScienceWorld, use-thermometer).
| Route | Executed Action | Interpretation |
|---|---|---|
| Direct | go to hallway | Historical destination |
| Verify | go to bathroom | State-grounded destination refinement |
Both actions preserve the go action structure but use different destination arguments. The task requires reaching the bathroom, Direct fail to advance the task while Verify can succeed.
3. Three-Route Supervision Statistics.
We analyze 3,000 decision samples using Direct (D), Verify (V), and Fresh (F) counterfactual evaluations. The supervision label selects the lowest-cost successful route, excluding cases where all routes fail.
| Success (D,V,F) | Samples | Ratio | Label |
|---|---|---|---|
| (1,1,1) | 1,327 | 44.23% | Direct |
| (0,1,1) | 503 | 16.77% | Verify |
| (0,0,1) | 776 | 25.87% | Fresh |
| (0,0,0) | 187 | 6.23% | Excluded |
| (1,1,0) | 71 | 2.37% | Direct |
| (1,0,1) | 58 | 1.93% | Direct |
| (1,0,0) | 32 | 1.07% | Direct |
| (0,1,0) | 46 | 1.53% | Verify |
Ratios for supervised classes are calculated over 2,813 valid samples.
The 549 Verify-class samples comprise 503 (0,1,1) and 46 (0,1,0) cases, where Direct fails but Verify succeeds. Verify success requires observed progress from the independently executed refined action, not merely verifier approval. If verification rejects the candidate and execution falls back to Fresh, any subsequent success is attributed to Fresh.
4. Counterfactual State Matching.
The three routes are evaluated under strictly matched complete environment states. For each counterfactual sample, Direct, Verify, and Fresh are evaluated from the same reconstructed decision state, with identical task configurations, random seeds, action histories, candidate trajectories, and initial world knowledge. Environment states are reconstructed through deterministic replay, and state equivalence is verified before executing each route. Knowledge updates are disabled during comparison to prevent cross-route interference. Thus, the only difference between the three branches is the selected reasoning route, ensuring that supervision labels reflect genuine differences in route outcomes rather than inconsistencies in environmental states.
We thank the reviewer for the careful evaluation and helpful suggestions for improving our work.
W1/Q1. Methodological Novelty and Baseline Comparisons
1. Methodological Novelty.
The core innovation of CWME is not a simple combination of memory and reasoning components, but addressing two interconnected challenges in continual world-model evolution: insufficient knowledge refinement and unreliable experience reuse.
First, existing methods can accumulate interaction experience, but newly observed outcomes may reveal that previously acquired knowledge is incomplete or outdated. To address this, Experience-Driven Knowledge Evolution uses execution feedback to reinforce, revise, or suppress existing knowledge and its validity conditions, rather than simply appending new memories.
Second, historically successful experience may remain task-relevant but become inapplicable when environmental conditions change. To address this, Knowledge-Informed Trajectory Generation aligns and grounds historical behaviors in the current state, while Reliability-Guided Decision Routing assesses their applicability and dynamically allocates reasoning effort, avoiding unreliable reuse and unnecessary fresh reasoning.
The key technical advance is the closed-loop coupling of knowledge evolution and reliable experience reuse: evolving knowledge guides current decisions, while execution outcomes continuously refine the knowledge and applicability conditions used in subsequent interactions. This enables CWME to jointly address knowledge obsolescence and state-dependent experience validity, rather than treating memory updating and experience reuse as independent processes.
2. Advantages over Existing Memory and Continual-Learning Methods.
Compared with existing approaches, CWME provides three distinctive advantages. First, unlike PatchWorld, which emphasizes constructing executable structured world models, CWME further integrates interaction feedback to continually refine world knowledge and its reuse conditions. Second, compared with ExpeL, CLIN, and MemP, which emphasize transferable experience, continual knowledge acquisition, or procedural memory evolution, CWME explicitly adapts historical trajectories to current environmental states rather than relying primarily on retrieved knowledge or reusable procedures. Third, unlike MemRL, which learns memory utility from historical feedback, CWME additionally assesses current-state trajectory applicability and dynamically determines whether experience should be reused or fresh reasoning is required.
To evaluate these advantages, we compare CWME with these representative methods using Qwen3.5-9B as the full-reasoning backbone, under matched benchmark settings and interaction budgets. The following table reports mean ± standard deviation over three seeds.
CWME achieves the highest performance across all three settings. On ScienceWorld, it outperforms MemRL, the strongest baseline, by 9.39 score points. On ALFWorld, CWME also surpasses MemP and MemRL on Seen tasks and achieves a substantial advantage over CLIN on Unseen tasks. These results indicate that continually refining knowledge while explicitly validating and adapting historical experience offers advantages over knowledge accumulation, procedural memory evolution, and feedback-based memory utility estimation alone.