- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper introduces EmbodiedCLBench, a benchmark for evaluating continual learning in self-evolving embodied agents. Built on TongSim, the benchmark contains 168 basic tasks and 100 advanced tasks, where advanced tasks require composing skills from multiple basic-task categories. To enable offline evaluation, the authors construct state-transition graphs with visual observations and candidate actions. They evaluate four agent harnesses with two backbone models under zero-shot, in-context learning, and skill learning settings. The results show that experience from basic tasks substantially improves advanced-task success rates, but additional learning samples do not consistently yield further gains. The paper also analyzes compositional complexity, action-selection errors, and step-budget sensitivity.
Critical strengths
The experiments cover four harnesses, two backbone models, and two learning methods. Additional analyses of compositional complexity, learning sample scaling, action-selection errors, and step-budget sensitivity provide further insights into agent behavior.
Critical weaknesses
- Limited novelty compared to prior embodied continual learning benchmarks. The benchmark mainly evaluates whether prior experience improves performance on unseen compositional tasks. Given prior embodied continual learning benchmarks (Wołczyk et al., 2021; Liu et al., 2023; Kim et al., 2024) study broader sequential learning settings, including knowledge retention and forgetting, the proposed basic-to-advanced task evaluation appears to be a restricted setting that potentially be built by reorganizing their tasks.
- Weak justification of self-evolution. The paper considers reusing prior interaction histories or summarized skills to improve performance on subsequent tasks as self-evolution. However, prior work such as (Wang et al., 2023) demonstrates a more comprehensive form of self-evolution through autonomous exploration, an ever-growing skill library, and iterative skill refinement based on environmental feedback. In contrast, EmbodiedCLBench evaluates one-time experience transfer from predefined basic tasks to advanced tasks, without assessing autonomous skill acquisition or progressive refinement.
References
Wołczyk et al., "Continual World: A Robotic Benchmark For Continual Reinforcement Learning," NeurIPS 2021.
Kim et al., "Online Continual Learning for Interactive Instruction Following Agents," ICLR 2024.
Wang et al., "Voyager: An Open-Ended Embodied Agent with Large Language Models," NeurIPS 2023.
Questions for authors
- What capabilities of continual learning and/or self-evolution does EmbodiedCLBench uniquely provide compared to prior embodied continual learning work? In addition, why is it nontrivial to extend such prior work to the proposed, basic-to-advanced, setup?
- How does retaining interaction histories or generating skill summaries constitute self-evolution rather than conventional in-context learning or memory-augmented task solving? Could the benchmark evaluate iterative skill refinement or cumulative improvement over a longer sequence of tasks?
- For the offline evaluation, to what extent do predefined action candidates and deterministic graph transitions reflect actual embodied decision-making? Limited scale of predefined transitions may not fully cover the diverse scenarios of the real-world applications.
- Are basic and advanced tasks constructed using disjoint objects or scenes? Could the authors evaluate transfer to held-out environments or object configurations to distinguish reusable skill acquisition from familiarity with recurring task patterns?
12 comments
Response to Reviewer jUgt (1/3): Novelty and Unique Capabilities
W1 & Q1: Novelty and unique capabilities
Our novelty is to evaluate continual learning through compositional generalization. Prior embodied continual learning benchmarks evaluate policies that learn through weight updates. In contrast, EmbodiedCLBench evaluates whether general-purpose agents can learn from their own experience and recombine it to solve unseen, harder tasks. This requires new tasks, a new action interface, and a new evaluation protocol, so it cannot be obtained by reorganizing prior tasks.
A different learner. Prior benchmarks evaluate continual learning algorithms that update policy weights. There, retention and forgetting are central, since new training can overwrite old knowledge. EmbodiedCLBench evaluates self-evolving agents, which keep their experience outside the model as interaction histories or skill files. The key question thus becomes whether agents can use this experience on unseen, harder tasks. Table R1 summarizes the differences.
Table R1. Comparison with prior embodied continual learning benchmarks.
| Benchmark | Evaluated learner | Learning without weight updates | Visual observations | Test on unseen, harder tasks | Offline evaluation |
|---|---|---|---|---|---|
| Continual World (Wołczyk et al., 2021) | RL policy | ✗ | ✗ | ✗ | ✗ |
| LIBERO (Liu et al., 2023) | Visuomotor policy | ✗ | ✓ | ✗ | ✗ |
| CL-ALFRED (Kim et al., 2024) | Instruction-following policy | ✗ | ✓ | ✗ | ✗ |
| EmbodiedCLBench (ours) | Self-evolving agents (4 harnesses) | ✓ | ✓ | ✓ | ✓ |
Unique capabilities (Q1).
- Compositional generalization. Agents learn from basic tasks and are tested on unseen, harder advanced tasks with controlled complexity.
- Continual learning of agent harnesses. We evaluate how general-purpose agent harnesses (OpenClaw, Claude Code, Codex, and Hermes) learn from their own experience. Evaluating such agents in embodied simulation is new.
- Offline evaluation on state-transition graphs. Predefined state-transition graphs need no running simulator and label every decision as optimal, suboptimal, or invalid.
Why reorganizing prior tasks is not enough (Q1). Prior benchmarks use low-level control, such as continuous arm actions (Continual World, LIBERO) or low-level navigation and interaction (CL-ALFRED). Agent harnesses instead choose among language action candidates grounded in images. The prior task sets are also not designed to test whether learned skills recombine into unseen, harder tasks. We build this structure explicitly, from atomic actions to basic tasks to advanced tasks.
We have added this comparison to Section 2 and Appendix D.1.
Response to Reviewer jUgt (2/3): Self-Evolution and Longer Task Sequences
W2 & Q2: Self-evolution and longer task sequences
ICL and Skill Learning are forms of self-evolution, and EmbodiedCLBench can evaluate cumulative improvement. In our new experiments on longer task sequences, success rate (SR) keeps improving with iterative accumulation.
Self-evolution. Recent surveys define a self-evolving agent as one that improves from its own experience without human supervision (Gao et al., 2025; Fang et al., 2025). Our agents learn from their own trajectories, including failed ones, rather than from curated demonstrations. In Skill Learning, they also write their own skills. Unlike Voyager, which lets the agent choose its tasks, EmbodiedCLBench fixes the learning tasks so that all agents can be compared under the same experience.
Setting. As requested by the reviewer, we evaluate longer task sequences with two ways of accumulating experience.
- One-time accumulation (Table 1 and Section 5.2): the agent learns from each learning task separately, without the experience from other tasks.
- Iterative accumulation (new): the agent learns from the learning tasks one after another and applies earlier experience to each new one.
- Learning samples: basic tasks, or other advanced tasks (excluding the test task). With advanced tasks, iterative accumulation forms a chain of 6 distinct tasks. Both modes use the same learning tasks in the same order. At test time, the agent receives all k interaction histories (ICL) or all k skills (Skill Learning).
- Evaluation: the same 50 advanced tasks as in Section 5.2, with Hermes and OpenClaw (Qwen3.8-Flash), each under ICL and Skill Learning.
Table R2. Average SR (%) on advanced tasks over the four settings. k is the number of learning samples, and k=0 is zero shot.
| Experience from | Accumulation | k=0 | k=1 | k=2 | k=3 | k=4 | k=5 | k=6 |
|---|---|---|---|---|---|---|---|---|
| Basic tasks | One-time | 9.0 | 34.0 | 39.5 | 29.5 | 38.0 | 34.0 | 30.0 |
| Basic tasks | Iterative | 9.0 | 33.0 | 49.0 | 50.5 | 50.0 | 53.5 | 54.0 |
| Advanced tasks | One-time | 9.0 | 31.5 | 39.0 | 36.5 | 39.0 | 40.5 | 41.5 |
| Advanced tasks | Iterative | 9.0 | 31.5 | 37.0 | 40.0 | 47.5 | 53.5 | – |
Results. With experience from basic tasks, iterative accumulation exceeds one-time accumulation by 17 points on average for k≥2. With experience from advanced tasks, it improves at every step and leads by 13 points at k=5. One-time accumulation gains little beyond 1–2 samples, consistent with Finding 3.
The advanced chain directly evaluates progressive self-evolution. The agent learns only from its own attempts on previous advanced tasks. Under Skill Learning, its skill set grows after every task, and each new skill builds on all previous ones.
We have added these results to Section 5.3 and Appendix D.10, and the discussion of self-evolution to Appendix D.1.
Overall Response to Reviewer jUgt
We sincerely thank the reviewer for the detailed comments and constructive questions. Below we briefly summarize our key responses; details for each point are in the follow-up comment replies.
We have incorporated all these clarifications and supplementary results into the revised paper, highlighted in blue. All section, figure, and appendix numbers below refer to the revised paper.