acceptodds
Under review as a conference paper at ICLR 2027

: Evaluating Continual Learning for Self-Evolving Embodied Agents

Abstract

Continual learning requires models to accumulate past experience and transfer it to new tasks. Despite substantial research on continual learning, existing work remains largely confined to non-embodied settings, without extending to embodied environments. To address this gap, we introduce , the first benchmark in embodied simulation for evaluating the continual learning capability of self-evolving agents. With compositional generalization as the core evaluation guidance, our benchmark evaluates whether agents can first learn from basic tasks and then re-combine the acquired skills to solve complex, unseen advanced tasks. Based on , we conduct extensive experiments across multiple agent harnesses and learning methods, revealing consistent limitations and distinctive behaviors. Our results show that while advanced tasks can be hardly solved under zero-shot setting, the experience from basic tasks enables consistent improvement across different harnesses. The benefit of experience remains stable as the task complexity increases. However, this improvement does not scale accordingly with the number of learning samples, and current agents cannot extract meaningful information from additional experience. Collectively, our benchmark provides a valuable resource for future research, and our findings highlight continual learning as a fundamental yet underexplored capability and offer insights for designing more effective self-evolving agents in embodied environments.

Primary area:
datasets and benchmarks
Keywords:
Continual Learning, Self-Evolving Embodied Agents, Compositional Generalization

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_jUgtOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces EmbodiedCLBench, a benchmark for evaluating continual learning in self-evolving embodied agents. Built on TongSim, the benchmark contains 168 basic tasks and 100 advanced tasks, where advanced tasks require composing skills from multiple basic-task categories. To enable offline evaluation, the authors construct state-transition graphs with visual observations and candidate actions. They evaluate four agent harnesses with two backbone models under zero-shot, in-context learning, and skill learning settings. The results show that experience from basic tasks substantially improves advanced-task success rates, but additional learning samples do not consistently yield further gains. The paper also analyzes compositional complexity, action-selection errors, and step-budget sensitivity.

Critical strengths

The experiments cover four harnesses, two backbone models, and two learning methods. Additional analyses of compositional complexity, learning sample scaling, action-selection errors, and step-budget sensitivity provide further insights into agent behavior.

Critical weaknesses
  • Limited novelty compared to prior embodied continual learning benchmarks. The benchmark mainly evaluates whether prior experience improves performance on unseen compositional tasks. Given prior embodied continual learning benchmarks (Wołczyk et al., 2021; Liu et al., 2023; Kim et al., 2024) study broader sequential learning settings, including knowledge retention and forgetting, the proposed basic-to-advanced task evaluation appears to be a restricted setting that potentially be built by reorganizing their tasks.
  • Weak justification of self-evolution. The paper considers reusing prior interaction histories or summarized skills to improve performance on subsequent tasks as self-evolution. However, prior work such as (Wang et al., 2023) demonstrates a more comprehensive form of self-evolution through autonomous exploration, an ever-growing skill library, and iterative skill refinement based on environmental feedback. In contrast, EmbodiedCLBench evaluates one-time experience transfer from predefined basic tasks to advanced tasks, without assessing autonomous skill acquisition or progressive refinement.

References
Wołczyk et al., "Continual World: A Robotic Benchmark For Continual Reinforcement Learning," NeurIPS 2021.
Kim et al., "Online Continual Learning for Interactive Instruction Following Agents," ICLR 2024.
Wang et al., "Voyager: An Open-Ended Embodied Agent with Large Language Models," NeurIPS 2023.

Questions for authors
  • What capabilities of continual learning and/or self-evolution does EmbodiedCLBench uniquely provide compared to prior embodied continual learning work? In addition, why is it nontrivial to extend such prior work to the proposed, basic-to-advanced, setup?
  • How does retaining interaction histories or generating skill summaries constitute self-evolution rather than conventional in-context learning or memory-augmented task solving? Could the benchmark evaluate iterative skill refinement or cumulative improvement over a longer sequence of tasks?
  • For the offline evaluation, to what extent do predefined action candidates and deterministic graph transitions reflect actual embodied decision-making? Limited scale of predefined transitions may not fully cover the diverse scenarios of the real-world applications.
  • Are basic and advanced tasks constructed using disjoint objects or scenes? Could the authors evaluate transfer to held-out environments or object configurations to distinguish reusable skill acquisition from familiarity with recurring task patterns?
12 comments
AuthorsOpenReview ↗

Overall Response to Reviewer jUgt

We sincerely thank the reviewer for the detailed comments and constructive questions. Below we briefly summarize our key responses; details for each point are in the follow-up comment replies.

  • W1 & Q1 (Novelty and unique capabilities). Our novelty is to evaluate continual learning through compositional generalization. Prior embodied continual learning benchmarks evaluate policies that learn through weight updates. In contrast, EmbodiedCLBench evaluates whether general-purpose agents can learn from their own experience and recombine it to solve unseen, harder tasks. This requires new tasks, a new action interface, and a new evaluation protocol, so it cannot be obtained by reorganizing prior tasks.
  • W2 & Q2 (Self-evolution and longer task sequences). ICL and Skill Learning are forms of self-evolution, and EmbodiedCLBench can evaluate cumulative improvement. In our new experiments on longer task sequences, success rate keeps improving with iterative accumulation.
  • Q3 (Offline evaluation). Our offline evaluation reflects actual embodied decision-making. It also brings three advantages over online evaluation: reproducibility, efficiency, and fine-grained analysis. With 268 tasks across 6 scenes and 51 object categories, it covers a broad range of household decisions.
  • Q4 (Held-out scenes and objects). When the learning tasks come from different scenes and involve different object categories, success rate stays nearly the same and remains far above zero shot. This indicates that agents acquire reusable skills rather than familiarity with recurring scenes or objects.

We have incorporated all these clarifications and supplementary results into the revised paper, highlighted in blue. All section, figure, and appendix numbers below refer to the revised paper.

AuthorsOpenReview ↗

Response to Reviewer jUgt (1/3): Novelty and Unique Capabilities

W1 & Q1: Novelty and unique capabilities

Our novelty is to evaluate continual learning through compositional generalization. Prior embodied continual learning benchmarks evaluate policies that learn through weight updates. In contrast, EmbodiedCLBench evaluates whether general-purpose agents can learn from their own experience and recombine it to solve unseen, harder tasks. This requires new tasks, a new action interface, and a new evaluation protocol, so it cannot be obtained by reorganizing prior tasks.

A different learner. Prior benchmarks evaluate continual learning algorithms that update policy weights. There, retention and forgetting are central, since new training can overwrite old knowledge. EmbodiedCLBench evaluates self-evolving agents, which keep their experience outside the model as interaction histories or skill files. The key question thus becomes whether agents can use this experience on unseen, harder tasks. Table R1 summarizes the differences.

Table R1. Comparison with prior embodied continual learning benchmarks.

BenchmarkEvaluated learnerLearning without weight updatesVisual observationsTest on unseen, harder tasksOffline evaluation
Continual World (Wołczyk et al., 2021)RL policy✗✗✗✗
LIBERO (Liu et al., 2023)Visuomotor policy✗✓✗✗
CL-ALFRED (Kim et al., 2024)Instruction-following policy✗✓✗✗
EmbodiedCLBench (ours)Self-evolving agents (4 harnesses)✓✓✓✓

Unique capabilities (Q1).

  1. Compositional generalization. Agents learn from basic tasks and are tested on unseen, harder advanced tasks with controlled complexity.
  2. Continual learning of agent harnesses. We evaluate how general-purpose agent harnesses (OpenClaw, Claude Code, Codex, and Hermes) learn from their own experience. Evaluating such agents in embodied simulation is new.
  3. Offline evaluation on state-transition graphs. Predefined state-transition graphs need no running simulator and label every decision as optimal, suboptimal, or invalid.

Why reorganizing prior tasks is not enough (Q1). Prior benchmarks use low-level control, such as continuous arm actions (Continual World, LIBERO) or low-level navigation and interaction (CL-ALFRED). Agent harnesses instead choose among language action candidates grounded in images. The prior task sets are also not designed to test whether learned skills recombine into unseen, harder tasks. We build this structure explicitly, from atomic actions to basic tasks to advanced tasks.

We have added this comparison to Section 2 and Appendix D.1.

AuthorsOpenReview ↗

Response to Reviewer jUgt (2/3): Self-Evolution and Longer Task Sequences

W2 & Q2: Self-evolution and longer task sequences

ICL and Skill Learning are forms of self-evolution, and EmbodiedCLBench can evaluate cumulative improvement. In our new experiments on longer task sequences, success rate (SR) keeps improving with iterative accumulation.

Self-evolution. Recent surveys define a self-evolving agent as one that improves from its own experience without human supervision (Gao et al., 2025; Fang et al., 2025). Our agents learn from their own trajectories, including failed ones, rather than from curated demonstrations. In Skill Learning, they also write their own skills. Unlike Voyager, which lets the agent choose its tasks, EmbodiedCLBench fixes the learning tasks so that all agents can be compared under the same experience.

Setting. As requested by the reviewer, we evaluate longer task sequences with two ways of accumulating experience.

  • One-time accumulation (Table 1 and Section 5.2): the agent learns from each learning task separately, without the experience from other tasks.
  • Iterative accumulation (new): the agent learns from the learning tasks one after another and applies earlier experience to each new one.
  • Learning samples: basic tasks, or other advanced tasks (excluding the test task). With advanced tasks, iterative accumulation forms a chain of 6 distinct tasks. Both modes use the same learning tasks in the same order. At test time, the agent receives all k interaction histories (ICL) or all k skills (Skill Learning).
  • Evaluation: the same 50 advanced tasks as in Section 5.2, with Hermes and OpenClaw (Qwen3.8-Flash), each under ICL and Skill Learning.

Table R2. Average SR (%) on advanced tasks over the four settings. k is the number of learning samples, and k=0 is zero shot.

Experience fromAccumulationk=0k=1k=2k=3k=4k=5k=6
Basic tasksOne-time9.034.039.529.538.034.030.0
Basic tasksIterative9.033.049.050.550.053.554.0
Advanced tasksOne-time9.031.539.036.539.040.541.5
Advanced tasksIterative9.031.537.040.047.553.5–

Results. With experience from basic tasks, iterative accumulation exceeds one-time accumulation by 17 points on average for k≥2. With experience from advanced tasks, it improves at every step and leads by 13 points at k=5. One-time accumulation gains little beyond 1–2 samples, consistent with Finding 3.

The advanced chain directly evaluates progressive self-evolution. The agent learns only from its own attempts on previous advanced tasks. Under Skill Learning, its skill set grows after every task, and each new skill builds on all previous ones.

We have added these results to Section 5.3 and Appendix D.10, and the discussion of self-evolution to Appendix D.1.

Reviewer_QfhROpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents EmbodiedCLBench, a benchmark for evaluating continual learning capability of self-evolving agents in embodied simulation (TongSim), through whether agents can perform compositional generalization and learn skills from basic tasks that can be re-combined to solve more complex, unseen tasks. Experiments over different agentic harnesses and evaluation schemes (zero-shot, in-context learning, skill learning) show the benefit of experience from basic tasks with improvement to solve advanced tasks. However, the benchmark still offers a test-bed to further evaluate self-evolving agents as performance is far from saturation on advanced tasks and also plateaus beyond a single learning sample.

Critical strengths
  • The benchmark appears to be reasonably well-structured and accessible. The fully offline reproducible evaluation scheme using state-transition graphs seems to be more scalable than running a simulator online. Evaluating optimal actions with shortest path search over the graph structure and suboptimal actions that deviate from the shortest path seems to work reasonably in practice (from experiments in Section 5.3 and 5.4). The main results in Table 1 also show that EmbodiedCLBench demonstrates that agents benefit from experience on basic tasks to solve advanced tasks (zero-shot vs. in-context learning or skill learning columns).
Critical weaknesses
  • The only evaluated metric is success rate. There have been a variety of other continual learning metrics proposed to evaluate catastrophic forgetting and knowledge transfer (https://arxiv.org/abs/1810.13166). Furthermore, the agent is only given two basic tasks as experience and evaluated on one additional advanced task in the non zero-shot settings. This seems more like compositional transfer rather than a broader continual learning capability? For instance, what happens when scaling the number of basic tasks used for experience, or the number of advanced tasks used for evaluation?

  • The authors claim that EmbodiedCLBench is the first benchmark to evaluate continual learning capability of self-evolving agents. This claim should be narrowed, considering the prior work of LIBERO (Liu et al., 2023), OMNI (https://arxiv.org/abs/2306.01711, Zhang et al., 2024), and OMNI-EPIC (https://arxiv.org/abs/2405.15568, Faldor et al., 2025). The authors should also consider discussing Huang et al., 2022 (https://arxiv.org/abs/2201.07207) and VirtualHome (https://arxiv.org/abs/1806.07011, Puig et al., 2018) which appear to use a task construction and evaluation scheme.

Questions for authors
  • How well would agents perform without image observations? It seems like a VLM could be used to produce text descriptions of the agent state from the given image observations, such that the benchmark would have a “text-only” evaluation setting.

  • Section 5.2: Is there any additional understanding of why additional basic task learning samples do not improve success rates? To me this seems that there is not sufficient diversity in the learning samples, which may result from the offline state-transition graph evaluation protocol.

  • L272: How are the two related basic tasks selected for each advanced task?

6 comments
AuthorsOpenReview ↗

Overall Response to Reviewer QfhR

We sincerely thank the reviewer for the careful reading and constructive suggestions. Below we briefly summarize our key responses; details for each point are in the follow-up comment replies.

  • W1a (Metrics beyond success rate). Success rate directly measures continual learning in our setting. Our analyses further evaluate agents along four dimensions, including forward transfer.
  • W1b & Q2 (Scaling the learning samples). We add experiments that scale both basic and advanced learning samples. They reveal a new finding: how experience is accumulated is important. With iterative accumulation, success rate keeps improving with more learning samples.
  • W2 (Novelty). Our novelty is to evaluate continual learning through compositional generalization. The suggested works train policies, generate tasks for RL training, or evaluate planning on individual tasks. In contrast, EmbodiedCLBench evaluates whether general-purpose agents can learn from their own experience and recombine it to solve unseen, harder tasks. We have toned down the "first" claim.
  • Q1 (Text-only setting). Visual observations matter in EmbodiedCLBench. In the added text-only settings, replacing images with captions lowers success rate in all cases, and removing them lowers it further.
  • Q3 (Selection of basic tasks). For each advanced task, we select the two most related basic tasks by three criteria: shared capability categories, coverage of its atomic action types, and no repeated atomic actions. These criteria make the agent transfer skills rather than replay them.

We have incorporated all these clarifications and supplementary results into the revised paper, highlighted in blue. All section, figure, and appendix numbers below refer to the revised paper.

AuthorsOpenReview ↗

Response to Reviewer QfhR (1/2): Metrics and Scaling the Learning Samples

W1a: Metrics beyond success rate

Success rate (SR) directly measures continual learning in our setting. Our analyses further evaluate agents along four dimensions, including forward transfer.

SR measures whether accumulated experience enables the agent to solve unseen, harder tasks. This is the core of continual learning in our setting. Our analyses add four dimensions:

  • Gain over zero shot (Section 5.1): the SR of ICL or Skill Learning minus the zero-shot SR. It measures forward transfer.
  • Scaling with learning samples (Section 5.2): how SR changes with more learning samples.
  • Error analysis (Section 5.4): how often the agent selects optimal, suboptimal, and invalid actions.
  • Step budget sensitivity (Section 5.5): how SR changes with extra steps beyond the shortest path.

We present them as trends in figures, which carry more information than single numbers.

Forgetting. Forgetting metrics, such as backward transfer in the cited work, are central when the learner updates its weights. Our agents learn without weight updates and keep past experience as interaction histories or skill files. We thus focus on forward transfer.

We have added this discussion to Appendix D.2.


W1b & Q2: Scaling the learning samples

As requested by the reviewer, we scale both basic and advanced learning samples. Our new experiments reveal that how experience is accumulated is important: with iterative accumulation, SR keeps improving with more learning samples.

Setting.

  • One-time accumulation (Table 1 and Section 5.2): the agent learns from each learning task separately, without the experience from other tasks.
  • Iterative accumulation (new): the agent learns from the learning tasks one after another and applies earlier experience to each new one.
  • Learning samples: basic tasks, or other advanced tasks (excluding the test task). With advanced tasks, iterative accumulation forms a chain of 6 distinct tasks. Both modes use the same learning tasks in the same order. At test time, the agent receives all k interaction histories (ICL) or all k skills (Skill Learning).
  • Evaluation: the same 50 advanced tasks as in Section 5.2, with Hermes and OpenClaw (Qwen3.8-Flash), each under ICL and Skill Learning.

Table R1. Average SR (%) on advanced tasks over the four settings. k is the number of learning samples, and k=0 is zero shot.

Experience fromAccumulationk=0k=1k=2k=3k=4k=5k=6
Basic tasksOne-time9.034.039.529.538.034.030.0
Basic tasksIterative9.033.049.050.550.053.554.0
Advanced tasksOne-time9.031.539.036.539.040.541.5
Advanced tasksIterative9.031.537.040.047.553.5–

Results. With experience from basic tasks, iterative accumulation exceeds one-time accumulation by 17 points on average for k≥2. With experience from advanced tasks, it improves at every step and leads by 13 points at k=5. One-time accumulation gains little beyond 1–2 samples, consistent with Finding 3.

  • Scaling (W1b). EmbodiedCLBench supports scaling both the learning samples and the number of advanced tasks in a sequence. Compositional transfer is how we measure continual learning (Section 3.2), and the new chain extends it from one test task to a task sequence.
  • Why more samples did not help (Q2). The results indicate that the cause lies in how experience is accumulated, not in the diversity of learning samples or the offline protocol. With iterative accumulation, the same basic tasks on the same offline graphs keep improving SR. We conjecture that trajectories under one-time accumulation do not build on each other, so the agent cannot make good use of the extra information in additional samples.

We have added these results to Section 5.3 (Figure 6) and Appendix D.10.

AuthorsOpenReview ↗

Response to Reviewer QfhR (2/2): Novelty, Text-Only Setting, and Task Selection

W2: Novelty of EmbodiedCLBench

Our novelty is to evaluate continual learning through compositional generalization. The suggested works train policies, generate tasks for RL training, or evaluate planning on individual tasks. In contrast, EmbodiedCLBench evaluates whether general-purpose agents can learn from their own experience and recombine it to solve unseen, harder tasks.

We have toned down the "first" claim. The revised paper describes EmbodiedCLBench as a new benchmark (abstract, Section 1, and Section 6).

Comparison with the suggested works.

  • LIBERO evaluates visuomotor policies that learn each task by behavior cloning with weight updates. It does not evaluate agents that learn from their own experience without training.
  • OMNI and OMNI-EPIC generate tasks for training RL agents. They are methods for open-ended learning, not benchmarks for evaluating agents.
  • VirtualHome and Huang et al. (2022) build household tasks from atomic actions, similar to our task construction. However, they evaluate planning on individual tasks, not learning from experience across tasks.

Three capabilities. EmbodiedCLBench provides three capabilities that prior work does not offer together:

  1. Compositional generalization. Agents learn from basic tasks and are tested on unseen, harder advanced tasks with controlled complexity.
  2. Continual learning of agent harnesses. We evaluate how general-purpose agent harnesses (OpenClaw, Claude Code, Codex, and Hermes) learn from their own experience. Evaluating such agents in embodied simulation is new.
  3. Offline evaluation on state-transition graphs. Predefined state-transition graphs need no running simulator and label every decision as optimal, suboptimal, or invalid.

We have added this comparison to Section 2 and Appendix D.1.


Q1: Text-only setting

Visual observations matter in EmbodiedCLBench. In an embodied environment, the image shows the agent its current state (e.g., whether it is near the cabinet and whether the cabinet is open). The agent needs this information to decide the next action.

As requested by the reviewer, we add two text-only settings. Our new results show that replacing images with captions lowers SR in all cases, and removing them lowers it further. Specifically, Qwen3.8-Flash writes a caption for each state image, and the caption replaces the image (Caption). We also test without any observation (No Image). Other settings follow Table 1.

Table R2. SR (%) of Hermes with Qwen3.8-Flash.

ObservationBasic (zero shot)Advanced (zero shot)Advanced (ICL)Advanced (Skill Learning)
Image (Table 1)58.98.044.033.0
Caption54.85.037.028.0
No Image42.93.030.022.0

For example, ICL on advanced tasks drops from 44.0% to 37.0% with captions and to 30.0% without observations. These results validate that agents rely on visual observations in EmbodiedCLBench. We have added this setting to Appendix D.8.


Q3: Selection of the two related basic tasks (L272)

For each advanced task, we select the two most related basic tasks by three criteria: shared capability categories, coverage of its atomic action types, and no repeated atomic actions. These criteria make the agent transfer skills rather than replay them.

  1. Shared capability categories. The basic tasks come from the capability categories that the advanced task combines.
  2. Coverage of atomic action types. Together, the two basic tasks cover as many atomic action types of the advanced task (e.g., open, pick up) as possible.
  3. No repeated atomic actions. The basic tasks avoid the exact atomic actions of the advanced task (the same action on the same object in the same scene).

We have added these criteria to Appendix D.4.


If our responses have addressed your concerns, we would appreciate it if you could consider raising your score.

Reviewer_rtvPOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

EmbodiedCLBench tests whether LLM agents reuse experience in TongSim household tasks: 168 basic tasks (about 2 atomic actions each) and 100 advanced tasks (about 4). Tasks are offline state-transition graphs with images. At each step the agent picks among 4 options: 1 shortest-path action, with the rest drawn from detours and/or invalid actions. After 2 basic tasks, advanced-task SR rises from about 8% to about 30–35% (ICL or skill summary) across 4 harnesses × 2 backbones. Gains saturate at k=1 and come mostly from fewer invalid actions.

Critical strengths
  1. The offline graph makes evaluation deterministic and cheap, and per-step logging enables the informative Fig. 6 breakdown.
  2. The anti-shortcut rewriting pipeline (App. B) and the 3-annotator filtering address text-only cues.
Critical weaknesses
  1. Compositional transfer vs. protocol familiarity. Every action follows look_at → walk_to → operate (Table 3, State 0). App. B Stage 3 hides this rhythm only in the wording. The paper’s own evidence fits a simpler explanation, that agents mainly learn this precondition protocol: The gain is mostly fewer invalid actions (−13.3 to −19.8 pp, Fig. 6). A single basic task gives most of the gain (Fig. 5). The gain stays flat up to 6-action targets (Fig. 4). Needed controls: learning tasks unrelated to the target, and a text-only description of the rule.
  2. Overstated “first.” CL-ALFRED (ICLR 2024), LENL (ICLR 2026), LLCRM, and LIBERO already exist. EmbodiSkill has evaluated training-free skill evolution on ALFWorld and EmbodiedBench. The “embodiment” here is a 4-way choice over symbolic actions. The claim should be qualified.
  3. Statistics and consistency. Results are single runs with no CIs (SE ≈ 4.6 pp), so 1–6 pp gaps are within noise. App. B defines tasks as 3/6 L1 goals vs. 2/4 in the main text. The “>30%” claim fails for 5 of 16 configurations. Sec. 5.1 reports 32.0%, while Fig. 4 shows about 28%.
Questions for authors
  1. What is SR under the two controls above?
  2. How much of each advanced task’s atomic actions do the learning tasks cover, especially at k=1?
  3. What happens after an invalid choice, and what is SR during the learning phase?
13 comments
AuthorsOpenReview ↗

Overall Response to Reviewer rtvP

We sincerely thank the reviewer for the positive assessment and constructive suggestions. Below we briefly summarize our key responses; details for each point are in the follow-up comment replies.

  • W1 & Q1 (Compositional transfer vs. protocol familiarity). Agents learn more than the precondition protocol. With the same protocol, related basic tasks outperform both unrelated basic tasks and the text-only rule. Success rate also keeps rising when agents accumulate experience iteratively from more related tasks.
  • W2 (Novelty). Our novelty is to evaluate continual learning through compositional generalization. The suggested works train policies through weight updates or evaluate a method on existing benchmarks. In contrast, EmbodiedCLBench evaluates whether general-purpose agents can learn from their own experience and recombine it to solve unseen, harder tasks. We have toned down the "first" claim.
  • W3 (Statistics and consistency). Our main findings hold under paired statistical tests. We have also corrected the three consistency points.
  • Q2 (Coverage of atomic actions). By design, the learning tasks provide the types of skills that each advanced task needs. Their coverage of its atomic action types grows from 50% at k=1 to 86% at k=2 and 100% at k≥3.
  • Q3 (Invalid choices and learning-phase SR). An invalid choice consumes one step without changing the state. Most learning trajectories succeed (learning-phase success rate: 58.9%–84.9%).

We have incorporated all these clarifications and supplementary results into the revised paper, highlighted in blue. All section, figure, and appendix numbers below refer to the revised paper.

AuthorsOpenReview ↗

Response to Reviewer rtvP (1/3): Compositional Transfer vs. Protocol Familiarity

W1 & Q1: Compositional transfer vs. protocol familiarity

Agents learn more than the precondition protocol. With the same protocol, related basic tasks outperform both unrelated basic tasks and the text-only rule. Success rate (SR) also keeps rising when agents accumulate experience iteratively from more related tasks.

Controls (preliminary evidence). As requested by the reviewer, we add both controls. Our new results show that related basic tasks outperform both controls under the same protocol. Specifically, we follow Table 1 with Hermes and Qwen3.8-Flash, as in Sections 5.1 and 5.5, and change only the learning phase.

  • Unrelated tasks: 2 basic tasks that share no capability category and no object with the advanced task. They follow the same look_at → walk_to → operate protocol.
  • Text-only rule: no learning phase. At test time, the agent receives a text description of this protocol.

Table R1. SR (%) on advanced tasks. Without a learning phase, ICL and Skill Learning coincide.

Learning phaseICLSkill Learning
None (zero shot, Table 1)8.08.0
Text-only rule30.030.0
2 unrelated basic tasks35.027.0
2 related basic tasks (Table 1)44.033.0

With the same protocol, related basic tasks outperform unrelated ones (44.0% vs. 35.0% with ICL, 33.0% vs. 27.0% with Skill Learning). They also outperform the text-only rule (30.0%).

More related tasks (further evidence). We also scale the number of related basic tasks (k) on the 50 advanced tasks of Section 5.2 (a supplementary experiment requested by Reviewers QfhR and jUgt; details are in our responses to those reviewers). In the original setting, the agent learns from each basic task separately (one-time accumulation). We add iterative accumulation, where the agent learns from the basic tasks one after another and applies earlier experience to each new one.

Table R2. Average SR (%) on advanced tasks, with experience from basic tasks. Results are averaged over Hermes and OpenClaw (Qwen3.8-Flash), each under ICL and Skill Learning.

Accumulationk=0k=1k=2k=3k=4k=5k=6
One-time (original)9.034.039.529.538.034.030.0
Iterative (new)9.033.049.050.550.053.554.0

One basic task already shows the full protocol. Yet with iterative accumulation, SR keeps rising from 33.0% at k=1 to 54.0% at k=6, a further gain of 21.0 points. This gain beyond the first task cannot be explained by protocol familiarity. This indicates that agents keep learning task knowledge from additional related tasks.

Analysis. Protocol familiarity is part of what agents learn: the text-only rule alone raises SR from 8.0% to 30.0%. In ICL and Skill Learning, agents discover this precondition knowledge from their own experience. The remaining gain comes from the compositional structure of our tasks. Each advanced task combines 2–4 of the six capability categories (Figure 3), and related basic tasks come from these categories. They show how to complete each category (e.g., picking up an object before placing it in a container), and agents reuse this knowledge in the advanced task. These results validate that agents learn more than the precondition protocol.

We have added the controls and this analysis to Appendix D.7, and the accumulation results to Section 5.3.

AuthorsOpenReview ↗

Response to Reviewer rtvP (2/3): Novelty

W2: Novelty of EmbodiedCLBench

Our novelty is to evaluate continual learning through compositional generalization. The suggested works train policies through weight updates or evaluate a method on existing benchmarks. In contrast, EmbodiedCLBench evaluates whether general-purpose agents can learn from their own experience and recombine it to solve unseen, harder tasks.

We have toned down the "first" claim. The revised paper describes EmbodiedCLBench as a new benchmark (abstract, Section 1, and Section 6).

Comparison with the suggested works.

  • CL-ALFRED trains instruction-following policies online, with weight updates.
  • LENL and LLCRM train navigation and manipulation policies over task sequences with weight updates, and focus on forgetting.
  • LIBERO trains visuomotor policies by behavior cloning.
  • EmbodiSkill evolves skills without weight updates. However, it is a method evaluated on existing benchmarks (ALFWorld and EmbodiedBench), which are not designed to test transfer from simpler tasks to unseen, harder ones.

Three capabilities. EmbodiedCLBench provides three capabilities that prior work does not offer together:

  1. Compositional generalization. Agents learn from basic tasks and are tested on unseen, harder advanced tasks with controlled complexity.
  2. Continual learning of agent harnesses. We evaluate how general-purpose agent harnesses (OpenClaw, Claude Code, Codex, and Hermes) learn from their own experience. Evaluating such agents in embodied simulation is new.
  3. Offline evaluation on state-transition graphs. Predefined state-transition graphs need no running simulator and label every decision as optimal, suboptimal, or invalid.

Embodiment. The embodiment of EmbodiedCLBench lies in its tasks and scenes. Agents act in simulated household scenes, observe first-person images, and must approach objects and change their states (e.g., open a cabinet or switch off a lamp) to reach a goal. The multiple-choice format is one form of acting in these scenes. We adopt it for three advantages. First, it focuses the evaluation on high-level decisions, separate from low-level control. Second, all agents face the same choices at each state, so results are reproducible and comparable. Third, every choice is labeled as optimal, suboptimal, or invalid, which enables fine-grained error analysis.

We have added this comparison to Section 2 and Appendix D.1, and the discussion of embodiment to Appendix D.3.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Rebuttal

Dear ACs and Reviewers,

We sincerely thank you and all the reviewers for your time and constructive feedback on our submission.

We are encouraged that, following the rebuttal, Reviewer QfhR raised the rating to a positive score, noting that our responses resolved most of the concerns, and Reviewer rtvP maintained the positive rating after several rounds of in-depth discussion. We also appreciate the reviewers' recognition of our offline evaluation on branching state-transition graphs, which is deterministic, scalable, and interactive (rtvP, QfhR, jUgt), our anti-shortcut task construction with human verification (rtvP), and our comprehensive evaluation and analyses, which provide insights into agent behavior and the benefits of accumulated experience (rtvP, QfhR, jUgt).

During the rebuttal, we conducted extensive additional experiments and analyses and revised the paper in the following areas:

  • Scaling to longer task sequences (QfhR, jUgt). We extended the evaluation to longer learning sequences of up to six tasks, which reveals a new finding: how experience is accumulated is key to continual learning. When agents learn from tasks one after another and build on earlier experience, success rate keeps rising from 33.0% to 54.0%, whereas learning from each task in isolation quickly plateaus (Section 5.3).
  • Learning task-specific skills from experience (rtvP, jUgt). We added control experiments showing that agents learn task-specific skills from related tasks, beyond the generic action routine shared by all tasks. With in-context learning, agents that learn from related tasks reach a 44.0% success rate, significantly higher than those learning from unrelated tasks (35.0%) or given only a text description of the routine (30.0%; Appendices D.7 and D.11).
  • Generalization to unseen scenes and objects (jUgt). We added a held-out evaluation in which learning and test tasks share no scenes or object categories, showing that agents acquire skills that transfer across environments. Success rate stays nearly unchanged (42.0% vs. 44.0% with in-context learning) and remains far above zero-shot performance (8.0%; Appendix D.9).
  • Importance of visual observations (QfhR). We added text-only variants of the benchmark, confirming that solving EmbodiedCLBench requires grounding decisions in visual observations. Replacing images with captions or removing them consistently lowers success rate, e.g., from 44.0% to 37.0% and 30.0% with in-context learning (Appendix D.8).
  • Statistical robustness of the main results (rtvP). We added paired significance tests and confidence intervals, confirming that the gains from experience are statistically robust. Under exact McNemar tests, the gain over zero-shot performance is significant in 15 of 16 agent configurations (13 with p < 0.001), and we now report bootstrap 95% CIs for the main results (Appendices D.6 and D.12).
  • Unique contributions relative to prior work (rtvP, QfhR, jUgt). We added a systematic comparison with prior embodied continual learning benchmarks, highlighting the combination of capabilities that EmbodiedCLBench uniquely offers. Unlike prior benchmarks that train policies through weight updates, EmbodiedCLBench evaluates whether general-purpose agents learn from their own experience to solve unseen, harder tasks, with fully offline and reproducible evaluation (Section 2, Appendix D.1).
  • Additional details on benchmark design (rtvP, QfhR, jUgt). We added further details on the benchmark design, showing how it enables reproducible and fine-grained evaluation of embodied decision-making. These include how the offline graphs capture real observations and action outcomes, how learning tasks are selected to require recombining skills rather than replaying them, and how our analyses go beyond success rate (Appendices D.2–D.5).

We believe these revisions and additional experiments have significantly improved our paper. We sincerely appreciate your time and effort in coordinating the review process.

Best regards,

EmbodiedCLBench Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.