acceptodds
Withdrawn from ICLR 2027

World Action Models Enable Continual Imitation Learning with Generative Replays

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Going beyond predicting robot actions, World Action Models (WAMs) can also generate future visual observations. We build on this generative capability to propose Recurrent Generative Replay (REGEN), a continual imitation learning framework that synthesizes pseudo-replay trajectories, enabling a robot policy to rehearse previously learned tasks without storing their original human demonstrations.During continual adaptation, REGEN recursively queries the WAM to synthesize pseudo-replay trajectories conditioned only on prior task instructions and current-task observations. Experiments in both simulation and real-world manipulation settings show that REGEN reduces catastrophic forgetting by up to 50% relative to sequential fine-tuning, while approaching the performance of privileged experience replay methods that require access to real replay data. Finally, we analyze the factors limiting generated replay, identifying long-horizon visual degradation and action-observation inconsistency as the primary bottlenecks. Our results establish WAMs as a promising foundation for continual robot learning without stored demonstrations.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Continual Learning, World Action Models, Robot Manipulation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_jLbNOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies continual imitation learning, where a robot is sequentially trained on new tasks while (optimally) retaining performance on previously learned tasks. In line with continual learning litreatures, it assumes that demonstrations from previous tasks are unavailable during adaptation, although storing them is possible in other continual-learning methods. ReGen uses a World Action Model to generate pseudo-demonstrations for previous tasks, predicting, not accessing future visual observations recursively. These synthetic trajectories are replayed with new-task demonstrations to reduce forgetting. The method improves retention over sequential fine-tuning, but generated observations can degrade and become inconsistent with the predicted actions.

Critical strengths

The RAR comparison and the gap between imagined and executed success clearly expose the limitations of generated replay, particularly visual degradation and action–observation inconsistency. I found this interesting, as I have not seen it in SOTA before.

Critical weaknesses

Soundness option: I have selected the option "I have minor or fixable concerns about the paper’s soundness." as I have only minor or fixable concerns regarding the paper’s technical soundness. However, the main weaknesses are not primarily about correctness: the methodological novelty is limited, and the experimental findings are largely unsurprising given prior work on generative replay and continual imitation learning. The paper does not convincingly establish a substantial advance over existing methods.

--- Weaknesses ---

The paper addresses a relevant problem, but its proposed solution is largely a straightforward application of recursive WAM generation to replay, and the experiments do not establish sufficient novelty or superiority over existing continual imitation-learning methods. Details are below:

W1) The central idea is not fundamentally new. Generative replay of previous robot-imitation trajectories has already been explored, e.g., in CRIL, which generated states and actions for replay using a generative model and an action-conditioned video predictor. ReGen mainly transfers this idea to a WAM backbone.

W2) The claimed novelty is therefore overstated. The paper should clearly distinguish the WAM-specific implementation from the broader, already established idea of generative replay for continual robot imitation learning.

W3) The experimental comparison is incomplete and outdated. Important continual-imitation baselines, including CRIL and more recent methods such as LOTUS, ISCIL, M2Distill, and MLR-IFA, are not evaluated. These papers report substantially broader comparisons on LIBERO-Object/Goal/50/100/LONG.

W4) The experimental setting is relatively narrow: six base tasks followed by only four continual stages in simulation, and three highly similar pick-and-place tasks on the real robot. The paper does not demonstrate robustness on longer or more diverse task sequences, e.g., comparable to LIBERO-50.

W5) The baselines (not SOTA, which is completely discarded as mentioned above) are not sufficiently strong to support the paper’s broad claims. The comparison is dominated by sequential fine-tuning, LoRA, EWC, and PackNet, while modern task-agnostic replay, distillation, latent-replay, and skill-based methods are absent.

W6) The method relies on strong and incompletely acknowledged scene assumptions. ReGen is initialized from observations containing the objects required by previous tasks; when these objects are absent, the method uses prior-task object metadata and simulator-rendered observations. This weakens the claim that replay requires no previous environment information.

W7) The paper does not isolate whether the gains come from ReGen itself or from the additional WAM architecture and auxiliary future-observation/reward objectives. A matched action-only policy and matched-capacity controls are needed. It is not clear whether other baselines are also based on WAM; thus, I am mentioning this.

W8) The generated replay is not reliably behaviorally grounded. The paper reports that 83% of generated rollouts appear successful visually, while only 42% succeed when their predicted actions are executed. It seems to me this means the replay targets can be visually plausible but physically incorrect.

Questions for authors

I do not have specific questions as I believe the paper is not yet mature and requires a lot of experiments and results, which might trigger questions, but not at the moment, tho all the weaknesses already listed above.

Reviewer_YZLcOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents Recurrent Generative Replay (REGEN), a framework that leverages the generative capabilities of WAM to synthesize pseudo-trajectories of previously learned tasks and replay them alongside new-task demonstrations to mitigate catastrophic forgetting in continual imitation learning. Experiments on three LIBERO suites and real-world manipulation tasks show that REGEN substantially reduces forgetting compared to sequential fine-tuning. The authors also identify visual degradation and action-observation inconsistency as key limitations of generative replay.

Critical strengths

Important problem and interesting motivation:

This paper addresses continual learning for robot foundation models without access to previous training demonstrations, a practically relevant problem. Leveraging the WAM's inherent generative capabilities for replay provides a simple and interesting solution.

Effective integration of generative replay with WAMs:

REGEN integrates generative replay with joint action-observation prediction in WAMs, eliminating the need for a separate generative model. The approach is straightforward and demonstrates substantial improvements in mitigating catastrophic forgetting without storing previous demonstrations.

Critical weaknesses

Gap between the motivation and experimental setting:

My primary concern is the mismatch between the paper's motivation and its experimental validation. The paper targets continual adaptation of pretrained WAMs whose original training data are unavailable. However, instead of evaluating an existing pretrained WAM checkpoint, the authors train a base policy on six LIBERO tasks using the Cosmos-Policy architecture and then artificially restrict access to those demonstrations. While this controlled setting is useful, it does not fully reflect the intended real-world scenario. Given that LIBERO demonstrations are publicly available and relatively small-scale, storing them for experience replay would be a simple and effective alternative. Evaluating REGEN on an existing pretrained WAM checkpoint would better support the motivation.

Furthermore, the LIBERO experiments use a fixed 6+4 task split and a single predefined task order, raising two concerns:

1. Lack of cross-suite continual learning evaluation. Each LIBERO suite is evaluated independently, without testing transitions such as Object→Spatial or Object→Goal. Tasks within LIBERO-Object and LIBERO-Spatial share similar manipulation structures, potentially making it easier to generate past-task trajectories from current-task observations. It remains unclear whether REGEN works when new and previous tasks differ substantially in goals, objects, or scene configurations.

2. Sensitivity to task selection and ordering. Only one base-task split and task sequence are evaluated per suite. Although all baselines share the same setting, task selection and ordering may significantly affect forgetting and replay effectiveness. Additional task splits and randomized orders would strengthen the conclusions.

Significant performance gap compared with real experience replay:

Although REGEN substantially outperforms sequential fine-tuning, it remains far behind real experience replay (ER). On LIBERO-Object, REGEN achieves an AUC of 65.5 compared to 93.4 for ER; on LIBERO-Goal, the corresponding results are 40.8 and 92.4. These gaps suggest that generated trajectories are not yet an effective substitute for real demonstrations. While the authors analyze visual degradation and action-observation inconsistency, they do not provide mechanisms to mitigate these issues.

Limited generalization and long-term evaluation:

The real-world experiments involve only three structurally similar pick-and-place tasks. Moreover, evaluation over longer continual learning sequences is limited. These results are insufficient to establish the method's robustness and scalability under more diverse, long-term continual learning scenarios.

Questions for authors

My main question concerns the initialization of pseudo-trajectories. Why is it reasonable to use observations from the current task to initialize trajectories for previous tasks? When tasks differ substantially in objects, goals, or scene configurations, could this introduce inconsistent state-action pairs? How would REGEN handle cross-suite continual learning?

What is the additional computational cost of recurrent trajectory generation? Have the authors considered comparing REGEN against limited-buffer ER with only 1, 5, or 10 real demonstrations per previous task? Such comparisons would help clarify whether eliminating replay storage justifies the additional computation and performance loss.

The paper reports a considerable gap between imagined task success and actual action execution success. Could action-observation consistency scores be used to filter or reweight generated trajectories? Would this help narrow the performance gap between REGEN and real experience replay?

Reviewer_LSBmOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper investigates continual imitation learning and builds on the generative capabilities of World Action Models. A method (REGEN) is proposed that enables a robot policy to rehearse previously learned tasks without storing their original human demonstrations. Experiments in both simulation and real-world manipulation settings show that REGEN reduces catastrophic forgetting.

Critical strengths

The method is applied for alleviating catastrophic forgetting to policy post-training and enable continual learning across several tasks without replaying demonstrations from previous tasks. Experiments demonstrate that the proposed methods are effective on both real-world and simulated tasks.

Critical weaknesses

It is challenging to understand the core insight behind the research question: why does post-training require one model for all tasks, and why are the post-training data unavailable? Pretraining focuses on learning foundational representations by training the model on hundreds of thousands of tasks with greater diversity in objects, scenes, and trajectories. Post-training deploys a pretrained model on local robots to achieve high accuracy and robustness. Therefore, it is hard to understand why a pretrained model needs continual learning across several tasks.

Questions for authors
  1. It is recommended to use real demonstrations of previous tasks. Aside from visual artifacts and blurriness, observation–action label accuracy is more important. Specifically, the training data do not exhibit shaking, but rollouts will, because model inference introduces latency. Back-and-forth shaking will introduce erroneous actions, such as inconsistencies between the action direction and the observed motion.

  2. There should also be comparisons with mixed training on all post-training tasks and zero-shot inference on both tasks.

  3. The scope of tasks should be more diverse. The current real-world tasks include only simple pick-and-place operations, involving only the atomic actions of picking and placing. Generalization across these tasks has been demonstrated in recent harness work. Therefore, evaluation on contact-rich and highly dynamic tasks, such as towel folding and tossing, is necessary.

  4. Comparing Tables 2 and 3, why does Seq-FT achieve a higher FWT than the proposed REGEN method in simulation, while the opposite holds in the real world? During post-training, REGEN only adds generated pseudo-demonstrations, which are usually of low quality, so it is unusual that adding low-quality data improves performance. The above results also demonstrate that mixed training on all high-quality data will yield the best results on real-world tasks.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.