acceptodds
Under review as a conference paper at ICLR 2027

Data-Free Continual Learning for World Action Model with Rectified Generative Replay

Abstract

World action models (WAMs) built on pretrained video generation models adapt quickly to new robotic tasks, but they forget previously learned tasks under sequential fine-tuning. Experience replay (ER) mitigates such forgetting by rehearsing data of previous tasks, yet this data is often unavailable in robotics, where a policy is deployed as weights only and its training data is not shared along with it. In this work, we ask whether a WAM can achieve data-free continual learning by regenerating its own past. Our key observation is that a WAM can indeed autoregressively regenerate complete rollouts of previously learned tasks and replay them to mitigate forgetting. However, replaying such raw rollouts still lags far behind replaying real data, which we attribute to two issues. First, the WAM is unable to recognize task completion and keeps generating purposeless motion after the task is done. Second, under open-loop generation, the generated actions gradually drift away from the generated observations. To address both issues, we present rectified generative replay (R-WAM), which truncates generated rollouts at task completion with an agentic evaluator and filters out rollouts whose actions are physically inconsistent with the observations. On the four LIBERO suites and five real-world tasks, R-WAM reduces forgetting by over 90% relative to sequential fine-tuning and nearly matches ER without accessing any data from previous tasks. Our analysis further reveals that forgetting in WAMs concentrates in inverse dynamics modeling: a forgotten WAM still produces visually plausible observations of previous tasks while its generated actions fail.

Primary area:
applications to robotics, autonomy, planning
Keywords:
World Action Model, Robotic Manipulation, Continual Learning, Experience Replay

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_HMA3OpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This work leverages the WAM for generative replay, in continual robot learning tasks.

R2-WAM uses the WAM (e.g., LingBot-VA) to imagine rollouts per learned task, then does some filtering to remove most inconsistent ones according to a simulator, and discards unsuccessful traces.

Results wise, on four ten-task LIBERO suites, it reaches 92.7% average final success, 20% above without generative replay, and 2 points below than storing real equal size data (equal in the sense of imagined traces). The authors further analyzed that the forgetting concentrates in inverse dynamics.

My own concern is that this is still not a very scalable way for true continual learning, as learning every task requires a few hours of finetuning the WAM. And though the buffer size is matched, it only means the authors compare using the same amount of imagined versus real data. However, this calculation omits the additional storage overhead of the WAM, which significantly exceeds the base data storage in short task sequences (though this is a common problem in all generative replay methods).

Critical strengths
  1. Solid and strong results in a practical setting: without any old-task data, R2-WAM reaches 92.7% average SR (NBT 2.33), against 94.6% for ER and 70.4% for raw generated replay, all measured with multiple seeds.

  2. Each rectification step has direct support. On LIBERO-Spatial, removing truncation drops SR from 94.0 to 70.8 and removing filtering to 80.8. Also, I particular thank the authors for providing the per-task tuning cost in terms of gpu hours.

Critical weaknesses

As I mentioned, my own concern is that this is still not a very scalable way for true continual learning, as learning every task requires a few hours of finetuning the WAM. And though the buffer size is matched, it only means the authors compare using the same amount of imagined versus real data. However, this calculation omits the additional storage overhead of the WAM, which significantly exceeds the base data storage in short task sequences (though this is a common problem in all generative replay methods).

On the other hand, some minor things worth checking, for REGEN (Sec 4.1–4.2 of the paper), it should start from real observations of the current task and retain no previous-task trajectories, i.e., it is already data-free?

The claim that forgetting “concentrates in inverse dynamics” compares quantities on different scales. Fig. 7(a) contrasts 8% retained task success with 78% retained PSNR, but a PSNR ratio is not a fraction of ability retained: PSNR is logarithmic and stays well above zero even for poor predictions, while success can fall to zero. Please report raw PSNR/MSE against a trivial-predictor baseline, or limit the claim to the qualitative evidence in Fig. 7(b).

Questions for authors

See weakness.

Reviewer_EkzeOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This paper studies continual learning for world action models without retaining real demonstrations from previous tasks. Using LingBot-VA, the method generates videos and actions from previous task instructions, rectifies the trajectories through completion-aware truncation and robot-motion consistency filtering, and mixes them with real demonstrations of new tasks for training. Experiments cover four LIBERO suites and five real-robot tasks.

Critical strengths
  1. Generative replay achieves strong performance close to real-data replay. Average final success across the four LIBERO suites reaches 92.7%, compared with 94.6% for ER and 70.4% for unrectified replay.
  2. Ablations directly support the benefits of trajectory rectification. With the same LingBot-VA backbone, the full method achieves 94.0% on LIBERO-Spatial, dropping to 80.8% without filtering and 70.8% without truncation.
Critical weaknesses
  1. The practical benefits and motivation of the setting need stronger justification. The method still maintains a synthetic replay buffer with the same trajectory capacity as real-data replay, while adding generation, evaluation, and filtering costs. It demonstrates neither a storage advantage nor better performance over real-data replay, and the privacy benefits of synthetic data remain unverified.
  2. The novelty is mainly an incremental improvement over existing generative replay, and the comparison with prior work is incomplete. ReGen already uses the same WAM to generate previous-task trajectories for replay and identifies inconsistencies between generated videos and actions. All evaluations are also limited to LingBot-VA, leaving the applicability to other WAMs, such as Cosmos-Policy, unclear.
Questions for authors

1.What practical benefits justify synthetic replay, given its additional computational costs and the absence of demonstrated storage or performance advantages over real-data replay? 2. Could the authors correct the description of ReGen and compare under matched initialization and rectification conditions to isolate the contributions of the proposed improvements? 3. Do completion-aware truncation and motion-consistency filtering provide similar benefits on another WAM, such as Cosmos-Policy?

1 comment

Dear Reviewer Ekze,

We understand how busy this period is and are truly grateful for your time. We have worked hard to address each of your comments, with new experiments and a revised paper. As the discussion period closes on November 18 (AoE), even a brief note from you would mean a great deal to us. If our responses have resolved your concerns, we would sincerely appreciate it if you would consider raising your score.

Best regards,

The Authors

Reviewer_m11gOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies continual learning for world action models (WAMs). The authors first show that, like other networks, WAMs suffer from catastrophic forgetting when fine-tuned sequentially on new tasks without access to data from previous tasks. To mitigate this, they propose R²-WAM, a data-free continual learning scheme tailored to WAMs. The key idea is to replay rollouts generated by the WAM itself from the task instructions alone, since data from previous tasks is often unavailable. Before replay, these rollouts are rectified in two ways. First, a VLM truncates each rollout at the point of task completion and discards failed ones. Second, physically inconsistent rollouts are filtered out by comparing the robot’s optical flow simulated from the generated actions in a robot-only simulator (MuJoCo) with the flow estimated from the generated video. The proposed method substantially reduces catastrophic forgetting compared with naive sequential fine-tuning, and it achieves performance comparable to the oracle setting (experience replay), which assumes access to ground-truth data from previous tasks.

Critical strengths
  • The paper is generally well-written and easy to follow.
  • The problem setting, data-free continual learning for WAMs, is interesting and practically important, since deployed policies are often - released as weights alone, without their training data.
  • The proposed method is simple and requires no additional training, which makes it appealing for real-world deployment.
  • The diagnostic finding that forgetting in WAMs concentrates in inverse dynamics modeling while video prediction is largely preserved is insightful, and it explains why a WAM can regenerate its own past tasks for replay.
Critical weaknesses
  • The evaluation is limited to a single simulation benchmark (LIBERO) and a small-scale real-world setup with five tasks that share the same scene. I encourage the authors to evaluate on additional benchmarks (e.g., RoboCasa and/or RoboLab) and to conduct more extensive real-world experiments with more diverse tasks.
  • All experiments are conducted with a single backbone, LingBot-VA, so it remains unclear whether the method generalizes to other WAMs, such as DreamZero or FastWAM.
  • Each task sequence contains at most 10 tasks (and only five in the real world), whereas a practical continual learning scenario would involve far more tasks. It is unclear how R²-WAM scales to longer sequences, where the replay buffer and the generation cost grow linearly with the number of tasks, and where errors in the generated rollouts may accumulate over many stages.
Questions for authors
  • Could the authors provide more results on other real/sim benchmarks and models?
3 comments

Dear Reviewer m11g,

We understand how busy this period is and are truly grateful for your time. We have worked hard to address each of your comments, with new experiments and a revised paper. As the discussion period closes on November 18 (AoE), even a brief note from you would mean a great deal to us. If our responses have resolved your concerns, we would sincerely appreciate it if you would consider raising your score.

Best regards,

The Authors

Thanks for the rebuttal with extensive experiments. My concers are mostly resolved and would like to raise my score from 2 to 3.

Thank you very much for taking the time to reply and for recognizing our work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.