Scaling Continual Learning to 125 Tasks with Experience Replay

The ability to continually acquire new skills while retaining existing capabilities is important for the long-term deployment of vision-language-action (VLA) models. However, scaling VLA continual learning beyond a hundred tasks remains underexplored. Prior work mainly studies task-wise updates on small task sets. Growing task pools and the need to incorporate new capabilities in batches motivate phase-wise continual learning, where each update jointly learns a batch of new tasks. We systematically study which continual learning recipes scale to this setting. Experiments on LIBERO and RoboCasa365 show that task-wise findings do not directly transfer to phase-wise updates. For example, adding low-rank adaptation (LoRA) to experience replay can benefit some task-wise settings through better old-task retention, but this advantage does not reliably persist across phases. Higher ranks and tuned adapter learning rates improve acquisition, yet LoRA still trails full-model ER and can forget more of the capabilities learned in intermediate phases. Further analysis shows that task composition and task scale both shape learning dynamics. We find that simple full-model experience replay (ER) effectively balances new-task acquisition and old-task retention in phase-wise VLA continual learning, making it the most practical choice among the recipes we evaluate. Across four phases and 125 tasks in RoboCasa365, full-model ER continually acquires new capabilities while preserving old ones, using only $2\%$ of each old task's data for replay and requiring neither additional teacher training nor online data collection. Our results establish full-model ER as a practical, scalable recipe for phase-wise VLA continual learning.