Hierarchical Replay Memory Construction for Continual Imitation Learning

Experience replay (ER) offers a simple and effective strategy for continual imitation learning, but its effectiveness depends critically on what is retained under a limited budget. Robot demonstrations provide complementary supervision across granularities: complete trajectories preserve execution context across task stages, recurring behaviors and transitions form shared structure across demonstrations, and local windows capture fine-grained state–action variation within these behaviors. Although prior work has explored data selection at different temporal scales, how to jointly organize them within replay memory remains underexplored. To address this gap, we propose Hierarchical Memory Retention (HMR), a hierarchical replay memory construction framework that progressively organizes retention from global context to local variation under a fixed budget. HMR first constructs a task-progress-aware behavioral structure by aligning demonstrations along task progress and clustering local action–state sequences, revealing recurring modes, stage-specific occurrences, and transitions. Guided by this structure, HMR retains a small set of complete demonstrations as contextual anchors and supplements them with local windows for reliable occurrences and transitions they underrepresent, establishing task-wide structural coverage. Remaining capacity is allocated according to behavioral frequency, event importance, and current coverage to preserve representative local variations. Extensive experiments on LIBERO task suites, together with cross-architecture and cross-domain evaluations, demonstrate consistent improvements over baseline methods under matched replay budgets, while ablations validate its key design choices.