- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This work investigates how to efficiently store data for episodic replay in robot lifelong imitation learning. The convention is to store a few representative traces, but this work goes further by analyzing which “chunks” of demonstrations are more worthy of storing. And the proposed method stores a few representative demonstrations and then selects other chunks to mitigate the “weakness” of those demonstrations for each task.
Specifically, the anchor demonstrations are ranked by how many recurring behavioral changes they contain or how representative they are, the additional chunks are chosen to cover reliable behaviors and transitions that the anchors miss, and the remaining budget is filled with representative and diverse chunks from each behavioral group.
As a result, with a fixed 1000-demonstration budget, the proposed method works better. With two VLAs, it improves final success by about 5.5 points over prefix-based ER, the only VLA baseline.
Critical strengths
-
The method only changes which chunks are stored for ER, so it is easy to plug in. It works with both a small BCTransformer trained from scratch and pretrained VLAs, and it explicitly keeps “rare” behaviors (e.g., regrasps and transitions between stages) that simply keeping the first or random chunks can miss.
-
The gains are consistent. With BCTransformer, it achieves the best forward transfer and AUC on all four LIBERO suites, including over GSS, the strongest baseline (+3.5 AUC on average). With two VLAs, it also improves final success over ER by about 5.5 points.
Critical weaknesses
A simple baseline is missing: just keep 1,000 random chunks from each task. How does that perform? In other words, which design in the proposed method is the most critical?
The method is quite complicated, and its components are not clearly necessary. It relies on many hand-designed steps and hyperparameters, yet uniform filling beats the proposed filling on LIBERO-Spatial, and anchors add little on LIBERO-Goal. In addition, the anchor ratios used in the main table match the best ablation settings, which were computed on two of the three evaluation seeds.
In general, I feel that more ablations to understand which parts make the proposed method work, and only highlighting a few top contributors, would be better than presenting a complex method with many different small updates. I believe this is a very practical method, and data selection (or coreset selection, which is already quite a small field in continual learning) is what people would actually use in practical continual learning. However, the current work may mix too many things, and the insights are less clear.
Questions for authors
See weakness.