- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces Retrieval-Anchored Representation Alignment (RARA), a two-stage adaptation method for fine-tuning pre-trained robot policies while preserving their generalization ability. RARA first aligns the representations of the target demonstrations with those of retrieved demonstrations from pretraining, and then fine-tunes the policy on the target demonstrations while limiting representation drift. The method is evaluated on 13 tasks in RoboTwin, RoboCasa, and a real robot, showing clear improvements in success rates both in-distribution and out-of-distribution compared to baseline methods.
Critical strengths
-
The method offers a well-motivated way to reuse task-relevant pre-training experience without directly imposing source-task action supervision. The paper is also well-presented and clearly written, making it easy to follow the proposed method.
-
The empirical evaluation covers two policy architectures, two simulation environments, and real-world manipulation across 13 tasks. RARA improves average ID and OOD success in all three settings, with larger gains under OOD conditions.
Critical weaknesses
-
The evaluation does not clearly establish robustness to shifts beyond pre-training coverage. In RoboTwin, pre-training already includes broad variation in backgrounds, clutter, and object positions. The results therefore support recovering pre-trained generalization, but leave robustness to previously unseen shifts unresolved.
-
The contribution of the retrieval design is not fully isolated. STRAP differs in both retrieval and training objectives, making it difficult to attribute the performance gap. Comparing alternative retrieval mechanisms, such as Behavior Retrieval [1] or Importance Weighted Retrieval [2], under the same RARA objectives would clarify this contribution. The restriction of negatives to instruction-filtered demonstrations also lacks comparison with cross-task negative sampling.
[1] Du, M., Nair, S., Sadigh, D., Finn, C. "Behavior Retrieval: Few-Shot Imitation Learning by Querying Unlabeled Datasets." RSS 2023.
[2] Xie, A., et al. "Data Retrieval with Importance Weights for Few-Shot Imitation Learning." CoRL 2025.
Questions for authors
-
Why does RETAIN perform substantially worse than FT, particularly on RoboCasa? How was the interpolation coefficient selected, and which parameters were merged?
-
What is the effect of removing only the Stage-1 anchor loss, and how sensitive is RARA to (\lambda_1) and (\lambda_2)?
4 comments
Q2: Stage-1 anchor loss and sensitivity to the loss weights
Without the Stage-1 anchor, the contrastive loss can also be reduced by moving the retrieved representation, so the target frames are no longer aligned with a fixed pre-trained representation. The result agrees with this (table below, and Appx. P of the revision): RARA drops from 76.0 / 58.3 to 72.4 / 44.1, so the OOD gain is lost, and the result is similar to Stage 2 without Stage 1.
RARA is not sensitive to the loss weights. When changes from 0.1 to 10, RARA stays 5.6 to 8.6 points above FT in ID and 6.0 to 11.0 points in OOD. We train both stages until the losses converge, and the weights then have little effect on the result.
Success rate (%) on RoboTwin, averaged over the four tasks:
| Method | ID | OOD |
|---|---|---|
| RARA () | 76.0 ± 1.0 | 58.3 ± 1.2 |
| Without the Stage-1 anchor () (new) | 72.4 ± 1.5 | 44.1 ± 2.2 |
| (new) | 75.0 ± 1.8 | 57.9 ± 1.6 |
| (new) | 73.0 ± 0.9 | 53.3 ± 1.8 |
| Stage 2 only, no Stage 1 (Table 3) | 73.0 ± 3.2 | 46.2 ± 4.1 |
| FT | 67.4 ± 1.4 | 47.3 ± 2.4 |
We hope that these responses address the reviewer's concerns, and we are happy to answer any further questions.
Thank you for the additional experimental results provided in the authors' response. I will maintain my original score.
Thank you for reading our response and the additional experiments, and for your positive assessment of our work. Your comments helped us improve the paper, in particular the comparison of retrieval designs and the analysis of the Stage 1 anchor and loss weights.
We thank the reviewer for the comments. We uploaded a revised paper in which all changes are marked in blue.
W1: Robustness to shifts beyond pre-training coverage
We agree. The RoboTwin pre-training data cover a broad range of backgrounds, clutter and object positions, and the OOD backgrounds can overlap with them. What is new in RoboTwin is the target task and its objects, which the policy sees only in 50 demonstrations with a narrow layout. RoboTwin therefore tests whether fine-tuning on a new task keeps the generalization of the pre-trained policy, and the revision now states this (Appx. M).
Shifts that the pre-training data do not cover are tested in RoboCasa, where the policy is pre-trained on real-world DROID data and adapted to simulated kitchens. RARA improves OOD success over FT by 11.2 points there (Table 2), and by 19.2 points on the real robot (Fig. 5).
W2: Isolating the retrieval design
We agree that STRAP differs from RARA in both retrieval and training objective. The table below keeps the two RARA objectives fixed and changes only the retrieval or the negatives (Appx. O of the revision). With Behavior Retrieval, RARA stays close to the original (2.5 points lower in ID and 4.1 in OOD) and well above FT (67.4 / 47.3). The gain therefore comes mainly from the two objectives, and a simple retrieval by instruction and action similarity, which needs no additional model, is sufficient. A likely reason for the small drop is that the embedding of Behavior Retrieval is trained only on the pre-training data, so its image part is less reliable for the new target objects, whereas the instruction and the actions do not depend on their appearance.
Replacing the negatives of RARA with frames from other tasks gives almost the same result. Unlike standard contrastive learning, RARA does not need negatives to prevent collapse, because the target frames are aligned to a pre-trained representation that the anchor keeps fixed, and frames of other tasks are already far away in this representation. The positives matter more: with random Stage-1 pairs, RARA falls below FT (Table 3).
Success rate (%) on RoboTwin, averaged over the four tasks. All rows use the two RARA objectives:
Q1: Why RETAIN is worse than FT
In our experiments, how well weight merging works depends on how far the target task is from the pre-training data (Appx. H of the revision). In the original setting of RETAIN, from LIBERO pre-training to new LIBERO tasks, the two are similar, and merging also works well in our test. On RoboTwin, where the pre-training and target data share the simulator, robot and cameras, RETAIN keeps about 70% of the success of FT. On RoboCasa, where the policy is pre-trained on real-world DROID data and adapted to simulated kitchens, it keeps less than half. A likely explanation is that this larger gap requires a much larger change of the weights, so the interpolated weights lie between two solutions and preserve neither behavior. RARA constrains the representation of the retrieved demonstrations instead of the weights, so the policy can still change as much as the target task needs.
For the coefficient, we follow the released implementation and interpolate all parameters with one coefficient α. We tested α∈{0.25,0.5,0.75}, the values that the original paper uses on DROID, and report α=0.75, which is the best of the three. Merging only the visual encoder or only the remaining parameters does not improve over merging all parameters (Appx. K of the revision).