- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies how we can improve VLA sensitivity to semantically-equivalent prompts. The authors propose MOSAR - a method that uses the best-performing prompt as a teacher for the other student prompts. This is done via on-policy self-distillation using the teacher action outputs. However, since distillation alone cannot exceed the performance of the teacher, the authors propose to alternate the distillation steps with PPO updates. Experiments use \pi_0.5 on RoboLab, REALM, and a real Franka robot, improving success on held-out prompts and unseen tasks against baselines.
Critical strengths
-
Studies an important problem of prompt sensitivity in VLAs. The main idea behind the paper (using best-performing prompt as a teacher model) is quite intuitive and nice.
-
The practical implementation of the idea is based on the sensible observation regarding gradient discrepancies between the RL and BC objectives.
-
The presentation is very clear and easy-to-follow
Critical weaknesses
-
I happened to notice that some of the result numbers aren't matching. For instance, in the abstract and the introduction, you claim "MOSAR achieves 53.2% success on unseen prompts in RoboLab"; however, Table 1 reports 56.8%. Perhaps, I am missing something?
-
If I read App. A6 correctly, the unseen prompt results for pi0.5 and OPSD-M come from a different evaluation. Would it be possible to re-evaluate these two baselines under exactly the same protocol as the rest?
-
All results seem to come from a single training run and without error bars. It would be quite important to report multiple seeds for establishing the superiority of the method.
-
I believe it would also be informative to add a couple of baselines. For instance, PPO from teacher prompt alone and PPO from all 11 prompts.
-
Why isn't PPO / GRPO reported in Tables 3-4?
-
As far as I understand, MOSAR has access to the stronger specific instruction (RoboLab specific instruction), extra distillation steps, and an adaptive step size. An important question for me is how much of the gain remains if the baselines get the teacher prompt and a matched compute budget?
Minor:
- In Sec. 4.3: "We cast training as improving return (ii) ...". Not sure if this is a typo, but what does (ii) refer to?
Questions for authors
See weaknesses.
3 comments
Rebuttal for reviewer P48c [2/2]
[W4,6] I believe it would also be informative to add a couple of baselines. For instance, PPO from teacher prompt alone and PPO from all 11 prompts. As far as I understand, MOSAR has access to the stronger specific instruction (RoboLab specific instruction), extra distillation steps, and an adaptive step size. An important question for me is how much of the gain remains if the baselines get the teacher prompt and a matched compute budget?
As suggested, we evaluated teacher-only PPO and PPO with MOSAR’s teacher/student split using 1, 3, or 10 student prompts per iteration. MOSAR outperforms all these controls, as well as the Table 1 PPO and GRPO baselines, on both trained and unseen prompts (Table R3). The added PPO controls match MOSAR’s interaction budget and PPO optimization settings. In the split controls, half of the environments use the teacher prompt, while the remainder use student prompts sampled from the ten-prompt pool.
Teacher-only PPO achieves 30.6% on trained prompts and 33.2% on unseen prompts. The strongest PPO control uses three students per iteration and achieves 47.2% / 44.8%, compared with MOSAR’s 53.0% / 56.8%. MOSAR therefore retains gains of 5.8 and 12.0 percentage points, respectively, supporting the additional benefit of multi-student distillation. MOSAR additionally performs teacher-target inference and a distillation backward pass on the collected rollouts, adding computation without requiring additional environment interactions.
We added these comparisons and a discussion of interaction budgets and distillation costs to Appendix C.5–C.6 and Tables A16–A17 (L1560–L1644).
Table R3. RoboLab success rates (%) on trained and unseen prompts for PPO controls with teacher-prompt access and the Table 1 baselines.
| Method | Trained | Unseen |
|---|---|---|
| PPO, fixed best teacher prompt | 30.6 | 33.2 |
| PPO, teacher/student split (1 student) | 35.3 | 30.1 |
| PPO, teacher/student split (3 students) | 47.2 | 44.8 |
| PPO, teacher/student split (10 students) | 40.9 | 38.2 |
| PPO, 10 training prompts (Table 1) | 43.0 | 33.7 |
| GRPO, 10 training prompts (Table 1) | 31.5 | 33.0 |
| MOSAR | 53.0 | 56.8 |
[W5] Why isn't PPO / GRPO reported in Tables 3-4?
We added PPO and GRPO to both evaluations under the same protocols in Table R4, R5. On goal swap, they achieve 36.5% and 34.4%, compared with 37.5% for and 54.2% for MOSAR. Under visual distractors, they improve over but remain below OPSD-S and MOSAR. These results highlight the benefits of MOSAR’s combination of RL and multi-student distillation for following changed instructions and maintaining robustness to visual distractors.
In the revision, we added PPO and GRPO rows to Table 3 (L385–L386) and Table 4 (L398–L399), together with one sentence summarizing them in Sec. 5.4 (L376–L377).
Table R4. Goal-swap success rates (%) under default instructions, reproduced from Table 3.
| Method | BBQ → salad | Right → left bin | Larger raisin → smaller butter | Left-on-right → right-on-left | Overall |
|---|---|---|---|---|---|
| π0.5 | 37.5 | 41.7 | 50.0 | 20.8 | 37.5 |
| PPO | 41.7 | 41.7 | 58.3 | 4.2 | 36.5 |
| GRPO | 37.5 | 37.5 | 54.2 | 8.3 | 34.4 |
| OPSD-S | 16.7 | 45.8 | 66.7 | 33.3 | 40.6 |
| OPSD-M | 12.5 | 4.2 | 41.7 | 20.8 | 19.8 |
| MOSAR | 37.5 | 50.0 | 66.7 | 62.5 | 54.2 |
Table R5. Success rates (%) under visual distractors, reproduced from Table 4.
| Method | Sauce | Mustard | RaisinBox | BowlStacking | Overall |
|---|---|---|---|---|---|
| π0.5 | 58.3 | 37.5 | 4.2 | 0.0 | 25.0 |
| PPO | 66.7 | 62.5 | 4.2 | 4.2 | 34.4 |
| GRPO | 62.5 | 41.7 | 0.0 | 20.8 | 31.3 |
| OPSD-S | 62.5 | 66.7 | 16.7 | 16.7 | 40.6 |
| OPSD-M | 62.5 | 33.3 | 12.5 | 8.3 | 29.2 |
| MOSAR | 75.0 | 66.7 | 20.8 | 41.7 | 51.1 |
[W7] In Sec. 4.3: "We cast training as improving return (ii) ...". Not sure if this is a typo, but what does (ii) refer to?
(i)–(iii) are the three goals from the introduction — (i) close the gap among phrasings, (ii) raise the ceiling of the strongest phrasing, (iii) preserve other tasks. We restate them at the start of Sec. 4 and rewrite the sentence as “we cast training as improving return, i.e. raising the ceiling of the strongest phrasing (goal (ii)), while …”. In the revision, we clarified in Sec. 4.3 (L274–L275) that “(ii)” refers to goal (ii) in the Introduction, namely raising the ceiling of the strongest phrasing.
A gentle note on our response
Dear Reviewer,
Thank you again for your thoughtful review and helpful comments on our paper.
We just wanted to share a brief note on our response. Following your comments, we corrected the inconsistent success rate, clarified that all baselines were evaluated under the same protocol, reported 95% confidence intervals with McNemar's tests (Tables R1–R2), added PPO controls with teacher-prompt access and a matched interaction budget (Table R3), and added PPO and GRPO to the goal-swap and visual-distractor evaluations (Tables R4–R5).
We understand this is a very busy time, and we sincerely appreciate the time and effort you have already devoted to reviewing our paper. If time permits before the discussion period closes, we would be grateful if you could read through our response. Any further thoughts or feedback would be very welcome, and we would be happy to clarify anything.
Best regards, The Authors
Rebuttal for reviewer P48c [1/2]
We thank the reviewer for the positive assessment and constructive suggestions, and for recognizing the importance of addressing prompt sensitivity in VLAs and the intuitive appeal of using the best-performing prompt as a teacher. We also appreciate the reviewer’s recognition of the sensible observation regarding gradient discrepancies between the RL and BC objectives underlying our implementation, and of our clear and easy-to-follow presentation. We address the numerical consistency, evaluation protocol, additional baselines, and clarification requests below.
We have aligned the reported RoboLab success rates throughout the manuscript with Table 1 to ensure consistency. The corresponding updates appear in the Introduction (L84) and Table A13 (formerly Table A12, L1512–L1525 in the revision).
We clarify that both π0.5 and OPSD-M were evaluated on the same 100 unseen prompts and under the same protocol as all other methods, including MOSAR. This evaluation subset was randomly selected from an initial pool of 360 unseen prompts to make evaluation across a broader range of baselines computationally feasible. We have revised Appendix A.6 (L784–L792) to distinguish the initial 360-prompt pool from the final 100-prompt evaluation subset and clarify that all methods used the same protocol.
Our RoboLab prompt evaluations already use four evaluation seeds, with multiple rollouts across different prompts for each task. To make evaluation uncertainty explicit, we report success rates with Wilson 95% confidence-interval half-widths in Tables R1–R2. Tables R1 and R2 add these intervals to the trained- and unseen-prompt results of Table 1 in the main paper, respectively.
From the tables, MOSAR achieves the highest overall success rate in all two tables, with confidence-interval widths comparable to those of the baselines. For the reported MOSAR–baseline comparisons, McNemar’s tests yield p<2×10−4 on trained prompts and p<10−5 on unseen prompts, providing statistical evidence for the observed improvements under the evaluation protocol.
We added these results in the revision as a new App. C.4 (L1528–L1537), with Table A14 (L1538–L1558) and Table A15 (L1566–L1576).
Table R1. RoboLab success rates (%) on trained prompts from Table 1, with Wilson 95% confidence-interval half-widths, from four evaluation seeds.
Table R2. RoboLab success rates (%) on unseen prompts from Table 1, with Wilson 95% confidence-interval half-widths from four evaluation seeds.