- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies whether dual-arm policies can execute familiar skills in combinations not seen during training. It introduces ACG-Bench, which tests changes in execution order, synchronization, and cross-task combinations. A shared scheduler provides the current instruction for each arm, so the evaluation focuses on execution rather than planning.
The proposed AE-VLA combines arm-specific action-token groups, skill-specific LoRA adapters, and an arm-wise attention mask on a shared π0.5 backbone. In simulation, it achieves higher average success on unseen compositions than Dual π0.5, the best-performing of the three main execution baselines. On real robots, it achieves 39% versus 10% for Dual π0.5 across five unseen conditions. AE-VLA has lower mean in-domain success than Dual π0.5 in both settings.
Critical strengths
-
S1: The benchmark checks whether the robot follows the required order and timing; it also checks whether the robot reaches the goal. This catches failures that a final-state check could miss. All methods are told what each arm should do, so the comparison focuses on how well they carry out those instructions.
-
S2: AE-VLA improves average success on unseen compositions in both simulation and real-robot tests. In the simulation ablations, adding SkillLoRA or AWA alone gives a small gain on unseen compositions. Adding both gives a much larger gain.
Critical weaknesses
-
W1: The ablations show that the full model performs better on unseen compositions, but several changes are tested together. The authors acknowledge this limitation. Further controls would help explain the gains:
- AWA changes several attention connections at once. It blocks direct attention between the arms' action tokens, restricts access to the other arm's local observations, and prevents attention to later action positions within each arm. The ablation does not separate these effects.
- Without AWA, right-arm action tokens can attend to left-arm action tokens, but left-arm action tokens cannot attend to right-arm action tokens. This asymmetry makes the comparison harder to interpret.
- SkillLoRA adds skill-specific adapters, extra parameters, and a skill-classification loss. The experiments do not separate the effects of these changes.
-
W2: The contribution beyond prior work and some evaluation details need clarification.
- MA-VLA already tests unseen skill combinations, including changes in order and synchronization. SkillVLA studies skill reuse and controls communication between arms. The paper explains how the methods differ, but it is still unclear what ACG-Bench adds to existing tests and what we learn from AE-VLA that prior work has not shown.
- In simulation, synchronization windows range from 15 to 45 seconds. The paper does not explain how these compare with typical skill durations or whether tighter windows change the results.
- Two Dual π0.5 conditions had their success scores changed after a manual collision check. The paper does not say whether all methods were checked in the same way.
- For the real-robot tests, the paper does not clearly explain how skill completion is detected or how instructions are updated. It is also unclear whether either step involves human input.
Questions for authors
- Q1: Could you test the following action-token attention settings while keeping all attention connections involving observation-prefix tokens fixed?
- Block attention between the arms' action tokens, while keeping attention within each arm unchanged.
- Prevent each action token from attending to later positions within the same arm, while keeping cross-arm action attention unchanged.
- Allow both arms' action tokens to attend to each other, while keeping attention within each arm unchanged.
How does each setting affect success on unseen compositions? The third setting would help check whether the current one-way access between the two action-token groups affects the results.
-
Q2: How does SkillLoRA compare with the following alternatives?
- Use one adapter shared by all skills, with the same number of added parameters.
- Keep the skill-classification loss, but remove the skill-specific adapters.
- Choose the adapter directly from the skill named in the prompt, instead of using the learned router.
These comparisons would help show whether separate skill adapters and learned routing are needed, and how much the extra parameters and classification loss contribute.
-
Q3: Please clarify the collision checks and the real-robot evaluation:
- Were successful runs from all methods checked for collisions in the same way?
- How many apparent successes were removed in each of the two affected Dual π0.5 conditions?
- On real robots, how is skill completion detected and how are instructions updated? Does either step involve human input?
-
Q4: Please clarify the benchmark's added value and synchronization thresholds:
- What does ACG-Bench test beyond the settings already studied in MA-VLA? What additional design insights does AE-VLA provide beyond SkillVLA?
- How were the synchronization windows chosen? How long do the relevant skills usually take, and do tighter windows change the relative performance?