- Rating
- 2(+1)
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 3
Read review
Paper summary
The paper builds a library of short robot movements taken from demonstrations. It takes an already trained robot controller and replaces each movement it proposes with a similar movement from this library, allowing limited adjustments. The authors report that this usually preserves the controller’s ability to complete tasks.
They then train another controller to choose a stored movement and predict how to rotate and adjust it for the current situation. They test which part causes failures by replacing these predictions with ones obtained from the original controller. For their ACT-based controllers, most of the performance loss comes from how the chosen movement is rotated and adjusted; the choice of movement matters much less.
Critical strengths
The authors show that several trained robot controllers can have their proposed movements restricted to stay near demonstrated movements, with limited adjustments, while largely retaining task performance
Critical weaknesses
-
The small loss from movement choice is measured with substantial help from ACT. ACT still predicts the full action sequence. MPQ uses this prediction to rotate and adjust the chosen movement. The experiment shows that the learned choices work well with this help. It does not show that movement choice is generally a minor source of failure. Section 3.1 acknowledges this condition. The paper’s broader conclusions should state it too.
-
The predicted adjustments do not depend on the movement chosen. This design choice could explain their poor performance. The tests across six architectures only show that MPQ largely preserves task success. They do not establish the same cause of failure across six architectures. Appendix F also shows difficulties in applying the method to policies that use trained action decoders. Using stored movements and adding corrections are already established ideas. The paper therefore needs to provide new insight into why learned controllers fail. The evidence does not show that its explanation holds across other strong controller designs. The authors also do not demonstrate a better controller built using this insight.
Questions for authors
-
What should a researcher designing a robot controller learn or do differently after reading this paper? Please give one concrete example supported by your experiments. What does the analysis establish beyond identifying a weakness in the particular ACT-based controllers you trained?
-
The controller you train predicts adjustments without using the chosen movement. Could this design explain its poor performance? Do you have results where the adjustment depends on the chosen movement?
-
Why call ACT’s score after applying MPQ a “ceiling”? Couldn't another controller perform better with the same movement library?
3 comments
Thank you for the new experiments which answer my question about conditioning on the chosen movement and resolve my concern about calling the reference a “ceiling.” The extra tests across controllers make the results more convincing. My main concern is still how much we learn beyond the familiar point that lower offline loss does not always mean better task success. The smaller correction network used for the Diffusion Policy selector also leaves open how much of its performance gap comes from that design choice. I am raising my score from 1 to 2, but I still find the new insights and practical benefits too limited to support acceptance.
Thank you for raising your score. We are glad that the new experiments resolve the concerns about conditioning and the term "ceiling".
Beyond offline loss versus task success. We agree that this gap is familiar in general (Mandlekar et al., CoRL 2021), and it is not our main finding. The main finding is where discrete skill policies lose success: not in the vocabulary or the choice of skill, but in placing and correcting the chosen motion. In latent-codebook methods, a learned decoder entangles these costs, so they cannot be separated.
Size of the Diffusion Policy selector's correction network. The reviewer is right that its U-Net is narrower than Diffusion Policy's. We retrained it at Diffusion Policy's width, with down dimensions 512 and 1024 (its first two levels, since a third does not fit the ten control points). This raises the generator from 6.5M to 72.7M parameters, about eleven times the capacity. Over three seeds, success rises from 82.0 to 85.8 (, positive on every seed), so the narrow network explains about a third of the gap to the library reference. Even at eleven times the capacity, a gap of 7.1 points remains, and the decomposition again attributes most of it to execution: 6.3 points, against at most 1.9 for the choice. The revised PDF includes this result (Sec. 5.3, App. E).
We appreciate your time and the constructive questions.
We thank the reviewer for the detailed review and questions. Revisions in the PDF are marked in blue. Apart from the points below, the revision reports both substitution orders across seeds (Table 3, SBWK) and limits its claims to selectors built on a reference's network or features (Mf55). New results use three seeds (mean ± sd).
Q1 / W2: What should a researcher do differently, beyond our ACT-based controllers?
We apologize that the manuscript did not make this clearer. The revision states these consequences more explicitly in the introduction and conclusion. Our experiments support three recommendations, and the external library enables a new kind of edit.
1. Validate a skill library in closed loop, not by reconstruction. As Mandlekar et al. (CoRL 2021) found for policy checkpoints, offline criteria fail for skill libraries. Reconstruction would choose a library several times larger than needed, rank first a design that costs 5 points, and miss a transfer loss of 4 to 8 points (Sec. 6).
2. When a discrete, inspectable and editable skill library is required, wrap a trained continuous policy instead of learning a discrete output head. On all three references, the wrapped policy matches or exceeds its learned selectors. ACT's selector, with regression heads, falls 13.1 points below the wrapped policy (67.1 against 80.2). Heads that generate placement and correction by flow matching or diffusion, as the reference generates actions, close the gap on the DiT (83.0 against 83.4) and leave 10.9 points on Diffusion Policy (82.0 against 92.9, Table 3).
3. To improve a learned selector, work on placement and correction, not on the choice. The decomposition shows that on every reference the choice of prototype costs at most 2.7 points, and that the loss lies in how the selector places and corrects it (Table 3). Changes to these heads do improve a selector. Conditioning them on the chosen prototype raises the ACT selector's success by 4.1 points (Sec. 5.3).
Example application. Because the library sits outside the policy, it can be edited after training. In a new experiment (Sec. 4.2), we delete the fastest 25% of the prototypes. Without retraining, the share of executed chunks above that speed falls from 28% (bare ACT) to 12% at unchanged success (79.0 against 79.3). Halving the bound lowers this to 4.5%, at a cost of 10 points. This constrains commanded motion but is not a safety guarantee.
Q2: Adjustments that depend on the chosen movement.
We thank the reviewer for this suggestion and added the experiment to the revision (Sec. 5.3). When the yaw and residual heads of the ACT selector are conditioned on the chosen prototype, success rises from 65.6 to 69.7 over a control with the same capacity (+4.1±3.1, positive on every seed). The conditioned selector still stays about 10 points below the library reference (80.2), almost all of it in execution. Diffusion Policy's selector, whose heads also see the chosen prototype, still loses 10.3 points in execution (Table 3). Conditioning therefore explains only part of the execution loss.
W1: The small loss from movement choice is measured with ACT's help.
This is correct, and the abstract, introduction and conclusion now state the condition. We also tested whether ACT's help hides poor choices (Sec. 5.2, Fig. 6). If it did, poor choices would also cost little. Under ACT's placement and correction, forcing the 5th-nearest or a random prototype instead of the nearest costs 18.9 and 68.7 points, while the selectors' choices cost at most 2.1. Without correction, the 5th-nearest costs about 25 points and an ACT selector's choices 2.9. The selection cost is small because the learned choices are good.
Q3: Why "ceiling"?
We agree that "ceiling" was too strong, since another selector could do better with the same library. Sec. 3.1 now calls the quantity the library reference, "a baseline rather than an upper bound".
W2: Established ideas and trained action decoders.
We do not claim stored movement prototypes or corrections as contributions. The contribution is placing them after a trained policy with no learned decoder in between, and the closed-loop findings that this placement makes measurable. A trained decoder would entangle selection and execution, as App. F shows for QueST, so the decoder-free design is deliberate.
We hope these results address the reviewer's questions, and we are glad to discuss any remaining points.