- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper addresses the issue of imitation learning policies executing slower than necessary and proposes an approach to accelerate performance at the controller level beneath the policy layer. The core method, CGDO (Coupled Gain–Duration Optimization), freezes the pre-trained base policy and additionally learns an execution adapter that outputs pairs of "controller gain g and execution duration d" for each predicted action block. Training is conducted using off-policy RL (SAC) within a semi-Markov decision process (SMDP) framework, with physical time cost as the reward. Section 3 of the paper also provides an analysis of the motivation. The experiments cover nine simulation tasks and three real-world tasks on the Piper robotic arm; baselines include DemoSpeedup, SpeedAug, SAIL, RACE, SpeedTuning, and policy-level RL methods DSRL/ResFiT. Results show that CGDO achieves 1.42–12.29× speedup compared with the base policy while all task success rates are maintained or improved; when combined with policy-level RL, it achieves the best TPR in eight out of nine tasks.
Critical strengths
i) The problem is identified in a novel and persuasive way, and the method can be combined with previous work. It shifts the acceleration issue from 'what action the policy outputs' to 'how the controller executes the action,' addressing the real pain point of commanded-pose policies where the reached-pose assumption does not hold, while keeping the base policy frozen and allowing for further improvements when combined with DSRL/ResFiT and RTC. ii) The motivation analysis is solid, and the experiments are extensive. Section 3 and the gain-duration coupling analysis in Appendix F clearly explain why joint learning is necessary; the experiments span two types of policy architectures, two categories of action labels, nine simulation tasks, and real hardware, including ablations on multiple seeds variance, time penalty sensitivity, and damping learning, which enhance the credibility of the conclusions.
Critical weaknesses
i) The real-machine verification is insufficient. The real-machine tests involved only 3 tasks, 10–20 trials per method, using a single Piper arm, with no observed improvement in success rate, and the statistical power is insufficient to support strong conclusions at the "deployed system" level. ii) Each task requires online RL retraining, and the costs and generalizability are not sufficiently explained. Each task needs to train the execution policy separately, and combinations such as DSRL on Can also require hyperparameter configuration adjustments; the SR for the three seeds of Relocate fluctuates between 74–88%, indicating limited training stability. The authors acknowledge that cross-task/cross-dynamics transfer has not been validated, but the paper does not report the wall-clock cost of online interactions or a comparison of sample efficiency, making it difficult to evaluate 'the actual overhead of this method relative to retraining the policy itself.'
Questions for authors
i) Please provide the total amount of online interaction and wall-clock time for training the execution strategy of each task, and compare it with the training cost of strategy-level RL methods like DSRL/ResFiT; how sensitive is the adapter training to hyperparameters? ii) On Adroit, RACE directly uses a commanded-pose target for waypoint planning, which leads to infeasibility and all fallback to a 10ms joint-PD controller — should this result be attributed to the method itself or the transfer setup? Did you try retraining the Adroit policy using the reached-pose target originally required by RACE? iii) In real-world experiments, how are safety margins for gain and duration determined? Under high gain, did you ever encounter joint torque saturation, vibration, or hardware protective shutdown?
5 comments
Response to Reviewer BFdS (2)
Q4. How are controller bounds and safety margins determined?
In real-world experiments, how are safety margins for gain and duration determined? Under high gain, did you ever encounter joint torque saturation, vibration, or hardware protective shutdown?
All six joints use a proportional gain of . CGDO selects a shared proportional gain within and action execution durations of 1–50 ms.
We select the gain bounds empirically through parameter sweeps. At larger gains, we observe joint vibrations at approximately 35 Hz. The selected range allows gain adaptation while limiting vibration.
We use a manual stop procedure when vibration is observed. No such stop is triggered within the selected gain range, where we observe no signs of torque saturation and no hardware protective shutdowns.
Gentle Reminder
Thanks for your time and effort in reviewing our work. As the discussion draws to a close, we would be grateful if you could have a look at the rebuttal and let us know if any concerns remain.
Following our point-by-point response, we would like to share a new real-world evaluation on clothes folding, a long-horizon, contact-rich task involving deformable-object manipulation. These task properties makes clothes folding a particularly demanding test of CGDO, further broadening the scope of our real-world evaluation, and better addressing the concern you raised in weakness (i).
| Method | Success Rate (%) ↑ | ATR (s) ↓ | Throughput () ↑ |
|---|---|---|---|
| Fixed | 98% (49/50) | 37.45 ± 1.62 | 15.73 ± 0.31 |
| CGDO | 94% (47/50) | 8.94 ± 1.84 | 60.84 ± 3.76 |
( ATR: mean successful physical completion time. : estimated successful completions per 600 seconds )
With as the base policy and 50 evaluation trials per method, CGDO reduces mean successful completion time from 37.45s to 8.94s (4.19× speedup) and increases throughput from 15.73 to 60.84 (3.87×). These results further support CGDO’s practical value for challenging tasks in the real world. We also provide an experiment video on this anonymous supplementary website and would appreciate your taking a look.
Response to Reviewer BFdS (1)
We appreciate your insightful feedback. We have carefully considered your comments and provided a point-by-point response, and we hope it addresses your concerns.
Q1. Can you evaluate more real-world tasks with more trials per method?
We add a real-world task requiring precise cube alignment, with 50 trials per method. CGDO reduces mean successful wall-clock completion time from 9.36s to 1.32s (7.11× speedup), while observed success increases from 82% (41/50) to 88% (44/50).
( ATR: mean successful physical completion time )
Q2. What is CGDO’s interaction overhead, and how sensitive is training to hyperparameters?
Figure 6b (Appendix C.1) compares sample efficiency on Lift. CGDO reaches a 200-episode average return of −40 after approximately 21.6k online environment steps, compared to 832k for DSRL, a 38.5× reduction during the online phase. Even including initial warm-up data collection, CGDO requires about 103.9k steps, roughly 1/8 of DSRL’s online interactions alone.
Notably, DSRL is the strongest RL baseline in this comparison, achieving higher returns and sample efficiency than both ResFiT and SpeedTuning.
In addition, we use a single shared set of hyperparameters within each benchmark, supporting easy reuse across tasks (Appendix A.1).
Q3. Does the Adroit RACE result reflect the method or the transfer setup?
RACE is designed for policies trained on reached targets. Directly applying it to commanded targets can alter the intended motion, because these commands need not correspond to states attained during demonstration. When planning fails, RACE defaults to PD control, according to the paper [1]. Therefore, RACE is not directly applicable to commanded targets.
In our manuscript, we use reached targets on RoboMimic and MimicGen, and commanded targets on Adroit. The former use RACE’s intended target type, whereas the latter do not. On Adroit, nearly all RACE planning attempts fail, triggering PD fallback as specified in its original paper, so the low scores are not surprising.
We will make this distinction explicit in the revised manuscript.
That said, to directly address this concern, we follow the reviewer's suggestion and re-train the Adroit Door base policy using reached targets. Results show CGDO still outperforms RACE in both success rate and speed, consistent with our findings on RoboMimic and MimicGen.
( TP₆₀₀: estimated successful completions per 600 seconds )
[1] Time Optimal Execution of Action Chunk Policies Beyond Demonstration Speed. ICLR 2026.