- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies when to launch inference for a fixed action-chunk generator under latency. It formulates Wait/Launch decisions as a decision-stage SMDP incorporating the action buffer and pending inference, and learns the timing policy with fitted Q-iteration. It gives a construction showing that value can vary non-monotonically with launch timing and a sub-optimality bound separating generator quality and fitted-Q errors. Experiments on Hidden-Danger GridWorld, Kinetix, and DOM evaluate success and generator calls against immediate-launch schedules and timing heuristics.
Critical strengths
-
A useful and well-motivated formulation. The decision-stage SMDP makes the interaction between inference latency, buffered execution, and information acquisition explicit. The non-monotonic example establishes why immediate launch can be suboptimal even with a fixed generator, while the performance bound identifies relevant sources of learning error.
-
Promising evidence in dynamic settings. On DOM's Dynamic Adaptation task, learned timing improves success over immediate launch by 13.50–15.25 percentage points across the three delay laws while reducing generator calls. It also outperforms Visual-Delta on this task. Hidden-Danger provides a complementary controlled example with improved success and reveal-aligned launch behavior. These results support the practical value of timing decisions when new information can change the appropriate action chunk.
Critical weaknesses
-
Practical significance appears concentrated in timing-sensitive tasks. The clearest benefits occur when newly revealed information demands a timely response, as in DOM's abrupt motion changes and Hidden-Danger. For tasks where new observations rarely change the required action during an inference window, immediate replanning may already provide most of the control benefit. Average Kinetix success gains are modest: 1.02 percentage points with one generator and 0.54 with the other, although the reduction in calls is useful. Table 11 also shows that Visual-Delta achieves higher success than learned timing in two ordinary-tracking settings. I therefore view the contribution as novel and useful within a specific regime, with moderate demonstrated breadth of impact.
-
The Kinetix gains leave the latency mechanism unclear. Table 1 reports a 1.84-percentage-point improvement at zero delay, exceeding the gains of 0.52–1.25 points under positive or stochastic delays. At zero delay, a pending inference cannot block subsequent requests. Changes in replanning frequency or buffer overwriting could also contribute to these gains. Explaining this result would help distinguish general scheduling benefits from benefits specifically due to avoiding stale inference.
Questions for authors
-
Clarification of the Kinetix mechanism: What explains the largest success gain occurring at zero delay? Can existing rollout evidence distinguish benefits from avoiding stale inferences from benefits of less frequent replanning or buffer overwriting? This would help assess how much of the Kinetix improvement specifically supports the latency-based motivation.
-
Suggested experiment to characterize practical scope: In an existing DOM task, vary the frequency of abrupt motion changes while holding the action generator and inference delay fixed. Compare learned timing, immediate launch, and Visual-Delta in both success and generator calls. This would identify when learned timing adds substantial value and when simpler schedules suffice, giving readers a clearer basis for deciding whether to adopt the method.
4 comments
Response to Reviewer KGTc (Part 2/3)
Q2. What explains the largest Kinetix success gain occurring at zero delay, when a pending inference cannot block subsequent requests? Can existing rollout evidence distinguish benefits from avoiding stale inferences from benefits of less frequent replanning or buffer overwriting?
A2. The gains at zero delay are consistent with two considerations highlighted in prior work on action chunking: temporal consistency and imperfect prediction. These complement our main focus on the arrival of new information. BID [1] explains that independently sampled chunks can fail to preserve temporal dependencies in demonstrations, leading to inconsistent successive decisions; its analysis also accounts for errors in the policy's implicit dynamics model. Diffusion Policy [2] finds that executing eight actions before replanning performs best on most tested tasks, illustrating the balance between coherent action sequences and timely feedback. In our setting, these considerations help explain why launch timing can remain useful at zero delay: an imperfect generator's new prediction need not improve on the buffered plan, and replacing that plan can disrupt continuity. Choosing when to update therefore still balances preserving the current plan against incorporating new observations.
Theorem 1 (Section 5.2) connects to these considerations through its error decomposition. The term measures the mismatch between the expert-induced chunk policy and the optimal chunk policy under the same meta policy. As explained in the footnote on page 7 and Appendix C.2, the expert chunk policy is induced by an instantaneous expert; the example rollout construction does not condition on the existing buffer. Its chunks can therefore differ from those best suited to the ongoing execution plan, connecting expert chunk quality to the continuity considerations above. The term captures the additional mismatch between the learned generator and the expert chunk policy, which need not vanish at zero delay. Together, these terms account for both the quality of the expert-induced chunks within the execution protocol and the generator's imperfect imitation of them.
Our additional Kinetix walker experiments illustrate the value of choosing update times under both zero and nonzero inference delay. At Point0, the meta policy achieves 64.31% success versus 63.05% for Immediate, a gain of 1.25 percentage points. At Point2, a separate comparison using the original walker policy bank gives 59.18% success versus 53.87% for a baseline that independently samples voluntary waiting lengths from the meta policy's distribution, a gain of 5.31 percentage points. These results support useful timing gains in both regimes and are consistent with contributions from both the arrival of new information and the choice of when to update the current plan in closed loop.
Response to Reviewer KGTc (Part 3/3)
Q3. Suggested experiment to characterize practical scope: In an existing DOM task, vary the frequency of abrupt motion changes while holding the action generator and inference delay fixed. Compare learned timing, immediate launch, and Visual-Delta in both success and generator calls.
A3. Thank you for suggesting this insightful experiment. In our additional analysis, the meta policy achieved higher mean success than both Immediate and Visual-Delta in every frequency group under both delay distributions, while saving 5.1–9.0 full generator calls per episode relative to Immediate. The size of the gains across frequency groups varied with the definition of a motion change.
We extended the DOM Dynamic Adaptation (DA) comparison in Section 6.2 (Table 3) by grouping scenes according to how often the target changed velocity. With our definition of an abrupt change, the average success gain was larger in the higher-frequency scenes. With a 0.01 m/s threshold and a four-second window, the lower-frequency group showed larger gains, though both groups benefited. As you suggest, motion frequency provides a useful way to group scenes and characterize performance. These results support the value of the meta policy in dynamic scenarios where new information matters. We give the details below.
We report results for two delay distributions: Uniform4:8 and the measured Empirical distribution. To measure motion independently of the schedules, we recorded 100 target trajectories with the arm parked, with ten resets of each of the ten official DA scenes. An event began when the change in horizontal velocity over one control step at 25 Hz reached the threshold. We then grouped the original scenes by changes per second. Each cell below gives success rate / full generator calls per episode (400 episodes per schedule and distribution).
Abrupt changes (0.05 m/s). Seven scenes had two changes and three had three changes over the 12-second episodes:
| Delay distribution | Motion frequency | Immediate | Visual-Delta | Meta policy |
|---|---|---|---|---|
| Uniform4:8 | Low (7 scenes) | 20.36% / 41.90 | 21.79% / 33.92 | 28.93% / 35.61 |
| Uniform4:8 | High (3 scenes) | 47.50% / 35.73 | 60.83% / 29.28 | 72.50% / 28.09 |
| Empirical | Low (7 scenes) | 16.07% / 43.23 | 16.79% / 34.44 | 27.14% / 35.91 |
| Empirical | High (3 scenes) | 45.83% / 34.70 | 34.17% / 33.45 | 70.83% / 28.26 |
On average across the two delay distributions, the meta policy improved success over Immediate by 9.82 points in the lower-frequency group and 25.00 in the higher-frequency group. Its mean gains over Visual-Delta were 8.75 and 24.17 points, respectively. In the higher-frequency group under Empirical delay, the meta policy achieved 70.83% success versus 34.17% for Visual-Delta, a significant gain of 36.67 points, while using 5.19 fewer calls per episode. The estimated gain over Visual-Delta was 26.31 points larger in the higher-frequency group under Empirical delay.
Including smaller changes (0.01 m/s). The four-second window yielded five lower-frequency scenes and five higher-frequency scenes:
| Delay distribution | Motion frequency | Immediate | Visual-Delta | Meta policy |
|---|---|---|---|---|
| Uniform4:8 | Low (5 scenes) | 31.00% / 41.05 | 36.50% / 33.00 | 50.00% / 33.01 |
| Uniform4:8 | High (5 scenes) | 26.00% / 39.05 | 30.50% / 32.06 | 34.00% / 33.70 |
| Empirical | Low (5 scenes) | 24.00% / 43.40 | 25.00% / 34.35 | 46.00% / 34.42 |
| Empirical | High (5 scenes) | 26.00% / 37.95 | 19.00% / 33.95 | 34.50% / 32.81 |
The meta policy again achieved the highest mean success in both groups, with larger mean gains in the lower-frequency group. There, it significantly improved success over Immediate under both Uniform4:8 (+19.00 points) and Empirical (+22.00 points). Against Visual-Delta under Empirical delay, it gained 21.00 points at nearly identical call counts (34.42 vs. 34.35). These results also show that the meta policy's benefits extend to scenes with less frequent small changes.
Thank you again for your thoughtful questions and constructive suggestions. We hope these responses address your concerns, and we welcome further discussion.
References
[1] Liu et al. Bidirectional Decoding: Improving Action Chunking via Guided Test-Time Sampling. ICLR, 2025.
[2] Chi et al. Diffusion Policy: Visuomotor Policy Learning via Action Diffusion. Robotics: Science and Systems, 2023.
Dear Reviewer KGTc,
We have uploaded the revised PDF. Section 6.1 discusses how action continuity may explain the Kinetix gains at zero delay. Section 6.2 summarizes the DOM comparison across motion frequencies, and Appendix D.5 (Table 13) reports success rates and generator calls.
Thank you again for helping us strengthen the paper. We hope the revision addresses your concerns and welcome any further questions in the remaining two days of discussion.
Authors
Response to Reviewer KGTc (Part 1/3)
Thank you for taking the time to carefully review our paper. We appreciate your perceptive questions and will incorporate relevant additional results and discussion into the manuscript as space permits. Below, we summarize your concerns and address them in turn.
A1. Thank you for highlighting the settings in which launch timing matters most. We agree that launch timing is particularly useful when important new information arrives during inference. As discussed in Section 1, our motivation centers on settings where new information can substantially change the optimal launch timing and action plan, such as highly dynamic environments or situations where safety is critical. Proposition 1 illustrates this value of information: waiting for a branch to be revealed improves value over launching immediately, but once the branch is known, further waiting lowers value (Section 4.1). The results you highlight also help identify settings in which the gains are smaller. We discuss these regimes in more detail below.
First, our empirical results support the value of launch timing when new information matters. On DOM Dynamic Adaptation (DA), a collision can change the target's direction during inference; learned timing improves success over immediate launch by 13.50–15.25 percentage points across the three delay laws (Section 6.2, Table 3). In Hidden-Danger, the scheduler launches at the first free decision after the reveal in 96.4% of episodes, and the same planner's success rises from 54.1% to 78.7% across 4,096 maps (Appendix D.1, Tables 5 and 7). On ordinary CR tracking, Visual-Delta is competitive and the ordering varies with the delay law (Appendix D.5, Table 11), consistent with your observation that a simple schedule can work well when new observations rarely change the required action. Together, these results support our motivation and analysis in Sections 1 and 4: when new information can change the appropriate action, learned launch timing can substantially improve performance.
Second, launch timing also yields larger improvements on harder tasks. This is intuitive: tasks already near 100% success leave little room for improvement. The average Kinetix gain for TrainingRTC is indeed approximately 1 percentage point, but this average obscures larger gains on harder tasks. Table 2 ranks task–delay pairs by independently measured baseline success for each backbone (Section 6.1). In the hardest quartile, learned timing raises success over immediate launch from 63.56% to 67.46% for TrainingRTC (+3.91 points) and from 72.49% to 74.23% for FutureRTC (+1.74 points). The gain on the harder TrainingRTC tasks also holds against a state-based trigger: in that quartile, learned timing exceeds State-Delta by 2.38 percentage points.