acceptodds
Under review as a conference paper at ICLR 2027

When to Launch Inference for Action Chunking under Latency?

13 views

Abstract

Policies parameterized by expressive generative models promise a powerful interface for real-time decision making. To boost performance and promote execution smoothness, these models predict a chunk of future actions in a single inference call. Yet, the non-negligible inference latency introduces a delicate trade-off. Early inference may prevent the model from reacting to critical information revealed shortly afterward; late inference suffers from staleness before the next action chunk prediction takes effect. Existing work mainly improves the chunk prediction policy itself, but sidesteps the equally essential question: when to launch inference under latency? This paper bridges this gap by formulating inference timing as a meta control policy in a semi-MDP induced by the original decision-making problem. We show that value can vary non-monotonically with inference timing, illustrate its consequences in a hidden-danger example, and adapt a fitted -iteration algorithm to learn the meta control policy. Theoretically, we prove a performance bound that disentangles sources of sub-optimality, notably including action chunk generation quality and fitted estimation error. Empirically, experiments on Kinetix and DOM span different environments and action-policy scales. Adding learned launch timing improves task success while reducing action-model calls compared with the same action generators under their default launch schedules, with larger gains on harder tasks and abrupt dynamic adaptation.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Inference latency, Action chunking, Meta-control, Reinforcement learning, Generative policies

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_KGTcOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies when to launch inference for a fixed action-chunk generator under latency. It formulates Wait/Launch decisions as a decision-stage SMDP incorporating the action buffer and pending inference, and learns the timing policy with fitted Q-iteration. It gives a construction showing that value can vary non-monotonically with launch timing and a sub-optimality bound separating generator quality and fitted-Q errors. Experiments on Hidden-Danger GridWorld, Kinetix, and DOM evaluate success and generator calls against immediate-launch schedules and timing heuristics.

Critical strengths
  1. A useful and well-motivated formulation. The decision-stage SMDP makes the interaction between inference latency, buffered execution, and information acquisition explicit. The non-monotonic example establishes why immediate launch can be suboptimal even with a fixed generator, while the performance bound identifies relevant sources of learning error.

  2. Promising evidence in dynamic settings. On DOM's Dynamic Adaptation task, learned timing improves success over immediate launch by 13.50–15.25 percentage points across the three delay laws while reducing generator calls. It also outperforms Visual-Delta on this task. Hidden-Danger provides a complementary controlled example with improved success and reveal-aligned launch behavior. These results support the practical value of timing decisions when new information can change the appropriate action chunk.

Critical weaknesses
  1. Practical significance appears concentrated in timing-sensitive tasks. The clearest benefits occur when newly revealed information demands a timely response, as in DOM's abrupt motion changes and Hidden-Danger. For tasks where new observations rarely change the required action during an inference window, immediate replanning may already provide most of the control benefit. Average Kinetix success gains are modest: 1.02 percentage points with one generator and 0.54 with the other, although the reduction in calls is useful. Table 11 also shows that Visual-Delta achieves higher success than learned timing in two ordinary-tracking settings. I therefore view the contribution as novel and useful within a specific regime, with moderate demonstrated breadth of impact.

  2. The Kinetix gains leave the latency mechanism unclear. Table 1 reports a 1.84-percentage-point improvement at zero delay, exceeding the gains of 0.52–1.25 points under positive or stochastic delays. At zero delay, a pending inference cannot block subsequent requests. Changes in replanning frequency or buffer overwriting could also contribute to these gains. Explaining this result would help distinguish general scheduling benefits from benefits specifically due to avoiding stale inference.

Questions for authors
  1. Clarification of the Kinetix mechanism: What explains the largest success gain occurring at zero delay? Can existing rollout evidence distinguish benefits from avoiding stale inferences from benefits of less frequent replanning or buffer overwriting? This would help assess how much of the Kinetix improvement specifically supports the latency-based motivation.

  2. Suggested experiment to characterize practical scope: In an existing DOM task, vary the frequency of abrupt motion changes while holding the action generator and inference delay fixed. Compare learned timing, immediate launch, and Visual-Delta in both success and generator calls. This would identify when learned timing adds substantial value and when simpler schedules suffice, giving readers a clearer basis for deciding whether to adopt the method.

4 comments
AuthorsOpenReview ↗

Response to Reviewer KGTc (Part 1/3)

Thank you for taking the time to carefully review our paper. We appreciate your perceptive questions and will incorporate relevant additional results and discussion into the manuscript as space permits. Below, we summarize your concerns and address them in turn.

Q1. I view the contribution as novel and useful within a specific regime: the clearest benefits occur in DOM's abrupt motion changes and Hidden-Danger, while immediate replanning may already provide most of the benefit when new observations rarely change the required action. Kinetix shows useful call reductions but modest average success gains, and Visual-Delta achieves higher success in two ordinary-tracking settings.

A1. Thank you for highlighting the settings in which launch timing matters most. We agree that launch timing is particularly useful when important new information arrives during inference. As discussed in Section 1, our motivation centers on settings where new information can substantially change the optimal launch timing and action plan, such as highly dynamic environments or situations where safety is critical. Proposition 1 illustrates this value of information: waiting for a branch to be revealed improves value over launching immediately, but once the branch is known, further waiting lowers value (Section 4.1). The results you highlight also help identify settings in which the gains are smaller. We discuss these regimes in more detail below.

First, our empirical results support the value of launch timing when new information matters. On DOM Dynamic Adaptation (DA), a collision can change the target's direction during inference; learned timing improves success over immediate launch by 13.50–15.25 percentage points across the three delay laws (Section 6.2, Table 3). In Hidden-Danger, the scheduler launches at the first free decision after the reveal in 96.4% of episodes, and the same planner's success rises from 54.1% to 78.7% across 4,096 maps (Appendix D.1, Tables 5 and 7). On ordinary CR tracking, Visual-Delta is competitive and the ordering varies with the delay law (Appendix D.5, Table 11), consistent with your observation that a simple schedule can work well when new observations rarely change the required action. Together, these results support our motivation and analysis in Sections 1 and 4: when new information can change the appropriate action, learned launch timing can substantially improve performance.

Second, launch timing also yields larger improvements on harder tasks. This is intuitive: tasks already near 100% success leave little room for improvement. The average Kinetix gain for TrainingRTC is indeed approximately 1 percentage point, but this average obscures larger gains on harder tasks. Table 2 ranks task–delay pairs by independently measured baseline success for each backbone (Section 6.1). In the hardest quartile, learned timing raises success over immediate launch from 63.56% to 67.46% for TrainingRTC (+3.91 points) and from 72.49% to 74.23% for FutureRTC (+1.74 points). The gain on the harder TrainingRTC tasks also holds against a state-based trigger: in that quartile, learned timing exceeds State-Delta by 2.38 percentage points.

AuthorsOpenReview ↗

Response to Reviewer KGTc (Part 2/3)

Q2. What explains the largest Kinetix success gain occurring at zero delay, when a pending inference cannot block subsequent requests? Can existing rollout evidence distinguish benefits from avoiding stale inferences from benefits of less frequent replanning or buffer overwriting?

A2. The gains at zero delay are consistent with two considerations highlighted in prior work on action chunking: temporal consistency and imperfect prediction. These complement our main focus on the arrival of new information. BID [1] explains that independently sampled chunks can fail to preserve temporal dependencies in demonstrations, leading to inconsistent successive decisions; its analysis also accounts for errors in the policy's implicit dynamics model. Diffusion Policy [2] finds that executing eight actions before replanning performs best on most tested tasks, illustrating the balance between coherent action sequences and timely feedback. In our setting, these considerations help explain why launch timing can remain useful at zero delay: an imperfect generator's new prediction need not improve on the buffered plan, and replacing that plan can disrupt continuity. Choosing when to update therefore still balances preserving the current plan against incorporating new observations.

Theorem 1 (Section 5.2) connects to these considerations through its error decomposition. The term Bexp\mathcal{B}_{\mathrm{exp}} measures the mismatch between the expert-induced chunk policy and the optimal chunk policy under the same meta policy. As explained in the footnote on page 7 and Appendix C.2, the expert chunk policy is induced by an instantaneous expert; the example rollout construction does not condition on the existing buffer. Its chunks can therefore differ from those best suited to the ongoing execution plan, connecting expert chunk quality to the continuity considerations above. The term Bimit\mathcal{B}_{\mathrm{imit}} captures the additional mismatch between the learned generator and the expert chunk policy, which need not vanish at zero delay. Together, these terms account for both the quality of the expert-induced chunks within the execution protocol and the generator's imperfect imitation of them.

Our additional Kinetix walker experiments illustrate the value of choosing update times under both zero and nonzero inference delay. At Point0, the meta policy achieves 64.31% success versus 63.05% for Immediate, a gain of 1.25 percentage points. At Point2, a separate comparison using the original walker policy bank gives 59.18% success versus 53.87% for a baseline that independently samples voluntary waiting lengths from the meta policy's distribution, a gain of 5.31 percentage points. These results support useful timing gains in both regimes and are consistent with contributions from both the arrival of new information and the choice of when to update the current plan in closed loop.

AuthorsOpenReview ↗

Response to Reviewer KGTc (Part 3/3)

Q3. Suggested experiment to characterize practical scope: In an existing DOM task, vary the frequency of abrupt motion changes while holding the action generator and inference delay fixed. Compare learned timing, immediate launch, and Visual-Delta in both success and generator calls.

A3. Thank you for suggesting this insightful experiment. In our additional analysis, the meta policy achieved higher mean success than both Immediate and Visual-Delta in every frequency group under both delay distributions, while saving 5.1–9.0 full generator calls per episode relative to Immediate. The size of the gains across frequency groups varied with the definition of a motion change.

We extended the DOM Dynamic Adaptation (DA) comparison in Section 6.2 (Table 3) by grouping scenes according to how often the target changed velocity. With our definition of an abrupt change, the average success gain was larger in the higher-frequency scenes. With a 0.01 m/s threshold and a four-second window, the lower-frequency group showed larger gains, though both groups benefited. As you suggest, motion frequency provides a useful way to group scenes and characterize performance. These results support the value of the meta policy in dynamic scenarios where new information matters. We give the details below.

We report results for two delay distributions: Uniform4:8 and the measured Empirical distribution. To measure motion independently of the schedules, we recorded 100 target trajectories with the arm parked, with ten resets of each of the ten official DA scenes. An event began when the change in horizontal velocity over one control step at 25 Hz reached the threshold. We then grouped the original scenes by changes per second. Each cell below gives success rate / full generator calls per episode (400 episodes per schedule and distribution).

Abrupt changes (0.05 m/s). Seven scenes had two changes and three had three changes over the 12-second episodes:

Delay distributionMotion frequencyImmediateVisual-DeltaMeta policy
Uniform4:8Low (7 scenes)20.36% / 41.9021.79% / 33.9228.93% / 35.61
Uniform4:8High (3 scenes)47.50% / 35.7360.83% / 29.2872.50% / 28.09
EmpiricalLow (7 scenes)16.07% / 43.2316.79% / 34.4427.14% / 35.91
EmpiricalHigh (3 scenes)45.83% / 34.7034.17% / 33.4570.83% / 28.26

On average across the two delay distributions, the meta policy improved success over Immediate by 9.82 points in the lower-frequency group and 25.00 in the higher-frequency group. Its mean gains over Visual-Delta were 8.75 and 24.17 points, respectively. In the higher-frequency group under Empirical delay, the meta policy achieved 70.83% success versus 34.17% for Visual-Delta, a significant gain of 36.67 points, while using 5.19 fewer calls per episode. The estimated gain over Visual-Delta was 26.31 points larger in the higher-frequency group under Empirical delay.

Including smaller changes (0.01 m/s). The four-second window yielded five lower-frequency scenes and five higher-frequency scenes:

Delay distributionMotion frequencyImmediateVisual-DeltaMeta policy
Uniform4:8Low (5 scenes)31.00% / 41.0536.50% / 33.0050.00% / 33.01
Uniform4:8High (5 scenes)26.00% / 39.0530.50% / 32.0634.00% / 33.70
EmpiricalLow (5 scenes)24.00% / 43.4025.00% / 34.3546.00% / 34.42
EmpiricalHigh (5 scenes)26.00% / 37.9519.00% / 33.9534.50% / 32.81

The meta policy again achieved the highest mean success in both groups, with larger mean gains in the lower-frequency group. There, it significantly improved success over Immediate under both Uniform4:8 (+19.00 points) and Empirical (+22.00 points). Against Visual-Delta under Empirical delay, it gained 21.00 points at nearly identical call counts (34.42 vs. 34.35). These results also show that the meta policy's benefits extend to scenes with less frequent small changes.

Thank you again for your thoughtful questions and constructive suggestions. We hope these responses address your concerns, and we welcome further discussion.

References

[1] Liu et al. Bidirectional Decoding: Improving Action Chunking via Guided Test-Time Sampling. ICLR, 2025.

[2] Chi et al. Diffusion Policy: Visuomotor Policy Learning via Action Diffusion. Robotics: Science and Systems, 2023.

Dear Reviewer KGTc,

We have uploaded the revised PDF. Section 6.1 discusses how action continuity may explain the Kinetix gains at zero delay. Section 6.2 summarizes the DOM comparison across motion frequencies, and Appendix D.5 (Table 13) reports success rates and generator calls.

Thank you again for helping us strengthen the paper. We hope the revision addresses your concerns and welcome any further questions in the remaining two days of discussion.

Authors

Reviewer_95nTOpenReview ↗
Rating
1(−1)
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper looks at how to schedule inference of policies with action chunking, taking into account the latency of inference. To address this, the paper proposes a meta-policy that has the task of determining when to trigger inference, by either waiting or launching inference.

Critical strengths

Reasonably well explained motivation and method.

Critical weaknesses
  • Past work such as the citations below do something quite similar, and this problem has been well-discussed in many other papers.
  • Given that the problem is well-known, the experimental results are quite weak, with all success rates being within approximately 1% of the baseline without the meta policy. The only big gap is in number of policy calls, which is outweighed in practice by the need for an extra policy.

Most relevant and not cited that undercut the novelty:

When to Replan? An Adaptive Replanning Strategy for Autonomous Navigation using Deep Reinforcement Learning, https://arxiv.org/abs/2304.12046

Dynamic Execution Horizon Prediction for Chunk-based Robot Policies, https://arxiv.org/abs/2606.11408

Finding the Time to Think: Learning Planning Budgets in Real-Time RL, https://arxiv.org/abs/2606.26463

Questions for authors

N/A

6 comments
AuthorsOpenReview ↗

Response to Reviewer 95nT (Part 1/2)

We appreciate the time and effort you have devoted to reviewing our paper, as well as your suggestions and questions, which help us improve it. Below, we address each of your concerns in turn.

Q1. Past work, including the three papers mentioned in the review, does something similar.

A1. Thank you for pointing out these three closely related works. We will definitely cite and discuss all of them in the revised manuscript. These works advance adaptive path replanning [1], execution-horizon selection [2], and planning-budget allocation [3], respectively. Our work still stands out through two connected contributions that advance the formulation and theoretical understanding of inference-launch timing under latency. First, we formulate inference-launch scheduling for a fixed action-chunk generator under latency as a decision-stage SMDP, enabling standard reinforcement learning methods to learn when to launch. Second, the same formulation supports a theoretical analysis of both the value of launch timing and the performance of the learned timing policy. To our knowledge, ours is the first analysis in this setting to jointly characterize the non-monotonic value of launch timing as new information becomes available and provide a suboptimality bound separating expert chunk quality, action-generator imitation error, and timing-policy estimation error. We elaborate on these two aspects below.

  1. SMDP formulation and algorithmic adaptation. The interaction protocol in Section 3 and the transition kernel in Appendix A.1 turn launch scheduling into a well-defined control problem. A Wait advances the system by one physical step; a Launch occupies the generator while the system continues to execute buffered actions. Because these choices can induce different holding times, the composed reward Rk=∑t=hkhk+1−1rt(st,at)R_k=\sum_{t=h_k}^{h_{k+1}-1} r_t(s_t,a_t) assigns each intervening reward to the corresponding decision. These decision-stage transitions let us learn launch timing directly from physical-time trajectories using reinforcement learning. In particular, Algorithm 1 fits the Wait and Launch values with a Bellman target consisting of RkR_k plus the estimated continuation value at the next free decision (Section 3; Section 5.1).

  2. Timing-specific theoretical analysis. The same formulation also lets us analyze both the value of choosing when to launch and the performance of the learned timing policy. Proposition 1 (Section 4.1) shows why the launch decision matters even for a fixed generator: in the construction, waiting two steps for information to arrive improves value by at least H/5H/5 over immediate launch, while each additional step of waiting after the reveal costs one unit of value over the stated range. Under its stated assumptions, Theorem 1 (Section 5.2) bounds the fitted policy's suboptimality by three terms: expert chunk quality, imitation error of the action generator, and fitted-QQ estimation error for the timing policy. The proof uses Bellman telescoping at two time scales (Appendix C.3): over physical steps to bound the downstream effect of generator errors at launch, and over decision stages to bound the timing-policy error in terms of replay coverage and visitation shift.

Q2. The experimental results are weak, with all success rates being within ~1% of the baseline w/o the meta policy.

A2. We respectfully disagree that the success-rate gains are all within approximately one percentage point. The roughly one-percentage-point gain is the overall Kinetix average for TrainingRTC: across twelve tasks and eight delay laws, changing only the launch schedule raises success from 86.15% to 87.17% (Section 6.1, Table 1). The gains are substantially larger on harder Kinetix tasks, in Hidden-Danger, and on the DOM Dynamic Adaptation task with DynamicVLA.

First, in Hidden-Danger, learned timing raises the same Local Planner's success rate from 54.1% under Immediate to 78.7% across 4,096 maps, an increase of 24.54 percentage points (Appendix D.1, Table 5).

Next, Table 2 groups Kinetix task–delay pairs by independently evaluated baseline success (Section 6.1). In the hardest quartile, success rises from 63.56% to 67.46%, a gain of 3.91 percentage points (95% CI [2.39, 5.47]). The larger gain occurs where the default schedule has greater difficulty. Tasks already near 100% success leave little room for improvement, while harder tasks more clearly reveal the benefits of learned timing.

Finally, on DOM, we evaluate the larger DynamicVLA generator on the Dynamic Adaptation (DA) task (Section 6.2, lines 425–429). With the same generator, switching from immediate launch to learned timing raises success from 28.50% to 42.00% under Uniform4:8, 24.25% to 38.25% under Point6, and 25.00% to 40.25% under Empirical delay (Section 6.2, Table 3). These gains of 13.50, 14.00 and 15.25 percentage points show the value of the launch decision on this task.

AuthorsOpenReview ↗

Response to Reviewer 95nT (Part 2/2)

Q3. The only big gap is in the number of policy calls, and that advantage is outweighed in practice by the need for an extra policy.

A3. As discussed in A2, learned timing already yields substantial gains in success rate. In addition, fewer generator calls translate into lower estimated inference time on DOM, even after accounting for the extra timing policy.

On Kinetix, learned timing reduces full generator calls from 63.99 to 46.02 per episode (28.08%; Section 6.1, Table 1). On DOM, calls fall from 38.13 to 28.97 (24.03%) when averaged equally across the two tasks and three delay laws in Table 3 (Section 6.2). The DynamicVLA timing measurements below show how these reductions translate into lower estimated inference time per episode.

The timing policy reuses DynamicVLA's features and has 1.21M parameters, just 0.28% of the 430.19M-parameter backbone. On an RTX A6000, feature acquisition takes 18.22 ms and the timing decision itself takes 0.88 ms (0.37% of a full Launch). The complete Wait path takes 18.76 ms; a Launch also fills the KV cache (20.95 ms) and runs ten flow steps (193.51 ms), taking 234.12 ms in total (Section 6.2, Table 4; all times are medians).

For example, on DA/Empirical, learned timing increases prefix evaluations from 40.67 to 69.82 per episode and reduces full calls from 40.67 to 33.62 (Appendix D.5, Table 12). Counting prefill once and applying the measured mean path times to each subsequent launch and prefix-only check gives 9.62 s of estimated model time per episode for immediate launch versus 8.60 s for learned timing, corresponding to a 10.57% reduction in estimated model time.

Thank you again for taking the time to review our paper. We hope these responses address your concerns, and we welcome further discussion.

References

[1] Honda, Yonetani, Nishimura, and Kozuno. When to Replan? An Adaptive Replanning Strategy for Autonomous Navigation using Deep Reinforcement Learning. arXiv:2304.12046, 2023.

[2] Zhao, Bogdanovic, Sohal, Tao, Darvish, Aspuru-Guzik, Shkurti, and Garg. Dynamic Execution Horizon Prediction for Chunk-based Robot Policies. arXiv:2606.11408, 2026.

[3] Muppidi, Darwish, Cope, Henriques, and Foerster. Finding the Time to Think: Learning Planning Budgets in Real-Time RL. arXiv:2606.26463, 2026.

Dear Reviewer,

We would like to kindly follow up on our rebuttal and additional experimental results. As the discussion period is approaching its end (in around 3 days), we would greatly appreciate any further feedback and would be happy to address any remaining concerns or questions.

Thank you again for your time and consideration.

Reviewer_VVxCOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies when to launch inference for action-chunking policies under latency. The authors formulate inference timing as a meta-control problem using a semi-MDP and propose a fitted Q-iteration approach to learn Wait/Launch decisions. Theoretical analysis and experiments on Kinetix and DOM demonstrate the potential benefits of adaptive inference timing.

Critical strengths
  1. The paper addresses a practical and important problem in real-time action chunking. Treating inference timing as a separate control decision is a meaningful perspective.

  2. The experiments cover different action generators and latency settings. The results show improvements in success rates while reducing action-generation calls, particularly in challenging dynamic environments.

Critical weaknesses
  1. The applicability of the theoretical guarantee is unclear. Theorem 1 requires positive sample counts for all decision stages and meta-actions. However, since a launch can span multiple physical time steps, some decision-stage indices may be unreachable under certain delay settings. It is unclear how the stated coverage condition applies in these cases. Moreover, the relationship between the stage-wise fitted Q-iteration in Algorithm 1 and the neural-network training procedures described in the appendix is not sufficiently explained.

  2. The experimental advantages over simple heuristics are not always clear. In Table 11, Visual-Delta achieves higher success rates on DOM Closed-loop Reactivity under two latency settings. Furthermore, reducing full action-generation calls does not necessarily imply proportional reductions in total computation, since the proposed method requires additional observation-prefix evaluations. More comprehensive efficiency comparisons and uncertainty estimates would strengthen the results.

Questions for authors
  1. How is the positive sample coverage condition in Theorem 1 satisfied when inference occupies multiple physical time steps?

  2. How does the practical neural-network training procedure correspond to Algorithm 1, and does the theoretical guarantee apply to this implementation?

  3. Can the authors further clarify the advantages of learned timing over heuristic scheduling in terms of both success rate and overall computational cost?

2 comments
AuthorsOpenReview ↗

Response to Reviewer VVxC (Part 1/2)

We thank you for the time you spent reviewing our paper and for recognizing our framework and empirical performance. We also appreciate your detailed questions, which help us improve the completeness and clarity of the paper. We now summarize the points made and address each question in turn.

Q1. Theorem 1 requires positive sample counts for all decision stages and meta-actions. How is this condition satisfied when a launch spans multiple physical time steps and some decision-stage indices may be unreachable?

A1. Thank you for raising this important question. We design our data-collection behavior policies to encourage coverage of feasible Wait/Launch choices across reachable decision stages.

We agree that coverage is an important consideration in practical training design. In our formulation, as in the classical options/SMDP framework [1], a multi-step Launch advances physical time, while the decision index kk increases by one at the next free decision (Section 3). Accordingly, the coverage requirement concerns feasible meta-actions at reachable decision stages. Theorem 1 makes the dependence on sample counts and visitation mismatch explicit through nk,un_{k,u} and CkC_k. A well-designed exploratory behavior policy can ensure positive sampling probability for all reachable stage–action pairs. We thus design our behavior policies to provide coverage of feasible Wait/Launch choices across reachable decision stages.

Our final collection design includes fixed waiting, balanced Wait/Launch exploration, and Slack-Uniform in Kinetix (Appendix D.3). DOM collection starts with the default DynamicVLA schedule and fixed-wait episodes, then uses Slack-Uniform (Appendix D.5). These behaviors provide experience at different launch times and buffer states. Additionally, we conducted a matched-budget ablation on the Kinetix trampoline setup. The mixed collection described above achieved 88.80% success versus 73.31% with Immediate-only collection: an improvement of 15.49 percentage points. These results further support the importance of behavior-policy design and action coverage, as well as our use of mixed collection to provide experience for both timing choices.

As discussed in Section 5.1 (lines 332–338), the offline procedure can also be extended with an online loop: deploy an exploratory softmax version of the learned policy and append newly collected decision transitions to replay. This additional experience can further improve the coverage of the data used to train the neural network.

Q2. How does the practical neural-network training procedure correspond to Algorithm 1, and does the theoretical analysis apply?

A2. Our implementation for training the meta policy follows the decision-stage SMDP and fitted-QQ structure of Algorithm 1. Each replay transition connects successive free decisions and accumulates rewards over the intervening physical steps, including the pending interval after Launch. A shared QQ-network predicts Wait/Launch values from state or visual features, the action buffer, and timing features (Appendices D.3 and D.5); this parameter sharing is already noted in Section 5.1 (lines 329–331). In Kinetix and DOM, we pool these transitions and fit the network using targets spanning up to eight decision transitions, holding the target network fixed during each fitting iteration and optimizing on minibatches. The resulting policy greedily selects among feasible actions.

Since our implementation follows the algorithm, our theoretical analysis applies to it. Our implementation satisfies the formulation's fixed-generator and action-feasibility conditions. Its replay construction preserves the SMDP transition and reward structure, and our behavior-policy mixture supplies training experience for both timing choices. Theorem 1 permits neural critic classes and provides a finite-sample guarantee for stage-wise fitted-QQ regression under its stated assumptions. The analysis in Appendix C further shows that small Bellman residuals on well-covered decision contexts suffice to control scheduling error, accounting for visitation mismatch. Together, these results provide a theoretical foundation for our training design by connecting value-estimation accuracy and data coverage to the performance of the learned timing policy.

AuthorsOpenReview ↗

Response to Reviewer VVxC (Part 2/2)

Q3. The advantages over simple heuristics are not always clear: Visual-Delta achieves higher success on DOM Closed-loop Reactivity under two latency settings (Table 11), and fewer full calls need not reduce total computation proportionally once observation-prefix evaluations are counted. Can the authors clarify the comparisons in success rate and overall computational cost and provide uncertainty estimates?

A3. We first clarify the success-rate advantages from two angles: newly revealed information and task difficulty, including uncertainty estimates for Hidden-Danger and the harder Kinetix groups. We then show that estimated model time per episode decreases even after accounting for the meta policy and its additional observation checks.

First, the success-rate comparisons show two patterns. Learned timing has a clearer advantage over Visual-Delta when new information arriving during inference can change the appropriate action chunk, and its gains over Immediate are larger on harder Kinetix tasks.

  1. When new information matters. Closed-loop Reactivity (CR) tests ordinary tracking, where Visual-Delta and the learned meta policy achieve similar success rates, with their relative ordering varying across delay distributions. Dynamic Adaptation (DA) adds abrupt motion changes, such as collision-induced deflections, that provide useful new information for the timing decision (Section 6.2, lines 425–429). A call launched just before a deflection uses the old observation and remains pending as the object changes direction; waiting can let the generator use the new direction (Section 6.2, lines 450–455). On DA, learned timing consistently achieves higher mean success than Visual-Delta under all three delay laws. Under Empirical delay, the meta policy achieves 40.25% success, compared with 22.00% for Visual-Delta and 25.00% for Immediate, with respective gains of 18.25 percentage points (95% CI [3.75, 35.75]) and 15.25 percentage points (95% CI [1.00, 31.50]) (Appendix D.5, Table 11). Hidden-Danger shows this behavior at an identifiable event: across 4,096 maps, the scheduler launched at the first free decision after the danger was revealed in 96.4% of episodes (95% map-bootstrap CI [95.8%, 96.9%]). Changing only the launch schedule raised the same planner's success from 54.1% to 78.7% (Appendix D.1, Tables 5 and 7).

  2. Task difficulty. Table 2 groups Kinetix task–delay pairs by independently evaluated baseline success (Section 6.1). With TrainingRTC, learned timing improves success over Immediate by 3.91 pp in the hardest quartile (95% CI [2.39, 5.47]); calls per episode fall by 33.40%. These intervals resample task–delay pairs and paired episodes. The harder tasks also distinguish learned timing from a state-based trigger: in the hardest quartile, it improves success over State-Delta by 2.38 pp [0.91, 3.94] at nearly identical call counts (69.93 versus 69.85 per episode; see Appendix D.3, Tables 8–9 for the full-panel comparison).

Second, the meta policy reduces estimated total model time per episode even after accounting for its own decisions and observation checks. An extra observation check is much cheaper than a full generator call. On an RTX A6000, the 1.21M-parameter timing policy reuses features from the 430.19M-parameter DynamicVLA backbone and takes 0.88 ms to decide. Including feature acquisition, a Wait takes 18.76 ms, versus 234.12 ms for a Launch that fills the KV cache and runs the flow model (medians; Section 6.2, Table 4). On the DA/Empirical episodes above, learned timing uses 69.82 prefix evaluations and 33.62 full calls per episode, while Immediate uses 40.67 of each (Appendix D.5, Table 12). Counting prefill once and applying the measured mean path times to every subsequent launch and prefix-only check, we estimate 8.60 s of model time per episode for learned timing versus 9.62 s for Immediate. This 10.57% reduction includes the extra observation checks and covers computation from prepared inputs to returned decisions or chunks (Appendix D.5).

Thank you again for reviewing our paper. We hope these responses address your concerns, and we welcome further discussion.

References

[1] Sutton, Precup, and Singh. Between MDPs and semi-MDPs: A Framework for Temporal Abstraction in Reinforcement Learning. Artificial Intelligence, 1999.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal Summary

Dear AC and Reviewers,

Thank you for guiding and participating in the discussion. We summarize the main feedback and how we addressed it in the rebuttal and revised manuscript.

Taken together, the reviews highlight the importance of the problem and the merits of our approach as a reasonable, well-motivated formulation. VVxC and KGTc also recognized that learned timing achieves higher success rates with fewer generator calls. The roughly one-percentage-point gain noted by 95nT describes the overall TrainingRTC average on Kinetix. On DOM Dynamic Adaptation, the gains over immediate launch were 13.50, 14.00, and 15.25 percentage points under Uniform, Point6, and Empirical delays, respectively. Hidden-Danger success rose from 54.1% to 78.7%, and the hardest Kinetix quartile gained 3.91 percentage points (95% CI [2.39, 5.47]).

Our rebuttal clarified how the decision-stage SMDP models real-time inference and how our training procedure implements it. To our knowledge, ours is the first analysis in this setting to jointly characterize the non-monotonic value of launch timing and bound the suboptimality of a learned timing policy. Section 2 now discusses all three papers suggested by 95nT. Following KGTc’s suggestion, Section 6.2 and Appendix D.5 also compare DOM scenes by motion-change frequency. Learned timing achieved higher mean success than both baselines in both frequency groups under both stochastic delay laws. We appreciate your time and the suggestions that helped strengthen the paper.

Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.