- Rating
- 2
- Confidence
- 4(+1)
- Informativeness
- 2
- Presentation
- 2(−1)
- Soundness
- 1
Read review
Paper summary
This paper proposes PaceVLA, a framework for accelerating VLA execution through adaptive temporal action quantization. The key idea is to dynamically adjust action granularity and prediction horizon according to the precision requirements of different manipulation phases. Specifically, the method introduces mode-specific action decoders built upon a shared action expert, together with a quantizability judge and a context-conditioned router for selecting execution modes. The quantizability judge is supervised using LVLM-generated annotations, while the router is trained based on mode-specific action prediction errors. Experiments across three simulation benchmarks and six real-world tasks demonstrate improved execution efficiency while largely preserving task success rates.
Critical strengths
-
Practical and well-motivated problem. The paper addresses an important limitation of imitation-learned robot policies: inefficient execution inherited from conservative human demonstrations. Adaptive temporal granularity is a reasonable approach to balancing execution speed and control precision.
-
Extensive experimental evaluation. The experiments cover multiple simulation benchmarks, two VLA backbones, and six real-world tasks across three robot platforms, including dexterous and humanoid manipulation. The real-world results are particularly valuable and demonstrate meaningful improvements in execution efficiency.
-
Relatively lightweight integration. The shared action expert and mode-specific decoders provide a practical way to support multiple temporal resolutions without duplicating the entire policy. The ablation studies demonstrate the contributions of the main components, and the additional inference overhead is small.
Critical weaknesses
-
Fundamentally questionable routing objective. The router is supervised by a target distribution constructed from mode-specific Flow Matching prediction errors, i.e., . However, these errors are not directly comparable across different temporal granularities and prediction horizons. Shorter prediction horizons are inherently less affected by long-term uncertainty, while temporally coarsened actions often discard fine-grained motion details and can be easier to predict. Consequently, a mode may receive a lower prediction loss simply because its prediction target is easier, rather than because it leads to better execution. Averaging losses over valid action elements does not resolve this fundamental issue. More importantly, prediction accuracy does not necessarily correlate with execution success or efficiency. The proposed routing objective therefore lacks a convincing justification, and the experiments do not establish whether it learns meaningful mode preferences or merely exploits differences in prediction difficulty.
-
Insufficient experimental disentanglement. The method jointly changes action granularity and prediction horizon, but their individual contributions are not isolated. Moreover, the SAIL baseline is evaluated with action scheduling disabled, and the accelerated policies benefit from controller gain adjustments while the base VLA retains its default gains in simulation. These choices complicate the interpretation of the reported improvements.
-
Limited methodological novelty. The framework mainly combines existing ideas, including temporal action quantization, multi-horizon action prediction, and expert routing. Although the integration is practically useful, the technical contributions appear relatively incremental. In particular, the routing formulation does not provide a principled objective for optimizing the speed–accuracy trade-off.
Questions for authors
-
How are the Flow Matching losses calibrated across different modes? Shorter and coarser action predictions can naturally have lower errors. Have the authors analyzed whether the routing targets are systematically biased toward certain modes? How strongly do these losses correlate with actual execution success and efficiency?
-
Can the authors compare the learned router with simple alternatives, such as always selecting the shortest horizon within the group chosen by the quantization judge, or using uniform/random routing? An oracle router based on rollout outcomes would also be informative.
4 comments
[Response to Reviewer z39S (2/2)]
[W2] Disentangling temporal quantization from horizon selection, the disabled action scheduling of SAIL, and the controller gains.
[A3] The speedup comes from temporal quantization, and neither the disabled action scheduling of SAIL nor the controller gains work in PaceVLA's favor.
Restricting, at evaluation time, the modes the router may select on the same checkpoint separates the two factors. Without the quantized modes the speedup disappears, and without horizon selection the policy executes poorly predicted chunk ends and success drops (RoboCasa-Kitchen; Tables 6a and 6b in Section 4.3):
\hline \text{Variant} & \text{SR} \uparrow & \text{TPR} \uparrow & \text{ES} \uparrow \newline \hline \colorbox{#EDE5F7}{PaceVLA (ours)} & \colorbox{#EDE5F7}{\textbf{66.9}} & \colorbox{#EDE5F7}{\textbf{2.75}} & \colorbox{#EDE5F7}{\textbf{1.14}} \newline \text{No horizon selection} & 62.0 & 2.33 & 0.96 \newline \text{No quantization} & 64.1 & 2.05 & 0.99 \newline \hline \end{array}$$ Action scheduling only hides inference latency when inference and execution run asynchronously. The simulation benchmarks and the Franka and OpenArm tasks run synchronously, so it has nothing to hide there, and on the asynchronous ALLEX task every method, SAIL included, used the same real-time chunking (Appendix C.6). A quantized action moves further per control step, so the higher arm gain is part of executing quantized actions. The base model with the same gains loses success, because its chunks are not produced for those gains (Table 14 in Appendix E.3): $$\begin{array}{lccc} \hline \text{RoboCasa-Kitchen} & \text{SR} \uparrow & \text{TPR} \uparrow & \text{ES} \uparrow \newline \hline \text{Base model, default gains} & 65.7 & 1.82 & 1.00 \newline \text{Base model, PaceVLA's gains} & 63.3 & 1.96 & 0.96 \newline \colorbox{#EDE5F7}{PaceVLA (ours)} & \colorbox{#EDE5F7}{\textbf{66.9}} & \colorbox{#EDE5F7}{\textbf{2.75}} & \colorbox{#EDE5F7}{\textbf{1.14}} \newline \hline \end{array}$$ --- **[W3] Limited methodological novelty, and no principled objective for the speed-accuracy trade-off.** **[A4]** **PaceVLA introduces a capability that prior acceleration methods lack:** one shared action DiT is trained on the same motion at two temporal granularities, and learning both makes the policy more accurate than the base model it was fine-tuned from, not just faster. On RoboCasa-Kitchen the full model reaches 66.9 against 65.7 for the base model at an ES of 1.14, while removing the quantized decoders and quantizing the base prediction instead drops it to 63.3 (Table 5a). Temporal quantization [1] fixes one granularity for the whole trajectory, and MoH [2] fuses heads of several horizons into a single granularity. PaceVLA's mode-specific decoders, quantizability-supervised judge and context-conditioned router are designed together to switch between granularities within a task, adding only about 2 ms of inference latency (Table 8; Section 2 and Appendix E.5). This design is what lets acceleration keep success. On the real robots, fixed quantization loses 21.9 to 28.2 points on Air Fryer and Stack Cup, while PaceVLA shortens execution by 33 to 45 percent with success close to the base policy (Table 2). The speed-accuracy trade-off has a dedicated objective for each side. The judge learns when quantized execution is safe, and its threshold sets how often it is used (Table 6c). The router learns which mode predicts most accurately within that group, and its objective has no term for speed, so a router trained without the judge does not accelerate execution (ES 1.01, Table 5a). [1] Tang et al., VLASH: Real-Time VLAs via Future-State-Aware Asynchronous Inference, arXiv 2025 [2] Jing et al., Mixture of Horizons in Action Chunking, arXiv 2025Thank you for the detailed rebuttal and additional experiments. However, my main concern regarding the theoretical justification of the routing objective remains unresolved.
- The cited imitation learning theory does not justify comparing Flow Matching losses across different action horizons.
The authors argue that lower prediction error leads to better closed-loop performance, citing Xu et al. (2020), Block et al. (2023), and Foster et al. (2024). However, these references establish substantially different results:
- Xu et al. (2020) derive an upper bound on the policy value gap based on the KL divergence between the learned and expert policies.
- Block et al. (2023) establish bounds on imitation trajectory discrepancies using distributional errors under specific low-level stability and policy continuity assumptions.
- Foster et al. (2024) analyze the relationship between behavior cloning with log loss, sample complexity, and task horizon. Notably, their Figure 6 also shows that MSE validation loss can be poorly correlated with rollout reward, even within a fixed task setting.
None of these works establishes that Flow Matching regression losses are directly comparable across different action horizons, let alone that their relative ordering predicts closed-loop performance.
The fundamental distinction is between bounding the performance gap using an imitation error under a given formulation and using errors from different prediction targets and execution horizons to rank competing policies. Even if lower imitation error yields a tighter theoretical upper bound, this does not imply better actual performance. Furthermore, different action horizons change the prediction targets, replanning frequency, and closed-loop error accumulation.
The authors' routing objective relies precisely on this unproven relationship. Therefore, I find the theoretical argument in the rebuttal insufficient and potentially misleading. The authors should either provide a theoretical justification specifically addressing cross-horizon loss comparability or acknowledge that this is an empirical heuristic rather than a theoretically supported routing criterion.
- The rebuttal claims that losses are compared only within the same quantization group, but Eq. (2) still defines the KL supervision over all K modes; restricting selection to a group at inference does not eliminate cross-granularity effects during training.
Overall, while I appreciate the additional experiments, I remain unconvinced by the central methodological justification.
Reply Rebuttal Comment by Authors
We appreciate the reviewer's response and the careful reading of the cited works. We acknowledge that these works do not directly justify comparing flow matching losses across different action horizons in robotic manipulation tasks. Our intent in citing these works was only to support a general point that a lower imitation error during training reduces the possibility to failure at test phase. We therefore address the reviewer's concerns directly below, with additional analysis.
(1) Comparing Flow Matching losses across action horizons. We note that the two losses within a group have the same scale, because each is an average of per-step errors and does not grow with the horizon. One might expect the longer horizon to have the larger loss in most chunks, since its later steps are more affected by long-term uncertainty, as the reviewer noted in the initial review. However, as reported in our initial response, the shorter horizon has the lower loss in only 53% of the fine-grained chunks and 55% of the quantized chunks, so which mode wins depends on the chunk, not on the horizon length.
We therefore train the router to predict this comparison for each chunk, and test it in closed loop. On the same checkpoint (RoboCasa-Kitchen, 24 tasks x 50 episodes), we replaced the learned router with simpler rules (random selection, short-horizon only, long-horizon only, and the router's less preferred horizon). As a result, every alternative lowers SR, TPR and ES (see the Table below). This indicates that if the policy always selects the longer action horizons, error accumulates even on the next replanning step. Likewise, if the policy always selects the shorter action horizons, it loses the chance to select the longer plans that were predicted well on the given phase. This makes the policy replan more often, the inconsistency between consecutive chunks grows, and more error accumulates over the full trajectory, which lowers the success rate.
\hline \text{Horizon choice within the group} & \text{SR} \uparrow & \text{TPR} \uparrow & \text{ES} \uparrow \newline \hline \text{Learned router (ours)} & 66.9 & 2.75 & 1.14 \newline \text{Uniform random} & 64.2 & 2.64 & 1.05 \newline \text{Always the shortest horizon} & 63.9 & 2.57 & 1.03 \newline \text{Always the longest horizon} & 62.0 & 2.33 & 0.96 \newline \text{Inverted router (less preferred horizon)} & 63.8 & 2.55 & 1.02 \newline \hline \end{array}$$ **(2) KL supervision over all K modes.** We note that training over all $K$ modes does not change which horizon the router learns to prefer within a group, and it becomes clear when training and inference are separated. In our earlier response, we wrote that losses are compared only within a group. What we meant there was how the router is used at inference, and we should have said so more clearly. During training, the reviewer notes that the KL supervision of Eq. (2) normalizes its target over all $K$ modes, so the losses of the other quantization group also affect the target values. However, they give more weights on the targets of the two modes in the same group by the same factor, so the preference of the two modes is decided by their own two losses within the group. The losses of the other group only change the target probability of their group, not changing the other group. At inference, the quantization judge decides the group, and the router only compares the two horizons inside it. The router's preference between the groups is not used, and its choice within the selected group follows the same preference as a target computed only within that group.
[Response to Reviewer z39S (1/2)]
Dear Reviewer z39S,
We sincerely appreciate your insightful comments and efforts in reviewing our manuscript. We address your comments and questions one-by-one below. In the revised draft, we mark our major revisions as “purple”.
[Q1, W1] Whether the per-mode losses are comparable across horizons and granularities, whether the router target is biased toward easier targets, and how the losses relate to execution success and efficiency.
[A1] The routing targets are not biased toward shorter or coarser modes, and following them improves both success and efficiency. Losses are compared only between the two horizons of the same quantization level, so coarser predictions are not compared with finer ones, and the judge decides quantization (Section 3.4).
The target is a softmax of the negative losses, the soft assignment analyzed in annealed multiple choice learning [1], and the balance term is the load-balancing loss of sparse mixture-of-experts routing [2], which keeps any single mode from being selected everywhere. Within each group, the shorter horizon has the lower loss in 53 percent of the chunks among fine-grained modes and 55 percent among quantized modes (9,747 demonstration chunks, RoboCasa-Kitchen; Appendix E.2).
A lower per-step prediction error keeps the executed actions closer to the demonstration, and imitation learning theory bounds the closed-loop performance gap by this error [3, 4, 5]. In closed loop, routing by the losses gives higher success and ES than random or fixed-horizon routing (66.9 and 1.14 vs 62.0 to 64.2 and 0.96 to 1.05, Q2).
[1] Perera et al., Annealed Multiple Choice Learning: Overcoming limitations of Winner-takes-all with annealing, NeurIPS 2024
[2] Fedus et al., Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity, JMLR 2022
[3] Xu et al., Error Bounds of Imitating Policies and Environments, NeurIPS 2020
[4] Block et al., Provable Guarantees for Generative Behavior Cloning: Bridging Low-Level Stability and High-Level Behavior, NeurIPS 2023
[5] Foster et al., Is Behavior Cloning All You Need? Understanding Horizon in Imitation Learning, NeurIPS 2024
[Q2] Learned router versus simple alternatives such as always-shortest-in-group, random routing, or a rollout-based oracle.
[A2] The learned router gives higher success and ES than every simple alternative, so the gain does not come from the judge alone. We replaced only the horizon choice within the judge's group on the same checkpoint (RoboCasa-Kitchen; Table 6a in Section 4.3):
\hline \text{Horizon choice within the group} & \text{SR} \uparrow & \text{TPR} \uparrow & \text{ES} \uparrow \newline \hline \colorbox{#EDE5F7}{Learned router (ours)} & \colorbox{#EDE5F7}{\textbf{66.9}} & \colorbox{#EDE5F7}{\textbf{2.75}} & \colorbox{#EDE5F7}{\textbf{1.14}} \newline \text{Uniform random} & 64.2 & 2.64 & 1.05 \newline \text{Always the shortest horizon} & 63.9 & 2.57 & 1.03 \newline \text{Always the longest horizon} & 62.0 & 2.33 & 0.96 \newline \hline \end{array}$$ Always taking the longer horizon keeps executing the end of a chunk even when it is poorly predicted, so errors accumulate until the next replan. Always taking the shorter one replans in the middle of motions that were predicted well. The learned router keeps the long plan when its predicted error is low and replans early when it is not. A rollout-based oracle would need to re-run every chunk under every mode from the same state, which is not possible in closed loop.