- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper addresses the stale-observation problem in asynchronous execution of action-chunking flow-based VLAs. PACT-VLA generates N candidate chunks from the inference-start observation , and selects one using the pre-commit observation , i.e., one control step before the planned chunk switch. Selection uses FlowRes, the flow-matching velocity residual of each candidate under the frozen policy conditioned on , computed with shared verification noise over a few flow times. Encoding of overlaps with candidate generation and all candidates are scored in one batch, so on the evaluated hardware the cycle stays within the same 30 Hz delay bucket (d=2) as single-candidate generation. Experiments on Kinetix (vs. many delay-handling baselines, and on top of VLASH/DEFLECT/REMAC), LIBERO (π0.5 and GR00T N1.7), and three real-robot bimanual tasks with π0.5 show gains that grow with delay (e.g., +15.0 pts over RTC at d=4 on Kinetix; 60%→82% on real robots).
Critical strengths
- Simple, training-free, and complementary. The method reuses the frozen policy as the verifier, requires no extra model, and improves several existing delay-robust methods (Table 2). This makes it practically attractive and easy to adopt.
- Broad empirical coverage with a targeted control. Table 1 compares against a wide set of delay-handling methods under a common checkpoint and protocol. The same-pool comparison that changes only the scoring observation ( vs. ; Tables 7–9, Old-FlowRes on the robot) directly supports the paper’s central claim, and the gain consistently grows with delay.
Critical weaknesses
-
Limited explanation of why the method works. The paper describes how PACT-VLA is implemented in detail but gives little analysis of why FlowRes under the pre-commit observation should identify better chunks. The authors themselves note that FlowRes "is a policy-consistency score rather than a direct estimate of task success" and defer its justification to experiments (l.272–274). There is no discussion of its relation to conditional likelihood, no analysis of when the candidate pool (generated from ) contains a chunk that fits (e.g., ablations on or candidate diversity), and no qualitative examples of selected vs. rejected candidates. Relatedly, the single-decision gain in the fixed-pool evaluation is small (+0.56 pts, D.2), while episode-level gains reach +15 pts (Table 9). Without an explanation of this gap, it is hard to attribute the improvement to the stated mechanism.
-
Unclear temporal alignment in FlowRes. Candidates are generated from , so index corresponds to step . The velocity predicted under presumably corresponds to a chunk starting at . The paper does not state whether candidates are re-indexed before computing Eq. 6. This matters most on Kinetix, where the largest gains are reported: with RTC-guided proposals the first actions are pulled toward the committed prefix and are nearly identical across candidates, yet with , and they carry ~94% of the score weight. If no re-indexing is done, it is unclear what the score actually discriminates. This is a soundness question rather than a presentation issue.
Questions for authors
Clarifications (could change my assessment)
-
Is the candidate re-indexed (e.g., comparing with the prediction at index ) before computing FlowRes? Please specify the delay-simulation protocol for each benchmark (a time-indexed queue as in RTC, or executing each new chunk from index 0). For RTC-guided candidates, is the verification query also conditioned on the committed prefix?
-
How do you reconcile the small single-decision gain in D.2 (+0.56 pts) with the large episode-level gain in Table 9 (+15 pts at d=4)?
-
Statistical reliability of the real-robot results. With 50 trials per method, the standard error of each success rate is roughly 5–7 pts. Over Old-FlowRes, the averaged gain (+17.3 pts, z≈3.5) and Pouring Coffee Beans (+28 pts, z≈3.4) are clearly significant, but Toasting Bread (+14) and Loading Dishwasher (+10) are within about 1.5 standard errors of the difference. The varying baseline rankings across tasks are also likely within noise; for example, Old-FlowRes is best among baselines on Dishwasher but below Naive Async on Coffee Beans. Please report:
- confidence intervals;
- the trial protocol (ordering, initial-state matching);
- how the VLASH checkpoint was trained;
- how PACT's hyperparameters (, the 25-step scoring window, flow times) were chosen, and whether they were tuned on the evaluation tasks.
-
What is the effective delay on the robot? With , the pre-commit observation is only ~33 ms newer than , yet the gain over Old-FlowRes (+17.3 pts) is much larger than the d=2 gains in simulation (+1.1 on LIBERO, +4.0 on Kinetix).
Missing comparisons (would strengthen the paper)
-
LIBERO and the real-robot experiments compare only against single-candidate async (plus VLASH on the robot). Comparisons against RTC in these settings, and against test-time selection methods (e.g., RoboMonkey, MG-Select, τ0-WM), would help. Prior work has also used the latest observation during chunk execution through lightweight learned residual correction without candidate switching (e.g., arXiv:2509.23224). How does pre-commit selection compare with such per-step correction, in both success and latency?
-
Sensitivity to , , the flow-time set, and , which differ across models (Table 6). Since simulations fix externally and do not reflect the added compute, please also report latency for and for GR00T.
We thank the reviewer for recognizing the training-free design and controlled comparisons. We have highlighted the revisions in blue and hope our responses address your concerns.
Comment/Question:
Answer: FlowRes favors actions consistent with demonstrated behavior under the updated state, reducing the mismatch caused by stale observations. Following this helpful suggestion, we expanded Section 3.2 to discuss the mechanism and conditional likelihood. The flow-matching objective provides a likelihood-related interpretation, while FlowRes uses its residual as a policy-consistency surrogate rather than an exact likelihood estimate. Appendix D.2 (Table 10) shows that 14.1% of decisions with large candidate-outcome differences contribute 56.8% of the gain. For such candidate pairs, pre-commit scoring identifies the better candidate 56.2% of the time, compared with 51.7% using the original observation. Appendix G links larger gains to higher single-candidate failure rates, while Appendix J (Fig. 9) shows candidate rankings and outcomes.
Comment/Question:
Answer: Kinetix candidates are re-indexed before FlowRes scoring, so the 94% positional weight falls on post-switch actions rather than the committed prefix. We remove the first d already-committed positions, which the new chunk cannot change, then shift the executable suffix forward and zero-pad the sequence. Padded positions retain the remaining weight and can contribute nonzero, candidate-dependent residuals. We clarify this in Appendix B.
Comment/Question:
Answer: Verification uses the frozen policy without RTC prefix guidance, while action indexing follows each benchmark's execution protocol. Kinetix uses RTC's time-indexed queue. LIBERO executes new chunks from index 0, while the real robot uses LeRobot's timestep-based queue merging. We use o(t+d−1) as the latest observation that leaves one control step for verification before the switch at t+d. The execution protocols are detailed in Appendices B and H.
Comment/Question:
Answer: Full-rollout gains reflect repeated selections, while the fixed-pool experiment isolates one decision. Appendix D.2 (Table 11) varies selection frequency: selecting only once adds 0.7 points, every second decision adds 8.0, and every decision adds 15.6. This demonstrates accumulation across decisions.
Comment/Question:
Answer: The 17.3-point average gain over Old-FlowRes is statistically supported, with a 95% CI of [+7.7, +27.0] (Appendix H, Table 13). Task-level CIs are also reported. All methods use LeRobot's default asynchronous execution settings, with rotated trial blocks and matched initial placements. Appendices E and H detail checkpoint training and deployment parameters, which were not tuned on the real-robot tasks.
Comment/Question:
Answer: The larger real-robot gain over simulation is explained by the longer execution horizon affected by each selection, despite both operating at d=2. At 30 Hz, the observation is 33 ms newer. Robot chunks are requested every 25 steps, and each choice shapes ~1.6 s of motion, versus replanning every 2–8 actions in simulation (Appendix H).
Comment/Question:
Answer: We have added RTC comparisons on LIBERO and real robots, alternative-selector controls, and A2C2 success and latency measurements. On LIBERO, RTC + PACT-VLA improves success from 93.2% to 94.2% at d=4 and from 73.8% to 76.1% at d=20 (Appendix M). On real robots, PACT-VLA reaches 82.0% versus RTC's 67.3% (Fig. 5). On GR00T, PACT-VLA achieves 56.4% versus 51.5% for τ0-WM's RCS (Table 7). RoboMonkey reports ~650 ms for 16 candidates under a different setup, while MG-Select's action-token score does not directly apply to flow policies (Appendix D). For A2C2, reproduced using the authors' code on our shared Kinetix policy, PACT-VLA achieves higher success at all four delays, from 91.1% versus 78.8% at d=1 to 77.2% versus 75.8% at d=4 (Table 19). On RTX 5090, P99 latency is 43.4 ms for generation plus the first A2C2 correction versus 60.2 ms for PACT-VLA with N=4, both retaining d=2 at 30 Hz (Table 20). A2C2's adapted correction head runs at every control step, while PACT-VLA verifies once per action chunk and requires no additional training.
Comment/Question:
Answer: The revision reports sensitivity, candidate-count, and GR00T P99 latency results. Fig. 7 varies flow times, step weights, scored positions, and action dimensions. Every tested setting beats its no-selection baseline. Appendices G/I vary N. Tables 4/16 cover N=1,2,4,8 on three GPUs. On RTX 5090, GR00T takes 34.1 ms (N=1) versus 62.5 ms (N=4), both d=2 at 30 Hz. Appendix K covers 25/20 Hz.