- Rating
- 3
- Confidence
- 4(+1)
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
While policy is computing actions, scene and robot’s state continue to change, and inference latency can cause the action to correspond to outdated observations.
This work establishes Dynamic-LIBERO and Dynamic-RoboTwin benchmarks and proposes PACE, a caching acceleration method, to investigate how faster inference can improve dynamic manipulation.
Critical strengths
The motivation is sound, and dynamic manipulation is a valuable research topic; the benchmark is designed as a simple modification of commonly used benchmarks, making it easy to understand and use; PACE improving dynamic manipulation through accelerated reasoning is intuitive.
The benchmark is expected to facilitate research in this area within the robotics community;
Critical weaknesses
Some of figures and discussion are rather rough, making it difficult to read for readers unfamiliar with the background knowledge relevant to this work; for example, Figure 3 and Section 6.2. I have no objection to using LLMs to generate figures or text, but they should be reviewed and polished by humans to ensure that it is clearly expressed and easy to understand.
Experiments in simulation are based on a limited number of policies, real-world experiments are also conducted with few repetitions; since the method does not involve large scale training, consideration should be given to supplementing the work with additional experiments to support its main idea.
PACE’s acceleration stems from a combination of various approaches, such as reusing different types of computations; however, the specific contributions of each of these approaches to the overall performance require further discussion.
Questions for authors
It is reasonable to implement a new benchmark by adding a layer of lightweight modifications to an existing benchmark; however, considering portability to other benchmarks, such as RoboDojo, it would be helpful to discuss what interface support this benchmark requires for underlying benchmark.
See weakness; consider providing some discussion or additional details for each point to support the main idea of the work.
3 comments
Thank you to the author for the detailed response; the explanation has addressed my main concerns. I will maintain my current positive rating and increase the confidence level for that rating.
Thank you for your thoughtful review and constructive feedback. We are glad that our response has addressed your main concerns, and we sincerely appreciate your time and careful consideration of our work.
Response to Reviewer 2xti
We sincerely thank you for your careful reading of our paper and your constructive comments.
W1: Clarity of Figure 3 and Section 6.2
We redrew Figure 3 with larger labels and separate input, encoding, and action-generation panels. Sections 5.1-5.3 and Appendix E.1, Table 11 explain the order: velocity reuse first, then residual reuse or full evaluation on a miss, with every native integration update retained. Section 6.2 distinguishes paused-world Original/Modified-only from measured-delay Static. FastWAM's Q=32, r=10 queue leaves 22 old actions that can move the robot before installing a chunk conditioned on an earlier observation. Appendix B.2 specifies replay and fallback. Zero-tail Q=r=10 yields 90.0%/92.0% Baseline/PACE Static success and PACE moving-target gains of 14.0/18.0 percentage points at v1/v2 (Appendix E.2, Table 12).
W2: Policy coverage and experimental repetitions
Simulation covers FastWAM and π0.5 across 50 Dynamic-LIBERO and 45 Dynamic-RoboTwin variants. We added queue, fewer-step, periodic-reuse, ungated-reuse, and component comparisons (Appendices E.2-E.4, Tables 12-14), plus direct BAC/VLA-Cache evaluations on Dynamic-LIBERO (Appendix E.8, Table 18; v1=1.2 cm/s):
PACE reaches 43.0%/81.2% v1 success on FastWAM/π0.5 (Table 2), exceeding BAC by 35.2/4.6 and VLA-Cache by 23.7/20.6 percentage points. This supports PACE's dynamic-performance advantage over existing caching methods.
Physical evaluation now covers five tasks with 25 trials per method and condition. Conveyor-belt grasping and rotating cup-rack placement test moving-object acquisition and moving-target alignment (Appendix E.5, Figure 6, Table 15):
Dynamic success gains are 12/8 percentage points with 56.1%-56.8% lower latency, approximately 2.3x acceleration. Together with 2.86-3.76x simulation speedups (Section 6.3, Table 2), these results show substantial acceleration and higher observed dynamic success in both simulated and physical environments. On T1-T3, Static success is maintained or improved; at 3.5 cm/s, each task improves from 0/25 to 3/25 (Section 6.5, Table 4; Appendix D.1). Appendix E.6, Table 16 reports two-sided 95% Wilson intervals.
We have also uploaded videos of the newly added real-world experiments as supplementary material.
W3: Contributions of individual reuse mechanisms
Observation reuse saves encoding; residual reuse skips hidden transformations while retaining the current embedding and output head; velocity reuse bypasses both but retains every integration update (Appendix E.1, Table 11). Appendix E.4, Table 14 reports latency increases when each component is removed:
Every removal increases latency and lowers both backbones' Dynamic-LIBERO dynamic success (Section 6.4, Table 3). Each module contributes to acceleration and closed-loop performance, supporting PACE's ordered runtime architecture across three complementary boundaries: observation encoding, cross-call hidden transformations, and within-chunk velocities.
Q1: Portability to benchmarks such as RoboDojo
The RoboCasa examples show that localized simulator adaptation can add controlled target motion to existing scenes (Appendix E.7, Figure 7, Table 17). Adapters connect observation and inference timing, world stepping, action execution, and motion hooks under a shared world clock while preserving task semantics and success criteria (Appendices A.2, B.1-B.2). We will continue expanding the dynamic datasets and task suites, adding Dynamic-RoboCasa and Dynamic-RoboDojo in subsequent releases and publishing usable benchmark-modification tools.
Q2: Additional discussion addressing the weaknesses
The revised presentation, expanded comparisons, physical trials, and component analysis address these points (Sections 5.1-5.3, 6.2-6.5; Appendices E.1-E.8).