- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes R$^2$, a flow policy that combines asynchronous vision-language processing with frequent proprioceptive updates. A partially denoised action buffer and a delay-dependent schedule allow actions to be emitted with one denoising step per call. Experiments include simulated cube reorientation and four real-world manipulation tasks using GR00T-N1.7, with approximately 25Hz proprioceptive feedback and reported success improvements of up to 30 percentage points.
Critical strengths
- Separating slow visual processing from fast proprioceptive feedback addresses a practical bottleneck in large action-chunking policies and allows corrections during execution.
- The action buffer combines incremental denoising with conditioning on already committed actions. This provides a useful way to reduce per-call computation while maintaining continuity.
- The four real-world tasks demonstrate the method on contact-rich manipulation. The force traces provide concrete examples of how faster feedback can prevent excessive gripping or missed contact.
Critical weaknesses
- The hardware comparison does not isolate the algorithmic gain. πR² uses two RTX A5000 GPUs, while the baselines use one. A real-world comparison with RTC using the same asynchronous visual processing and hardware is missing.
- Robustness to changing latency is not directly established. The main study sweeps fixed delays, while the deployment algorithm uses a rolling estimate and repeats actions when predictions arrive late. Performance under latency spikes and the frequency of this fallback are not reported.
- The simulation latency accounting needs clarification. Section 4.1.2 derives the delay budget from four denoising steps, whereas Appendix A.1 uses 15 steps for Flow and RTC. Since the assigned delays strongly affect the results, the mapping from NFE counts to simulated latency should be explicit.
Questions for authors
- How does R$^2$ perform on a single GPU? Does its advantage remain against RTC with the same asynchronous visual processing and hardware allocation?
- Can you report results with delay changing within an episode, including sudden spikes and the frequency of repeated-action fallback?
- Section 4.1.2 derives the simulated latency budget from four denoising steps, while Appendix A.1 uses 15 steps for Flow/RTC. How are the actual NFE counts mapped to the delays assigned to each method?
6 comments
Response to Reviewer eSo2 (2/2)
R4. Policy latency and cache age charged to each method in simulation, in control ticks, with and . "–" means the charged exceeds , so the schedule cannot fit a clean prefix and a noise tail of length inside the chunk. For FASTER, is the ready-step of the last position it emits under its own schedule (, ), with steps on Leap Cube and on Kinetix; because the schedule depends on the delay, is the smallest self-consistent solution. 's cache age is charged as , the lower end of its real-world range.
Leap Cube, ,
| Method | Charged policy latency | ||
|---|---|---|---|
| Standard Flow, RTC, Train-time RTC | 2 / 4 / 6 / – | – (in ) | |
| FASTER | 1 / 3 / 4 / – | – (in ) | |
| SDP | , stalls | 1 / 2 / 3 / 5 | – (in ) |
| 1 / 1 / 1 / 2 | 1 / 2 / 3 / 4 | ||
| w/o async | 1 / 2 / 3 / 5 | – (in ) | |
| w/o staircase | 1 / 2 / 3 / 5 | 1 / 2 / 3 / 4 | |
| w/o latency-adaptive | , stalls | 1 / 1 / 1 / 2 | 1 / 2 / 3 / 4 |
Kinetix, ,
| Method | Charged policy latency | ||
|---|---|---|---|
| Standard Flow, RTC, Train-time RTC | 2 / 4 / – / – | – (in ) | |
| FASTER | 1 / 4 / – / – | – (in ) | |
| SDP | , stalls | 1 / 2 / 3 / 4 | – (in ) |
| 1 / 1 / 1 / 1 | 1 / 2 / 3 / 3 | ||
| w/o async | 1 / 2 / 3 / 4 | – (in ) | |
| w/o staircase | 1 / 2 / 3 / 4 | 1 / 2 / 3 / 3 | |
| w/o latency-adaptive | , stalls | 1 / 1 / 1 / 1 | 1 / 2 / 3 / 3 |
Thanks for clarifying how the simulation delays were set. I had missed that Flow and RTC run 15 denoising steps on Leap Cube but are charged the cost of four GR00T steps. That answers my question. Please include R4 and this explanation in the paper.
The rollout numbers also answer my question about the fallback. The robot repeated the last action on 58 of 5,401 control ticks. I was glad to see the per-rollout counts too, since both Box trials with 17 holds still succeeded.
My remaining question is about task success on the robot. With both methods on two GPUs and using asynchronous vision, R$^2$ takes 31.5 ms per call and RTC takes 94.4 ms. That addresses my concern about the latency comparison. Do you also have success rates for those two setups on any of the robot tasks? I’d like to know how much of a difference the shorter call time made in practice.
Response to Reviewer eSo2 (1/2)
We thank the reviewer for recognizing the value of the action buffer and the force-trace analysis.
[W1] + [Q1] Two GPUs. The second GPU cannot shorten the baselines' policy latency, which is a serial chain that includes the VLM latency. Train-time RTC with asynchronous VLM processing, as the reviewer asks, is our "w/o staircase" ablation (Leap Cube in the submission, and Kinetix in this rebuttal), where hardware plays no role, and πR2 beats it at every unit delay on both Leap Cube and Kinetix, by a margin that widens with the delay. On the same two GPUs on the robot, RTC with asynchronous VLM still has a policy latency of 94.4 ms vs. 31.5 ms for πR2, and even on a single GPU πR2's policy latency (69.3 ms) is lower than that of every RTC configuration (R3). See [C2] for details.
[W2] + [Q2] Robustness to varying latency. Both the policy latency and the cache age vary during our real-world rollouts, and our reported results were obtained under this variation (R2). Across the twelve evaluation rollouts shown in R2 (3,999 policy calls), the policy call interval fluctuates with spikes of up to 6.0× its nominal value (2.6% of calls deviate by more than ±10%), and the cache age varies continuously by a median of 0.67 ticks between consecutive calls. The hold-last-action fallback fires on 58 of 5,401 control ticks (1.07%), including two Box rollouts that hold 17 times each and still succeed. This is by design: πR2 measures d at every call and the schedule follows it, so the buffer is not rebuilt; only its per-position noise targets shift, and it settles within a few calls (App. A.4). See [C3] for details.
[W3] + [Q3] Simulated latency accounting. We apologize for the confusion. In simulation, the policy latency and cache age are set by a fixed cost model from GR00T-N1.7 (VLM latency dvlm=0.75d0 and denoising-step latency ddn=0.25d0, where the unit delay d0 is the cost of GR00T's default 4 denoising steps), not by how many denoising steps the small simulation network runs. Flow and RTC run 15 denoising steps on Leap Cube and up to 5 on Kinetix for quality, but are charged only GR00T's 4 steps, which favors them. R4 lists the policy latency and cache age charged to each method; we will add it to the main draft. A method is excluded at a given d0 when its charged policy latency exceeds H/2 (H=16 on Leap Cube, H=8 on Kinetix), as it then no longer allows continuous real-time execution.