- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes RTA (Relative Temporal Alignment), a lightweight correction method to address observation staleness during the asynchronous execution of VLA policies. By keeping the base VLA model frozen, RTA captures relative environmental and robot changes during inference by combining optical flow from global images taken at the start and end of inference with the difference in the robot's states. It then applies a one-shot residual correction to the action chunk right before the new actions are handed over for execution. Experiments across LIBERO, DOMINO, and real-robot environments demonstrate that RTA is robust to delays while reducing computational cost by roughly 4 orders of magnitude compared to per-step correction methods such as A2C2.
Critical strengths
- The methodology is clearly explained. The motivation, system design, and implementation details are presented in a straightforward manner.
- The proposed correction mechanism is extremely lightweight, operating with minimal latency.
Critical weaknesses
- Because RTA captures relative changes using low-resolution optical flow, it can be sensitive to lighting changes, reflections, occlusions, motion blur, and textureless regions. This susceptibility can lead to incorrect predictions.
- When correcting an absolute action, the residual is linearly scaled and added along the horizon to the valid suffix of the action chunk. Consequently, if the target makes non-linear movements (e.g., sudden acceleration, stops, or sharp turns) during inference, a significant discrepancy from the actual target position may occur in the latter half of the chunk, which is an obvious weakness of the method and has dependencies on the environment and tasks. I would like to know how robust the proposal is.
Questions for authors
- RTA captures relative changes using 2D optical flow computed from low-resolution global camera images and maps it to an action-space residual. However, the logical connection explaining how the robot's actions can be accurately corrected in a 3D space based solely on 2D pixel displacements from a single viewpoint is not adequately justified. In tasks where movement along the depth axis is prominent, how does the limitation of this 2D representation affect performance?
- The inference of the A2C2 correction head has been confirmed to be lightweight enough to fit within a single step. Please clarify the real-world scenarios in which further reduction of this computational cost would be necessary.
3 comments
Response to Reviewer zPXR
Thank you for highlighting the clear description of the method and the low overhead of the correction module.
Question 2
The inference of the A2C2 correction head has been confirmed to be lightweight enough to fit within a single step. Please clarify the real-world scenarios in which further reduction of this computational cost would be necessary.
Authors’ Response
-
Thank you for asking when lower correction overhead matters in practice. As you note, a single A2C2 correction can fit within one control step.
-
When perception and correction share a local GPU, fewer correction calls can free GPU time for perception. High control rates also leave less time to schedule a correction at each step. In our real-robot setup (Section 5.4), we run the RTA correction on a local RTX 4060 and the base VLA on a remote RTX 5090 server.
-
Backbone independence. A2C2's VLA correction head uses internal features from the base policy, such as the SmolVLM latent representation in its LIBERO implementation [1]. RTA predicts the residual from optical flow and the measured change in robot state, independently of the VLA backbone's internal features. The remote VLA can use its existing action-chunk interface, and the robot can apply the correction locally.
-
In our RTX 5090 microbenchmark (Section 5.3, Table 3), A2C2 performs 30 corrections per action chunk and RTA performs one at handover. The FP32 P95 total correction time per chunk is 269.411 ms for A2C2 and 2.510 ms for RTA.
-
We also compared the methods on DOMINO Level 1 dynamic manipulation tasks (Table 2). At the same eight-step delay, A2C2 achieved 3.40% success, Naive Async achieved 8.74%, and RTA achieved 12.29%.
Reference
[1] Sendai, Kohei, et al. "Leave no observation behind: Real-time correction for vla action chunks." arXiv preprint arXiv:2509.23224 (2025).
Thank you for your response and for clarifying the architectural advantage of RTA with respect to backbone independence in client-server setups (Question 2). That is a valid practical point. However, my main concerns regarding the empirical validation and robustness of the proposed method remain largely unaddressed:
- While I appreciate the plan to evaluate RTA on DOMINO L2 and L3 in the revised paper, promises of future results cannot be verified during the review period. Without seeing the actual quantitative results on L2/L3 now, it is difficult to assess how well the linear residual scaling holds up under non-linear trajectories.
- Simply acknowledging the fragility of 64×64 optical flow to lighting changes, occlusions, and motion blur as a limitation for future work does not resolve the concern about real-world reliability.
Because the rebuttal primarily relies on acknowledging limitations and deferring experiments to the camera-ready version or future work rather than providing new empirical evidence, I am unable to raise my score at this stage.
Response to Reviewer zPXR
Thank you for highlighting the clear description of the method and the low overhead of the correction module.
Weakness 1
Authors’ Response
Weakness 2
Authors’ Response
Thank you for raising this concern about abrupt changes in target motion. In the real-robot rolling-can task (Section 5.4, Table 4), the can rolls at a nonuniform speed. RTA achieved a 28% success rate on this task, compared with 20% for Naive Async.
To evaluate robustness across different target trajectories, we will report comparative results on DOMINO at two difficulty levels in the revised paper.
Question 1
Authors’ Response
Thank you for asking how RTA handles target motion along the camera’s depth axis.
RTA learns the correction in the base policy’s action space. The correction head takes global-camera optical flow and the measured robot-state change during inference (Eqs. (3) and (4)). The state change captures the robot’s own motion. We train the head by matching the corrected suffix of the frozen VLA’s action chunk to expert actions at the corresponding execution times (Eq. (8)).
Single-view flow leaves the target’s metric depth displacement ambiguous. Near the optical axis, a target moving toward or away from the camera may mainly expand or contract in the image, with little shift of its center. This cue can be subtle at the 64 × 64 flow resolution. We therefore expect the correction to be less reliable when it depends on such weak depth-motion cues.
On DOMINO Level 1, the overall success rate at the same eight-step delay is 12.29% for RTA and 8.74% for Naive Async (Table 2).