acceptodds
Under review as a conference paper at ICLR 2027

Relative Temporal Alignment for Real-Time Execution of Vision-Language-Action Policies

Abstract

Vision-Language-Action (VLA) policies commonly leverage action chunking to amortize the high computational cost of model inference. Action chunking reduces amortized computation but also lowers the rate at which fresh observations enter the policy. Synchronous execution may pause the robot between chunks, whereas asynchronous execution maintains continuous control but hands over a chunk conditioned on an earlier observation. Existing approaches address this challenge through chunk-boundary smoothing, future robot-state prediction, or stepwise visual feedback, yet they do not explicitly relate robot–environment changes during asynchronous inference to action residuals (i.e., corrections to the pending action sequence), limiting low-cost temporal alignment of pending actions. We propose Relative Temporal Alignment (RTA), a lightweight correction method decoupled from the base VLA policy. RTA keeps the base policy frozen and uses relative robot–environment changes during inference to efficiently correct each incoming action chunk before execution. This aligns actions conditioned on the observation at inference onset with the robot–environment state available when inference completes. Experiments across static simulation, dynamic simulation, and real-robot platforms show that RTA maintains robust performance under varying inference delays and improves task completion rates while incurring minimal correction overhead.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-Language-Action Policies, Asynchronous Execution, Action Chunking, Temporal Alignment

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_zPXROpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RTA (Relative Temporal Alignment), a lightweight correction method to address observation staleness during the asynchronous execution of VLA policies. By keeping the base VLA model frozen, RTA captures relative environmental and robot changes during inference by combining optical flow from global images taken at the start and end of inference with the difference in the robot's states. It then applies a one-shot residual correction to the action chunk right before the new actions are handed over for execution. Experiments across LIBERO, DOMINO, and real-robot environments demonstrate that RTA is robust to delays while reducing computational cost by roughly 4 orders of magnitude compared to per-step correction methods such as A2C2.

Critical strengths
  • The methodology is clearly explained. The motivation, system design, and implementation details are presented in a straightforward manner.
  • The proposed correction mechanism is extremely lightweight, operating with minimal latency.
Critical weaknesses
  • Because RTA captures relative changes using low-resolution optical flow, it can be sensitive to lighting changes, reflections, occlusions, motion blur, and textureless regions. This susceptibility can lead to incorrect predictions.
  • When correcting an absolute action, the residual is linearly scaled and added along the horizon to the valid suffix of the action chunk. Consequently, if the target makes non-linear movements (e.g., sudden acceleration, stops, or sharp turns) during inference, a significant discrepancy from the actual target position may occur in the latter half of the chunk, which is an obvious weakness of the method and has dependencies on the environment and tasks. I would like to know how robust the proposal is.
Questions for authors
  • RTA captures relative changes using 2D optical flow computed from low-resolution global camera images and maps it to an action-space residual. However, the logical connection explaining how the robot's actions can be accurately corrected in a 3D space based solely on 2D pixel displacements from a single viewpoint is not adequately justified. In tasks where movement along the depth axis is prominent, how does the limitation of this 2D representation affect performance?
  • The inference of the A2C2 correction head has been confirmed to be lightweight enough to fit within a single step. Please clarify the real-world scenarios in which further reduction of this computational cost would be necessary.
3 comments

Response to Reviewer zPXR

Thank you for highlighting the clear description of the method and the low overhead of the correction module.

Weakness 1

Because RTA captures relative changes using low-resolution optical flow, it can be sensitive to lighting changes, reflections, occlusions, motion blur, and textureless regions. This susceptibility can lead to incorrect predictions.

Authors’ Response

  • Thank you for raising this concern. As discussed in Appendix E, the sensitivity of low-resolution optical flow is a limitation of RTA. Changes in lighting and reflections can lead to spurious flow estimates, occlusions can disrupt correspondences between frames, and motion blur or low texture can make displacement difficult to estimate. The resulting flow errors can affect the direction or magnitude of the predicted action residual.
  • In future work, we plan to assess how these conditions affect action correction and explore more robust motion representations. We will consider alternative flow estimators such as RAFT and investigate the use of depth cues, object tracking, or flow confidence estimates.

Weakness 2

When correcting an absolute action, the residual is linearly scaled and added along the horizon to the valid suffix of the action chunk. Consequently, if the target makes non-linear movements (e.g., sudden acceleration, stops, or sharp turns) during inference, a significant discrepancy from the actual target position may occur in the latter half of the chunk, which is an obvious weakness of the method and has dependencies on the environment and tasks. I would like to know how robust the proposal is.

Authors’ Response

  • Thank you for raising this concern about abrupt changes in target motion. In the real-robot rolling-can task (Section 5.4, Table 4), the can rolls at a nonuniform speed. RTA achieved a 28% success rate on this task, compared with 20% for Naive Async.

  • To evaluate robustness across different target trajectories, we will report comparative results on DOMINO at two difficulty levels in the revised paper.

    • L2. Targets follow polynomial curves of degree 2–5, with varying curvature and acceleration.
    • L3. Each trajectory concatenates two or three independent L1 or L2 motion segments, each with a randomly sampled duration.

Question 1

RTA captures relative changes using 2D optical flow computed from low-resolution global camera images and maps it to an action-space residual. However, the logical connection explaining how the robot's actions can be accurately corrected in a 3D space based solely on 2D pixel displacements from a single viewpoint is not adequately justified. In tasks where movement along the depth axis is prominent, how does the limitation of this 2D representation affect performance?

Authors’ Response

  • Thank you for asking how RTA handles target motion along the camera’s depth axis.

  • RTA learns the correction in the base policy’s action space. The correction head takes global-camera optical flow and the measured robot-state change during inference (Eqs. (3) and (4)). The state change captures the robot’s own motion. We train the head by matching the corrected suffix of the frozen VLA’s action chunk to expert actions at the corresponding execution times (Eq. (8)).

  • Single-view flow leaves the target’s metric depth displacement ambiguous. Near the optical axis, a target moving toward or away from the camera may mainly expand or contract in the image, with little shift of its center. This cue can be subtle at the 64 × 64 flow resolution. We therefore expect the correction to be less reliable when it depends on such weak depth-motion cues.

  • On DOMINO Level 1, the overall success rate at the same eight-step delay is 12.29% for RTA and 8.74% for Naive Async (Table 2).

Response to Reviewer zPXR

Thank you for highlighting the clear description of the method and the low overhead of the correction module.

Question 2

The inference of the A2C2 correction head has been confirmed to be lightweight enough to fit within a single step. Please clarify the real-world scenarios in which further reduction of this computational cost would be necessary.

Authors’ Response

  • Thank you for asking when lower correction overhead matters in practice. As you note, a single A2C2 correction can fit within one control step.

  • When perception and correction share a local GPU, fewer correction calls can free GPU time for perception. High control rates also leave less time to schedule a correction at each step. In our real-robot setup (Section 5.4), we run the RTA correction on a local RTX 4060 and the base VLA on a remote RTX 5090 server.

  • Backbone independence. A2C2's VLA correction head uses internal features from the base policy, such as the SmolVLM latent representation in its LIBERO implementation [1]. RTA predicts the residual from optical flow and the measured change in robot state, independently of the VLA backbone's internal features. The remote VLA can use its existing action-chunk interface, and the robot can apply the correction locally.

  • In our RTX 5090 microbenchmark (Section 5.3, Table 3), A2C2 performs 30 corrections per action chunk and RTA performs one at handover. The FP32 P95 total correction time per chunk is 269.411 ms for A2C2 and 2.510 ms for RTA.

  • We also compared the methods on DOMINO Level 1 dynamic manipulation tasks (Table 2). At the same eight-step delay, A2C2 achieved 3.40% success, Naive Async achieved 8.74%, and RTA achieved 12.29%.

Reference

[1] Sendai, Kohei, et al. "Leave no observation behind: Real-time correction for vla action chunks." arXiv preprint arXiv:2509.23224 (2025).

Thank you for your response and for clarifying the architectural advantage of RTA with respect to backbone independence in client-server setups (Question 2). That is a valid practical point. However, my main concerns regarding the empirical validation and robustness of the proposed method remain largely unaddressed:

  • While I appreciate the plan to evaluate RTA on DOMINO L2 and L3 in the revised paper, promises of future results cannot be verified during the review period. Without seeing the actual quantitative results on L2/L3 now, it is difficult to assess how well the linear residual scaling holds up under non-linear trajectories.
  • Simply acknowledging the fragility of 64×64 optical flow to lighting changes, occlusions, and motion blur as a limitation for future work does not resolve the concern about real-world reliability.

Because the rebuttal primarily relies on acknowledging limitations and deferring experiments to the camera-ready version or future work rather than providing new empirical evidence, I am unable to raise my score at this stage.

Reviewer_HzHzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the problem that when a VLA model executes asynchronously, actions are generated based on stale observations, and proposes RTA: it freezes the main model and uses optical flow between images captured at the start and end of inference, together with the corresponding change in the robot’s actual state, to predict a correction vector with a small network. The correction vector is then used to modify the remaining actions in one pass before a new action segment takes over. The experiments include static sim tasks on LIBERO, dynamic sim tasks on DOMINO, and two dynamic real-robot tasks.

Critical strengths
  1. The problem is valuable, and the method is simple. The correction module does not rely on the main model's internal features and is called only once per action segment, deployment cost is very low.
  2. The experimental coverage is relatively broad. It includes two main models, static and dynamic sim tasks, and real-robot experiments. 3, The dynamic tasks show clear improvement. Compared with naive asynchronous execution, the DOMINO success rate increases from 8.74% to 12.29%; in real-robot grasping, conveyor block increases from 34% to 54%, and rolling can increases from 20% to 28%.
Critical weaknesses
  1. The claimed mechanism is unclear. Optical flow and the change in the robot’s actual state show env change but not the difference between plan and execution. The correction network does not receive the original actions, the current absolute state, or the task instruction, so it is unclear how it learns how to correct. In particular, I do not understand where the improvements in static environments come from.
  2. The correction rule for absolute actions is not clearly justified. Equation (7) explicitly applies to absolute action targets, yet it directly adds the change in the robot’s actual state during inference.
  3. The training objective does not separate delay-induced errors from the base policy’s existing errors. As described, training uses observations at two time steps from expert trajectories and encourages the corrected actions to match the expert actions. However, it is unclear how the training data captures the state deviations caused by executing the previous action chunk. The gains may come from correcting general policy errors, not inference delays.
  4. Key comparisons and ablations are missing. There are no input ablations using only the state difference or only optical flow. On DOMINO, A2C2 performs much worse than naive asynchronous execution, but the reason is not explained. The real-robot evaluation includes only two dynamic tasks and compares RTA only against sync and naive async.
Questions for authors
  1. What coordinate frame is used for the absolute targets in Equation (7)? Why add the actual state change, rather than keep the original targets or adjust them based on the deviation from the original plan?
  2. Why is A2C2 on DOMINO lower than naive async?
  3. In Table 2, why are FLOPs for FP16 and FP32 different?
  4. Does the reported 1.16 ms latency include optical flow computation and preprocessing?
3 comments

Response to Reviewer HzHz

Thank you for highlighting the simple deployment, the breadth of the evaluation, and the gains on dynamic tasks.

Weakness 1

The claimed mechanism is unclear. Optical flow and the change in the robot’s actual state show env change but not the difference between plan and execution. The correction network does not receive the original actions, the current absolute state, or the task instruction, so it is unclear how it learns how to correct. In particular, I do not understand where the improvements in static environments come from.

Authors’ Response

  • Thank you for raising this point.

  • Optical flow and Δs=st+d−st\Delta s=s_{t+d}-s_t describe the observed changes during inference. A plan–execution error would reference the planned state at t + d. RTA uses the observed changes to predict a residual and applies it to the valid suffix of the base VLA’s chunk, which is conditioned on the observation, absolute state, and task instruction (Eqs. 1, 3–7).

  • The head learns how to correct by minimizing the squared error between the corrected suffix and the demonstration actions at the corresponding execution times t + d + j (Eq. 8).

  • Even with a stationary target, the previous chunk can move the gripper during inference, changing its pose relative to the target at handover. On LIBERO (Table 1), RTA improves mean success over Naive Async by 0.7 percentage points at d = 0 and 56.1 percentage points at d = 8, consistent with a benefit from correcting the chunk at delayed handover.

  • We are currently evaluating a matched input ablation on DOMINO to assess each cue’s contribution in dynamic tasks. The comparison includes state-difference-only, flow-only, and full RTA, with Naive Async as the reference.

Weakness 2 and Question 1

The correction rule for absolute actions is not clearly justified. Equation (7) explicitly applies to absolute action targets, yet it directly adds the change in the robot’s actual state during inference.

What coordinate frame is used for the absolute targets in Equation (7)? Why add the actual state change, rather than keep the original targets or adjust them based on the deviation from the original plan?

Authors’ Response

  • Thank you for the question.

  • Absolute actions are targets in the base policy's action coordinates with a reference shared across the chunk. Examples include absolute joint positions, end-effector poses in the controller's Cartesian frame, and joint positions relative to a fixed initial configuration. Equation (6) maps the observed state change into those coordinates via PP (P=IP=I when state and action coordinates and dimensions match).

  • Equation (7) recovers Eq. (5) for single-step increments. Subtracting adjacent corrected targets in the valid suffix (j≥1j \ge 1) cancels the shared state offset and leaves one residual per step.

    = \left(a_{d+j\mid t}-a_{d+j-1\mid t}\right)+r_{t,d}.$$
  • The state offset anchors the first increment at handover. When Eq. (6) gives the difference between the two observed states in additive action coordinates, Δst,d(a)=st+d(a)−st(a)\Delta s^{(a)}_{t,d}=s^{(a)}_{t+d}-s^{(a)}_t, Eq. (7) at j=0j=0 gives

    = \left(a_{d\mid t}-s^{(a)}_t\right)+r_{t,d}.$$ We measure the first increment from the observed state at inference onset and apply the corrected increment from the observed handover state. Summing the corrected increments yields $(j+1)r_{t,d}$ in Eq. (7).
  • Keeping the original targets preserves their positions in the action frame. An offset based on plan deviation defines the first increment relative to the planned handover state.

Weakness 3

The training objective does not separate delay-induced errors from the base policy’s existing errors. As described, training uses observations at two time steps from expert trajectories and encourages the corrected actions to match the expert actions. However, it is unclear how the training data captures the state deviations caused by executing the previous action chunk. The gains may come from correcting general policy errors, not inference delays.

Authors’ Response

  • Thank you for raising this point.

  • The correction implied by Eq. (8), at+d+j∗−ad+j∣ta^*_{t+d+j}-a_{d+j\mid t}, includes the delay-related prediction change and the refreshed policy's error relative to the expert. Training pairs come from demonstrations. At deployment, the preceding policy chunk affects the handover state (Section 4.3 and Algorithm 1).

  • We are currently evaluating an ablation that uses only the delay-related change aj∣t+d−ad+j∣ta_{j\mid t+d}-a_{d+j\mid t} as the training target. The two terms are time-aligned predictions from the same frozen VLA using inputs at handover (t+dt+d) and inference onset (tt), respectively.

Response to Reviewer HzHz

Thank you for highlighting the simple deployment, the breadth of the evaluation, and the gains on dynamic tasks.

Weakness 4 and Question 2

Key comparisons and ablations are missing. There are no input ablations using only the state difference or only optical flow. On DOMINO, A2C2 performs much worse than naive asynchronous execution, but the reason is not explained. The real-robot evaluation includes only two dynamic tasks and compares RTA only against sync and naive async.

Why is A2C2 on DOMINO lower than naive async?

Authors’ Response

  • As described in our response to Weakness 1, we are evaluating a matched input ablation on DOMINO to assess how each input contributes to task success. We compare RTA variants that use only the robot-state difference, only optical flow, or both inputs (full RTA) under the same base policy, tasks, and delays, with Naive Async as the reference.

  • We ran 50 trials per method per task for RTA, blocking Sync (OpenPI), and Naive Async on the conveyor-block and rolling-can tasks (Section 5.4 and Table 4). We are expanding the evaluation to include tasks in static scenes and a conveyor task at a higher belt speed, with RTC and A2C2 as additional baselines.

  • Thank you for asking about A2C2's performance on DOMINO. At the same eight-step delay, A2C2 achieved a lower success rate than Naive Async on 24 of 35 Level 1 tasks, with the largest gaps on two bottle manipulation tasks (Table C5).

    DOMINO Level 1A2C2 success rateNaive Async success rate
    Overall (Table 2)3.40%8.74%
    adjust bottle (Table C1)8%45%
    shake bottle horiz. (Table C1)11%47%
  • A2C2 predicts a residual at each control step using the latest observation, the base action, base-policy features, and the action's position in the chunk [1]. Its head must infer how the scene changed during inference from these inputs. RTA computes optical flow between the images at inference onset and handover. It supplies the flow and the robot-state difference over that interval directly to its correction head.

    We speculate that when a target moves during inference, A2C2 may misjudge the direction or magnitude of the adjustment needed. The resulting residual can steer the end effector off an otherwise viable approach. Naive Async executes the base actions directly and avoids this source of error.

Question 3

In Table 2, why are FLOPs for FP16 and FP32 different?

Authors’ Response

  • Thank you for your careful observation. We traced most of the discrepancy to double counting under FP16 autocast, where the profiler included both the outer and inner calls to the same convolution.
  • For A2C2, the profiler also missed FLOPs from some fused operators, and the Transformer took different execution paths in FP16 and FP32.
  • The original FP32 benchmark measured 424.17 GFLOPs per action chunk for A2C2 with SmolVLA. For the revised benchmark, we switched the VLA backbone to π0.5, which changed the architecture of A2C2's correction network [1]. Table 3 now reports 573 GFLOPs for A2C2 and 20.9 MFLOPs for RTA per action chunk in FP32.

Question 4

Does the reported 1.16 ms latency include optical flow computation and preprocessing?

Authors’ Response

  • Thank you for the question. The original 1.16 ms is the P95 forward-pass latency of the RTA correction network with SmolVLA. We computed optical flow and completed the other input preprocessing before starting the timer.

  • In the revised Table 3, we benchmark RTA with π0.5\pi_{0.5} in FP32. The P95 latencies per action chunk are 1.939 ms for input preprocessing, 0.486 ms for network inference, and 2.510 ms for the full correction, including preprocessing.

  • We have updated the latency figures in the Introduction and Conclusion and clarified the scope of each measurement in Section 5.3.

Reference

[1] Sendai, Kohei, et al. "Leave no observation behind: Real-time correction for vla action chunks." arXiv preprint arXiv:2509.23224 (2025).

Thanks for the response. I still have the following concerns:

  1. It‘s’ still unclear why the observed change improve static tasks. The ongoing ablation is important.
  2. Without the original actions and current state as inputs, it's still unsure how the correction network be able to learn the right adjustment. Also the generalization ability could be a problem.

I'll keep my score.

Reviewer_D6TkOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper targets asynchronous action-chunking VLAs. Different from previous works, such as RTC and VLASH that mainly handle chunk-boundary continuity and predict future state based on current state and robot motion, or A2C2 which corrects the action at every control step using base-policy features, this paper focuses more on the environment change over the asynchronous inference window. Besides the robot state, it encodes a low-resolution optical flow, and utilizes a lightweight head to predict one action residual that corrects the whole valid suffix of the chunk before handover.

The authors conduct extensive experiments. On LIBERO with SmolVLA, RTA has the highest mean success rate among all asynchronous baselines. On DOMINO with dynamic objects, RTA reaches 12.29% SR, compared with 8.74% for naive method and surpasses the 11.71% for sync baseline. On two dynamic grasping tasks on real robot, mean SR rises from 27.0% to 41.0%. For efficiency, each correction costs 20.88 MFLOPs with a P95 latency of 1.16 ms, only about 1/368 of A2C2's per-chunk latency (Sec. 5.3).

Critical strengths
  1. The motivation is intuitive and well illustrated (Sec. 4.1, 4.2, Fig. 1). The error caused by stale observations depends on the relative robot-environment change during the inference window. This directly highlights the importance of external modeling, which RTC, VLASH, and A2C2 do not explicitly model.
  2. The method is cheap and independent from the base policy. One forward pass per chunk (1.16ms, 1.94MB) makes it easy to deploy on a local GPU next to a remote policy server, which is examined by real-robot experiments.
  3. The experimental results are quite impressive.
Critical weaknesses
  1. Unclear training signal for external signals. The corrector is trained on demonstrations, with (It+d,st+d)(I_{t+d}, s_{t+d}) and the target a∗t+d+ja*_{t+d+j} taken from the demo trajectory. However, at deployment the motion over [t,t+d][t, t+d] comes from the previous chunk executed by the policy, not from the demonstrator. On LIBERO the scene is static, so what the predictive head learns is only robot motion. This is exactly what VLASH does. Besides, the formula Crel=2Cint−(Cint+Cext)C_rel = 2C_int − (C_int + C_ext) (Ln. 218 - 220) contradicts the implementation, where the two features are simply concatenated. Besides, no regarding ablations show that the model actually learns to separate the internal and external components.
  2. Comparisons and ablations which should be added:
  • On DOMINO the authors only compare against naive method and A2C2, with RTC and VLASH missing.
  • On real-robot, experiment includes only blocking sync and naive async, and the number of trials is not reported.
  • There is no flow-only vs state-only comparison, no flow resolution or flow-method study (Farnebäck vs. RAFT), no constant-vs-per-step residual comparison, so it is hard to know which component produces the gains.
  1. Suboptimal demonstrations, including
  • The LIBERO results are shown only as curves (Fig. 3, C1, D1), with no numerical table, so the size of the gap between RTA, A2C2 and VLASH cannot be quantified.
  • Fonts and texts in Fig. 2, 3 are too small, with large space left.
Questions for authors
  1. A2C2 (3.40%) falls far below Naive Async (8.74%), could the authors provide an intuitive explanation?
  2. In Table 2, the FP16 FLOPs are roughly twice the FP32 FLOPs for both methods (A2C2: 779.16G vs. 424.17G; RTA: 40.94M vs. 20.88M). Why?
3 comments

Response to Reviewer D6Tk

Thank you for recognizing the motivation for modeling relative motion, the low correction overhead, and the empirical results.

Weakness 1

Unclear training signal for external signals. The corrector is trained on demonstrations, with (It+d,st+d)(I_{t+d}, s_{t+d}) and the target a∗t+d+ja*_{t+d+j} taken from the demo trajectory. However, at deployment the motion over [t,t+d][t, t+d] comes from the previous chunk executed by the policy, not from the demonstrator. On LIBERO the scene is static, so what the predictive head learns is only robot motion. This is exactly what VLASH does. Besides, the formula Crel=2Cint−(Cint+Cext)C_rel = 2C_int − (C_int + C_ext) (Ln. 218 - 220) contradicts the implementation, where the two features are simply concatenated. Besides, no regarding ablations show that the model actually learns to separate the internal and external components.

Authors’ Response

  • Thank you for raising these questions about the training signal and the role of the two motion cues.

  • Training supervision and deployment. We generate the action chunk offline with the frozen base VLA. Demonstrations provide the paired images and robot states, and Eq. (8) trains the corrector to match the corrected suffix to time-aligned expert actions. During deployment, the measured changes accumulate while the robot executes the preceding chunk from the policy (Section 4.3 and Algorithm 1).

    We are evaluating whether the delay-related prediction change alone can train the corrector. The ablation uses aj∣t+d−ad+j∣ta_{j\mid t+d}-a_{d+j\mid t} as the sole training target. Both terms are predictions from the same frozen VLA for execution step t+d+jt+d+j, using inputs at handover (t+dt+d) and inference onset (tt), respectively.

  • Static LIBERO scenes and VLASH. Even with a stationary target, the preceding chunk can move the robot relative to the target during inference. VLASH rolls the robot state forward using that chunk and conditions generation on the predicted handover state (Section 5.1). It fine-tunes the base VLA over a predefined range of temporal offsets and selects an offset based on the deployment device's measured inference latency. Switching to a slower device whose delay exceeds that range may require widening the range and fine-tuning the VLA again. [1]

    RTA measures the delay when inference completes and corrects the returned chunk's valid suffix before handover using the changes observed over that interval (Section 4.3 and Algorithm 1).

  • Relative-change identity and feature fusion. Section 4.1 rewrites the relative change accumulated during inference as

    Ct,drel=2Ct,dint−(Ct,dint+Ct,dext).C_{t,d}^{\mathrm{rel}}=2C_{t,d}^{\mathrm{int}}-\bigl(C_{t,d}^{\mathrm{int}}+C_{t,d}^{\mathrm{ext}}\bigr).

    This form motivates two complementary cues. The robot-state difference Δst,d=st+d−st\Delta s_{t,d}=s_{t+d}-s_t measures the internal change Ct,dintC_{t,d}^{\mathrm{int}} due to robot motion. Low-resolution optical flow between the global-view images at tt and t+dt+d characterizes the apparent image-plane motion associated with the combined change Ct,dint+Ct,dextC_{t,d}^{\mathrm{int}}+C_{t,d}^{\mathrm{ext}}. Eqs. (3)–(4) specify how the correction head uses these cues. Separate branches encode them, and the fusion network predicts one action-space residual from their concatenated features.

    To assess each cue's contribution to task success, we are evaluating a matched input ablation on DOMINO. We compare state-difference-only, flow-only, and full RTA, with Naive Async as the reference.

Reference

[1] Tang, Jiaming, et al. "Vlash: Real-time vlas via future-state-aware asynchronous inference." arXiv preprint arXiv:2512.01031 (2025).

Response to Reviewer D6Tk

Thank you for recognizing the motivation for modeling relative motion, the low correction overhead, and the empirical results.

Weakness 2

Comparisons and ablations which should be added:

  • On DOMINO the authors only compare against naive method and A2C2, with RTC and VLASH missing.
  • On real-robot, experiment includes only blocking sync and naive async, and the number of trials is not reported.
  • There is no flow-only vs state-only comparison, no flow resolution or flow-method study (Farnebäck vs. RAFT), no constant-vs-per-step residual comparison, so it is hard to know which component produces the gains.

Authors’ Response

  • Thank you for suggesting these additional comparisons and ablations.

  • DOMINO baselines. We are evaluating RTC and VLASH on the DOMINO tasks from Table 2, matching the π0.5 backbone, horizons (H = 50, K = 30), delay (d = 8), and metrics. Table 2 compares RTA, Naive Async, and A2C2 at this delay and includes Sync at d = 0 as a reference.

  • Real-robot evaluation. We ran 50 trials per method per task for RTA, blocking Sync (OpenPI), and Naive Async on the conveyor-block and rolling-can tasks (Section 5.4 and Table 4). We are expanding the evaluation to include tasks in static scenes and a conveyor task at a higher belt speed, with RTC, VLASH, and A2C2 as additional baselines.

  • Input ablation. As described in our response to Weakness 1, we are running a matched ablation on DOMINO to assess how each input contributes to task success. We compare RTA using only the robot-state difference, only optical flow, or both inputs (full RTA), with Naive Async as the reference.

Weakness 3

Suboptimal demonstrations, including

  • The LIBERO results are shown only as curves (Fig. 3, C1, D1), with no numerical table, so the size of the gap between RTA, A2C2 and VLASH cannot be quantified.
  • Fonts and texts in Fig. 2, 3 are too small, with large space left.

Authors’ Response

  • Thank you for your suggestions on presenting the LIBERO results and improving figure readability.

  • Numerical LIBERO results. Table 1 in Section 5.1 reports the unweighted mean success rates across the four LIBERO suites. We have added references to Table 1 alongside the per-task delay and execution-horizon plots in Figs. C1C4 and D1D5. The values at two delays are reproduced below.

    Methodd = 8d = 20
    RTA70.8%61.0%
    A2C266.8%57.5%
    VLASH69.5%61.4%
  • Figure readability. In Figs. 1 and 2 of the revised manuscript, we have enlarged the labels and other text and adjusted the layouts to reduce excess whitespace.

Question 1

A2C2 (3.40%) falls far below Naive Async (8.74%), could the authors provide an intuitive explanation?

Authors’ Response

  • Thank you for asking about the performance gap on DOMINO.

  • DOMINO results. At the same eight-step delay (d = 8), A2C2 had a lower success rate than Naive Async on 24 of 35 Level 1 tasks. The largest gaps were on the two bottle manipulation tasks shown below (Table C5).

    DOMINO Level 1A2C2 success rateNaive Async success rate
    Overall (Table 2)3.40%8.74%
    adjust bottle (Table C1)8%45%
    shake bottle horiz. (Table C1)11%47%
  • Why per-step correction may lower success. A2C2 predicts a residual at each control step from the latest observation, the base action, base-policy features, and the action's position in the chunk [1]. From these inputs, its correction head must infer how the scene changed during inference. RTA supplies its correction head with optical flow and the change in robot state measured between inference onset and handover.

    We speculate that when a target moves during inference, A2C2 may adjust the base action in the wrong direction or by the wrong amount. This can divert the end effector from an otherwise viable approach to the target. Naive Async executes the base actions directly and avoids this source of error.

Reference

[1] Sendai, Kohei, et al. "Leave no observation behind: Real-time correction for vla action chunks." arXiv preprint arXiv:2509.23224 (2025).

Response to Reviewer D6Tk

Thank you for recognizing the motivation for modeling relative motion, the low correction overhead, and the empirical results.

Question 2

In Table 2, the FP16 FLOPs are roughly twice the FP32 FLOPs for both methods (A2C2: 779.16G vs. 424.17G; RTA: 40.94M vs. 20.88M). Why?

Authors’ Response

  • Thank you for your careful observation. We traced most of the discrepancy to double counting under FP16 autocast, where the profiler included both the outer and inner calls to the same convolution.
  • For A2C2, the profiler also missed FLOPs from some fused operators, and the Transformer took different execution paths in FP16 and FP32.
  • The original FP32 benchmark measured 424.17 GFLOPs per action chunk for A2C2 with SmolVLA. For the revised benchmark, we switched the VLA backbone to π0.5\pi_{0.5}, which changed the architecture of A2C2's correction network [1]. Table 3 now reports 573 GFLOPs for A2C2 and 20.9 MFLOPs for RTA per action chunk in FP32.

Reference

[1] Sendai, Kohei, et al. "Leave no observation behind: Real-time correction for vla action chunks." arXiv preprint arXiv:2509.23224 (2025).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.