- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper proposes RACE, a method for improving the reliability of long action chunks in vision-language-action (VLA) models by explicitly modeling subskill transition timing. It uses automatic change-point detection to generate transition pseudo-labels and incorporates a transition prior into action generation through an auxiliary denoising pass and token-wise modulation. The method is evaluated primarily on , with results suggesting improved task success rates compared with standard fine-tuning at comparable chunk lengths and competitive performance with longer chunks. Real-robot experiments further demonstrate its potential to improve execution reliability while reducing idle time relative to shorter-chunk baselines. However, the evidence for reduced transition errors and improved long-horizon reliability remains limited, and the additional computational overhead and generalizability to other VLA architectures require further investigation.
Critical strengths
- Valuable research perspective: The paper identifies subskill transition timing within action chunks as a potentially important factor affecting the reliability of long-chunk action prediction.
- Intuitive method design: The method uses automatic change-point detection to generate transition labels and then leverages a transition prior to guide action generation.
- Clear writing and presentation: The paper is well organized, and the figures are generally clear and informative.
Critical weaknesses
-
Insufficient experimental evidence for the core motivation. The paper argues that action errors are concentrated around subskill transitions. However, the reduction in transition error reported in Figure 4 appears modest, providing insufficient evidence that the proposed method substantially improves action prediction at transition points.
-
Insufficient long-horizon evaluation. Although supports predictions of up to 50 steps, the experiments primarily focus on shorter execution horizons (e.g., 10 or 20 steps). This is insufficient to establish whether the method effectively mitigates error accumulation in long action chunks.
-
Potentially misleading claims regarding inference speedup. The speedup appears to come primarily from increasing the action chunk length rather than from RACE itself. Moreover, the additional transition-timing prediction and denoising pass introduce computational overhead. The paper should therefore emphasize improvements in action quality rather than inference speedup.
-
Lack of adaptive action chunk length selection. The transition prior only guides action generation; it does not dynamically adjust chunk length or determine when to switch chunks based on predicted subskill transitions. Consequently, the method does not explicitly balance execution efficiency and action accuracy through adaptive chunking.
-
Unclear reliability of transition-timing predictions. The transition labels are generated through automatic change-point detection, which may itself be inaccurate. The paper lacks sufficient quantitative evaluation and qualitative visualizations to establish the reliability of the predicted transition timings.
-
Lack of cross-model validation. The experiments use as the sole backbone, leaving the generalizability of the method to other VLA architectures and action-generation mechanisms unclear.
Questions for authors
- How accurate is the transition-timing prediction? How do prediction errors, missed transitions, and temporal misalignment affect the final performance?
- Can the method consistently reduce action errors around transition points over longer execution horizons? Are the improvements statistically significant, and can the authors provide more comprehensive qualitative analyses?
- How sensitive is the method to the transition loss coefficient and pseudo-label quality? What is the additional training overhead compared with standard fine-tuning?
- Can the method be extended to other VLA frameworks? Which types of VLA architectures are suitable or unsuitable for this approach?
11 comments
Response to Reviewer Hg7g(2/8): W2
W2. Evaluation with longer action chunks.
Thank you for the suggestion. As suggested, we evaluate chunks longer than and report the results below.
(1) Results beyond . We compare RACE with the fine-tuned at and on VLABench. The results of RACE are from Tab. 11 (Appendix B.2), and those of the fine-tuned are newly added. We report the average success rate, with from Tab. 1 for reference.
| fine-tuning | RACE (ours) | Gain | |
|---|---|---|---|
| 20 | 28.1 | 33.0 | +4.9 |
| 30 | 26.1 | 28.5 | +2.4 |
| 40 | 23.8 | 24.0 | +0.2 |
RACE achieves a higher success rate than the fine-tuned up to . However, the gain becomes smaller as the chunk grows, and the two models are comparable at . Thus, RACE makes longer chunks more reliable up to a moderate length, but not at very long ones.
(2) Why the gain becomes small at . To understand this, we analyze the action errors at by position within the chunk, as in W1.
| Actions | Model | All | 0–9 | 10–14 | 15–19 | 20–24 | 25–29 | 30–34 | 35–39 |
|---|---|---|---|---|---|---|---|---|---|
| Transitions | fine-tuning | 4.572 | 1.092 | 2.484 | 2.953 | 3.013 | 3.171 | 7.496 | 19.421 |
| Transitions | RACE (ours) | 3.044 | 1.004 | 1.823 | 1.797 | 1.612 | 1.230 | 4.724 | 15.001 |
| Transitions | Reduction | 1.529 | 0.088 | 0.661 | 1.156 | 1.400 | 1.941 | 2.772 | 4.420 |
| Non-transitions | fine-tuning | 1.757 | 0.656 | 1.436 | 1.674 | 1.830 | 2.020 | 2.585 | 2.993 |
| Non-transitions | RACE (ours) | 1.044 | 0.540 | 1.119 | 1.161 | 1.019 | 1.349 | 1.320 | 0.934 |
| Non-transitions | Reduction | 0.713 | 0.117 | 0.317 | 0.513 | 0.811 | 0.671 | 1.265 | 2.060 |
| Transitions − Non-transitions | Reduction gap | 0.816 | −0.029 | 0.344 | 0.643 | 0.589 | 1.270 | 1.507 | 2.360 |
RACE reduces the errors in every position range, even at . However, the errors at transitions grow sharply over the last ten positions for both models. At positions 35–39, the error of RACE (15.001) is about four times its largest error at (3.702; W1). This observation is in line with our analysis in Sec. 4.4 (Fig. 4). Errors grow with the prediction horizon, and transitions add further difficulty on top. RACE reduces the errors at transitions, but it cannot fully compensate for the errors that grow with the chunk length (Appendix B.2). We believe that this is why the two models achieve comparable success rates at .
The gain returns when only the first 20 of the 40 predicted actions are executed. Below, the same models predict actions and execute either all of them or only the first 20 (Tab. 14 in Appendix B.4).
| fine-tuning | RACE (ours) | Gain | ||
|---|---|---|---|---|
| 40 | 40 | 23.8 | 24.0 | +0.2 |
| 40 | 20 | 31.2 | 35.2 | +4.0 |
In this setting, the last positions are not executed and the policy also replans more often, so both may contribute to the gain.
(Continued in the next comment: (3) chunk length of 50 and (4) why we focus on .)
Response to Reviewer Hg7g(3/8): W2 (continued)
W2. Evaluation with longer action chunks (continued).
(3) Chunk length of 50. As the reviewer notes, can predict up to 50 actions. However, the official checkpoints for LIBERO and VLABench are trained with and execute only the first 5 actions (Sec. 4.1). Without additional training, the official checkpoint fails when run with (0.4% success rate). To examine this setting, we train both and RACE on VLABench to predict actions and execute all of them, with the same training setup as in our main experiments (Sec. 4.1).
| Model | Avg. SR | ||
|---|---|---|---|
| baseline | 10 | 5 | 24.6 |
| trained with | 50 | 50 | 15.8 |
| RACE trained with | 50 | 50 | 16.2 |
Both models fall far below the baseline at this length, and they are comparable, as at . This is in line with (1) and (2): the gain of RACE becomes smaller as the chunk grows and disappears at very long chunks.
(4) Why we focus on . The main goal of extending the chunk is to remove stop-and-go execution while keeping the execution reliable. , four times the execution length of the official checkpoints, already meets this goal. On our real robot, it removes all pauses after the first policy call (Appendix D.5), and the gain of RACE over fine-tuning is largest at this length (Fig. 5b). An even longer chunk has no pauses left to remove on our robot. It only lowers the success rate of RACE, from 33.0 at to 28.5 at 30 and 24.0 at 40, since the errors at the end of the chunk grow sharply, as shown in (2).
In summary, RACE does not remove all errors of long-horizon prediction; at very long chunks, the errors that grow with the chunk length remain large. By reducing the errors at transitions, however, it makes chunk extension reliable up to the length needed in practice (i.e., ), which already removes stop-and-go execution on our robot.
We have added the results in (1) to Appendix B.2 (Tab. 11) and the error analysis in (2) to Appendix D.6 (Tab. 34) of the revised version (highlighted in red). We have also revised Sec. 4.4 and extended Fig. 5b to , and discuss the smaller gain with very long chunks in Appendix F.1 (Limitations) of the revised version (highlighted in red).
Response to Reviewer Hg7g(1/8): W1
We thank Reviewer Hg7g for the constructive feedback and for recognizing the value of our perspective on transition timing, the intuitive design of our method, and the clear writing and presentation. We address each concern and question below. Table, figure, section, and line numbers below refer to the revised version.
W1. Error reduction at transitions.
Thank you for raising this important point. In Fig. 4, the reduction may appear modest because the errors at early positions are small for both models. The reduction is larger near the end of the chunk, where the errors at transitions are the largest. To make this clearer, we quantify the reduction on VLABench training demonstrations with Hexec=20 and 10. As in Fig. 4, we compare errors at transitions with those of non-transition actions at the same chunk position. We report the errors over all positions (All) and over the early, middle, and late positions within the chunk.
Results with Hexec=20.
Results with Hexec=10.
RACE reduces the errors at transitions over all positions, and this reduction is statistically significant at both lengths (see our response to Q2). The reduction at transitions is also larger than that at non-transitions; their difference (i.e., reduction gap) is 0.196 at Hexec=20 and 0.159 at Hexec=10 (All). At both lengths, the gap is largest at the late positions (0.878 and 0.713), which are the least reliable part of a long chunk, since their actions are predicted farthest ahead in open loop. Accordingly, RACE achieves a higher success rate than fine-tuning at both lengths (33.0 vs. 28.1 and 34.4 vs. 32.2; Tab. 1).
As complementary evidence on held-out demonstrations, we measure when the generated gripper command switches (Tab. 31 in Appendix D.4). At Hexec=20, RACE switches within ±1 step of the demonstrated step more often than the fine-tuned π0.5 (91.5% vs. 86.9%). It also misses fewer switches (3.9% vs. 6.3%).
We have added these results to Appendix D.6 of the revised version (Tab. 33; highlighted in red).