acceptodds
Under review as a conference paper at ICLR 2027

Don't Throw Away the Tail: Action Upcycling for Policy Acceleration

Abstract

Modern robot policies predict a chunk of future actions from a single observation, execute only a prefix, and discard the rest before replanning. Choosing the length of this prefix, the execution horizon, poses a trade-off between reactivity and efficiency. A short horizon keeps the policy reactive to the environment, but requires frequent policy calls. Recent test-time methods adaptively select the horizon for each chunk, but they either read model internals, where the signal must be chosen for each architecture, or draw extra samples, which adds cost. We propose *Action Upcycling*, a training-free algorithm that reuses actions the policy would otherwise discard, without accessing model internals or drawing extra samples. We find that discarded actions stay close to their replanned versions as long as the action velocity remains smooth. Action Upcycling therefore extends the execution horizon up to the point where the velocity begins to fluctuate. Extensive experiments on simulated and real-world manipulation tasks show that Action Upcycling reduces policy calls by 1.2–1.7 with no loss in success rate, across multiple Vision-Language-Action Models (VLAs) and even a World Action Model (WAM). It applies to any chunked policy at negligible cost and is orthogonal to other policy acceleration methods such as few-step sampling and streaming action decoding, opening a new axis for policy acceleration. Videos are available at https://actionupcycling.github.io/.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-Language-Action Models, Action Chunking, Efficient Policy Inference

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 2 reviews, above 27% of the others.
Reviewer_RfkGOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The task is to reduce robot policy inference cost (diffusion/flow-based VLAs and world action models) by deciding how many predicted actions to execute before observing and replanning. The baselines use a fixed execution length or adapt it using attention weights or uncertainty across multiple predictions. The paper executes more actions from each predicted sequence when motion stays smooth. The method measures smoothness by summing the magnitudes of changes between consecutive predicted velocities. It assumes smooth action sequences remain reliable for longer without new observations.

Critical strengths
  • The method requires no retraining, model internals, or additional predictions.
  • Across three simulation benchmarks, policy calls decrease by 1.2-1.7x while reported success stays equal or improves. Physical robot tasks finish 11-14% faster. The experiments cover four policies, including a world action model.
Critical weaknesses
  • Against AutoHorizon, success increases by only 0.5-3.7 percentage points. Inference time improves only 1-2% on two benchmarks and worsens on the third. Real-robot tests use only 10 trials per task, for which the gains are not statistical significant assuming a 95% confidence interval.
Questions for authors
  1. How does the method select the target reuse ratio without success labels?
  2. Does smoothness predict reliability, or just easy motion? Maybe there is a way to test smooth but incorrect predictions and unexpected disturbances?
1 comment
AuthorsOpenReview ↗

Response by Authors

We sincerely thank the reviewer for the valuable comments. We have conducted new experiments on a label-free selection of τ\tau, a reliability analysis across motion difficulty, and robustness to unexpected disturbances. All new results are included in the revised paper (highlighted in blue). We address each concern below.


W1. Comparison with AutoHorizon and statistical significance.

We would like to clarify that the main advantage of our method over AutoHorizon [1] is that it requires no access to model internals. This allows our method to be applied directly to diverse models, including a World Action Model, which is not covered by AutoHorizon. On top of this practical advantage, our method achieves comparable or better performance gains. Notably, on RoboTwin 2.0, AutoHorizon falls below the baseline (59.7 → 56.8), whereas our method does not (59.7 → 60.5).

For the real-robot experiments, we note that AutoHorizon likewise reports 10 trials per task. Nevertheless, we agree that the gains in success rate are not statistically significant, and we have toned down our claims accordingly. Please refer to the revised Section 4.2.


Q1. Label-free selection of the upcycling ratio.

Thank you for this important question. The upcycling ratio, and implicitly τ\tau, can be selected without success labels. For example, we set τ\tau to the median velocity fluctuation of the prefix actions, computed from 10 random baseline episodes without success labels, and use this single τ\tau for all tasks. As shown in Table 1, this simple rule maintains or improves success rate while reducing policy calls by about 1.7×, further supporting its practicality. Please refer to the revised Appendix B.3.

Table 1. Label-free selection of τ\tau on LIBERO.

PolicyBaselineOursCall reduction
π0.596.997.81.69×
GR00T N1.796.196.11.73×

Q2. Reliability vs. motion difficulty, and unexpected disturbances.

In response to this question, we examine whether velocity smoothness indicates reliable tail actions or merely easy motion. We divide each episode into three phases of increasing difficulty: free-space motion, transport, and grasp/release. Within each phase, we compare tails executed and discarded by our method in terms of their error to the replanned actions.

As shown in Table 2, the executed tails show nearly 2× lower error in every phase, including hard motion such as grasp and release. This shows that smoothness predicts reliability, not merely easy motion. Please refer to the revised Appendix B.5.

Table 2. Error of tails executed and discarded by our method (lower is better; π0.5, LIBERO).

Phase (difficulty)Executed tailDiscarded tail
Free-space motion (easy)0.094 (×0.54)0.173
Transport (moderate)0.075 (×0.51)0.146
Grasp / release (hard)0.133 (×0.56)0.237

Furthermore, to test unexpected disturbances, we apply object displacement and end-effector push (2/4/6 cm) at a random step during execution. As shown in Table 3, our method performs comparably to or better than the baseline under all disturbances (95.3 → 95.8 on average), while reducing policy calls by about 1.5×. These results show that our method remains robust under unexpected disturbances. Please refer to the revised Appendix B.7.

Table 3. Success rate (%) under execution-time disturbances (π0.5, LIBERO).

DisturbanceMagnitudeBaselineOursCalls / ep.
End-effector push2 cm96.897.232.3 → 22.1
4 cm95.796.732.9 → 22.2
6 cm95.295.333.4 → 22.7
Object displacement2 cm96.896.732.6 → 22.1
4 cm94.294.833.3 → 22.7
6 cm92.993.934.5 → 23.2
Average95.395.833.2 → 22.5

Reference

[1] Wang et al., "VLA Knows Its Limits: Adaptive Execution Horizons for Robot Policies", ECCV 2026.

Reviewer_J2NWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3(+1)
Soundness
2
Read review
Paper summary

This paper studies the setting of robot policies (VLAs and WAM), trained with action chunking. The authors discuss a trade-off at execution time between executing the whole horizon chunk of size (H), versus executing only a short prefix (h). While executing a larger part of the chunk reduces the number of model inference, allowing for faster inference, it reduces policy reactively. Instead of committing to a fixed prefix size (h) to execute, the paper considers adaptative execution horizon methods which vary the executed prefix size at each prediction. Towards this, the paper investigates the relationship between actions that would be normally discarded, versus the replanned actions at the same timestep. The paper then demonstrates that discarded actions can be close to replanned actions, and that larger variation in robot velocity of the discarded actions, seems to relate to larger variation to the future action. This motivates their method, action upcycling, which tracks the norm of cumulative differences in velocities (acceleration) across a predicted action chunk. For each chunk, actions are taken until this cumulative difference exceeds a threshold. This threshold is calibrated via a target execution horizon set before execution. The paper then compares their method on both simulated and real-world experiments for several VLAs and a WAM. On simulated experiments, their method can reduce execution time while maintaining (or slightly increasing success) rate, and compares favorably to prior methods. The paper then demonstrate that this setup translates to real-world policies, where similar improvements hold.

Critical strengths
  • Presentation: The writing of the paper is clear, and easy to understand.

  • The paper makes an interesting observation that there may be a relationship between current and future predicted chunk, and the velocity smoothness of the current chunk, leading to a simple method that seems easy to use.

  • The paper evaluates across several policies, simulated benchmarks, and translates setup to real-world robotics.

Critical weaknesses
  • One of the main concerns I have, is how effective this strategy is versus finding a good fixed increase in mean execution length, as shown in Table 5. Here, the simple baseline of using an increased fixed-length execution maintains the same speed increase as your method, with a slightly lower success rate (roughly a percentage point).

  • Related to this point, my understanding of Table 5, is that it is comparing at the tuned \bar{h} as shown in Figure 6, i.e. at the best \bar{h} for adaptative, but this may not be the ideal tuning of \bar{h} for the adaptative method. I think it may be useful to see also a sweep, for fixed-length baseline, or even a pareto-curve to more fairly compare these, i.e. sweeping over various \bar{h} for both fixed-length and adaptative, and plotting all the frontier (i.e. showing like success rate vs execution speed tradeoff).

  • or are there specific tasks, more dynamic tasks, that could better highlight your method vs fixed \bar{h}? Or harder benchmark, i.e. extending the fixed-horizon comparison table 5 to Libero-plus or robotwin? Or conveying what tasks, or part of a task the policy is using a higher vs shorter horizon (e.g. moving in free space vs object interaction).

  • the upcycling ratio, and implicitly, \bar{h} is swept over, is it also possible to infer offline (not just threshold calibrated offline for a target r, to know a good value? Or held out evaluation, i.e. sweeping over r for 5/10 tasks, and reporting improvement on various heldout tasks?

  • Another concern is that it is partially unclear how reliable/general velocity fluctuation is. For example, could there be cases where velocity varies significantly, such as if this is natural in the training data or required for the task). It would be interesting to see/calibrate this metric with the model and training data, e.g. plotting velocity smoothness of the model’s predictions versus the smoothness of ground-truth actions.

Minor clarity: I think it would be useful to highlight or hint earlier in the paper (like the intro, figure 1,2), that when you mean velocity, it refers to the robot’s joint/end-effector velocity. Due to the use of flow matching, until reading Section 3.2, I assumed that velocity was referring to the learned velocity field of the policies (trained with flow matching).

Questions for authors

What is are the quantities of the x and y axis of the replanned action vs discarded action in Figure 1? I understand that this is showing a type of correlation between overlapping actions but it is unclear what is being varied (-1 to 1) in this plot.

Also in Figure 1, for velocity fluctuation vs RMSE, is this the cumulative metric used later in Equation 3, or an average, how is this computed? if this is cumulative, could this correlation reflect that these quantities both monotonically increase along horizon, rather than being correlated?

6 comments
AuthorsOpenReview ↗

Response by Authors (1/2)

We sincerely thank the reviewer for the thoughtful comments. We have conducted new experiments, including a Pareto-curve comparison, an evaluation on harder benchmarks, a phase-wise analysis, a sweep-free selection of τ\tau, and a velocity smoothness comparison. All new results are included in the revised paper (highlighted in blue). We address each concern below.


W1 and W2. Pareto-curve comparison.

Thank you for the constructive comment. In response, we compare fixed-length execution and our method over a range of mean execution lengths hˉ\bar{h} on RoboTwin 2.0 (π0.5). As shown in Table 1, our method slightly improves success over the baseline around hˉ\bar{h}=15 and degrades only gradually beyond it, whereas fixed-length execution degrades steadily and more steeply, placing our method on a better Pareto front. Please refer to the revised Appendix B.4 for the full comparison.

Table 1. Success rate (%) at the same mean execution length (π0.5, RoboTwin 2.0).

hˉ\bar{h}Fixed-lengthOurs
10 (baseline)59.759.7
1558.260.5 (+2.3)
2553.656.6 (+3.0)

W1 and W3. Harder benchmarks and phase-wise execution length.

In response to the suggestion of the reviewer, we extend Table 5 of the main paper to LIBERO-Plus and RoboTwin 2.0. As shown in Table 2, our method achieves a higher success rate on all benchmarks, and the gap is larger on the harder benchmarks, demonstrating the advantage of adaptive execution over fixed-length execution. Please refer to the revised Appendix B.4.

Table 2. Success rate (%) at a similar mean execution length (π0.5).

BenchmarkFixed-lengthAdaptive (Ours)
LIBERO97.197.9 (+0.8)
LIBERO-Plus84.186.5 (+2.4)
RoboTwin 2.058.260.5 (+2.3)

We also analyze the execution length of Action Upcycling in three task phases: free-space motion, transport, and grasp/release. As shown in Table 3, our method executes the whole chunk more often during free-space motion and executes only the prefix more often around grasp and release. This shows that it extends execution where motion is simple and replans where precise interaction is required, which fixed-length execution cannot do. Please refer to the revised Appendix B.5 for the full distribution over all execution lengths.

Table 3. Share of policy calls by executed length (π0.5, LIBERO).

Phase (difficulty)Prefix only (5)Whole chunk (10)
Free-space motion (easy)7.4%26.0%
Transport (moderate)9.2%19.0%
Grasp / release (hard)16.8%6.3%

W4. Sweep-free selection of the upcycling ratio.

Thank you for raising this important point. The upcycling ratio, and implicitly τ\tau, can be selected without a sweep. For example, we set τ\tau to the median velocity fluctuation of the prefix actions, computed from 10 random baseline episodes without success labels, and use this single τ\tau for all tasks. As shown in Table 4, this simple rule maintains or improves success rate while reducing policy calls by about 1.7×, further supporting its practicality. Please refer to the revised Appendix B.3.

Table 4. Sweep-free selection of τ\tau on LIBERO.

PolicyBaselineOursCall reduction
π0.596.997.81.69×
GR00T N1.796.196.11.73×

Following the held-out evaluation suggested by the reviewer, we further evaluate π0.5 only on the 30 LIBERO tasks not included in the 10 episodes used to set τ\tau. On these held-out tasks, our method still improves the success rate from 96.4% to 97.7% (+1.3%p) while reducing policy calls by 1.70×, indicating that τ\tau generalizes beyond the tasks used to set it.


W5. Reliability of velocity fluctuation.

In response to the suggestion of the reviewer, we compare the velocity fluctuation of the predictions of the policy with that of the ground-truth demonstrations in Table 5. Both show larger fluctuations during complex motion (e.g., grasp and release) and smaller fluctuations during simple motion (e.g., free-space motion), and the policy closely follows the velocity smoothness of the data. Please refer to the revised Appendix B.6 for the full distributions in each phase.

Even when a policy naturally exhibits large velocity fluctuations, our method remains effective. When τ\tau is set by the median rule in W4, it is automatically calibrated to the fluctuation level of the policy, so our method stops execution only at larger-than-usual changes.

Table 5. Median velocity fluctuation (π0.5, LIBERO).

PhasePredictionDemonstration
Free-space motion0.2430.282
Transport0.3240.347
Grasp / release0.3850.434
AuthorsOpenReview ↗

Response by Authors (2/2)

Minor. Clarification of "velocity".

Thank you for the suggestion. In response, we have clarified that velocity refers to the end-effector or joint velocity of the robot induced by consecutive actions, rather than the velocity field learned by flow matching. Please refer to the revised Section 1.


Q1. Axes of Figure 1.

Both axes show the action values themselves. For each point, the x-axis is the discarded action and the y-axis is the replanned action at the same timestep. Since LIBERO action values lie in [−1, 1], both axes span this range. Thus, the plot shows that actions at the same timestep are strongly correlated (Pearson correlation between 0.89 and 0.98 across all four policies). Please refer to the revised caption of Figure 1.


Q2. Correlation in Figure 1.

The velocity fluctuation in Figure 1 is not the cumulative metric in Eq. 3, but the magnitude of fluctuation of each discarded action. We compare the deviation (RMSE) from the replanned action according to this magnitude. Thus, the correlation shows that larger fluctuation is associated with larger deviation. Please refer to the revised caption of Figure 1.

First, I appreciate the effort for responding to my questions and providing additional results on the pareto curve, harder benchmark mark, and picking the upcycling ratio. Overall, such as on W1, I am still concerned about the effectiveness of the method versus a fixed increase in the mean execution, as performance is still relatively close as shown in Table 1 in the response ( 2-3 point increase). My concern on the small increase in performance over baselines also holds for the harder benchmarks (Libero-plus, RoboTwin 2.0) as shown in Table 2 in the response.

I appreciate that Table 3, and Table 5, which clarify where the chunk is extended and the similarity between demonstration and policy prediction’s velocities.

Overall, I think these updates increase the clarity of the method’s effect, however I am finding that there is still limited gains, even in harder scenarios. I also concur with Reviewer RfkG concerns on significance.

At this time, I have increased presentation 2→3, and maintained weak rejection overall.

Comments on the paper

6 comments
AuthorsOpenReview ↗

General Response by Authors

We sincerely thank the reviewers for their constructive and thorough comments. We are encouraged that the reviewers recognized that our paper makes an interesting observation that leads to a simple and easy-to-use method (J2NW, 31na), is genuinely model-agnostic, requiring no retraining, model internals, or additional predictions (RfkG, 31na), and is evaluated on several policies including a World Action Model and in real-world robotics (J2NW, RfkG).

In response to the comments, we have made the following major revisions, including additional experiments, highlighted in blue in the revised paper:

  • Automatic selection of the upcycling ratio without a sweep or success labels (Appendix B.3)
  • Pareto-curve comparison with fixed-length execution and evaluation on harder benchmarks (Appendix B.4)
  • Reliability analysis of the velocity fluctuation signal (Appendices B.5 and B.6)
  • Robustness under dynamic disturbances (Appendix B.7)
  • Clarifications on velocity, Figure 1, and the real-world results (Section 1 and Section 4.2)

Below, we address the comments of each reviewer in detail.

AuthorsOpenReview ↗

Response by Authors

We sincerely thank the reviewer for the insightful comments. We have conducted new experiments on an automatic selection of the upcycling ratio, robustness to dynamic disturbances, and adaptive behavior across task phases. All new results are included in the revised paper (highlighted in blue). We address each concern below.


W1. Automatic selection of the upcycling ratio.

Thank you for the constructive comment and great suggestion. The upcycling ratio, and implicitly τ\tau, can be selected automatically without task-specific tuning. For example, we set τ\tau to the median velocity fluctuation of the prefix actions, computed from 10 random baseline episodes. As shown in Table 1, this simple rule maintains or improves success while reducing policy calls by about 1.7×. Since τ\tau follows the fluctuation level of each policy or deployment, it yields a policy- or deployment-specific selection without manual tuning, further supporting its plug-and-play nature. Please refer to the revised Appendix B.3.

Table 1. Automatic selection of τ\tau on LIBERO.

PolicyBaselineOursCall reduction
π0.596.997.81.69×
GR00T N1.796.196.11.73×

Q1. Robustness to dynamic disturbances.

In response to this question, we apply object displacement and end-effector push (2/4/6 cm) at a random step during execution. As shown in Table 2, our method performs comparably to or better than the baseline under all disturbances (95.3 → 95.8 on average), while reducing policy calls by about 1.5×. This shows that our method remains robust to dynamic disturbances. Please refer to the revised Appendix B.7.

Table 2. Success rate (%) under dynamic disturbances (π0.5, LIBERO).

DisturbanceMagnitudeBaselineOursCalls / ep.
End-effector push2 cm96.897.232.3 → 22.1
4 cm95.796.732.9 → 22.2
6 cm95.295.333.4 → 22.7
Object displacement2 cm96.896.732.6 → 22.1
4 cm94.294.833.3 → 22.7
6 cm92.993.934.5 → 23.2
Average95.395.833.2 → 22.5

Q2. Identifiable failure modes.

We did not observe identifiable failure modes in which upcycled actions substantially deviate from the replanned trajectory. This is because our method does not always extend execution, but seeks a balance between speed and reactiveness, with an average length of only 1.2 to 1.7× the original horizon.

To examine this further, we analyze the execution length of Action Upcycling depending on the motion difficulty. We divide each episode into three phases of increasing difficulty: free-space motion, transport, and grasp/release. As shown in Table 3, our method executes the whole chunk more often during free-space motion and executes only the prefix more often around grasp and release. This shows that it extends execution in easy phases and replans frequently in difficult phases, which helps avoid such failure modes. Please refer to the revised Appendix B.5 for the full distribution over all execution lengths.

Table 3. Share of policy calls by executed length (π0.5, LIBERO).

Phase (difficulty)Prefix only (5)Whole chunk (10)
Free-space motion (easy)7.4%26.0%
Transport (moderate)9.2%19.0%
Grasp / release (hard)16.8%6.3%

Q3. Why does the method improve success?

We attribute this to the ability of Action Upcycling to adapt the execution length to each policy call. The optimal execution length differs across calls: during free-space motion, executing a longer portion of the chunk is beneficial as it preserves temporal consistency, as the reviewer suggested, whereas near contact transitions, frequent replanning is beneficial for reacting to the environment. As a result, our method achieves better performance at the same mean execution length hˉ\bar{h}, placing it on a better Pareto front.

To support this explanation, we compare our adaptive execution with fixed-length execution at the same hˉ\bar{h}. As shown in Table 4, our method slightly improves success over the baseline around hˉ\bar{h}=15 and degrades only gradually beyond it, whereas fixed-length execution degrades steadily and more steeply. This shows that the gain comes from adapting the execution length to each policy call, rather than from simply executing longer. Please refer to the revised Appendix B.4 for the full comparison.

Table 4. Success rate (%) at the same mean execution length (π0.5, RoboTwin 2.0).

hˉ\bar{h}Fixed-lengthOurs
10 (baseline)59.759.7
1558.260.5 (+2.3)
2553.656.6 (+3.0)
AuthorsOpenReview ↗

A Kind Reminder for Comments

Dear Area Chair and Reviewers,

We sincerely thank the reviewers for their constructive feedback and careful evaluation of our work. We have provided detailed point-by-point responses and updated the manuscript with additional experiments and clarifications addressing the reviewers’ concerns. The key changes are summarized in our individual reviewer responses, and all revisions in the manuscript are highlighted in blue.

We would greatly appreciate the Area Chair’s help in inviting reviewers to engage in further discussion and consider our responses. We welcome any remaining questions and would be happy to provide further clarification.

Thank you again for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.