acceptodds
Under review as a conference paper at ICLR 2027

Progress-aware Visual Token Pruning for Efficient Vision-Language-Action Manipulation

Abstract

Vision-language-action (VLA) models have shown great promise for embodied AI, yet repeatedly processing hundreds of visual tokens during robot execution incurs substantial computational overhead. Existing pruning methods reduce this cost by retaining task-relevant visual tokens and discarding irrelevant ones. However, the retention priority of task-relevant evidence changes with execution progress, particularly in long-horizon manipulation involving multiple subtasks. To address this issue, we introduce ProgPrune, a training-free method that adapts visual-token pruning to execution progress. Specifically, ProgPrune first parses instructions into ordered subtasks and uses execution feedback to track the active subtask and its execution phase. Guided by this progress, ProgPrune dynamically prioritizes visual tokens relevant to the active subtask over those associated with completed or future subtasks. This allows task-relevant but currently unnecessary tokens to be pruned, while full-instruction guidance preserves global context. As decoder representations evolve, we introduce Fresh-QK to recompute progress-aware token relevance at successive pruning layers. This keeps later pruning decisions from relying on outdated relevance estimates, without additional forward passes. Experiments on LIBERO, SimplerEnv, and CALVIN across multiple VLA architectures show consistent 1.20–1.44× speedups improvement with competitive task success rate. Notably, ProgPrune achieves particularly strong efficiency on long-horizon tasks (e.g., LIBERO-Long and CALVIN), demonstrating its effectiveness of progress modeling in visual token pruning.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-Language-Action Models, Pruning methods, Progress-awareness

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_xSJcOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes ProgPrune, a training-free visual token pruning framework for vision-language-action models. A deterministic instruction parser and an execution-feedback tracker identify the active subtask and phase, which determine the instruction spans used to guide token selection. The method combines active-context guidance with a full-instruction reserve, refreshes relevance scores using current-layer Q/K projections, and applies motion-dependent pruning schedules. Experiments evaluate task success and computational efficiency across multiple manipulation benchmarks and VLA backbones.

Critical strengths

Controlled experiments provide useful support for the proposed design. Under the same configured pruning schedule, combining active-context and full-instruction guidance improves LIBERO-Long success from 92.0% to 94.2% compared with full-instruction-only scoring. The appendix further reports switching accuracy, paired interventions, and uncertainty estimates, providing useful evidence about the progress mechanism. Complete action-call measurements also show practical speedups in several settings.

Critical weaknesses
  1. The broader benefit of progress-aware pruning is insufficiently established. Table 10 shows that CALVIN never enables the progress tracker, so its results cannot establish the benefit of internally tracked progress. On RoboCasa365, complete action-call latency is essentially unchanged relative to the dense policy, while success decreases from 57.72% to 51.16%. These results limit the evidence for the central mechanism and its practical value in long-horizon manipulation.

  2. The independent novelty of Fresh-QK is not sufficiently clear. ADP already uses text–visual QK relevance scores, and SpecPrune-VLA updates token importance across decoder layers. The Fresh-QK ablation supports its usefulness, but the additional methodological insight beyond progress-conditioned text selection remains unclear.

Questions for authors
  1. Attribution of long-horizon gains. Given that CALVIN never enables progress tracking, which components account for its improvements? What evidence isolates the benefit of internally tracked progress beyond LIBERO-Long?

  2. Practical efficiency on compositional tasks. What explains the essentially unchanged complete action-call latency on RoboCasa365 despite reduced analytical FLOPs? Given the accompanying success-rate decrease, what evidence supports a favorable measured performance–latency trade-off in this setting?

  3. Contribution of Fresh-QK. What is the new methodological insight in Fresh-QK beyond the QK-based scoring in ADP and the layer-wise importance updates in SpecPrune-VLA? Please distinguish this from the contribution of progress-conditioned text selection.

Reviewer_sDPZOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ProgPrune, a training-free visual-token pruning method for vision-language-action models. Its central idea is that visual evidence relevant to the overall instruction may become less important as execution progresses. A rule-based instruction parser and an execution-feedback state machine identify the active subtask and manipulation phase, which guide token selection alongside a full-instruction reserve. The proposed Fresh-QK mechanism refreshes relevance scores at successive pruning layers using current query/key projections. Experiments on LIBERO, SimplerEnv, and CALVIN demonstrate reduced LLM computation and decoder-prefill latency, with broadly comparable aggregate task performance, although some individual tasks exhibit notable degradation.

Critical strengths
  1. A well-motivated use of task progress for visual computation allocation. The distinction between instruction-wide relevance and relevance to the current execution stage is meaningful for multi-step robotic manipulation. The controlled ablation on LIBERO-Long provides supporting evidence: combining active-context and full-instruction guidance achieves 94.2% success, compared with 92.0% using full-instruction guidance alone under the same pruning schedule. This is the paper’s clearest conceptual contribution.

  2. A lightweight approach with evidence across multiple evaluation settings. ProgPrune does not require backbone retraining or an additional decoder forward pass, and the reported progress tracker avoids privileged object poses and simulator success predicates. Evaluation across multiple benchmarks and VLA backbones supports its potential as a practical inference-time modification, even though broader robustness remains insufficiently established.

Critical weaknesses
  1. The methodological advance is limited, and its central progress-tracking assumption is insufficiently validated. The framework largely builds on existing action-aware pruning and layer-wise relevance scoring; the main distinguishing element is explicit subtask/phase conditioning. However, this conditioning depends on a deterministic parser and a state machine driven by gripper and motion events. Such events do not by themselves establish semantic task completion: grasp failures, incorrect placements, retries, or execution orders that differ from the parsed order can produce incorrect progress estimates. An erroneous transition may then deprioritize visual evidence still needed for successful execution. The main paper does not quantify tracking accuracy or adequately demonstrate recovery from incorrect transitions. Consequently, it remains unclear whether the added mechanism offers a robust, general improvement beyond structured benchmark settings.

  2. The demonstrated practical efficiency gain is modest and accompanied by unresolved task-level regressions. The headline 1.20–1.44× speedups represent analytical LLM FLOPs ratios rather than complete action-inference acceleration. Measured decoder-prefill speedups are approximately 1.15–1.27×, and on LIBERO the latency improvement over the closest baseline, ADP, is only about 0.7% (83.99 ms versus 83.42 ms). Meanwhile, CALVIN SR@5 decreases from 55.1% to 53.7%, and CogACT’s DrawerApple success rate drops from 52.8% to 45.4%. These results do not establish a consistently favorable practical trade-off. Without complete action-call timing, statistical uncertainty, and an explanation of the larger regressions, the evidence is insufficient to demonstrate a substantial and reliable deployment benefit.

Questions for authors
  1. Clarification of the methodological contribution: Beyond changing the text context used for scoring, what distinguishes Fresh-QK and the pruning schedule from existing QK-based and layer-wise pruning approaches such as ADP and SpecPrune-VLA? Which controlled results establish the contribution of these differences?

  2. Evidence concerning progress-tracking reliability: What fraction of evaluated instructions is successfully parsed, and how accurately does the tracker identify subtask and phase transitions? How does it handle failed grasps, incorrect releases, retries, or a policy executing subtasks in a different order? Please clarify whether incorrect transitions can be reversed and what evidence supports robustness in these cases.

  3. Clarification of practical acceleration: What are the complete action-call latencies for the dense policy, ADP, and ProgPrune under matched execution settings, including visual encoding, action generation, and pruning/tracking overhead? The main text refers to Appendix A.1; please identify the relevant measurements and clarify whether the advantage over ADP remains meaningful at this level.

  4. Evidence concerning performance reliability: Are the reported gains and regressions consistent across repeated evaluations or random seeds? In particular, what explains the DrawerApple degradation, and do uncertainty estimates or failure analyses indicate whether it stems from incorrect progress tracking, removal of action-critical tokens, or evaluation variability?

Reviewer_yGmuOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes ProgPrune, a training-free visual-token pruning method that uses a deterministic instruction parser and execution-feedback state machine to prioritize visual evidence for the active manipulation subtask and phase, while reserving capacity for full-instruction context. Its Fresh-QK component refreshes relevance scores from current query/key projections at successive pruning layers within the same VLA forward pass. Experiments on LIBERO, SimplerEnv, CALVIN, and RoboCasa365 report 1.20–1.44× analytical LLM-compute speedups. The reported decoder-prefill measurements imply approximately 1.15–1.27× speedups, while Appendix A.1 reports 1.00–1.18× measured complete action-call speedups. Task-performance effects are mixed across benchmarks.

Critical strengths
  1. Coherent and training-free design. ProgPrune combines progress-conditioned instruction context, a global full-instruction reserve, and Fresh-QK relevance refresh within the same forward pass. The controlled ablations support the claimed mechanism: active-plus-full guidance reaches 94.2% on LIBERO-Long versus 92.0% for full-instruction guidance, while Fresh-QK reaches 94.2% versus 91.0% for entry-QK replay. This is consequential because the improvements are supported by component-level comparisons rather than only by end-to-end results.

  2. Broad evaluation with substantially improved reproducibility. The paper evaluates multiple benchmarks and policy configurations, and the appendix specifies the parser, tracker transitions, routing schedule, token budgets, and matched action-call timing protocol. This is consequential because it makes the method easier to assess and reproduce across different VLA settings.

Critical weaknesses
  1. The reliability and recovery behavior of the progress tracker remain insufficiently established. The appendix reports 81.04% task-macro F1 at a ±1-query tolerance, but also reports unsafe or spurious predictions and substantial variation in progress coverage across benchmarks. The paper does not evaluate how tracking errors affect token selection, whether incorrect transitions can be reversed, or how the absorbing COMPLETE state is handled after an erroneous transition. This is consequential because progress tracking is the central mechanism underlying the claimed advantage of progress-aware pruning.

  2. The practical efficiency advantage and accuracy trade-off remain uncertain. Analytical speedups are 1.20–1.44×, whereas measured complete action-call speedups are only 1.00–1.18×, with no measured speedup on RoboCasa365. Accuracy also regresses relative to Dense on several settings, including RoboCasa365, Google Robot, CALVIN, and DrawerApple. This is consequential because the central claim is that the method improves efficiency while preserving task performance, yet the practical benefit and failure cases are not fully characterized.

Questions for authors
  1. How do progress-estimation errors affect token selection and task success? In particular, can the tracker recover from a misplaced release, false grasp detection, or an erroneous transition to the absorbing COMPLETE state?

  2. How should the reported 1.20–1.44× analytical speedups be interpreted relative to the 1.00–1.18× measured complete action-call speedups? Do the complete-call measurements include all tracking, routing, scoring, and token-selection overhead, and can the authors provide timing variability or a matched complete-call comparison with VLA-ADP?

  3. What explains the performance regressions relative to Dense on RoboCasa365, Google Robot, CALVIN, and especially DrawerApple? Are these regressions associated with tracking errors, whole-instruction fallback, token budgets, or particular routing branches?

  4. Since CALVIN supplies atomic instructions externally and resets the tracker at each new instruction, what evidence supports the claim that the method autonomously tracks multiple subtasks within an unchanged instruction?

Reviewer_DuXLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes ProgPrune, a training-free visual token pruning method for efficient VLA manipulation. The key idea is to adapt token retention to task execution progress, rather than relying solely on full-instruction relevance. ProgPrune uses a rule-based progress tracker to identify the active subtask and execution phase. It then prioritizes visual tokens relevant to the current manipulation while preserving some global context. The proposed Fresh-QK mechanism further updates token relevance across decoder layers using current query-key projections.

Experiments on LIBERO, SimplerEnv, and CALVIN show 1.20–1.44× analytical speedups with generally competitive task performance. The paper addresses a relevant problem and presents encouraging results, especially on long-horizon tasks. However, the reliability of progress tracking and the practical benefits of the proposed design require stronger evidence.

Critical strengths
  1. The paper addresses a relevant problem in efficient VLA inference. Visual tokens that are useful for the overall instruction may not be equally important at every stage of execution. Prioritizing visual information based on the active subtask is a reasonable approach, particularly for long-horizon manipulation involving multiple subtasks.
  2. ProgPrune is training-free and does not require retraining the underlying VLA policy. The progress tracker uses robot execution feedback rather than privileged simulator information. The combination of active-subtask guidance and full-instruction relevance also provides a reasonable way to preserve global context while prioritizing the current task.
  3. The proposed Fresh-QK mechanism updates token relevance at successive pruning layers using current query-key projections, without additional forward passes. The ablation results in Table 3 support this design. Fresh-QK improves LIBERO-Long success from 91.0% to 94.2% compared with reusing entry-layer scores under the same pruning schedule and analytical compute budget.
  4. The evaluation covers several VLA architectures and manipulation benchmarks, including LIBERO, SimplerEnv, and CALVIN. The authors also provide ablations of progress conditioning, pruning schedules, and layer-wise scoring. On LIBERO-Long, ProgPrune matches the dense model's 94.2% success rate. It also achieves a 1.39× analytical speedup across LIBERO, showing a favorable accuracy–efficiency trade-off.
Critical weaknesses
  1. The reliability of progress tracking needs further validation. Section 3.2 uses gripper and motion feedback to identify execution transitions. Although the method includes validated grasp and release events, these signals do not necessarily guarantee successful manipulation. Incorrect progress estimates could cause important visual tokens to be discarded prematurely. A quantitative evaluation of tracking accuracy and its impact on pruning performance would strengthen the paper.
  2. It remains unclear how well the rule-based progress tracker generalizes to more complex instructions. The method relies on predefined subtask structures and transition rules. Unsupported or ambiguous instructions fall back to full-instruction guidance. The authors could report how frequently this fallback occurs across the evaluated benchmarks and how it affects pruning efficiency.
  3. The efficiency analysis could be clearer. On LIBERO, ProgPrune achieves a 1.39× analytical speedup, while the measured decoder-prefill speedup is approximately 1.18× (98.82 ms to 83.42 ms). Although these metrics measure different aspects of efficiency, a clearer discussion of this difference would be useful. In particular, the authors could clarify the overhead introduced by progress tracking and token selection, and how it affects the overall inference latency.
  4. Some benchmark settings show performance degradation after pruning. For example, DrawerApple success decreases from 52.8% to 45.4% on Google Robot VM, while CALVIN SR@5 drops from 55.1% to 53.7%. These reductions may be acceptable given the efficiency gains, but further analysis of the larger performance drops and uncertainty estimates for the reported success rates would make the experimental findings more convincing.
Questions for authors

N/A

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.