acceptodds
Under review as a conference paper at ICLR 2027

Complementary Future Token Pruning for World Action Models

Abstract

World action models (WAMs) jointly predict future visual states and robot actions, but processing dense future-frame tokens incurs substantial inference latency. We propose CF-Prune, a training-free framework for pruning future-frame tokens in pretrained WAMs. Our key insight is that future visual predictions support ac- tion generation and therefore need not preserve every visual detail required for high-fidelity video synthesis. CF-Prune combines instruction and action attention to identify task-relevant tokens during the initial denoising forward pass, even when future-frame and action representations remain highly noisy. An entropy- based phase identification mechanism further retains dense computation during precision phases to mitigate pruning-induced performance degradation. Exper- iments on RoboLab, RoboCasa, and real-world manipulation tasks demonstrate consistent inference speedups. At a 35% pruning ratio, CF-Prune achieves average speedups of 1.19–1.27× while improving task success rates. At a 50% pruning ratio, it achieves 1.29–1.33× average speedups on both simulation benchmarks, with overall success rates within 1.0 percentage point of the baseline.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Robot Policy, World action model, Token Pruning

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_qhVPOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes CF-Prune, a training-free method for accelerating pretrained world action models by removing future-video tokens while retaining observation tokens. It constructs a pruning mask during the first conditional forward pass using complementary instruction and action attention, with Gaussian smoothing and separate treatment of wrist and third-person views. An attention-entropy criterion disables pruning on calls identified as requiring precise interaction. The method is evaluated with Cosmos3-based policies on 120 RoboLab tasks, four RoboCasa tasks, and three real-robot tasks. At 35% pruning, the reported average inference speedups are 1.19-1.27x, with higher measured success rates. At 50%, the simulation speedups reach 1.29-1.33x and aggregate success remains close to the dense baseline, although some individual tasks deteriorate substantially.

Critical strengths
  1. The paper targets a useful source of computational redundancy: future visual representations used to support action prediction. Being able to remove these tokens from an existing checkpoint without retraining is practically appealing. I find the distinction between preserving visually detailed predictions and preserving information needed for control well motivated. The measured gains remain useful even with only four denoising steps on RoboLab and the real robot.

  2. The evaluation goes beyond a token-count or FLOPs argument. The authors report measured inference speedups and closed-loop success, distinguish coarse-phase from average speedup, and include both simulation and physical manipulation. The broad RoboLab evaluation is a particular strength. Table 4 also provides some evidence that combining the two attention sources is useful, rather than relying on either alone.

Overall, I lean toward weak acceptance. The results make a credible case for a useful training-free speed/accuracy tradeoff, even though the explanation of which components are responsible needs stronger controls.

Critical weaknesses
  1. The experiments do not fully isolate the benefit of the proposed selection and phase criteria. The speed-matched Obs + future VLA baselines prune observations as well as future tokens, whereas CF-Prune preserves observations. Matching average latency is helpful, but this still leaves open whether some of the advantage comes from where pruning is applied rather than how tokens are ranked. Similarly, the phase ablation raises success from 35.3% to 37.8% by restoring dense computation on 31.4% of calls. Without a control that restores dense computation on the same fraction of calls, or uses milder fixed pruning at comparable latency, this does not establish that the entropy criterion identifies especially valuable calls. These controls matter because complementary selection and phase identification are the main methodological contributions.

  2. The evidence for improved or preserved success is less secure than the headline averages suggest. No confidence intervals or repeated-seed variability are reported. The real-world gain from 85.3% to 90.7% corresponds to four additional successes across 75 trials, and the RoboLab gain at 35% pruning is 1.3 percentage points. These are encouraging observations, but they do not yet substantiate the proposed regularization explanation. At 50% pruning, RoboCasa navigation falls from 38% to 18%, despite an aggregate drop of only one point. The authors acknowledge this failure, but it places an important limit on the performance-preservation claim. All experiments also use Cosmos3-based policies, so the evidence currently supports this model family rather than WAMs generally.

Questions for authors
  1. With observation tokens preserved and the phase schedule held fixed, how does CF-Prune compare with the adapted VLA scoring rules, or a simple uniform/random future-token mask, at comparable latency? This would help separate the value of the proposed scoring rule from the choice to prune only future tokens.

  2. How were the three phase thresholds in Appendix A.1 selected, and were the evaluation tasks or rollouts used in that selection? On RoboLab, does entropy-based gating outperform random dense-call allocation or milder fixed pruning when average inference cost is matched?

  3. Were initial states and sampling seeds matched between methods? Could you provide uncertainty estimates for the main success-rate differences and the Table 4 ablations, using paired outcomes where available? I would also appreciate an analysis of the navigation failures at 50% pruning, particularly whether the assumption of a stable third-person scene breaks down there.

4 comments

We appreciate the reviewer's feedback and the positive assessment of our work. Before addressing the specific concerns, we note that the additional experiments, unless otherwise specified, were conducted on 40 tasks selected from the full set of 120 RoboLab tasks, considering task difficulty and key manipulation characteristics.

W1, Q1, Q2. Effectiveness of Token Selection and Phase Identification

First, we would like to highlight that one of the key contributions of our work is identifying computational redundancy in future-video tokens repeatedly processed in WAMs and introducing them as a new target for pruning. Unlike observation tokens in conventional VLAs, future tokens lack clear visual information during initial denoising, making their importance difficult to estimate. To address this challenge, CF-Prune introduces a new token selection criterion specifically designed for these characteristics.

To evaluate the effectiveness of our selection criterion, we adapted the scoring criteria of SpecPrune-VLA and SP-VLA to prune only future tokens, while preserving observation tokens and keeping the phase schedule fixed. On 40 RoboLab tasks, SpecPrune-VLA and SP-VLA achieved success rates of 32.3% and 34.4%, respectively, compared with 43.3% for CF-Prune. Random pruning also achieved a lower success rate of 37.8%. These results demonstrate that the benefits of CF-Prune extend beyond the choice of pruning future tokens, supporting the effectiveness of our proposed selection criterion itself.

Furthermore, to evaluate the effectiveness of phase identification, we compared CF-Prune with random phase identification and milder fixed pruning at a 25% pruning ratio under matched average inference latency. These methods achieved success rates of 40.5% and 41.8%, respectively, compared with 43.3% for CF-Prune. These results suggest that entropy-based phase identification is more effective at selectively allocating dense computation to important inference calls, rather than simply increasing the computational budget.

Finally, the phase thresholds reported in Appendix A.1 were selected using separate calibration rollouts on a small subset of tasks in each environment. Some calibration tasks overlapped with the evaluation tasks, but no evaluation rollouts were used for threshold selection.

W2, Q3. Statistical Reliability, Navigation Failure, and Backbone Generalization

Statistical Reliability. All methods were evaluated on the full set of 120 RoboLab tasks using identical initial states and random seeds. We computed 95% confidence intervals for the success rate differences between the dense baseline and CF-Prune using paired rollout results. The intervals were [−1.08, +3.83] percentage points for 35% pruning and [−3.25, +1.58] percentage points for 50% pruning. Neither setting showed a statistically significant difference in success rate. Therefore, rather than interpreting the observed improvements as definitive evidence of a regularization effect, we emphasize CF-Prune's primary contribution of accelerating inference while maintaining competitive task performance.

Navigation Performance Degradation. We acknowledge that the performance degradation on Navigation at 50% pruning highlights a limitation of aggressive pruning, which may not preserve performance across all tasks. Navigation is one of RoboCasa's 25 atomic tasks and, unlike typical manipulation tasks, requires broader spatial context and involves changes in spatial relevance as the robot moves. Indeed, we observed higher third-person attention entropy for Navigation than for the other manipulation tasks (0.809 vs. 0.712-0.762), suggesting that aggressive pruning may remove important spatial information when attention is more broadly distributed. Furthermore, CF-Prune's observation-derived shared mask may be less suitable for such dynamic settings. We therefore interpret the degradation as a possible consequence of the interaction between navigation-specific characteristics and aggressive pruning, and acknowledge this limitation.

Backbone Generalization. We agree that the original evaluation was limited to Cosmos3-based models. To address this concern, we additionally applied CF-Prune to Flux3 Action, a different WAM backbone. The results of 35% pruning across all 120 RoboLab tasks are shown below.

DifficultyBaselineOurs (35%)
Simple48.3%49.8%
Moderate35.1%35.4%
Complex28.8%27.6%
Overall41.2%42.0%

CF-Prune maintained a comparable overall success rate of 42.0%, compared with 41.2% for the dense baseline, while achieving a 1.24× inference speedup. These results demonstrate that future-token pruning is not limited to Cosmos3 and can also be applied to another WAM backbone.

We will incorporate these additional results and clarify the corresponding evaluation procedures and limitations in the revision.

Dear Reviewer qhVP,

Thank you again for your thoughtful review and valuable feedback. We have provided detailed responses to address each of your comments and concerns.

As the discussion period is coming to an end, we would greatly appreciate it if you could take another look at our responses. If any part of our explanation remains unclear or if we have misunderstood any of your concerns, we would be sincerely grateful for the opportunity to provide further clarification.

Thank you again for your time and consideration.

Best regards, The Authors

Thank you for the additional experiments and clarifications. My main concerns have been addressed, and I will maintain my score of 3 (Weak acceptance). Please incorporate the supplementary results and acknowledged limitations into the revised manuscript.

We thank the reviewer for confirming that the main concerns have been addressed. We briefly summarize the additional evidence and clarifications below.

1. Controlled comparisons of selection and phase identification. On 40 RoboLab tasks, with observation tokens preserved and the phase schedule fixed, CF-Prune achieved 43.3% success, compared with 32.3% for adapted SpecPrune-VLA scoring, 34.4% for SP-VLA scoring, and 37.8% for random pruning. Under matched average latency, random phase identification and milder fixed pruning at 25% achieved 40.5% and 41.8%, respectively, versus 43.3% for CF-Prune. Thresholds were selected using separate calibration rollouts; some calibration tasks overlapped with evaluation tasks, but no evaluation rollouts were used.

2. Statistical reliability. The full 120-task RoboLab evaluation used identical initial states and random seeds across methods. Paired 95% confidence intervals for the success-rate difference (CF-Prune minus Dense) were [−1.08, +3.83] percentage points at 35% pruning and [−3.25, +1.58] at 50%. Neither difference was statistically significant. Accordingly, we emphasize inference acceleration with competitive task performance rather than treating the observed gains as definitive evidence of regularization.

3. Navigation limitation. We acknowledged the success-rate drop from 38% to 18% on Navigation at 50% pruning. Its broader spatial dependencies and changing spatial relevance are consistent with higher third-person attention entropy (0.809 versus 0.712–0.762 for the other manipulation tasks). We discussed aggressive pruning and the observation-derived shared mask as possible contributing factors, rather than established causes.

4. Additional backbone evaluation. On Flux3 Action across all 120 RoboLab tasks, CF-Prune achieved 42.0% success versus 41.2% for Dense, with a 1.24× speedup, extending the empirical evidence beyond Cosmos3.

We have incorporated the supplementary results and acknowledged limitations into the revised manuscript.

Reviewer_VDvJOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CF-Prune, a training-free method that prunes future-video tokens in pretrained world action models using action and instruction attention.

Critical strengths

CF-Prune is training-free and easy to apply to pretrained WAMs, providing consistent inference speedups without architectural changes.

The action–instruction complementary criterion is intuitive and effective, preserving task-relevant tokens while reducing future-token computation.

Critical weaknesses

A mask derived from the current frame may not remain valid for future frames. In third-person views, the same spatial mask is reused across all future frames, even though the robot and objects may move. Locations that seem irrelevant now could become important later.

It is unclear whether actions actually depend on future tokens. The small performance change after pruning 35–50% of future tokens may mean that the action head barely uses the future branch. Random pruning, removing all future tokens, or blocking action–future attention would help distinguish these explanations.

The quality of future prediction is not evaluated. The paper reports only task success and inference speed, so it remains unclear how pruning affects temporal consistency, physical plausibility, and prediction accuracy. Preserving action performance does not necessarily mean that the model retains reliable world-modeling ability.

Questions for authors

See the Weakness

6 comments

We thank the reviewer for their feedback and the opportunity to clarify our approach. Before addressing the specific concerns, we note that all additional experiments were conducted on 40 tasks selected from the full set of 120 RoboLab tasks, considering task difficulty and key manipulation characteristics.

W1. Validity of Shared Spatial Masks in Third-Person Views

CF-Prune performs token selection at the first denoising step to maximize inference efficiency. At this stage, future representations remain highly noisy, particularly for later future frames where sufficient visual information may not yet have emerged. For wrist views, where substantial visual changes occur due to robot motion, we select tokens independently for each future frame using its corresponding attention information. In contrast, third-person views capture a relatively stable workspace. We therefore use attention from clean observations rather than noisy future representations to construct a shared spatial mask across future frames. To validate this design, we additionally evaluated frame-specific pruning for third-person views. On 40 RoboLab tasks, frame-specific pruning achieved a success rate of 36.0%, compared with 43.3% for our shared-mask approach. These results support the effectiveness of using observation-derived shared masks for third-person views during early denoising.

W2. Contribution of Future-Video Tokens to Action Generation

One of the key contributions of CF-Prune is introducing selective pruning based on the observation that not all future-video tokens need to be processed in WAMs. Nevertheless, we agree with the reviewer that it is important to distinguish whether future tokens contribute meaningfully to action generation and whether our proposed token selection effectively preserves relevant information. To investigate this, we compared CF-Prune with random pruning and complete future-token removal on 40 RoboLab tasks, using the same entropy-based phase gating. While the dense baseline and CF-Prune with 35% pruning achieved success rates of 42.3% and 43.3%, respectively, random pruning achieved 37.8%, and removing all future tokens reduced success to 12.5%. The 29.8 percentage-point drop relative to the dense baseline indicates that future-video tokens play a substantial role in action generation. Furthermore, the higher success rate of CF-Prune compared with random pruning supports the importance of selectively retaining action-relevant tokens rather than simply reducing the number of tokens.

W3. Evaluation of Future Prediction Quality

We agree with the reviewer that preserving action performance does not necessarily guarantee the preservation of future-video prediction quality. However, the central motivation of CF-Prune is that future-video prediction in WAMs serves as an intermediate representation supporting action generation, and that preserving every fine-grained visual detail is not necessary for accurate action prediction. Accordingly, our work does not aim to preserve the visual fidelity of future videos or the full world-modeling capabilities of the pretrained WAM. Instead, we focus on reducing inference computation while retaining the information necessary for accurate action generation. Following this objective, we use task success and inference latency as our primary evaluation metrics.

We will incorporate the additional experimental results and further clarify our research objectives, design rationale, and evaluation scope in the revision.

Dear Reviewer VDvJ,

Thank you again for your thoughtful review and valuable feedback. We have provided detailed responses to address each of your comments and concerns.

As the discussion period is coming to an end, we would greatly appreciate it if you could take another look at our responses. If any part of our explanation remains unclear or if we have misunderstood any of your concerns, we would be sincerely grateful for the opportunity to provide further clarification.

Thank you again for your time and consideration.

Best regards, The Authors

Reviewer_VDvJOpenReview ↗

Response to the rebuttal

Thanks for the response. The results support using shared masks over frame-specific masks estimated from noisy future representations. However, their reliability under substantial robot or object motion remains unclear.

Reviewer_XakZOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes CF-Prune, a training-free method for accelerating pretrained world action models by pruning future-video tokens during joint video-action denoising. It combines an observation-derived instruction relevance prior with action-to-visual attention, applies spatial smoothing, and selects complementary token subsets during the initial denoising forward pass. The selected subset is used throughout the remaining denoising process. An attention-entropy-based phase detector disables pruning for inference calls classified as requiring precise interaction. Experiments with Cosmos3-based policies on RoboLab, selected RoboCasa tasks, and three real-world manipulation tasks report average inference speedups of 1.19–1.27× at a 35% pruning ratio, together with higher aggregate success rates than the dense baseline.

Critical strengths
  1. The paper addresses a practically relevant source of WAM inference cost: dense future-video representations used to support action generation. Pruning these representations without additional training offers a useful way to explore whether full visual detail is necessary for downstream control.
  2. The evaluation includes both simulation and physical manipulation, and reports average speedups that account for inference calls where pruning is disabled. Ablations on complementary token selection, phase identification, and spatial smoothing provide evidence that these components improve aggregate task performance within the evaluated setting.
Critical weaknesses
  1. The incremental contribution over existing phase-adaptive pruning is insufficiently established. CF-Prune shares with VLA-ADP [1] the high-level strategy of pruning during coarse motion and retaining dense computation during precise interaction. The future-token selection and attention-entropy-based phase detector are meaningful differences, but their benefits over prior mechanisms are not isolated experimentally. Although VLA-ADP is cited, no direct comparison with its motion-based phase detector is reported. The phase-identification ablation improves success from 35.3% to 37.8% while restoring dense computation for 31.4% of inference calls. This supports the utility of selective dense computation, but does not establish that the proposed detector is better than an existing detector under a matched computational budget.
  2. The evidence supporting a fixed token subset selected at the initial denoising step is limited. In coupled video-action denoising, low initial attention does not necessarily imply low importance for later updates or for the final action prediction. Removing tokens can alter both attention normalization and the representations of retained tokens. Section 4.4 examines scoring-layer entropy across tasks and rollout stages at the first denoising step; it does not establish that token importance remains stable across subsequent denoising steps. The paper does not report a stepwise analysis of action-prediction deviations or comparisons with delayed pruning or refreshed token selection. The reported task-level results are encouraging, but leave the conditions under which early fixed pruning is reliable insufficiently characterized.
  3. The comparative benefits of token pruning versus other WAM acceleration strategies remain insufficiently characterized. The related-work section acknowledges alternatives such as reducing denoising steps, single-step inference in DreamZero [2], and avoiding explicit future-video generation in Fast-WAM [3] and GigaWorld-Policy [4]. However, the experiments primarily compare token pruning and sparse attention, without establishing when reducing the number of tokens per denoising evaluation is preferable to reducing the number of model evaluations, for example through fewer sampling steps or DreamZero-style DiT caching. Architectural alternatives may require retraining and are therefore not directly interchangeable with CF-Prune; nevertheless, controlled comparisons of compatible inference-time strategies on the same checkpoint are feasible and important. A latency breakdown and a joint evaluation of denoising steps and pruning ratios would clarify the relative contributions of repeated model evaluations, per-evaluation token processing, and other overheads. Without this analysis, the practical advantage of token pruning, particularly in the paper’s few-step settings, remains insufficiently established.

Ref

[1] Action-Aware Dynamic Pruning for Efficient Vision-Language-Action Manipulation.

[2] World Action Models are Zero-shot Policies.

[3] Fast-WAM: Do World Action Models Need Test-time Future Imagination?

[4] GigaWorld-Policy: An Efficient Action-Centered World–Action Model.

Questions for authors

With CF-Prune’s token-selection procedure held fixed, how does the proposed entropy-based phase detector compare with a motion-based detector adapted from VLA-ADP, under a matched computational budget? A comparison at a similar average latency or fraction of dense inference calls would help distinguish the benefit of the proposed detector from that of selectively allocating additional computation.

What evidence supports retaining a fixed token subset selected during the initial denoising forward pass? Do initially low-ranked tokens become important in subsequent steps, and how do action predictions deviate from the dense model as denoising proceeds under identical inputs and initial noise? A targeted comparison with delayed pruning or refreshed token selection would help establish when the proposed design is reliable.

Could the authors adapt representative acceleration strategies developed for other WAM backbones, such as DreamZero’s DiT caching, to the backbone used in this paper and include them as baselines? Evaluating compatible strategies on the same checkpoint, tasks, and hardware would help isolate the benefits of token pruning from differences in model architecture or training. Could the authors additionally apply CF-Prune to another compatible WAM backbone to assess its transferability? Within each evaluated backbone, varying both the denoising-step count and pruning ratio, together with an end-to-end latency breakdown that includes pruning overhead, would clarify the relative benefits of reducing model evaluations versus reducing tokens per evaluation, and whether these strategies provide complementary gains.

5 comments

We thank the reviewer for their detailed feedback. Unless otherwise specified, the additional experiments were conducted on 40 tasks selected from the full set of 120 RoboLab tasks, considering task difficulty and key manipulation characteristics.

W1, Q1. Effectiveness of Entropy-Based Phase Identification

To evaluate the effectiveness of our phase detector, we adapted the motion-based detector from VLA-ADP while keeping CF-Prune's token selection unchanged. We also compared random phase identification and milder fixed pruning under matched average inference latency. On 40 RoboLab tasks, VLA-ADP achieved a success rate of 41.5%, random phase identification achieved 40.5%, and milder fixed pruning at a 25% pruning ratio without phase identification achieved 41.8%, compared with 43.3% for CF-Prune. These results suggest that entropy-based phase identification can more effectively allocate dense computation to important inference calls under the same computational budget.

W2, Q2. Reliability of Initial-Step Token Selection

To evaluate the stability of initial token selection, we compared important tokens across denoising steps on five RoboLab tasks. The retained-token overlaps between the first denoising step (D1) and D2, D3, and D4 were 72.91%, 73.05%, and 73.37%, respectively. We also compared the final action predictions of Dense and CF-Prune under identical inputs and initial noise, obtaining an average relative L2 deviation of 4.38%. These results suggest that a substantial portion of important tokens remains consistent across denoising steps, while changes in the final action predictions remain limited.

Additionally, we compared Fixed and Delayed selection. At the same 35% pruning ratio, Delayed selection using information from D2 achieved a success rate of 42.8% on 40 RoboLab tasks, comparable to 43.3% for Fixed selection. However, Delayed selection offers limited acceleration because pruning is applied to fewer denoising steps. When increasing its pruning ratio to 67% to match inference latency, its success rate dropped to 22.3%. These results support that initial token selection provides an effective speed-accuracy trade-off by maintaining comparable action performance while reducing computation across more denoising steps.

W3, Q3. Comparison with Other WAM Acceleration Strategies

DiT Caching. We adapted DreamZero's DiT caching to the same Cosmos3 checkpoint. Under matched denoising steps and inference latency, DiT caching achieved a success rate of 38.0%, compared with 40.0% for CF-Prune with 50% pruning on 40 RoboLab tasks. These results suggest that CF-Prune achieves an effective speed-accuracy trade-off in few-step WAMs by reducing per-forward future-token computation rather than reusing denoising predictions.

Backbone Generalization. To evaluate the transferability of CF-Prune to another WAM backbone, we applied our method to Flux3 Action. The results of 35% pruning on all 120 RoboLab tasks are shown below.

DifficultyBaselineOurs (35%)
Simple48.3%49.8%
Moderate35.1%35.4%
Complex28.8%27.6%
Overall41.2%42.0%

CF-Prune maintained comparable task performance to the dense baseline while achieving a 1.24× inference speedup. These results support the applicability of CF-Prune beyond Cosmos3 to another WAM backbone.

Denoising Step Reduction and Compatibility. To investigate the relative benefits and compatibility of denoising step reduction and future-token pruning, we compared 3-step Dense inference with 4-step CF-Prune at 50% pruning under matched inference latency using the same Cosmos3 checkpoint. On 40 RoboLab tasks, 3-step Dense achieved a success rate of 38.3%, compared with 40.0% for CF-Prune. Furthermore, combining denoising step reduction with CF-Prune maintained a success rate of 37.5% while achieving a 1.71× speedup relative to the original 4-step Dense baseline. These results show that the two strategies target different sources of computational cost and can provide complementary acceleration benefits.

End-to-End Latency and Pruning Overhead. To analyze the acceleration of CF-Prune, we measured the latency of individual denoising forwards and pruning overhead on an NVIDIA RTX A6000. While the first conditional forward remained unchanged at 499.7 ms, the first unconditional forward decreased from 499.7 ms to 345.1 ms, and the subsequent six forwards (D2-D4) decreased from 2,998.5 ms to 2,070.9 ms. The additional overhead for scoring and phase identification was only 8.7 ms. Consequently, the total inference latency decreased from 3,997.9 ms to 2,924.4 ms, achieving a 26.9% latency reduction and 1.37× speedup, including the additional overhead.

We will incorporate these additional experimental results and further clarify the design rationale, evaluation procedures, and comparison with existing acceleration strategies in the revision.

Dear Reviewer XakZ,

Thank you again for your thoughtful review and valuable feedback. We have provided detailed responses to address each of your comments and concerns.

As the discussion period is coming to an end, we would greatly appreciate it if you could take another look at our responses. If any part of our explanation remains unclear or if we have misunderstood any of your concerns, we would be sincerely grateful for the opportunity to provide further clarification.

Thank you again for your time and consideration.

Best regards, The Authors

Thank you for the additional experiments and clarifications. The comparisons with VLA-ADP and the additional backbone evaluation strengthen the empirical support. However, my novelty concern remains: the overall phase-adaptive pruning strategy is close to VLA-ADP, and the WAM-specific changes still appear incremental. For DiT caching, the response claims matched latency but does not report the actual times for either method. It also remains unclear whether this comparison includes inference calls where pruning is disabled and whether the reported 1.37× speedup corresponds to this setting.

The token-overlap analysis and delayed-pruning comparison partly address my concern about subsequent denoising. However, the reported final action deviation does not show how prediction errors evolve across denoising steps, leaving this aspect insufficiently characterized.

Overall, the additional results improve the validation, but do not fully resolve these concerns. I will therefore maintain my score of 2 (Weak rejection). Please incorporate the supplementary results and clarify the remaining limitations in the revised manuscript.

Reviewer_JXUYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper leverages the assumption that not all pixels are necessary in robotic manipulation, therefore for World Action Models, the authors introduced a training-free framework CF-Prune to identify only task-relevant tokens during the initial denoising forward pass. The pruning-induced performance degradation is mitigated through an entropy-based approach.

Experiments on RoboLab and RoboCasa simulators and a real-world manipulation with three simple but basic tasks have demonstrated significant inference speedup without much of performance loss.

Critical strengths

The attentative latent token selection by

  • identifying the spatial regions relevant to the instruction, complemented by
  • action weighted visual regions by combining both wrist view and third person views.

The challenges in Lines 162-169 are clearly articulated and motivated.

Critical weaknesses

The results section in Lines 312-320 mentioned two VLA pruning/speedup methods adapted to WAM, and one sparse VideoGen. This compares the proposed CF-Prune only empirically with the current methods, lacking theoretical insights.

Ideally the literature review section should have a more detailed introduction of how these different VLA speed-up methods worked, to give a better context to justify the design of the CF-Pruning technique.

Questions for authors

The JEPA models are claimed to learn better world models, which to a certain extent resembles the selective attentions in the latent space, especially in separating the background scene from the manipulable objects, how would a JEPA based world model contributes to action generation?

Would video scene graph anticipation help in such cases?

4 comments

We appreciate the reviewer's feedback. We agree that a more detailed discussion of existing acceleration methods and related approaches would help clarify the motivation behind our design.

W1. Differences from Existing Acceleration Methods and Design Rationale of CF-Prune

SpecPrune-VLA exploits temporal consistency across inference calls by combining historical attention from previous inference calls with current attention and visual changes to select important observation tokens. It further performs layer-wise dynamic pruning and adjusts pruning aggressiveness based on end-effector velocity derived from previously predicted actions to account for different manipulation phases.

SP-VLA combines spatial importance, such as image edges, with semantic relevance to identify and retain important observation tokens. It further employs model scheduling based on historical action trajectories, switching between the full VLA model and a lightweight action generator according to the complexity of the current action.

However, neither method directly considers the action relevance of future-video tokens repeatedly processed during joint denoising in WAMs. Moreover, given the long action horizons of WAMs, decisions based on attention from previous inference calls or historical actions may not adequately reflect changes in currently relevant regions or manipulation phases. To address these limitations, CF-Prune is designed to leverage attention information from the initial denoising forward pass of the current inference call, rather than relying on historical information. Specifically, we account for the distinct characteristics of observations, which provide clear visual information, and noisy future representations, which encode evolving future states. We further combine instruction relevance, which reflects explicit task objectives, with attention from noisy action tokens to visual tokens. This complementary approach leverages both reliable spatial information and temporal relevance to action generation to select important future tokens, even at the first denoising step.

Meanwhile, Sparse VideoGen exploits spatial and temporal sparsity by sparsifying attention connections, but incurs additional computational overhead for sparsity profiling. Particularly in WAMs with few denoising steps, this overhead can offset the acceleration benefits, as also observed in our experiments, where Sparse VideoGen fails to achieve speedup over the dense baseline.

Q1. Relationship Between JEPA-Based World Models and Action Generation

JEPA-based world models predict future dynamics in latent representations rather than reconstructing pixel-level future videos, potentially providing useful information for action generation or planning. This is conceptually aligned with the core motivation of CF-Prune that not all visual details are necessary for accurate action prediction. However, while JEPA focuses on learning predictive representations, CF-Prune reduces future-token computation in pretrained WAMs without additional training.

Q2. Potential Benefits of Video Scene Graph Anticipation

Video scene graph anticipation predicts future object relationships and interactions, potentially complementing future-token selection by identifying task-relevant regions that change over time. However, incorporating such predictions may require an additional predictive model and computational overhead. Given CF-Prune's objective of accelerating inference without additional training, we consider this a potential direction for future work.

We will clarify these distinctions and the design rationale of CF-Prune more explicitly in the revision.

Dear Reviewer JXUY,

Thank you again for your thoughtful review and valuable feedback. We have provided detailed responses to address each of your comments and concerns.

As the discussion period is coming to an end, we would greatly appreciate it if you could take another look at our responses. If any part of our explanation remains unclear or if we have misunderstood any of your concerns, we would be sincerely grateful for the opportunity to provide further clarification.

Thank you again for your time and consideration.

Best regards, The Authors

Dear Reviewer JXUY,

Thank you again for your feedback. Our response explains how SpecPrune-VLA, SP-VLA, and Sparse VideoGen work and how they differ from CF-Prune, and how these distinctions motivate our selection of noisy future tokens using current-call instruction relevance and action-to-visual attention. We also discuss the potential connections to JEPA-based world models and video scene graph anticipation.

As the discussion deadline approaches, we would appreciate your thoughts on whether these explanations address your concerns. We would be happy to provide any further clarification.

Thank you for your time and consideration.

We briefly summarize our responses to the reviewer’s concerns.

1. Existing methods and CF-Prune’s design rationale. We explained that SpecPrune-VLA uses historical and current attention for observation-token selection, while SP-VLA combines spatial and semantic importance with action-history-based model scheduling. In contrast, CF-Prune targets noisy future-video tokens, combining clean-observation instruction relevance with action-to-visual attention from the current inference call. We also explained that Sparse VideoGen sparsifies attention connections, but its profiling overhead can offset acceleration in few-step WAMs.

2. Connections to other world-modeling approaches. We discussed how JEPA’s predictive latent representations could support action generation without detailed video reconstruction, whereas CF-Prune reduces computation in pretrained WAMs without additional training. Video scene graph anticipation could help identify task-relevant regions through predicted object interactions, but may require an additional model and computational overhead.

We have expanded the related-work discussion of VLA pruning methods and clarified CF-Prune’s design rationale in the revised manuscript.

Comments on the paper

2 comments

Dear Reviewers and Area Chair,

We have uploaded the revised manuscript incorporating the additional results and clarifications discussed during the rebuttal. Thank you for your time and constructive feedback.

Best regards, Authors

Dear Reviewers and Area Chair,

Thank you for the constructive reviews and engagement throughout the discussion period. We have uploaded the revised manuscript incorporating the additional experiments and clarifications. As the discussion closes, we briefly summarize the three core takeaways from the rebuttal:

1. Novelty: Training-free pruning of noisy future-video tokens in WAMs (XakZ, JXUY, qhVP, VDvJ). A central question was how CF-Prune differs from prior VLA pruning and phase-adaptive methods such as VLA-ADP, SpecPrune-VLA, and SP-VLA (XakZ, JXUY, qhVP). Our core contribution is not adapting phase-adaptive pruning to WAMs, but identifying future-video tokens as a new pruning target and estimating their importance during the first denoising pass while they remain highly noisy. First, removing all future tokens on pruning-enabled calls drops success from 42.3% to 12.5%, confirming that future tokens are essential for action generation (VDvJ). Second, in a controlled 40-task RoboLab comparison where observation tokens are preserved and the phase schedule is held fixed, CF-Prune achieves 43.3% success, outperforming adapted SpecPrune-VLA (32.3%), SP-VLA (34.4%), and random pruning (37.8%) (qhVP, XakZ). Entropy-based phase identification plays a complementary role and also outperforms the VLA-ADP motion detector (43.3% vs. 41.5%) and latency-matched fixed pruning (41.8%) (XakZ, qhVP).

2. Significance: Practical, training-free acceleration for pretrained WAMs (XakZ, JXUY). In video-diffusion WAMs, dense future-video tokens dominate per-step inference latency. Some existing approaches require retraining or architectural changes, while the profiling overhead of sparse-attention methods can outweigh their computational savings in few-step settings, as observed with Sparse VideoGen on RoboLab (0.97× speedup). CF-Prune accelerates existing checkpoints without retraining, with only 8.7 ms of scoring and gating overhead. On the same Cosmos3 checkpoint and GPU, four-step CF-Prune at 50% pruning achieves 40.0% success at 2,984.7 ms (1.34x average speedup, including pruning-disabled calls), outperforming latency-matched DiT caching (38.0% at 2,999.0 ms) and three-step dense inference (38.3%) (XakZ). Moreover, future-token pruning is orthogonal to step reduction: combining both achieves a 1.71x speedup while maintaining 37.5% success (XakZ).

3. Experimental Rigor: Generalization, high-motion robustness, and statistical transparency (qhVP, XakZ, VDvJ). To broaden the evaluation and address the reviewers’ concerns, we added:

  • Second WAM backbone (XakZ, qhVP): Applied to Flux3 Action across all 120 RoboLab tasks at 35% pruning, CF-Prune achieves 42.0% success versus 41.2% for Dense with a 1.24x average speedup.
  • High-motion robustness (VDvJ): On the top 25% (30 tasks) of RoboLab tasks with the largest third-person image-space motion, CF-Prune achieves 48.7% success versus 47.7% for Dense, supported by supplementary mask visualizations.
  • Initial-step selection (XakZ): Retained-token overlaps between the first and subsequent denoising steps are 72.91–73.37%. At 35% pruning, initial-step selection achieves comparable success to delayed selection (43.3% vs. 42.8%) with lower latency. These results support its practical effectiveness but do not characterize stepwise action-prediction deviations.
  • Statistical reliability and limitations (qhVP): Across all 120 RoboLab tasks, paired 95% confidence intervals for success-rate differences (CF-Prune minus Dense) are [−1.08, +3.83] pp at 35% pruning and [−3.25, +1.58] at 50%. Neither difference is statistically significant. We also report and analyze the Navigation success-rate drop from 38% to 18% at 50% pruning.

Throughout the rebuttal, we responded promptly to the reviewers’ feedback and updated the discussion as additional experiments were completed. We hope this summary assists in the final evaluation.

Best regards, Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.