acceptodds
Withdrawn from ICLR 2027

Divide and Prune: Geometry-Driven Token Pruning for Efficient VLA Driving Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Vision-Language-Action (VLA) models have emerged as a premier paradigm for embodied AI, enabling scene comprehension and auto-regressive action generation in autonomous driving, yet their deployment in the real-world system is hindered by the excessive number of visual tokens from high-resolution inputs. Previous general visual token pruning approaches primarily exploit text-visual attention, visual attention maps or token similarity, inherently blinding themselves to the non-uniform space and risk-asymmetric nature of driving environments. In this work, we propose Divide and Prune (DiP), a geometry-driven visual token pruning strategy for VLA driving models. DiP utilizes the depth estimation as spatial priors to divide the visual field into near-field and far-field regions. For near-field tokens, DiP fuses the edge energy with the token informativeness to retain local details critical for obstacle avoidance. For far-field tokens, we propose the depth-modulated diverse token selection to maximize semantic diversity while mitigating the semantic dilution caused by spatial overlap in 2D images. Experiments on the nuScenes and the Bench2Drive benchmarks show that DiP achieves strong performance while delivering an approximately 3.4 inference speedup.

Primary area:
applications to robotics, autonomy, planning
Keywords:
vision language navigation, cross-modal application, multimodality

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_kG4MOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes Divide and Prune (DiP), a training-free visual-token pruning method for driving VLAs. DiP uses Depth Anything v2 to divide image tokens into near- and far-field groups, retains near-field tokens through a rank fusion of Sobel edge energy and token L2L_2-norm, and selects far-field tokens through depth-modulated cosine farthest-point sampling. On Impromptu-VLA/nuScenes, DiP is compared with SparseVLM, VisPruner, DivPrune, and FastDriveVLA at several token budgets; on SimLingo/Bench2Drive, it evaluates aggressive pruning in closed loop. At 75% token pruning, the method reports an average L2 error of 31.97 cm and reduces LLM prefill time from 185 ms to 55 ms.

Critical strengths
  1. DiP provides a practical accuracy-efficiency trade-off. With 812 of 3,249 tokens retained, it preserves an average L2 error of 31.97 cm while reducing reported prefill time from 185 ms to 55 ms.

  2. The evaluation spans two VLA backbones and closed-loop driving. DiP retains 96.0% of the full model's Driving Score on Bench2Drive with only 25% of the visual tokens.

Critical weaknesses
  1. The published far-field algorithm is inconsistent with the stated method. FfarF_{\rm far} is never used in Algorithm 1, and all similarity matrices and selections are defined over V\mathcal V, which includes valid near-field tokens. The paper must clarify whether the implementation instead uses V∩Ffar\mathcal V\cap F_{\rm far} and how overlap with TnearT_{\rm near} is prevented (Section 3.4, Equations 7–8, p. 5; Algorithm 1, p. 14).

  2. The closed-loop and safety claims are stronger than the evidence. Table 2 compares DiP with heterogeneous full-token systems rather than pruning methods applied to the same SimLingo checkpoint, and the pruned model loses substantial Success Rate and Comfortness relative to SimLingo. Figure 5 shows qualitatively better token coverage but does not by itself establish safer control. In addition, the reported 3.36× speedup measures LLM prefill rather than complete action-generation or control-loop latency (Table 2 and Figure 5, pp. 7–9; Table 5, p. 8).

Questions for authors
  1. Which Table 2 baselines were reproduced and which were quoted from prior work? For methods with available implementations, can the authors provide at least one same-SimLingo, same-token-budget pruning comparison to contextualize DiP's closed-loop result (Table 2, p. 7; Appendix D, p. 15)?

  2. Can the authors provide a compact equal-budget ablation that separately removes the depth partition, the near-field energy/norm selector, and the far-field depth modulation? These three variants would be sufficient to identify the source of the gain without requiring an exhaustive factorial study (Sections 3.2–3.4 and Section 4.3, pp. 4–8).

  3. How do the authors reconcile the repeated safer-driving interpretation with the lower Success Rate and Comfortness and the slightly worse full-token collision/intersection rates under aggressive pruning (Tables 1–2, p. 7; Figure 5, p. 9)?

Reviewer_FSWqOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes Divide and Prune (DiP), a geometry-guided visual token pruning method for efficient VLA driving models. DiP uses monocular depth estimates to divide visual tokens into near-field and far-field groups. Near-field tokens are selected by combining image edge strength with feature norms, while far-field tokens are selected through depth-modulated diversity sampling. The retained tokens are then passed to the existing VLA model for action generation. Experiments on Impromptu-VLA with nuScenes and SimLingo with Bench2Drive evaluate open-loop and closed-loop driving performance. The reported results show favorable open-loop performance relative to competing pruning methods and a 3.36× prefill speedup at 75% token pruning, with some degradation in closed-loop driving performance.

Critical strengths
  1. Well-motivated pruning design for driving scenes. DiP uses depth to distinguish near-field regions, where local detail matters, from far-field regions, where broader scene coverage is useful. The corresponding selection rules provide a practical way to incorporate spatial information into token pruning. Using an existing depth estimator and explicit selection rules also allows integration into existing VLA models without training an additional pruning network.

  2. Favorable performance–efficiency trade-off under controlled comparisons. Table 1 compares pruning methods on the same Impromptu-VLA backbone at matched token budgets. At 75% pruning, DiP achieves the best average L2, collision, and intersection metrics among the listed pruning methods, while Table 5 reports a 3.36× prefill speedup including pruning overhead. Evaluation on an InternVL-based system in Bench2Drive further extends the study to a second model family and closed-loop driving.

Critical weaknesses
  1. The closed-loop evaluation lacks matched pruning baselines, leaving the performance trade-off insufficiently established. In Table 2, pruning 75% of visual tokens reduces the success rate from 67.27% to 60.00% and comfortness from 33.67 to 21.56. While the driving score remains relatively close to the full-token baseline, these losses warrant further analysis. The comparisons with other driving systems do not establish whether DiP offers a better trade-off than alternative pruning methods applied to the same SimLingo backbone. Matched comparisons at the same token budgets would clarify this trade-off and strengthen the evidence that DiP’s relative advantages generalize across architectures.

  2. The efficiency evaluation does not establish the end-to-end inference benefit. Table 5 reports a 3.36× prefill speedup and explicitly includes pruning overhead, but does not report latency from image input to complete trajectory output or timing on the closed-loop model. Moreover, DiP’s reported prefill time is higher than those of DivPrune and FastDriveVLA (55 ms versus 43 ms and 49 ms, respectively). Evaluating complete inference latency alongside driving quality under matched conditions would better establish the practical value of the proposed trade-off.

Questions for authors
  1. Could you compare DiP with alternative pruning methods, such as DivPrune and FastDriveVLA, on the same SimLingo backbone at matched token budgets? In particular, how do their success rate and comfortness compare with DiP’s, and does a less aggressive pruning ratio mitigate the degradation observed in Table 2?

  2. Could you clarify the timing boundaries of Table 5 and report end-to-end latency from image input to complete trajectory output, including depth estimation, token selection, and decoding? Reporting these measurements alongside driving quality for both Impromptu-VLA and SimLingo would help determine whether DiP provides a favorable practical trade-off relative to the pruning baselines.

Reviewer_Z8pXOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Divide and Prune, a geometry-driven visual token pruning method for VLA driving models. It uses monocular depth estimates to divide visual tokens into near-field and far-field groups. Near-field selection combines edge energy with token feature norms to preserve local details. Far-field selection uses depth-modulated diversity sampling to retain broader scene information. Experiments evaluate Impromptu-VLA on nuScenes and SimLingo on Bench2Drive.

Critical strengths

The overall pipeline is reasonably easy to follow.

Critical weaknesses
  1. The evidence for generalization across backbones is insufficiently systematic. Although the paper evaluates Impromptu-VLA and SimLingo, each backbone is paired with a different benchmark and evaluation protocol. Most pruning comparisons and ablations are restricted to Impromptu-VLA. The authors should evaluate multiple backbones under consistent protocols to establish the general applicability of the proposed strategy.

  2. The closed-loop results show non-negligible performance degradation. In Table 2, pruning reduces the Driving Score from 85.07 to 81.64 and the Success Rate from 67.27% to 60.00%, a decrease of 7.27 percentage points. Comfortness falls from 33.67 to 21.56, a relative decrease of approximately 36%. The Efficiency column is a driving-performance score for which higher is better; its decrease from 259.23 to 245.16 represents a 14.07-point performance loss. These results require a more careful justification of the claimed performance–efficiency trade-off.

  3. The computational advantage over existing pruning methods is unclear. In Table 5, DiP reduces prefill time from 185 ms to 55 ms, achieving a 3.36× speedup over the unpruned model. However, it is approximately 27.9% slower than DivPrune at 43 ms and 12.2% slower than FastDriveVLA at 49 ms. Comparisons at matched driving performance are needed to establish whether the additional complexity provides a worthwhile practical benefit.

  4. The efficiency analysis is incomplete. Table 5 reports prefill FLOPs and time but omits end-to-end latency and peak GPU memory. Although Section 4.4 states that pruning overhead is included, the measurement boundaries remain unclear. The authors should report a breakdown covering depth estimation, visual encoding, token selection, LLM prefill, and action decoding. The approximately 3.4× speedup should also be clearly identified as a prefill result rather than an established end-to-end speedup.

  5. Important driving-specific token compression baselines are missing. The comparison should include Prune2Drive [1] and MVPruner [2], with suitable adaptations and clearly specified evaluation settings. Their absence limits the evidence that DiP improves upon existing methods tailored to autonomous driving.

[1] Minhao Xiong, Zichen Wen, Zhuangcheng Gu, Xuyang Liu, Rui Zhang, Hengrui Kang, Jiabing Yang, Junyuan Zhang, Weijia Li, Conghui He, and Linfeng Zhang. Prune2Drive: A Plug-and-Play Framework for Accelerating Vision-Language Models in Autonomous Driving. CVPR, 2026.

[2] Nan Yang, Zhanwen Liu, Linfeng Zhang, Shangyu Xie, Yang Wang, Wenzhuo Zhou, and Xiangmo Zhao. MVPruner: Dynamic Token Pruning for Accelerating Multi-view Vision-Language Models in Autonomous Driving. ECCV, 2026.

Questions for authors

Could the authors provide systematic evaluations across multiple backbones, comparisons with driving-specific pruning methods, and complete end-to-end latency and peak-memory analyses to establish the method’s general applicability and practical advantages?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.