- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper focuses on reducing the computational costs of 3D VLMs from two directions, i.e., token reduction and layer skipping. In detail, it utilizes 3D voxel tokens to replace the multi-view tokens, which can merge the repeated tokens in multiple views and reduce the number of vision tokens. Then, it uses late entry and early exit with recoverable group-wise layer skipping to reduce the token computational costs. The experimental results demonstrate a 6× speedup in inference.
Critical strengths
- The paper is well-written, with a clear presentation of its experimental findings, motivation, and methodology. The analysis in Sec. 3 is particularly informative.
- The paper offers sufficient results to prove the efficiency of the proposed method.
Critical weaknesses
The novelty of the proposed method appears limited:
- Using 3D voxel representations to reduce the number of tokens is common in multi-view tasks, as demonstrated by Scaffold-GS [A] and DTC [B].
- The GroupRoute is similar to HiDrop.
For the Saliency-aware selection in Line 263, higher attention weights do not necessarily indicate greater token importance. For example, vision foundation models such as DINOv2 can assign high attention scores to task-irrelevant regions.
[A] Lu, Tao, et al. Scaffold-gs: Structured 3d gaussians for view-adaptive rendering. CVPR 2024.
[B] Huang, Hsiang-Wei, et al. Zero-shot 3d question answering via voxel-based dynamic token compression.CVPR 2025.
Questions for authors
- In the equation in Line 258, different operations are used to aggregate features and saliency scores within each voxel, which may cause a mismatch between the aggregated features and their corresponding saliency scores.
- There is no experimental results to demonstrate that saliency- and coverage-aware selection outperforms random selection.
7 comments
Response to Reviewer woYf (Part 2/4)
W1: The novelty of the proposed method appears limited: Using 3D voxel representations to reduce the number of tokens is common in multi-view tasks, as demonstrated by Scaffold-GS and DTC. The GroupRoute is similar to HiDrop.
A1 (Continued)
-
Layer-wise Visual Computation (GroupRoute):
-
As summarized in the Global Response, the key distinction from HiDrop lies in how visual tokens are processed within the visual computation window. Our analysis in Sec. 3 shows that visual token relevance is locally stable, but currently low-relevance tokens may become important at later layers. Building on HiDrop’s Late Entry and Early Exit [6], GroupRoute introduces recoverable group-wise skipping within a visual computation window adapted to multi-view 3D VLMs. Unlike HiDrop’s permanent pruning, low-priority tokens temporarily bypass attention and FFN computation while retaining their hidden states. At regrouping anchors, all retained visual tokens are updated and regrouped, allowing previously skipped tokens to resume computation.
-
To isolate the contribution of recoverable skipping, we additionally disable GeoSemZip and Late Entry/Early Exit and compare RGS with permanent pruning at matched average active visual token budgets per layer. On LLaVA-OneVision, RGS improves performance retention over the best tested pruning variant by 0.57/1.44 points when 50%/10% of visual tokens remain active after each anchor, respectively, supporting the value of retaining tokens for later regrouping. The table below further shows that full GeoGR outperforms HiDrop on both models after post-training at comparable average active-token budgets.
Table G2. Isolated RGS evaluation and comparison with HiDrop. Scores report average performance retention relative to the corresponding Dense model (100%), gains are in percentage points.
Setting Model Ours (%) Comparison Methods (%) Gain (pp) RGS only, 50% active LLaVA-OneVision-7B 100.38 Permanent pruning: 99.81 +0.57 RGS only, 10% active LLaVA-OneVision-7B 97.30 Permanent pruning: 95.86 +1.44 Full GeoGR LLaVA-OneVision-7B 95.18 HiDrop: 88.84 +6.34 Full GeoGR Video-3D LLM-7B 96.80 HiDrop: 95.21 +1.59
-
-
Joint Representation and Computation Compression:
-
GeoSemZip and GroupRoute address two distinct sources of redundancy: repeated scene observations across views and unnecessary visual computation across decoder layers. GeoSemZip consolidates cross-view evidence into a compact scene representation, while GroupRoute selectively processes the retained tokens according to their query relevance across layers. Together, they reduce both the number of tokens entering the LLM and the computation spent on those tokens. Compared with projector-only compression at a comparable average active-token budget, this joint design allows more scene tokens to be retained at the projector and processed selectively across layers.
-
Our original submission reports this comparison in Appendix E.2, Table 18, summarized below in Table G3. GeoGR achieves 95.18% retention with 638 average active visual tokens per layer, compared with 94.00% at 628 tokens for GeoSemZip alone and 94.65% at 2,128 tokens for GroupRoute alone, supporting the complementarity of the two stages.
Table G3. Component comparison on LLaVA-OneVision-7B. Visual tokens are averaged over all decoder layers. GroupRoute alone includes Late Entry/Early Exit and RGS but disables GeoSemZip.
Configuration Projector keep ratio Avg. active visual tokens/layer ScanQA CIDEr ScanQA EM SQA3D EM Avg. (%) GeoSemZip only 10% 628 87.44 25.41 56.69 94.00 GroupRoute only 100% 2,128 88.48 26.07 55.84 94.65 GeoGR (both) 30% 638 88.25 26.20 56.61 95.18
-
[6] Wu H, Fan Y, Dai J, et al. Hidrop: Hierarchical vision token reduction in mllms via late injection, concave pyramid pruning, and early exit[C]//International Conference on Learning Representations. 2026, 2026: 97296-97316.
Response to Reviewer woYf (Part 3/4)
W2: For the Saliency-aware selection in Line 263, higher attention weights do not necessarily indicate greater token importance. For example, vision foundation models such as DINOv2 can assign high attention scores to task-irrelevant regions.
A2: We agree that high attention weights alone do not guarantee greater token importance. GeoSemZip uses vision-encoder attention as a visual-saliency cue and complements it with spatial coverage and residual merging to preserve information from regions with low attention scores.
-
Visual-saliency Estimation: For each visual token, we average its received self-attention across heads and visual query positions in the last executed vision-encoder layer. We pool these scores to the projector grid and take the maximum within each voxel. This question-independent score reflects the attention a region receives during visual encoding. Encoder attention is also used for visual token selection in VisionZip [7] and VisPruner [8].
-
Spatial Coverage and Residual Merging: GeoSemZip uses attention to select dominant tokens and allocates the remaining budget to spatially distributed contextual anchors, sampled in Morton order from the remaining voxels independently of their saliency scores. This spatial-coverage branch helps retain regions that saliency-only selection may overlook. Residual merging further aggregates features from unselected voxels into these anchors, preserving additional scene context without increasing the token budget.
-
Component Ablation: Table 4 of our original submission evaluates these components at a fixed 30% token budget. Saliency-only selection retains 96.99% of Dense performance. Including cross-view voxel consolidation and spatial coverage increases retention to 98.19%, and adding residual merging further raises it to 99.33%. These results support combining attention-based selection with mechanisms that preserve complementary spatial context.
Q1: In the equation in Line 258, different operations are used to aggregate features and saliency scores within each voxel, which may cause a mismatch between the aggregated features and their corresponding saliency scores.
A3: The pooled feature and the saliency score summarize different properties of the same voxel. The feature integrates visual observations across views, while the score aggregates their saliency for voxel selection. This score is not an attention score recomputed from the averaged voxel feature.
-
Why We Use Different Pooling Operations: Mean feature pooling integrates observations of the same voxel across views and reduces dependence on any single observation. For saliency, visibility and occlusion can vary across views. A region may be clearly visible and highly salient in one view while receiving lower scores in others. Max saliency pooling preserves that strong response for voxel selection. The two operations therefore serve complementary purposes: fusing cross-view features and retaining the strongest observed saliency cue.
-
Ablation at a Fixed Token Budget: We evaluate all four combinations of feature and saliency pooling at the same 30% token budget (Table R2). Mean feature pooling with max saliency pooling achieves the highest performance retention (99.33%), compared with mean/mean (99.06%), max/max (98.29%), and max/mean (98.11%), with feature pooling listed first. Holding feature pooling fixed at mean, switching saliency pooling from mean to max improves retention by 0.27 percentage points. Holding saliency pooling fixed at max, switching feature pooling from max to mean improves retention by 1.04 percentage points. These comparisons support using mean pooling for cross-view feature fusion and max pooling for voxel selection in GeoSemZip.
Table R2. Feature and saliency pooling at a 30% token budget on LLaVA-OneVision-7B. Retention is normalized to Dense (100%).
Feature pooling Saliency pooling ScanQA CIDEr ScanQA EM SQA3D EM Avg. retention Mean Max (GeoSemZip) 93.95 27.51 57.63 99.33% Mean Mean 93.15 27.76 57.12 99.06% Max Max 91.69 27.29 57.63 98.29% Max Mean 92.05 27.17 57.37 98.11%
[7] Yang S, Chen Y, Tian Z, et al. Visionzip: Longer is better but not necessary in vision language models[C]//2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2025: 19792-19802.
[8] Zhang Q, Cheng A, Lu M, et al. Beyond text-visual attention: Exploiting visual cues for effective token pruning in vlms[C]//2025 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2025: 20857-20867.
Response to Reviewer woYf (Part 1/4)
We thank you for your thorough review and appreciate both the positive feedback and the constructive comments. Please find our responses to your points below. In the revised manuscript, newly added or substantially modified text is highlighted in blue, while previously existing passages that already address your concerns are highlighted in yellow for ease of reference. Please let us know if you have any further questions or suggestions.
A1: We respectfully disagree with the assessment of limited novelty. As summarized in the Global Response, GeoGR jointly addresses representational redundancy across views and computational redundancy across LLM layers in multi-view 3D VLMs. Its technical contributions include spatial–semantic compression that preserves salient content, spatial coverage, and information from unselected voxels, together with recoverable group-wise skipping motivated by our analysis of evolving token relevance across layers. Below, we clarify these contributions and present the supporting evidence.
Voxelization is a well-established technique for 3D representation [1-3]. Scaffold-GS [4] uses voxelized anchors to organize view-adaptive 3D Gaussians for rendering, while DTC [5] uses voxelization for cross-view visual token compression. Voxelization itself is not the defining innovation of either method; their distinctive contributions lie in the mechanisms built on this common geometric foundation.
The distinction between DTC and GeoSemZip lies in how they compress and preserve scene information using this geometric foundation. DTC progressively enlarges voxels and repeatedly performs similarity-based token matching to remove redundancy. GeoSemZip instead consolidates cross-view observations into voxels and allocates a fixed token budget to high-saliency dominant voxels and contextual anchors sampled in Morton order from the remaining voxels. It then aggregates features from unselected voxels into these anchors through residual merging. This design jointly preserves salient content, spatial coverage, and information from unselected regions, while avoiding iterative voxel resizing and matching.
Our original submission already reports comparisons between GeoSemZip and DTC at approximately 10%, 20%, and 30% projector token budgets in Appendix D.2, Table 12. We present the key results below for ease of reference. GeoSemZip achieves higher performance retention at all three budgets.
Table R1. Performance retention on LLaVA-OneVision-7B across projector token budgets. All configurations are post-trained on LLaVA-OneVision-7B, with retention normalized to Dense (100%).
Our updated latency measurements further demonstrate the practical efficiency advantage of GeoSemZip. With the same general inference optimizations applied to both methods, GeoGR achieves substantially lower projector-stage latency than DTC on both models.
Table G1 (excerpt from the Global Response). Projector-stage latency after inference optimizations (ms), including token compression, measured on NVIDIA RTX PRO 5000 Blackwell.
Together, these results demonstrate the benefits of GeoSemZip in both performance retention and practical efficiency.
[1] Liu Z, Tang H, Lin Y, et al. Point-voxel cnn for efficient 3d deep learning[J]. Advances in neural information processing systems, 2019, 32.
[2] Graham B, Engelcke M, Van Der Maaten L. 3d semantic segmentation with submanifold sparse convolutional networks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Ieee, 2018: 9224-9232.
[3] Chen Y, Liu J, Zhang X, et al. Voxelnext: Fully sparse voxelnet for 3d object detection and tracking[C]//2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2023: 21674-21683.
[4] Lu T, Yu M, Xu L, et al. Scaffold-gs: Structured 3d gaussians for view-adaptive rendering[C]//2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2024: 20654-20664.
[5] Huang H W, Chen F C, Chai W, et al. Zero-shot 3d question answering via voxel-based dynamic token compression[C]//2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2025: 19424-19434.