- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes UG-Pruner, a training-free visual token pruning framework that aims to unify pruning across images, offline videos, streaming videos, and multi-view 3D scenes. The method first selects candidate tokens, then constructs task-specific graphs based on semantic, 2D spatial, temporal, causal-access, and 3D geometric conditions. It estimates node importance through score propagation and selects tokens to retain. The 3D setting additionally incorporates IFPS to balance importance and spatial coverage. The authors evaluate the complete framework on four models and seventeen benchmarks, demonstrating competitive performance retention and efficiency at high pruning ratios.
Critical strengths
-
A configurable pruning framework covering multiple input types. The method represents the structural characteristics of different tasks as graph-construction constraints, sharing the candidate selection, graph scoring, and token selection pipeline while keeping model parameters frozen. This provides a clear structure for reusing pruning implementations across different visual tasks.
-
Broad experimental coverage. The experiments cover images, offline videos, streaming videos, and 3D scenes across four models and seventeen benchmarks. They include comparisons with task-specific methods and actual efficiency measurements, providing some support for the practical utility of the complete framework in the evaluated settings.
Critical weaknesses
-
Table 4 mainly compares semantic-only connections with the complete task-specific configuration, without adequately isolating the effects of the 2D spatial, temporal, and 3D geometric constraints. Consequently, the results do not clearly establish whether each constraint is necessary or how much it contributes to the performance gains.
-
The method uses OR for image and 3D tasks but AND for video tasks, without sufficiently explaining these choices or comparing alternative combinations. Semantic-threshold sensitivity experiments do not establish the validity of these combination rules.
-
In addition to geometric graph construction, the 3D setting introduces IFPS, but the experiments do not clearly separate their contributions. It is therefore difficult to determine whether the improvements primarily come from geometric constraints, IFPS, or their combination.
Questions for authors
-
When the candidate nodes, final token budget, and all other settings are held fixed, how does offline video performance change when the 2D spatial or temporal constraint is removed individually? Can these results establish the individual contributions of the two constraints and the benefit of combining them?
-
What is the rationale for using OR for image and 3D tasks but AND for video tasks? Have alternative combinations been compared, along with their corresponding graph densities and graph-construction overhead?
-
Do the Semantic and Constraints variants for ScanQA in Table 4 use the same final sampling algorithm? Can the benefit of geometric connections be isolated while holding the sampling algorithm fixed, and can the contribution of IFPS be reported separately?
4 comments
Response to Reviewer V1ab (Part 2 of 2)
W3&Q3:Do the Semantic and Constraints variants for ScanQA in Table 4 use the same final sampling algorithm? Can the benefit of geometric connections be isolated while holding the sampling algorithm fixed, and can the contribution of IFPS be reported separately?
R3: Thanks for your comment. Since IFPS introduces additional spatial information into token selection, the Semantic setting in Table 4 uses only semantic constraint and Top-K to evaluate semantic information alone, while the Constraints setting incorporates both 3D-geometric constraint and IFPS.
To better address your concern, we provide ablations to separately examine the contributions of geometric constraint and IFPS selection.
Selector ablation
| Selector | ScanQA CIDEr | Multi3DRefer [email protected] | SQA3D EM |
|---|---|---|---|
| Top-K | 92.95 | 53.97 | 56.03 |
| IFPS* | 94.27 | 54.88 | 56.46 |
Constraint ablation
| Constraints | ScanQA CIDEr | Multi3DRefer [email protected] | SQA3D EM |
|---|---|---|---|
| Semantic-only | 93.48 | 54.62 | 55.66 |
| Semantic + 3D-Geometric* | 94.27 | 54.88 | 56.46 |
From these results, we can see that both geometric constraints and IFPS improve performance. 3D-Geometric constraint connect neighboring nodes across views, while IFPS additionally considers spatial distance for broader scene coverage. Following your suggestions, we will add the above discussions to our new version.
Awaiting Your Feedback
Dear Reviewer V1ab:
Thanks again for your great effort in reviewing our paper.
In our rebuttal, we have carefully responded to all your concerns, including the contributions of graph constraints (Q1/R1), the rationale for the OR and AND settings (Q2/R2), and the role of the sampling algorithm in 3D (Q3/R3).
With the discussion period drawing to a close, we look forward to hearing from you and would be glad to answer any further questions you may have.
Best regards,
The Authors
Look Forward to Your Feedback
Dear Reviewer V1ab,
Thanks again for your valuable time and effort in reviewing our paper.
In our rebuttal, we have carefully responded to all your concerns in details.
-
Contributions of graph constraints (Q1/R1). We follow your suggestion to add component-wise ablations across image, offline video, streaming video, and 3D tasks, further confirming the effectiveness of our graph constraint designs for different pruning tasks.
-
Rationale for OR and AND (Q2/R2). We theoretically analyze the OR and AND combinations for image and video tasks from the perspective of graph construction and node estimation. Then, the quantitative ablations of two choices are also provided to further confirm their effectiveness.
-
Role of the sampling algorithm in 3D (Q3/R3). We follow your suggestion to supplement the controlled experimentations of 3D geometric constraint and the IFPS selection, which well validates the distinct advantages of two designs.
Moreover, in our rebuttal, we have addressed the above concerns shared by the other two reviewers, and received their positive feedback.
With the discussion period drawing to a close, we look forward to hearing from you, and we would be delighted to answer any further questions you may have.
Best regards,
The Authors
Response to Reviewer V1ab (Part 1 of 2)
We highly appreciate your valuable time and effort in reviewing our paper. Below we respond to you concerns point by point.
W1&Q1: Table 4 mainly compares semantic-only connections with the complete task-specific configuration, without adequately isolating the effects of the 2D spatial, temporal, and 3D geometric constraints. How does offline video performance change when the 2D spatial or temporal constraint is removed individually? Can these results establish the individual contributions of the two constraints and the benefit of combining them?
R1: Thanks for your constructive comment. Following your suggestion, we provide additional component-wise ablations on image, offline video, streaming video, and 3D scene understanding. Each variant removes one constraint from the full UG-Pruner configuration.
Image (90% pruning ratio)
Offline video (90% pruning ratio)
Streaming video (75% pruning ratio)
3D scene understanding (90% pruning ratio)
From these results, we can see that removing any task-specific constraint reduces performance. For offline video, removing either the spatial or temporal constraint lowers performance on both benchmarks, confirming their individual contributions and the benefit of combining them.
As noted in our paper, different pruning tasks have distinct properties, and these ablations validate the effectiveness of our graph constraints in handling these properties. We will include these results in the revised paper.
W2&Q2: What is the rationale for using OR for image and 3D tasks but AND for video tasks? Have alternative combinations been compared, along with their corresponding graph densities and graph-construction overhead?
R2: Thanks for your careful review. We mainly consider these choices from the perspectives of token pruning and graph modeling.
For image and 3D tasks, the token pruning are directed applied to all input tokens, and the AND combination are strong constraints for graph building, which will result in an excessive number of isolated nodes. These isolated nodes prevent an accurate assessment of their importance in the information propagation of our UGPruner, which further affects the final token selection. In this case, the OR combination impose looser constraint on the construction of the graph, effectively preventing the aforementioned situation from occurring.
In contrast, for offline and streaming video tasks, UG-Pruner constructs the graph frame by frame, and these tasks often involves more visual tokens than the image and 3D ones. Using OR in this setting can produce an overly dense graph, making it difficult to distinguish important tokens from redundant ones. In this case, the AND combination of constraints can obtain a better token graph.
To better respond to your concern, we supplement the OR and AND ablation below, of which results well validate our motivations. We will give these results to our new version.
Image
Offline video
3D scene understanding