- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes a training-free visual token pruning framework designed to accelerate VLM during inference without degrading performance. It addresses two major limitations of existing compression methods: hierarchy-blind pruning and attention drift. The reported experiment results achieves state-of-the-art accuracy-efficiency trade-offs across multiple video understanding and captioning benchmarks.
Critical strengths
- The paper reported strong performance results compared to existing training-free pruning methods across multiple backbones and retention ratios (Sec 3.2)
- It also reported noticeable speedups with minimal peak memory overhead compared to baselines (Sec 3.3)
- The proposed scheme is training free, easily adaptable to diverse VLM architectures.
Critical weaknesses
- The paper did not seem to have established a general optimal guarantee, but relies on heuristic design choices.
- Does the same hyperparameters work across different video resolution, length, etc?
- How does the performance change on rare events and unusual scenes?
- MGDE builds a quadtree on a per-frame base (Sec 2.2.) Temporal structure is not explicitly modeled. Does this spatial only hierarchy affect performance and would a spatiotemporal hierarchy improve the capability?
Questions for authors
Please see last section.
6 comments
Response to Weakness 2: Hyperparameter Robustness Across Video Resolutions and Durations
Reviewer concern: Does the same hyperparameters work across different video resolution, length, etc?
Thank you for raising this question. Yes, the same pruning hyperparameters apply across the video resolutions, durations, and datasets evaluated in this paper, without retuning for individual inputs. Our main experiments already cover these variations, as summarized below.
| Dataset | Duration | Source resolution / format |
|---|---|---|
| Video-MME | Short: ~0–2 min; medium: 4–15 min; long: 30–60 min | Mainly 1280×720 / 640×360; 16:9 |
| VDC | Typically tens of seconds; local range: ~8–163 s | 1080p / 720p / 4K; also 4:3 and portrait |
| MVBench | Mostly seconds to tens of seconds; some clips last minutes | 480×320 / 480×270 / 720p / 1080p / 224×224; varied formats |
| VNBench | Long-video retrieval; sample-dependent | Source-dependent |
| NExT-QA | Typically tens of seconds | Various resolutions; mainly landscape |
| VALOR32K-AVQA Visual | ~10 s | Various resolutions; mainly landscape |
These datasets span short clips and long videos, diverse resolutions, and landscape, portrait, and square formats. We use the same MGDE and ATP hyperparameters under each benchmark's established sampling and preprocessing protocol. The cross-dataset results in the main tables and appendix support the applicability of this configuration across these input conditions.
Response to Weakness 3: Rare Events and Unusual Scenes
Reviewer concern: How does the performance change on rare events and unusual scenes?
Thank you for raising this question. MVBench, included in our main experiments, contains the official Unexpected Action task, whose videos come from FunQA and evaluate the understanding of unexpected or unusual actions. Our evaluation therefore already includes samples relevant to unusual scenes and events. The 3,800 MVBench samples we evaluate exclude only the missing Fine-grained Pose task and include Unexpected Action.
We further summarize accuracy on Unexpected Action, comparing the full model with FoveaTree at different visual-token retention ratios:
| Model | Full | 50% retention | 30% retention | 10% retention |
|---|---|---|---|---|
| InternVL3.5-8B | 87.0% | 87.0% | 86.5% | 85.0% |
| Qwen3-VL-8B | 81.0% | 81.0% | 82.0% | 81.0% |
| Qwen2.5-VL-7B | 83.5% | 82.5% | 83.0% | 83.0% |
| LLaVA-OneVision-7B | 83.0% | 84.0% | 83.5% | 84.0% |
FoveaTree maintains accuracy close to the full model across all four backbones, with improvements in some settings. Even at 10% visual-token retention, it preserves strong understanding of unexpected and unusual actions.
Response to Weakness 1: Optimality Guarantees and Design Choices
Thank you for raising this question. For visual-token pruning, the optimal selection depends on the video, the question, and the model's subsequent reasoning. Pruning decisions based on feature or attention scores therefore cannot guarantee globally optimal task performance across all inputs and tasks. FoveaTree is a training-free compression algorithm for practical video understanding, designed to achieve a favorable accuracy–efficiency trade-off under a given token budget; we do not claim a general guarantee of global optimality. We evaluate this objective through task performance and computational cost across models and tasks, which form the main evidence for our contribution.