- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
STORS proposes a training-free method for video visual-token compression. It selects tokens by orthogonal residuals over a unified spatio-temporal candidate pool using vision-encoder-side features, then merges the post-projector features of unselected tokens into the kept tokens. The paper's central observation is that representation geometry differs substantially across the visual projector in Qwen2.5-VL, making pre-projector scoring more effective than post-projector scoring; no comparable scoring-space benefit is statistically detectable in LLaVA-OneVision.
Critical strengths
- The scoring-space comparison is valuable. On Video-MME with Qwen2.5-VL, holding the selection criterion fixed and changing only the scoring space, pre-projector scoring yields higher accuracy in all nine settings; eight meet the paper's paired-test significance threshold. The authors also report the absence of a detectable benefit on LLaVA-OneVision.
- The compression results have practical potential. At 10% visual-token retention, Qwen2.5-VL achieves an average score of 58.7, or 94.4% of the uncompressed score of 62.2. With 32 frames, time to first token (TTFT) decreases from 1.10 s to 0.60 s.
Critical weaknesses
- The scope and cause of the scoring-space benefit remain unclear. For Qwen, the default pre-projector scoring feature is the mean of the four sub-tokens within a cell, whereas the actual projector input is their concatenation. The current results do not adequately separate the effects of feature averaging, the projector's joint spatial transformation, and other architectural differences between backbones. I suggest comparing mean, concatenated, and post-projector scoring features on the same Qwen backbone under matched settings, and qualifying the cross-backbone mechanistic interpretation accordingly.
- The main LLaVA-OneVision results are less complete, limiting cross-backbone conclusions. Table 1 reports three retention ratios (5%/10%/20%), Video-MME duration splits, MLVU, and a broader set of compression baselines for Qwen. Table 2 reports only 10%/20%, average scores on three benchmarks, and a subset of the baselines for LLaVA. Although supplementary Table 21 includes STORS and scoring-space comparisons at 5% on LLaVA, it does not provide a full matched-budget, multi-method comparison analogous to Table 1. Please add the 5% baselines, Video-MME duration splits, MLVU, and missing methods where feasible, and explain any inapplicable settings.
- Efficiency gains should be distinguished from those over a matched-budget baseline. At 32 frames and 10% retention, TTFT is 0.60 s for STORS, 0.64 s for VisionZip, and 1.10 s without compression. The approximately 1.82× speedup over the uncompressed model is valid, but the latency improvement over VisionZip at a matched token budget is about 6%. Please break down the time spent on visual encoding, token selection and merging, and LLM prefill. The current budget-matched comparison disables FlashVID's second, in-LLM compression stage; I also recommend reporting the quality–latency results and per-layer token counts for the original two-stage configuration.
Questions for authors
- On the same Qwen backbone, data, and token budget, can you compare accuracy, feature geometry, coverage-fill fraction, and computational cost when the scoring features are the four-sub-token mean, concatenation, or post-projector features? Can these experiments separate the effects of scoring-feature construction from those of the projector itself?
- Can you provide LLaVA-OneVision comparisons aligned with Table 1 as closely as possible? If some baselines or datasets cannot be included, please explain the limitations.
- What is the relationship between the optimization objectives in Eqs. (2) and (1)? Please also check the explanation of the stopping criterion following Eq. (4).
- Please report a stage-wise breakdown of TTFT, end-to-end inference latency, peak memory, and the quality–latency results and per-layer token-retention profile of the original two-stage FlashVID.