- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This is an empirical study of frame selection, spatial compression, and reinvestment of a fixed answerer visual-token budget. It compares six baselines using a shared scorer. The adjustment of the design choices reveals how the token budget should be spent for better long-video understanding performance. During the study, the authors also discovered that "Orthogonal Matching Pursuit" as a central mechanism of token compression/selection works surprisingly well.
Critical strengths
- A useful controlled decomposition of selection, compression, and reinvestment, with OMP as a strong, simple reference.
- The experiments breadth looks OK, covering several base models and benchmarks.
Critical weaknesses
-
The paper has clear signs of pure AI writing. I am not objecting to AI writing, but the defensive wording and broken logic make it hard to read and interpret. For example, the abstract even has things like "an implementation bug in our own AKS baseline and a 0.07–3.74-point gap between two harnesses"
-
From Table 2 and Figure 1, it is unclear what is the main conclusions and whether the results are significant
-
As a crucial experiment to verify that token compression has not abandoned useful information, please also apply this to some captioning benchmarks
-
New scientific insight needs to be clarified. The controlled decomposition and OMP baseline are useful. However, the main findings, e.g., query-aware selection and reducing spatial resolution for more timestamps, are largely existing motivations and observations. The paper should clarify which important uncertainty this study resolves and what researchers should do differently because of its findings.
Questions for authors
See above
3 comments
I appreciate the efforts from the authors, but 1/ the current writing quality is still below the bar of this venue; 2/ lacking the captioning result is crucial to justify whether the method in this paper is generalizable; 3/ what the authors have clarified as new insights was not convincing enough for me. Therefore, I keep my current rating of 2.
Response to Reviewer qpYu
Thanks for reading the revision and for clearly stating the remaining concerns.
On captioning: we agree it is the most direct test of whether compression loses information. For the next version, we plan to caption videos using a detailed captioning benchmark at fixed timestamps, with full-resolution and half-resolution frames at a matched cost/token budget, and compare caption scores on a per-video basis. Until then, the paper limits its claims to multiple-choice question answering, as stated in Section 8.
On writing and new insights: we take both points and will continue revising the presentation to ensure the new findings are stated more clearly. Thank you again for the time spent on our paper.
Response to Reviewer qpYu
We thank the reviewer for the constructive comments and for finding the decomposition and the OMP reference useful.
Re: The revised PDF is rewritten with a new structure around one question: how a fixed visual-token budget should be spent. The abstract now states the question and three findings; implementation corrections (the AKS fix, the harness gap) are moved to Appendix G. The revised PDF also expands the AI Use Statement to describe how AI tools were used.
Re: In the submission, Figure 1 was the protocol diagram and Table 2 the reinvestment table; neither stated a conclusion, and some of Table 2's rows had no paired test. In the revision:
Re: We agree this is the right check. We could not run it in this period. Section 8 now states it as a limitation: without a captioning benchmark, "compression is close to free" does not show that no information is lost. We plan to add it in the next version.
Re: We agree that each idea is known on its own. Prior work shows that spending tokens on more, smaller frames helps; it leaves open whether choosing frames still matters then, and what it is good for. Our answer: choosing matters most when the budget buys few frames, and mostly for questions about one moment. In practice (Section 7 of the revision):