- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper proposes a visual token compression method by predicting masked content in latent space of a frozen vision encoder to force it to generate fewer tokens while preserving the original semantics. After training the compressor, it can adapt to other downstream Vision-Language models (VLMs). Experimental results show a certain level of effectiveness, but not quite interesting. The paper's organization and writing is poor and reading or understanding it is difficult. It might be an auto-research paper.
Critical strengths
Developing visual token compression methods that can adapt to different VLMs without requiring task-specific annotations is an important research direction. This paper proposes a rather effective method to address this issue. The method is motivated by JEPA (Joint-Embedding Predictive Architecture). The authors propose or adopt simple interpretable training objectives to learn a compressor for a frozen vision encoder.
Critical weaknesses
- The paper's organization and writing is a disaster. It doesn't look like a formal conference paper, and might be generated by AI. For example, there are so many unnecessary detailed experimental result numbers in abstract, meaningless italics, unusual wording like 'competitor', 'host', etc. The presentation of experiments is also poor. I expect that there should be a clear table that shows the proposed method's results and other compared methods, while the main table is organized in a chaos.
- The method is not novel. The authors simply combine existing methods without proposing any new one.
- The experimental results are not good. In table 3, the proposed method cannot exceed ToMe (ICLR 2023), and the compared methods are not the latest algorithms.
Questions for authors
Please do research by yourself. It is obvious that this paper cannot match the ICLR-level. It is meaningless to submit your AI-generated paper to a top-tier venue, because there is no chance to be accepted. You should read your submission at first to understand what and how it is.
2 comments
ToMe in Table 3. We agree that ToMe is at par with or slightly ahead of our compressor in Table 3, and we will state this in the abstract. ToMe leads by 0.45–1.81 points on POPE, GQA, MMBench and MME, our compressor leads on SEED (+2.45), and over the eight tasks of Tables 2–3 outside TextVQA the two match (94.7% against 94.9% mean retention). ToMe's strength, however, is specific to this host. No training-free heuristic is strong on every host, whereas our compressor has the highest retention of the four methods on every host outside TextVQA (Table R2). ToMe keeps only 50.3% of the uncompressed captioning score on the 135M decoder, against 84.9% for ours, and trails ours by 5.0 points on the Q-Former bridge. Choosing a heuristic for a new host therefore requires a labelled evaluation on that host, whereas our compressor is trained once per encoder, without labels. ToMe is also not cheaper at inference: it uses 31.6% of the uncompressed GFLOPs, against 31.1% for ours. On TextVQA, the 2×2 grid averaging discards the fine detail that reading text requires; our attention-pool variant raises retention from 45.0% to 66.1%, close to ToMe (67.6%), and we will add it to Table 3.
Table R2. Retention (% of the uncompressed host) at matched token count, on every host in the paper. 7B: mean over the nine tasks of Tables 2–3 except TextVQA. *Below ToMe, the strongest competitor in this cell (Table 2). †Grid / attention-pool.
| Host | Task | ToMe | ToMe 1-shot | VisionZip | Ours | |
|---|---|---|---|---|---|---|
| LLaVA-1.5-7B | 8 tasks, mean | 144 | 94.7 | 90.9 | 91.4 | 94.9 |
| BLIP-2 host (Q-Former) | COCO CIDEr | 144 | 91.6 | <91.6* | <91.6* | 96.6 |
| Research host (135M) | COCO CIDEr | 144 | 50.3 | 73.6 | 74.0 | 84.9 |
| CLIP ViT-B/16 | Flickr30k | 49 | 93.3 | 73.8 | 63.8 | 94.1 |
| CLIP ViT-B/16 | Flickr30k | 98 | 98.6 | 96.5 | 97.6 | 99.5 |
| Worst case over the five rows above | 50.3 | 73.6 | 63.8 | 84.9 | ||
| LLaVA-1.5-7B | TextVQA | 144 | 67.6 | 34.9 | 71.9 | 45.0 / 66.1† |
We thank the reviewer for recognising that label-free visual token compression that adapts to different VLMs is an important direction, and that our training objectives are simple and interpretable.
The goal of this paper is a transferable, self-supervised compressor that is trained once and inserted into a frozen host VLM, so the host never has to be retrained. Our aim is performance comparable to the uncompressed host and to existing token-compression methods, with little or no degradation, not necessarily to outperform trained compressors that co-adapt the host. The saving comes from transfer: one training run serves every host that shares the same vision encoder.
We acknowledge that our wording overstated the scope of this transferability. Our experiments cover hosts built on CLIP-based vision transformers, not all vision encoders. CLIP-based VLMs are widely used in research and practice, but we will revise the abstract and Introduction to state this scope precisely, and we plan to extend the approach to other encoder architectures in future work. Below, we address the reviewer’s comments from this perspective.
While the individual components of our method are conventional, we claim novelty in combining them to address a new problem. We show that this combination produces a compressor that is trained once and transfers across host models without modification, while performing comparably to state-of-the-art methods.
Regarding the comment that the paper may be AI-generated: we confirm that the work, including the method, experiments, and analysis, is our own. As stated in the AI assistance disclosure, we used an AI tool only to help improve the wording. All authors reviewed and edited the manuscript, and the paper was proofread several times. We take the feedback on readability seriously and we can further improve the paper’s readability considering the reviewer’s suggestions about the choice of some words. We respectfully ask that the discussion focus on the technical content.
Comparison with recent methods. Table 3 already includes VisionZip (CVPR 2025). To compare with the latest methods without re-implementing them, Table R1 uses the numbers published by VisionZip and EvoComp (CVPR 2026). Individual benchmarks move in both directions: against VisionZip at 128 tokens we lead on GQA (+0.83) and POPE (+1.77) and trail on MMBench (−1.74); against EvoComp we lead on POPE (+1.77) and trail on GQA (−0.79) and MMBench (−3.44). The mean over the three benchmarks gives the overall picture: at K=144 our compressor retains 95.53%, within one point of VisionZip (95.25%) and EvoComp (96.35%) at 128 tokens, their nearest published budget. EvoComp, however, is trained for each host with labelled data, whereas our compressor uses no labels, never sees LLaVA-1.5-7B and is inserted unchanged. It therefore performs comparably to the state of the art without any training in the host. We will add this table to the paper.
*Table R1. LLaVA-1.5-7B, zero-shot, retention (%) against the uncompressed host of each evaluator. Published rows are from the VisionZip and EvoComp papers; our row uses our evaluator (Table 3 of the submission). Mean: mean of GQA, MMBench and POPE.