- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies how to allocate a fixed visual-token budget between compressed representations and recovered original tokens.
TokenGraft selects original tokens using visual importance and spatial distance from the compressed representation, combines their percentile ranks, and inserts the selected tokens with their original features and positional information. Recovery requires no training and does not depend on the text query.
The evaluation compares recovery with native compression at matched budgets across three compressors, three VLM backbones, referring-expression grounding, and CV-Bench.
Critical strengths
-
The budget comparison is carefully controlled. Recovery and native expansion use the same final token count, and Nuwa also matches intermediate budgets. This makes the observed gains informative about representation choices under a fixed budget.
-
Recovery has credible benefits beyond the weakest baseline. Nuwa improves by 1+ percentage points across the three Qwen grounding datasets at 12.5% base retention, with positive paired intervals. All nine nonzero InternVL fixed-budget allocations also improve significantly. These results support the usefulness of recovery in several settings.
-
Conditioning recovery on the compressed representation has direct support. Removing base reference locations reduces VisionZip and PPE accuracy by 1.16 and 5.49 points in Table 10. This supports selecting tokens that complement the existing representation.
-
The method is simple to apply and the evaluation is reasonably transparent. It requires no parameter updates, and the recovered set can be reused across questions about the same image. Image-clustered paired intervals, invalid-output handling, and the ordering control help assess the results reliably.
Critical weaknesses
-
The contribution over existing recovery and selection methods is not sufficiently established. Recoverable Compression already restores original visual tokens. TopV includes uniform recovery after pruning and a recovery ablation under the same FLOPs-reduction target. SCOPE already combines saliency and coverage through greedy selection. These works are cited, but the distinction from their recovery and selection mechanisms needs more explanation. The paper also omits Reroute, which studies recoverable routing under matched budgets, although its layer-wise re-entry differs from TokenGraft's interface-level recovery. These connections make the novelty of budget allocation and the proposed selector less clear.
-
Recovery and sequence ordering are not fully separated in the main evidence. Table 11 shows that sorting alone explains most of the approximately 20-point OV1/PPE improvement. TokenGraft adds only 0.47, 1.24, and 0.16 points over sorted expansion, with only the RefCOCO+ interval strictly above zero. The existing control is useful, but Qwen/PPE lacks the same comparison despite producing the largest main-study gains. This leaves the contribution of original-token recovery to those gains unresolved.
-
The combined selector does not show a clear advantage over simpler recovery rules. In Table 9, TokenGraft is significantly worse than importance-only recovery for PPE, does not significantly outperform spatial-only recovery for VisionZip, and has no significant advantage over the three simple rules for Nuwa. A common rule without compressor-specific tuning is convenient, but the added value of the percentile product remains weakly supported.
-
Transfer and runtime results place limits on the practical contribution. VisionZip and Nuwa have no significant gains in the OV1 sweeps, and only two PPE allocations have positive overall intervals in the CV-Bench fixed-budget sweep. The reported sub-1% request-time increment is supported for the measured workload, but Table 14 shows larger relative increases in time to first token and no end-to-end speedup over Dense for PPE or Nuwa. These results narrow the demonstrated benefit of a general recovery module.
Questions for authors
-
How much of the Qwen/PPE improvement remains when native expansion uses the same original-anchor ordering as TokenGraft? This would directly affect my interpretation of the largest gains.
-
What does the budget-allocation diagnosis establish beyond TopV's recovery ablation? A comparison with uniform recovery or a closely related saliency-coverage rule, using the same candidates, ordering, and final budget, would help establish the incremental contribution.
-
Given Table 9, what evidence shows that the percentile-product selector adds value beyond a simpler recovery rule across compressors?