- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes CARE, a training-free framework for post-pruning visual token refinement in vision-language models. CARE constructs one-to-one token exchange proposals using query relevance and token redundancy, followed by counterfactual decision-guided selection based on the frozen model's prediction confidence. Experiments across multiple benchmarks, pruning methods, and model backbones show improved accuracy and reduced recovery-induced errors without increasing the visual token budget.
Critical strengths
-
The paper identifies an important limitation of existing token recovery methods: restoring relevant visual information can also disrupt originally correct predictions. This provides a meaningful motivation for decision-aware recovery.
-
The fixed-budget token exchange strategy is training-free and applicable to different pruning methods. The reported reductions in recovery-induced errors are encouraging.
Critical weaknesses
-
Several reported numbers are difficult to reconcile. Section 5.3 mentions 500 evaluated samples, but Table 2(b) reports 13 damaged samples with an RIE of 2.03%, implying approximately 640 originally correct samples. Similarly, the 1,500-sample setting in Section 5.4 is inconsistent with Table 3 (e.g., 221 damaged samples at 4.05% RIE). Moreover, Figure 4 shows 11–20 token exchanges, exceeding the stated maximum of eight. These inconsistencies undermine confidence in the experimental results.
-
Counterfactual beam search requires multiple VLLM evaluations, yet Table 4 reports nearly identical latency for CARE and the base pruner. It is unclear whether the complete search overhead is included. Furthermore, higher self-generated answer likelihood does not necessarily indicate greater correctness. The gains over RESTORE are relatively small, and stronger compute-matched comparisons are needed to establish the benefit of the proposed selection strategy.
Questions for authors
-
Can the authors reconcile the sample counts, RIE values, and token exchange numbers in Tables 2–3 and Figure 4 with the stated experimental settings?
-
Does the latency in Table 4 include all counterfactual model evaluations? Please report end-to-end inference cost and the average number of VLLM evaluations per sample.
-
How does CARE compare with simpler fixed-budget exchange strategies using confidence-based filtering under the same computational budget?