- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes PrimeFocus, a test-time reinforcement learning framework for vision-language models that addresses two limitations of majority-voting-based adaptation: the lack of visual grounding in pseudo-rewards and the use of fixed training schedules. The method augments TTRV's frequency-based reward with three attention-derived signals—visual enhancement, visual focus, and group visual consensus—and introduces a label-free early stopping criterion based on the stabilization of token-entropy deviations from an exponential moving average. Experiments on seven visual recognition and VQA benchmarks with two InternVL backbones report average accuracy improvements of 2.27 and 2.42 percentage points over TTRV. The paper also provides component ablations, attention diagnostics, and early stopping analyses.
Critical strengths
-
Clear motivation targeting concrete limitations of existing test-time RL methods. The paper identifies two relevant issues in consensus-based test-time adaptation: majority agreement does not necessarily imply visual grounding, and a fixed training schedule may be suboptimal across different datasets. Both problems are practically meaningful, and the proposed components are clearly connected to these motivations.
-
Simple and computationally practical reward design. The attention-derived rewards are computed from the model's own attention statistics without additional annotations or a separately trained reward model. They can be incorporated into the existing GRPO framework, and the reported additional computational overhead is relatively modest (approximately 12% per training step).
-
Consistent performance improvements with useful diagnostic analyses. PrimeFocus improves average accuracy over TTRV on both evaluated backbones, with positive gains in 13 of 14 model–benchmark combinations. The paper goes beyond final accuracy by analyzing individual reward components, attention smearing and collapse, attention–correctness correlations, and early stopping trajectories. These analyses provide some empirical support for the proposed design.
Critical weaknesses
-
Insufficient validation of visual grounding. The proposed rewards rely on attention statistics as proxies for visual grounding, but higher or more concentrated visual attention does not necessarily indicate reliance on relevant visual evidence. Moreover, the reported correlations between visual attention and correctness vary across tasks. The current experiments mainly establish correlations and changes in attention patterns, without directly demonstrating that PrimeFocus reduces visually unsupported predictions or corrects erroneous majority consensus.
-
The group visual consensus reward does not explicitly isolate inter-response agreement. Equation (7) measures the entropy of the averaged attention map, which combines individual attention concentration with inter-response disagreement. Consequently, its effectiveness may partly arise from additional attention sharpening rather than genuine visual consensus. Furthermore, when all responses share the same answer, the group-level reward becomes constant and cancels during GRPO advantage normalization. These limitations weaken the claimed interpretation and effectiveness of the proposed consensus mechanism.
-
Incomplete experimental validation and insufficient comparisons. The experiments do not adequately isolate the contributions of different reward components and early stopping, and comparisons with closely related VLM test-time RL methods are limited. The early stopping analysis covers only a few single-seed trajectories without comparisons against simpler alternatives, while generalization beyond InternVL backbones remains unexplored. Additionally, the reported adaptation samples, epochs, and training steps require clarification, raising concerns about reproducibility and fair comparison.
Questions for authors
-
Can the authors directly verify that the proposed attention-based rewards improve visual grounding rather than merely altering attention distributions? For example, could image perturbation or visual ablation experiments demonstrate reduced reliance on language priors and improved correction of incorrect majority predictions?
-
How much of the group visual consensus improvement comes from genuine inter-response agreement rather than individual attention concentration? Could the authors disentangle these effects through separate reward ablations and clarify how the mechanism handles confidently incorrect or unanimous answer clusters?
-
How does the proposed early stopping criterion compare with simpler label-free alternatives, such as entropy stabilization, reward plateaus, or policy KL divergence?
-
Could the authors provide more comprehensive and controlled evaluations, including comparisons with closely related methods such as TTIQ, complete reward-component ablations, and experiments on additional VLM families? Additionally, could they clarify the relationship between the reported adaptation samples, epochs, and training steps to ensure reproducibility and fair comparisons?
Thank you for the rebuttal and additional experiments. I appreciate the authors' efforts, particularly the new analyses of visual grounding and group consensus. However, concerns remain regarding the limited early stopping evaluation, insufficient comparisons with related methods, and unclear training settings. While the rebuttal addresses some of my concerns, I will maintain my original rating.