- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes VEC-TTA, a per-sample test-time adaptation method for small multimodal language models. The method first constructs five query-related pseudo-supervision samples, then updates a temporary LoRA adapter through invariant evidence consolidation, which enforces consistency between answer fitting and weak image augmentation, and counterfactual evidence calibration, which encourages pseudo-answers to receive stronger support when the image is present than when a fully black image is used. Experiments on two models of approximately 2 billion parameters and five benchmarks show that the method improves image captioning and question answering performance. The paper further validates the approach through module ablation, occlusion of matched evidence regions and control regions, analysis of the adaptation process, and measurement of computational cost.
Critical strengths
-
The paper provides a more direct test of visual reliance than overall task accuracy. Under a fixed post-adaptation model, Table 3 compares occlusion of evidence regions with occlusion of non-overlapping control regions of equal area. VEC-TTA improves original-image accuracy and the evidence selectivity gap on both models, while accuracy after control occlusion remains close to original-image performance. This offers substantive evidence for the intended use of visual grounding, going beyond merely suppressing answers on fully black images.
-
The method achieves generally favorable results across two models and multiple task types, with particularly clear gains for SmolVLM on image captioning and VizWiz. Table 2 supports the complementary roles of IEC and CEC, and the paper explicitly analyzes low-quality pseudo-supervision and failure cases. In addition, Table 6 reports that single-sample end-to-end time decreases from 207.50 seconds to 103.69 seconds relative to TTAdapt, giving the comparison clearer practical reference value.
Critical weaknesses
-
The core contribution has not yet been sufficiently distinguished from ordinary query-conditioned self-training. Table 2 removes IEC or CEC at the full-stage level, but does not include a baseline trained solely with cross-entropy under the same QEE sample set, LoRA placement, and update budget. Consequently, the large improvement attributed to IEC alone cannot be specifically tied to the cross-view consistency constraint. Without this control, the contribution of the evidence-constrained objective is difficult to separate from that of pseudo-supervision and the adaptation configuration itself.
-
Existing evaluations do not yet adequately distinguish improvements in visual understanding from effects arising from output style and differences in computational budget. Appendix C.1 acknowledges that caption length and style affect metrics, and several baselines in Table 5 show CIDEr-D values near zero. These anomalously low values require verification under unified prompting, decoding, and scoring settings. Table 6 valuably reports runtime comparisons with TTAdapt, but has not yet established accuracy-computation comparisons against all test-time computation baselines. These issues limit the interpretation of the large gains in image captioning and the assessment of the method's practical advantages over other test-time approaches.
Questions for authors
-
With QEE, LoRA placement, and update budget held fixed, how do cross-entropy fitting alone, the addition of weak-view augmentation, the further addition of symmetric KL, and the full method perform respectively?
-
If the no-visual-content reference is replaced with a mean-color image or a randomly paired unrelated image, do the benefits of CEC still persist? Table 3 already provides valuable local occlusion evidence; this additional control would help determine how much the effect depends on the fully black image as a specific reference.