- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a training-free framework, named ReMo, to improve visual perception and reasoning in vision-language models by combining language-guided localization, ViT-guided context completion, and attention reweighting. The method is evaluated on Qwen3-VL-4B/8B and Kimi-VL-16B-A3B across high-resolution perception, general visual understanding, and hallucination benchmarks. The reported results show improvements over the corresponding base models. Progressive ablations and qualitative visualizations provide evidence for the contributions of the proposed components.
Critical strengths
- The paper reports consistent improvements across multiple VLM backbones and diverse evaluation settings. Experiments cover both dense and MoE models, with gains on high-resolution perception, general visual understanding, and hallucination benchmarks.
- The language-guided localization and ViT-guided context completion is well motivated. Task-relevant language cues identify candidate evidence, while visual affinities help expand coverage to surrounding context. The framework implements this interaction without updating model parameters and preserves the full visual token sequence during attention modulation. This provides a useful approach to exploiting complementary information already available within pretrained VLMs.
Critical weaknesses
1. The novelty and advantages over closely related methods are insufficiently demonstrated.
For language-guided localization, internal attention-based localization and semantic guidance already appear in MLLMs Know Where to Look [1] and VGA [2], while VAALE [3] combines generated textual context with attention refocusing. The additional contribution of ReMo’s knowledge priors and semantic-anchor denoising requires clearer justification. For ViT-guided completion, VEGAS [4] already uses vision-encoder attention to guide the language model. AnchorPrune [5] supplements query-relevant tokens with complementary context, and HiPrune [6] preserves neighboring tokens around important regions. Although the latter two target pruning, their context-selection strategies remain relevant to ReMo’s claimed novelty. For evidence reweighting, training-free visual attention enhancement is established by PAI [7], VAR [8], and VGA [2]. The paper should clarify what its specific combination contributes beyond these precedents.
The experiments also omit these methods and primarily compare against base VLMs. Representative attention-enhancement baselines should be evaluated using the same backbone and protocol, while alternative context-selection strategies should be compared under controlled token budgets. Without such discussion and comparisons, the results demonstrate improvements over base models but do not varify the novelty or advantages of its proposed combination.
2. The experiments do not isolate the value of ViT-guided completion or support the claimed efficiency advantages.
The progressive ablation in Fig. 4 is useful, but adding ViT-guided completion also increases the number of visual tokens receiving an attention boost. Without comparisons that enhance the same number of tokens, such as spatial expansion or selection by the original relevance scores, it remains unclear whether the gains come from ViT affinities or simply from covering more visual content. Furthermore, the efficiency argument is not supported by end-to-end latency or memory measurements, though the additional 4B VLM is only used for prior generation.
Reference
[1] Khayatkhoei, Mahyar, Prateek Chhikara, and Filip Ilievski. "Mllms know where to look: Training-free perception of small visual details with multimodal llms." International Conference on Learning Representations. Vol. 2025. 2025.
[2] Zhao, Jianfei, et al. "Tell model where to look: Mitigating hallucinations in mllms by vision-guided attention." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2026.
[3] Wang, Jingyi, Fei Li, and Rujie Liu. "Attention to details, logits to truth: visual-aware attention and logits enhancement to mitigate hallucinations in LVLMs." ICASSP 2026-2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2026.
[4] Wang, Zihu, et al. "VEGAS: Mitigating Hallucinations in Large Vision-Language Models via Vision-Encoder Attention Guided Adaptive Steering." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2026.
[5] Oh, Kyuan, and Bumsoo Kim. "AnchorPrune: Relevance-Anchored Contextual Expansion for Visual Token Pruning." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2026.
[6] Liu, Jizhihui, et al. "HiPrune: Hierarchical Attention for Efficient Token Pruning in Vision-Language Models." Findings of the Association for Computational Linguistics: ACL 2026. 2026.
[7] Liu, Shi, Kecheng Zheng, and Wei Chen. "Paying more attention to image: A training-free method for alleviating hallucination in lvlms." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2024.
[8] Kang, Seil, et al. "See what you are told: Visual attention sink in large multimodal models." International Conference on Learning Representations. Vol. 2025. 2025.
Questions for authors
-
Novelty and comparative advantage. What specific technical advances distinguish ReMo’s semantic-anchor refinement, ViT-guided completion, and attention biasing from the related methods discussed above? Can the authors provide a comparison with these representative attention-enhancement methods using the same backbone and evaluation protocol to substantiate ReMo’s advantages?
-
Contribution of ViT-guided completion. Does the improvement in Fig. 4 persist when the seed set, number of reinforced tokens, and attention-bias magnitude are held fixed across completion strategies?
-
End-to-end efficiency. What are ReMo’s end-to-end latency and peak memory usage relative to the base VLM, including prior generation by the auxiliary 4B VLM? Can the authors separate the costs of prior generation and subsequent inference, and clarify which measurements support the claimed efficiency advantages?