acceptodds
Under review as a conference paper at ICLR 2027

Latent Noise Mask Eliminates Visual Redundancy of Multimodal Large Language Models

Abstract

Multimodal large language models (MLLMs) often fail in fine-grained visual reasoning, as question-relevant visual cues are diluted by dense and redundant image tokens. Recent multimodal reasoning methods usually extend chain-of-thought from language models into visual or latent spaces, seeking to add intermediate reasoning states while overlooking the negative impact of redundant visual tokens. We propose LatEnt Noise maSk (Lens), a question-conditioned visual evidence purification framework that empowers MLLMs to reason with cleaner visual cues in latent space. Lens introduces a lightweight Lens Evidence Token (LET) to score which visual tokens support the current question and preserve them during decoding. Guided by the LET scores, it injects adaptive latent noise into low-relevance tokens, softly suppressing distractors without changing the model backbone or token sequence. With only one temporary learnable control token and a lightweight noise generator, Lens adds minimal overhead while improving the base MLLM by 2.4-6.4 points on most VQA datasets and by 4.1-6.4 points on grounding tasks. These results show that multimodal reasoning can benefit more directly from cleaner question-relevant visual evidence than from simply extending the reasoning trace.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Multimodal Large Language Model, Latent Reasoning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_KfdxOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes LENS (Latent Noise Mask), a framework for improving multimodal reasoning by suppressing question-irrelevant visual tokens in latent space. The method introduces a learnable Lens Evidence Token (LET) to estimate visual token relevance and injects adaptive noise into low-relevance tokens while preserving the original token structure. The framework is optimized through supervised fine-tuning and reinforcement learning. Experiments across multiple MLLMs and ten benchmarks demonstrate improvements in visual question answering and grounding tasks.

Critical strengths

Effective visual evidence selection framework. LENS introduces a soft latent-space suppression mechanism that preserves relevant visual tokens without changing the token sequence structure, providing a practical alternative to hard token pruning.

Extensive experimental evaluation. The paper evaluates multiple model families and benchmarks, with comprehensive ablation studies demonstrating consistent improvements in both visual question answering and grounding performance.

Critical weaknesses

Limited methodological novelty. Question-conditioned visual token selection and suppression have been extensively explored in previous studies. LENS primarily combines a learnable evidence token with latent noise injection, which appears to be an incremental modification of existing visual token filtering and feature gating approaches.

Potential confounding effects of additional supervision. The LET module is trained using question-relevant bounding-box annotations, providing explicit supervision about important visual regions. It remains unclear whether the improvements primarily result from this additional supervision rather than the proposed latent noise mechanism. Comparisons with alternative methods under identical evidence supervision are necessary.

Insufficient justification for latent noise injection. The paper assumes that injecting noise into low-relevance visual tokens effectively suppresses redundant information. However, feature perturbation does not necessarily imply information removal or reduced interference. More direct evidence is needed to establish that latent noise genuinely suppresses irrelevant visual information rather than merely altering feature distributions.

Incomplete isolation of the proposed method's contribution. Although the paper includes SFT and GRPO ablations, the benefits of evidence selection, noise injection, and reinforcement learning are not fully disentangled under matched training conditions. Stronger comparisons with conventional soft masking, feature gating, and attention reweighting using the same LET predictions would better establish the necessity of the proposed mechanism.

Limited analysis of computational efficiency and generalization. LENS requires an additional evidence-probing pass and a noise generation module during inference. The practical computational overhead is not sufficiently quantified. Furthermore, its dependence on object-level evidence annotations raises questions about generalization to tasks requiring global visual understanding or complex multi-region reasoning.

Questions for authors

NA

Reviewer_PQy7OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes LENS, a question-conditioned framework for reducing "visual distractions" in MLLMs. A temporary evidence token predicts the relevance of visual tokens, and a learned noise generator perturbs low-relevance features before answer generation. The method uses supervised evidence labels and further refines evidence selection with GRPO. Experiments show improvements on VQA and grounding tasks across several model families.

Critical strengths
  1. The paper addresses a relevant problem: useful visual evidence can be overwhelmed by irrelevant image regions. I find the focus on reducing distractions well motivated.

  2. The method preserves the original visual token layout and requires relatively simple additional modules.

  3. The results cover both VQA and grounding, and the component ablations provide useful support for combining evidence prediction with latent suppression.

Critical weaknesses
  1. The evidence-policy optimization needs stronger justification. The method independently samples each token with a Bernoulli policy and uses only four rollouts per pompt. It is unclear whether this provides stable updates for high-dimensional masks or captures dependencies between evidence regions.

  2. The choice of intervention location is not sufficiently studied. Noise is applied to visual embeddings before decoder processing, but there is no comparison across different layers. Layer-wise experiments would help explain this design choice.

  3. Cross-dataset generalization remains unclear. Evaluations on unseen benchmarks would better support the broader claims. For example, MMStar would test comprehensive multimodal capbililty, while V*Bench and HR-Bench are particularly relevant to the claimed benefits for fine-grained perception.

Questions for authors

Please see the weakness section above.

Reviewer_SbvcOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

MLLMs are limited in fine-grained visual reasoning because problem-related local visual cues are easily diluted by dense and redundant background or distracting image tokens. Most existing methods add information by expanding textual, visual, or latent space thought chains, ignoring the negative impact of redundant visual interference, which requires the model to sift through information in a longer context. From the perspective of visual evidence purification, the authors propose the LENS (Latent Noise Mask) framework, which purifies the visual context before decoding through two coupled stages: evidence probing and latent suppression.

Critical strengths

S1. A novel perspective on refining visual evidence, combined with a soft intervention mechanism that preserves sequence topology. The paper accurately points out that the core bottleneck of multimodal fine-grained inference lies in visual selectivity rather than visual availability.

S2. Extensive validation across model families and scales, and robust substitution ablation.

Critical weaknesses

W1. The design of fixed linear dimensions for the probe head is mathematically in conflict with the implementation of dynamic resolution.

W2. Double prefill renders the KV cache invalid, causing the actual first-word latency (TTFT) to surge by 60%–71%, contradicting the claim of “minimal overhead”.

Questions for authors

The core motivation proposed in this paper, "purifying visual evidence through soft noise in latent space," is very appealing. However, there are some disconnects and logical gaps between the theoretical formulas presented in this paper and actual engineering deployments. I would be happy to raise the score if the authors can clarify the correspondence between theory and practical deployment and revise the relevant formulas and efficiency statements in the Rebuttal and revised version.

Q1. Conflict between fixed linear dimensions of the "Probe Head" and dynamic resolution deployment Could the authors explain how the single-layer Linear projection (with fixed weight dimensions, as seen in page4 eq(5) and Page 14 Appendix B) maps a single 1D vector hm∈Rdh_m \in \mathbb{R}^d to an NN-dimensional output—where NN varies dynamically between 3 and 1000 (as described in page13 Appendix B)—while maintaining spatial grid alignment for images with varying aspect ratios?

Q2. The contradiction between the actual deployment computation flow of dual prefill (KV Cache obsolescence) and the claim of "Minimal Overhead". If, based on the underlying feature replacement locations in Page 14 Appendix B and the measured TTFT surge of 60%–71% in *Page 20 Table G1, then the actual deployment requires two complete prefill forward computations ($2 times $Prefill FLOPs)**. Does this conform to the Minimal Overhead claim?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.