- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates two delimiter-related attention patterns in multi-image multimodal large language models: visual tokens attending to delimiters associated with other images (Visual-Delimiter Leakage), and text tokens concentrating image-related attention on delimiters, with relatively limited attention to visual tokens (Text-Delimiter Sink). The authors introduce VDLS and TDSS to quantify these patterns and propose a training-free method, Delimiter-Aware Regrounding (DAR).
DAR consists of two modules. Delimiter Ownership Correction (DOC) reallocates part of the attention assigned to non-own image delimiters to the corresponding own-image delimiters while preserving the total delimiter-attention mass. Visual Evidence Redirection (VER) selects high-TDSS attention heads and redirects part of their delimiter contributions toward image-local visual evidence constructed using an ownership prior. The paper evaluates DAR on four multi-image benchmarks across eight model configurations from four model families, and provides component ablations, attention analyses, inference-overhead measurements, and additional video hallucination experiments on VidHalluc.
Critical strengths
-
The method provides concrete, connected interventions for two distinct parts of the computation.
DOC adjusts the delimiter-based ownership of visual tokens, while VER redirects delimiter contributions toward image-local evidence in selected attention heads. The explicit attention-mass constraint and ownership prior provide a substantive design beyond uniform delimiter scaling. The component ablations offer initial support for the individual contributions and combined effectiveness of the two modules.
-
The improvements are consistent across models and tasks and have practical value.
Table 1 covers eight configurations from four model families. DAR improves accuracy in 31 of the 32 model–benchmark combinations and matches the baseline in the remaining combination, with a maximum absolute gain of 5.40 percentage points. Its training-free nature and supplementary VidHalluc results further support its practical relevance. The appendix also reports inference overhead: latency increases by approximately 17%–19% in the two measured settings, while peak GPU memory remains essentially unchanged.
Critical weaknesses
-
Evaluation on dedicated multi-image hallucination benchmarks is missing.
The main multi-image experiments use Mantis, BLINK, MIRB, and QBench2, which primarily evaluate general multi-image understanding. Appendix E acknowledges that task accuracy serves as a proxy for hallucination-related failures, but improvements on these benchmarks alone do not establish reductions in the specific hallucinations targeted by DAR. The VidHalluc experiments provide useful additional evidence, but concern video hallucination rather than a dedicated multi-image hallucination evaluation.
The paper would benefit from evaluation on dedicated benchmarks such as MIHBench or MIOH, the latter of which is already cited in the manuscript. Reporting results by hallucination category would help determine whether DAR reduces errors involving object existence, counting, attributes, or cross-image consistency, providing more direct support for its central claim.
-
Comparisons with two closely related baselines, FOCUS and DAB, are missing.
The main external comparison is limited to DTS. FOCUS is a training-free method that directly targets cross-image information leakage through image-wise focused decoding. DAB (Dynamic Attention Balancing) is also training-free and mitigates multi-image hallucinations by redistributing attention across images. Both are closely related to DAR's objective and provide important comparisons for assessing its incremental contribution.
Questions for authors
-
Could you evaluate DAR on a dedicated multi-image hallucination benchmark, such as MIHBench or MIOH, and report results by hallucination category to examine whether the improvements on general multi-image benchmarks translate into reductions in the targeted hallucinations?
-
Could you compare DAR with FOCUS and DAB on mutually supported models under the same evaluation protocol, reporting both performance and inference overhead to establish the relative benefits and computational trade-offs?