acceptodds
Withdrawn from ICLR 2027

Learning What to Align: Automatic Target Discovery for Hallucination Mitigation in MLLMs

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Multimodal large language models often suffer from visual hallucination, where generated responses are inconsistent with the input image. Existing preference-based alignment methods mainly rely on sequence-level supervision, making it difficult to localize visual evidence and response tokens responsible for hallucinated predictions. Recent target-level approaches improve alignment precision but usually depend on manually annotated regions or token spans. In this paper, we propose Learning What to Align with Target Discovery (LWA-TD), which leverages minimal visual contrast to identify visual evidence and response tokens responsible for grounded semantic differences. Given counterfactual image-response pairs, LWA-TD identifies question-relevant visual changes and estimates token sensitivity to counterfactual visual changes. The discovered targets are further incorporated into preference optimization and grounding regularization to improve visual-text consistency. Experiments on multiple hallucination benchmarks demonstrate that LWA-TD effectively reduces hallucination across different model scales and backbones, while maintaining strong general multimodal reasoning

Primary area:
foundation or frontier models, including LLMs
Keywords:
Multimodal Large Language Models, Hallucination Mitigation, Preference Optimization, Visual Grounding, Target Discovery

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_SUYSOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper identifies relevant image regions and response tokens from image pairs with minimal visual differences. It uses these discovered targets in bidirectional preference optimization and an evidence-consistency objective to reduce visual hallucinations.

Critical strengths

(1) Automatic target discovery could reduce the need for manually annotated regions and response spans. (2) The paper includes a controlled comparison using the same training data, component ablations, and results across several backbones. These experiments support the method's practical value.

Critical weaknesses

(1) The method is closely related to TL-DPO's target-level alignment and S-VCO's minimal visual contrasts. BiDPO also uses the same 12K paired dataset and token-level supervision. The controlled comparison is therefore important, but the MMHal score improves only from 3.09 for BiDPO to 3.14 for this method; repeated runs are needed to assess this increment. (2) The masking experiment shows that the selected regions matter to the model, but does not directly measure whether the discovered regions match the known image edits.

Questions for authors

Could the authors report multi-seed results for the controlled comparison with BiDPO and evaluate localization against the known edited regions? These results would materially clarify the incremental contribution of target discovery.

Reviewer_oTdCOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper presents Learning What to Align with Target Discovery (LWA-TD). LWA-TD is a new preference-based alignment methodology to reduce visual hallucinations in vision language models. It leverages minimal visual contrast to identify visual evidence and response tokens responsible for grounded semantic differences. It starts from a dataset composed of counterfactual image-response pairs. For each sample LWA-TD identifies question-relevant visual changes and estimates token sensitivity to counterfactual visual changes. The discovered visual and textual targets are used to guide preference optimization and grounding regularization to improve visual-text consistency. They bring various experiments across different multimodal hallucination benchmarks and model backbones. The results show substantial improvements over the corresponding base models and generally competitive performance against existing hallucination-mitigation approaches.

Critical strengths
  1. Relevant Problem and no explicit manual annotation needed. Weighting differently the textual and visual tokens is an interesting approach to improve preference optimization to reduce visual hallucinations. Furthermore, given a previously generated counterfactual image-response pairs dataset, LWA-TD is able to automatically identify textual and visual targets and exploit them during training by exploiting the way the dataset was artificially constructed.

  2. Strong experimental validation. The paper includes comparisons with several recent related approaches, extensive ablations of the individual components, and an interesting visual-region masking experiment showing the effectiveness of the identified target regions.

Critical weaknesses
  1. Limited evaluation on recent strong backbones. The experiments are conducted on various models from 2023 (Llava-1.5, Muffin), 2024 (Llava-NeXT) and the latest is released in January 2025 (Qwen2.5 VL). Vision language models have substantially advanced from those “not so recent” backbones.

  2. Reliance on synthetic counterfactual training pairs. The methodology relies on an artificially constructed dataset of counterfactual image-response pairs. My main concern is whether the proposed method can be applied to more standard preference datasets, or whether it fundamentally requires training data to be constructed in this specific way. This concern is particularly relevant for the visual target discovery component, which relies on patch-wise differences between paired images. Since BiDPO-data-12k is explicitly constructed through localized visual modifications of the original images, the dataset provides a particularly favorable setting in which corresponding patches remain largely aligned and the modified region can be highlighted through feature differences.

Questions for authors

Following my major concerns in the Critical Weaknesses section:

  • I would very much appreciate an additional evaluation on a more recent backbone like Qwen3-VL [1]. The improvement on Qwen2.5-VL is relatively modest compared with the gains observed on older backbones. An evaluation on a more recent model such as Qwen3-VL would help clarify whether the proposed alignment strategy remains beneficial as the underlying MLLM becomes stronger.

  • Can LWA-TD be applied to a more general preference-learning setting where the training data are not explicitly constructed as counterfactual tuples (Iw,Il,Q,Aw,Al)(I_w, I_l, Q, A_w, A_l) in this way? In particular, how dependent is the visual target discovery mechanism on the spatial alignment induced by localized synthetic edits?

[1]: Bai, Shuai, et al. "Qwen3-vl technical report." arXiv preprint arXiv:2511.21631 (2025).

Reviewer_3xtqOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes LWA-TD to reduce visual hallucinations in multimodal large language models. The method uses counterfactual image pairs to automatically find important visual regions and response tokens. These targets are then used in bidirectional preference optimization, token-level weighting, and attention alignment. Experiments on several models and benchmarks show improvements in hallucination reduction while maintaining general multimodal performance.

Critical strengths
  1. The paper addresses an important problem in visual grounding. Using counterfactual image pairs to discover visual regions and response tokens is a reasonable idea that reduces the need for extra fine-grained annotations. The joint use of visual and textual targets also provides more focused training signals than sequence-level preferences alone.
  2. The experimental evaluation is fairly complete. The paper includes comparisons across several models, useful ablation studies, and masking experiments. In particular, Table 7 compares with BiDPO using the same training data, while Tables 4, 5, and 8 provide evidence about the effects of visual targets and token weighting. The improvement in AMBER hallucination rate and coverage is also encouraging.
Critical weaknesses
  1. The independent contribution of joint target discovery is not fully established. The paper provides useful ablations, including target removal, visual masking, and different token weighting strategies. However, it remains unclear whether the improvements come specifically from discovering more useful visual and textual targets, rather than the model's existing attention patterns or changes in the overall token-level loss strength. In particular, the attention-based weighting in Equation (17) affects both token importance and the total loss weight. A controlled comparison with equal total weighting strength would help clarify this contribution. The visual target definition in Equations (9) and (10) also needs clarification, since cosine similarity can be negative and the resulting weights are not explicitly guaranteed to be non-negative.
  2. The additional improvements over BiDPO need stronger evidence of reliability. Table 7 provides a valuable controlled comparison using the same training data and settings. The MMHal improvements over BiDPO are relatively small, while the AMBER results are more encouraging, showing reduced hallucination together with improved coverage. However, the paper does not clearly report variability across repeated runs or sufficient evaluation details to assess the reliability of the smaller differences. Given the additional components introduced by LWA-TD, stronger evidence of consistent improvements would make its incremental contribution more convincing.
Questions for authors
  1. What is the main difference between LWA-TD and previous automatic token-level preference methods? Which existing experiment best shows the added benefit of jointly discovering visual and textual targets?
  2. How can the authors show that the discovered targets are useful for correcting hallucinations, rather than mainly reflecting the model's existing attention or visual sensitivity? Does the method still perform better when different weighting methods use the same total loss weight?
  3. Are repeated-run results available for the matched-data BiDPO comparison in Table 7? How much do the reported improvements vary across runs?
  4. How sensitive is target discovery to counterfactual data quality, such as editing artifacts or unrelated visual changes? Can the authors clarify this using existing data or results?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.