acceptodds
Withdrawn from ICLR 2027

Compositional Grounded Contrast for Fine-Grained Multi-Image Understanding

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Although Multimodal Large Language Models (MLLMs) have advanced rapidly, they still face notable challenges in fine-grained multi-image understanding, often exhibiting spatial hallucination, attention leakage, and failures in object constancy. Existing approaches typically rely on expensive human annotations or large-scale chain-of-thought (CoT) data generation to solve these challenges. In contrast, we propose Compositional Grounded Contrast (CGC), a low-cost full framework for boosting fine-grained multi-image understanding of MLLMs. Built on existing single-image grounding annotations, CGC constructs compositional multi-image training instances through Inter-Image Contrast and Intra-Image Contrast, which introduce unrelated visual distractor contexts for cross-image discrimination and correlated cross-view samples for object constancy, respectively. CGC further introduces a Rule-Based Spatial Reward within the GRPO framework to improve source-image attribution, spatial alignment, and structured output validity. Experiments show that CGC achieves state-of-the-art results on fine-grained multi-image benchmarks, including MIG-Bench and VLM2-Bench. The learned multi-image capability also generalizes to broader multimodal reasoning tasks, yielding consistent gains on MathVista (+2.90), MuirBench (+2.88), MMStar (+1.93), MMMU (+1.77), BLINK (+1.69) and Hallusion (+1.16) over the Qwen3-VL-8B.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Multimodal Large Language Models, Multi-Image Understanding, Visual Grounding, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_M2y9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Compositional Grounded Contrast (CGC), a low-cost framework for improving fine-grained multi-image understanding in MLLMs. CGC automatically constructs multi-image training data from existing single-image grounding annotations through Inter-Image Contrast and Intra-Image Contrast. It further employs GRPO with a rule-based spatial reward to improve source-image attribution and object localization. Experimental results demonstrate strong performance on MIG-Bench and VLM2-Bench, along with improvements on broader multimodal reasoning benchmarks.

Critical strengths

(1) The paper addresses important challenges in fine-grained multi-image understanding, particularly cross-image attention leakage, spatial hallucination, and object constancy. The motivation is clear, and the proposed method directly targets these issues.

(2) The method constructs multi-image training data from existing single-image grounding annotations without requiring additional human annotations or large-scale CoT generation. This makes the data construction process relatively simple.

(3) The experiments show substantial improvements on MIG-Bench and VLM2-Bench, as well as gains on several general multimodal reasoning benchmarks.

Critical weaknesses

(1) The proposed data synthesis strategy primarily relies on relatively simple image compositions and grounding tasks, which remain substantially different from the complex multi-image understanding and reasoning required in real-world scenarios. Although accurate visual grounding is an important foundation for fine-grained multi-image understanding, it does not necessarily imply reliable cross-image relational reasoning, semantic correspondence, or object identity preservation. Specifically, we have the following two concerns:

(a) Cross-view generalization of Intra-Image Contrast. This strategy constructs correlated views through multi-scale cropping of the same image, preserving nearly identical object appearances without introducing genuine viewpoint changes. Therefore, it remains unclear whether it can effectively improve cross-view object constancy in complex scenarios. For example, in the vehicle tracking case shown in Figure F.1 (Q1), although CGC ultimately predicts an accurate bounding box, its reasoning states: “The bounding box coordinates from the first image can be applied to the 6th image to locate the same car.” This reasoning is questionable, as the vehicle's position and scale may change across frames, and the final predicted coordinates actually differ from those in the first frame. This suggests that correct localization does not necessarily correspond to reliable cross-frame spatial reasoning.

(b) Effectiveness of Inter-Image Contrast for complex cross-image semantic reasoning. This strategy primarily constructs training instances by randomly combining semantically unrelated images, requiring the model to identify the corresponding image and localize the target based on a given label. However, real-world multi-image tasks often involve more complex cross-image semantic relationships. For example, Figure F.1 (Q2) requires the model to identify a shared semantic concept across different scenes, rather than simply locating a predefined target among unrelated distractors.

Although the ablation studies demonstrate the contributions of both data construction mechanisms to the final performance, we remain somewhat skeptical about the claimed improvements in cross-view object constancy and cross-image semantic reasoning.

(2) The paper claims state-of-the-art performance on fine-grained multi-image understanding benchmarks. However, several recent works, such as UniVG-R1 (2025)[1] and GeM-VG (AAAI 2026)[2], have reported stronger results on MIG-Bench using the Qwen2-VL-7B backbone. These highly relevant methods are neither cited nor discussed in the paper, raising concerns about the completeness of the related work and the validity of the SOTA claim. Furthermore, the comparison with MIRG-RL in Table 6 is not fully fair, as CGC-8B uses the stronger Qwen3-VL-8B backbone, while MIRG-RL is based on Qwen2-VL-7B.

(3) The paper emphasizes CGC as a low-cost framework, but this advantage primarily comes from avoiding additional human annotations and large-scale CoT data generation. The full training procedure still relies on GRPO and requires 8 NVIDIA H200 GPUs. However, the paper lacks a systematic comparison of computational costs, such as GPU-hours, training throughput, and total computational overhead.

[1] UniVG-R1: Reasoning Guided Universal Visual Grounding with Reinforcement Learning. Bai S, Li M, Liu Y, et al. Univg-r1: Reasoning guided universal visual grounding with reinforcement learning[J]. arXiv preprint arXiv:2505.14231, 2025.

[2] Zheng S, Zhu Y, Zhao H, et al. GeM-VG: Towards Generalized Multi-image Visual Grounding with Multimodal Large Language Models[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2026, 40(34): 28857-28865.

Questions for authors

Please refer to the Critical Weaknesses section.

Reviewer_VjecOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Compositional Grounded Contrast, a framework for improving fine-grained multi-image understanding in MLLM. The proposed method consist of two parts. first an automated contrastive data synth pipeline that transforms existing single-image grounding annotations into multi-image training instances through Inter-Image Contrast and Intra-Image Contrast. Inter-Image Contrast combines independently sampled images to encourage source-image attribution and distractor discrimination, while Intra-Image Contrast constructs multiple cropped views of the same object to improve cross-view consistency. Second, the authors employ Group Relative Policy Optimization (GRPO) with a rule-based spatial reward that combines source-aware bounding-box IoU and structured output validity.The method is evaluated using Qwen-based models and InternVL3-8B on fine-grained multi-image benchmarks and broader multimodal reasoning benchmarks.

Critical strengths
  1. Simple and annotation-efficient framework. The proposed approach reuses existing single-image grounding annotations to construct multi-image training data without requiring additional human annotations or expensive generated reasoning trajectories. Despite its simplicity, CGC achieves substantial improvements over the base models, particularly on MIG-Bench (+23.91) and VLM2-Bench (+7.05) with Qwen3-VL-8B.

  2. Comprehensive experimental evaluation. The authors evaluate the method across multiple fine-grained and general multimodal benchmarks. The ablation studies examine the effects of the two data construction strategies, reward components, and training configurations. The additional experiments on InternVL3-8B also provide evidence of cross-architecture generalization.

Critical weaknesses
  1. Missing comparisons with recent strong baselines. The paper omits important recent methods, including UniVG-R1 (S Bai et al. 2025) and GeM-VG (S Zheng et al. AAAI 2026), which report higher MIG-Bench scores (72.64 and 74.65 vs. 67.57), albeit under different evaluation protocols. Moreover, under backbone-matched comparisons, CGC-7B underperforms Migician and only marginally outperforms MiCo. The paper also overlooks DiG (Tao et al., 2026), which similarly employs synthetic multi-image differential grounding data and GRPO-based training to enhance fine-grained visual perception. These omissions weaken the claimed state-of-the-art performance and methodological novelty.

  2. Limited methodological novelty and unclear sources of performance gains. The framework mainly combines standard techniques, including random image composition, object-centered cropping, and GRPO with IoU-based rewards. And even this exact technique combination can be found in recent papers. While the improvements are substantial, it remains unclear why these relatively simple strategies are so effective. The synthesized tasks may not adequately reflect challenging real-world cross-image correspondence, and the ablations do not sufficiently isolate the benefits over simpler augmentation baselines. Further analysis is needed to distinguish genuine perception improvements from task adaptation and structured output learning.

Questions for authors
  1. Could the authors clarify why UniVG-R1 and GeM-VG are not included in the comparisons? Given their strong MIG-Bench results, it would be valuable to provide evaluations under a consistent benchmark version and evaluation protocol.

  2. Could the authors clarify the methodological novelty of CGC compared with recent approaches that combine grounding-based data augmentation and GRPO, such as DiG (Tao et al., CVPR 2026)? Furthermore, given the simplicity of these approaches and their substantial empirical gains, a deeper analysis of why such training strategies are effective would help establish the underlying mechanisms and the specific contributions of CGC.

  3. Could the authors provide additional evaluations on widely adopted MLLM benchmarks, such as MME, MMBench, or MME-RealWorld, to further validate the claimed generalization? It would also be useful to report the variance across independent training runs, especially for benchmarks showing relatively small improvements.

Reviewer_XT9GOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Compositional Grounded Contrast (CGC), a training framework for fine-grained multi-image understanding that repurposes existing single-image grounding annotations. Inter-Image Contrast combines independently sampled images to train source-image attribution and localization, while Intra-Image Contrast combines an original image with two object-centered crops and remaps the corresponding bounding boxes. The models are trained with GRPO using a source-aware set-wise IoU reward and an output-format validity reward, without additional CoT annotations. The paper evaluates CGC on MIG-Bench, VLM2-Bench, and broader multimodal benchmarks using Qwen and InternVL backbones. It reports improvements over the corresponding base models and provides ablations on data construction, reward components, training paradigm, and data scale.

Critical strengths
  1. Clear task formulation and accessible method presentation.
    • The paper clearly formulates the grounding task in terms of source-image indices, semantic labels, and bounding boxes. The descriptions of the two data-construction branches and the source-aware spatial reward make the main training pipeline straightforward to follow.
  2. Broad coverage of ablation experiments.
    • The paper examines several important design choices, including the training paradigm, the data branches, their mixing ratio, crop scale, reward components, and source-index supervision. These experiments provide useful information about how the reported performance varies with the training configuration and help readers assess the proposed recipe.
Critical weaknesses
  1. The incremental contribution and practical advantage over existing approaches are not sufficiently established.

    • Multi-image GRPO with augmented views has already been explored by MiCo, while Migician constructs multi-image grounding tasks from existing annotations, including grouped images and cropped regions. GeM-VG [1] (which also shows strong performance) is also closely related in combining multi-image grounding with GRPO. The contribution of CGC therefore needs to be established through its particular combination of data construction and source-aware supervision. Although the paper includes relevant baseline comparisons and a broad ablation suite, it does not sufficiently isolate the benefit of this combination from simpler image grouping, crop augmentation, and additional grounding training under matched conditions.
    • The SFT comparison is useful, but the supervised target used for the SFT baseline is not clearly described, particularly whether reasoning is included, how it is obtained, and which tokens contribute to the loss. Moreover, using the same 36k inputs does not establish comparable training cost, particularly when GRPO generates eight responses per input. Clarifying the SFT supervision and reporting end-to-end training costs, including rollout generation, would help establish the practical advantage of the proposed recipe beyond avoiding additional annotation and CoT generation.
  2. The explanations involving attention leakage, object constancy, and positional bias exceed the diagnostic evidence provided.

    • Figure 1(b) demonstrates incorrect image-comparison responses, but does not establish that these errors arise from cross-image attention leakage or that CGC specifically mitigates this mechanism. Likewise, Intra-Image Contrast creates crops of the same source image, and its reward can be satisfied by independently localizing the target in each crop; the experiments do not distinguish this improvement from learning cross-image identity consistency.
    • The positional-debiasing claim is also not directly tested: removing source-index supervision is different from removing image-order shuffling, and no permutation-robustness analysis is reported.
    • The stated reward evaluates source indices, boxes, and structural validity, without directly assessing semantic-label correctness or reasoning faithfulness. The broad ablations and downstream gains support the usefulness of the training recipe, but do not sufficiently connect its components to the specific failure modes used to motivate it. Targeted distractor interventions, order-robustness tests, and comparisons against independent crop grounding would provide more direct evidence for these central claims.

[1] GeM-VG: Towards Generalized Multi-image Visual Grounding with Multimodal Large Language Models. AAAI 2026.

Questions for authors
  1. What benefit does composing multiple images into the same grounding task provide over training on the same source images and crops as independent grounding examples? Which results support attributing the gains specifically to reduced cross-image interference or improved identity consistency, rather than general improvements in grounding and output validity?
  2. Does CGC’s low-cost claim extend beyond annotation preparation to the full post-training procedure? Given related multi-image preference-learning approaches such as MIA-DPO [2] and CcDPO [3], what existing evidence supports the performance–cost advantage of online GRPO over a DPO alternative using the same synthesized data and reward-derived preferences? Any comparison should account for total training cost, including response generation and preference-pair construction.

[2] MIA-DPO: Multi-Image Augmented Direct Preference Optimization For Large Vision-Language Models. ICLR 2025.

[3] Zooming from Context to Cue: Hierarchical Preference Optimization for Multi-Image MLLMs. NeurIPS 2025.

Reviewer_Y6rzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes CGC, which turns single-image grounding annotations (RefCOCO, SODA, LISA, OmniLabel, VAW) into about 36K multi-image grounding samples. Inter-Image Contrast combines unrelated images as distractors. Intra-Image Contrast pairs an image with two crops of the same target. The model is trained with GRPO using a source-aware set-wise IoU reward plus a format reward. On Qwen2-VL-7B, Qwen2.5-VL-7B, Qwen3-VL-8B and InternVL3-8B, CGC improves MIG-Bench and VLM2-Bench and gives small gains on general benchmarks.

Critical strengths
  • The data pipeline is cheap and fully rule-based: it reuses existing annotations, with no human labeling, LLM generation, or SFT stage.
  • The source-aware IoU reward is a sensible way to penalize attributing a box to the wrong image. The img_idx ablation (−4.96 on MIG-Bench) supports it.
  • Large gains over each base model, for example Qwen3-VL-8B going from 43.66 to 67.57 on MIG-Bench. The ablations cover the RL-vs-SFT comparison, the two data branches, mixing ratio, crop scale, and data scale. The method also transfers to InternVL3.
Critical weaknesses
  • On the same Qwen2-VL-7B backbone, CGC-7B reaches 55.50 on MIG-Bench, below Migician (63.82) and MIRG-RL (64.82). MIRG-RL is left out of Table 1 with a stronger performance. Table 6 then compares CGC-8B (Qwen3-VL) against Qwen2-VL-based MIRG-RL and Visual-RFT without stating the backbones. Fair comparison would require the same backbone model.
  • The method closely resembles MIRG-RL (multi-image grounding + GRPO with image-index and IoU rewards), and its data construction follows Migician’s conversion of single-image annotations. The main differences are dropping the SFT cold start and using a set-wise, index-gated IoU. Further discussion and experiments should be on the comparison with precious methods (i.e Why cold-start SFT is no longer required for CGC).
Questions for authors

See weaknesses

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.