- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces ReSynth, a framework for constructing multi-image supervision from generated images. ReSynth plans and generates candidate relations first, then verifies the resulting visual evidence and reuses it to construct questions across five multi-image abilities. The authors train Qwen2.5-VL-7B using GRPO and report improvements on BLINK, MUIRBench, and VLM2-Bench. Results show that RL on these generated images alone can improve multi-image understanding, even better than reported score obtained with real-world image data.
Critical strengths
The paper addresses a meaningful data bottleneck in multi-image understanding. Instead of relying only on existing image collections, ReSynth actively constructs cross-image relations such as viewpoint changes and localized edits.
Critical weaknesses
-
The "verification" mechanism is not validated sufficiently. The authors claim the pipeline relies on "verified" synthetic supervision, but the verification is performed by a MLLM rather than by human annotation or review. The paper doesn't report human agreement or verification accuracy for counts, attributes, edits or same-instance relations. Since verification is central to ReSynth, this is a major missing validation.
-
View-point generation is limited to simple and complete objects. The identity or cross-view supervision of constructed data are mainly from 'large, complete subjects in simple scenarios', which may limit generalization to real multi-image scenarios or real world cases.
-
The method shows limited scaling to stronger models. Results with more data, and the gains on Qwen3-VL-8B are much smaller. It shows that he current data may only compensate for weaker models.
Questions for authors
1.How reliable is the automatic verification process? Could the authors provide human-audited accuracy or error rates for counts, attributes and same-instance verification?
-
The scene-plan dimensions appear manually designed. The authors specify scene plan with main object, setting, activity, physcial relation, appearance, lighting, viewpoint and placement. Why these schema? Could the authors provide the reasons why these factors are sufficient or representative for multi-image understanding?
-
The generated QA is checked for correctness, but not sufficiently for difficulty or representativeness. The paper does not establish whether the generated questions are sufficient or representative of multi-image reasoning.
-
The five ability categories are not clearly justified as a comprehensive taxonomy. Identity, change, quantity, composition and correspondence are not mutually exclusive. For example, asking how many objects disappeared between two images belongs to "change" or "quantity"? Other dimensions of multi-image reasoning, such temporal ordering, motion, occlusion-based object permanence are not covered.
6 comments
W3. Scaling to Stronger Models
Slow HunyuanImage editing limited the original data scale. Adding examples also did not increase difficulty. We switch to Qwen-Image-2.1, expand to 440K examples, and add similar-instance distractors, reversed change descriptions, and spatial and temporal tasks.
Qwen3-VL-8B gains 6.10 percentage points across three main benchmarks. It exceeds the 235B model on BLINK and MUIRBench and also improves on additional benchmarks:
Model BLINK MUIRBench VLM2-Bench MMIU DiCoBench Qwen2.5-VL-7B Base 53.34 56.27 51.98 52.94 35.4 ReSynth 7B 68.60 70.54 75.62 57.33 43.4 Qwen3-VL-8B Base 65.49 73.65 78.78 57.8 40.8 ReSynth 8B 74.75 80.08 81.38 62.9 47.2 Qwen3-VL-235B 70.7 72.8 — — — We do not construct training QA specifically for DiCoBench, yet both models improve on it. The 8B model's four-benchmark single-image average also rises from 66.60 to 69.26, extending the gains beyond multi-image tasks.
Q2. Choice of Scene-Planning Factors
We choose these factors to control cross-image relations and scene diversity. Keeping an object fixed while changing its color creates an attribute-change pair. Keeping the same instance while changing the viewpoint creates a cross-view pair. Background, lighting, subject size, and scene complexity place these relations in varied visual conditions.
Planning also reduces reliance on free-form caption generation, which tends to repeat a few subjects. In our analysis, the five most frequent subject categories account for 29.4% of freely generated descriptions. We therefore sample subjects and compatible scene factors before expanding them into captions. The planning ranges are:
Dimension Range Objects 491 Scenes 76 location types and 6,185 specific scene descriptions Physical relations 27 templates Appearance 4–11 choices per object type Lighting 27 conditions Human activities 20 Viewpoint/pose 5–6 choices per type Subject size 5 levels Scene complexity 3 levels The nominal Cartesian product gives approximately 1.33×10¹³ to 4.38×10¹³ combinations. During construction, we sample compatible combinations and specify which factors remain fixed and which change. These factors support controllable construction of perception-based multi-image tasks. We assess the resulting data through task coverage and transfer to external benchmarks.
Q3. Question Difficulty and Representativeness
Question difficulty. Correct answers alone do not ensure that a question requires image comparison. We revise questions and distractors to reduce shortcuts. Change questions include reversed descriptions, requiring the model to identify what changed and in which direction. Counting questions include cross-image sums, quantity comparisons, and zero counts. Ordering questions use revised candidate permutations to reduce structural clues. Patch-completion distractors come from other images or outside the visible field of view, requiring the model to match the missing region to its context.
Task representativeness. W3 above reports transfer results on MMIU, DiCoBench, and single-image tasks. We do not construct training questions specifically for DiCoBench, yet both models improve on it. The 8B model also improves on all four single-image benchmarks. These results show that the training gains extend to other tasks beyond the question types directly constructed.
Improving generalization with RL. Beyond improving data construction, we also use RL to improve generalization to external tasks. On the same corpus, RL outperforms SFT on all three multi-image benchmarks:
Model Training BLINK MUIRBench VLM2-Bench Qwen2.5-VL-7B SFT 67.1 58.0 0.0 Qwen2.5-VL-7B RL 68.6 70.5 75.6 Qwen3-VL-8B SFT 62.1 58.4 0.0 Qwen3-VL-8B RL 74.8 80.1 81.4 SFT scores 0.0 on VLM2-Bench because its outputs do not follow the required evaluation formats.
Q4. Capability Groups and Uncovered Dimensions
Our groups draw on tasks in BLINK, MUIRBench, and related benchmarks. They are organized by image relations and construction methods, rather than mutually exclusive abilities.
For example, asking how many objects disappeared involves both change and counting. If editing changes only object count within the same scene, we assign the question to change. If different scenes are combined for counting or comparison, we assign it to quantity. The grouping reflects construction, while a task can involve multiple abilities.
We add viewpoint understanding and temporal understanding. Panorama rendering and geometric projection support surrounding-view and camera-turn questions. Video generation and program state sequences support action and state ordering.
We primarily construct perception-based multi-image tasks. Although some dimensions remain uncovered, training on these tasks still generalizes across tasks. Gains transfer to DiCoBench and extend to abstract pattern induction in BLINK IQ Test:
Model DiCoBench gain BLINK IQ Test gain Qwen2.5-VL-7B 8.00 6.00 Qwen3-VL-8B 6.40 11.33 The 7B and 8B models gain 6.00 and 11.33 points on BLINK IQ Test, respectively. Both models also improve on DiCoBench, for which we likewise construct no dedicated training questions.
Thank you for your suggestions on verification reliability and scene coverage. We add a human audit and supporting experiments, and respond point by point below.
Before addressing individual comments, we summarize the main updates. We switch to Qwen-Image-2.1 for generation and editing to address slow editing and inconsistent instruction following. We also use Qwen3-VL-8B for single-image annotation and local checks. This tests whether a model can help construct data that improves its own multi-image understanding. We expand the data from approximately 13K to 440K examples, with harder distractors and new viewpoint and temporal tasks. After training, 8B outperforms Qwen3-VL-235B on both BLINK and MUIRBench. We address your comments in light of these updates below.
W1 / Q1. Human Audit and Verification Reliability
W2. Simple Scenes and Complete Subjects