- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies the substantial performance drop in multimodal models when question text and corresponding images are combined into a single canvas. The paper introduces the PiP benchmark and uses a series of controlled experiments to analyze factors that may contribute to the performance drop. The paper proposes IGOA, which uses active Fisher statistics and accumulated Fisher memory to adjust the training weights of the two input formats online. Experiments evaluate the performance gap between the two input formats across multiple models and validate the effectiveness of the method on four backbone groups.
Critical strengths
The experimental design is thorough, with extensive comparisons and ablations, including fixed-mixture baselines, a static control matched to the average learned weights, component ablations, and multiple training seeds. These experiments provide substantial support for the paper’s claims and the proposed method.
Critical weaknesses
- Although the paper uses controlled experiments to analyze some possible factors contributing to the performance drop when question text and corresponding images are combined into a single canvas, the underlying mechanism remains insufficiently explained. It is still unclear which factors play the main role.
- The paper is motivated by documents, diagrams, and densely interleaved visual content, but lacks targeted evaluations on realistic dense-layout scenarios. Although the method improves performance on PiP and five standard benchmarks, these results do not establish its effectiveness on dense layouts in real-world scenarios.
Questions for authors
- Beyond the existing controls, is there further evidence identifying the main factors behind the performance drop under unified-canvas inputs?
- Could you provide results on benchmarks representative of naturally laid-out documents, diagrams, or pages with densely interleaved text and images to verify the effectiveness of the proposed method in the real-world scenarios mentioned in the introduction?
- Could you add a control experiment trained only on the original text-and-image data, keeping the training algorithm and budget the same, and compare the evaluation results to see how much improvement comes from adding unified-canvas training inputs?
7 comments
Response to Reviewer EHo9 (Part 2/2)
Q3 · Could you add an Original-only control with the same training algorithm and budget, and compare it with the mixed training setting to quantify the benefit of adding unified-canvas inputs?
A3. A3. We added Original-only DAPO for Qwen3-VL-8B-Instruct and Qwen3-VL-30B-A3B-Instruct, comparing it with fixed 1:1 DAPO mix and IGOA token on each backbone. All three methods follow the same RLVR training protocol, with 1,200 steps and .
Table 3. Two-backbone, three-arm training comparison of Original-only, DAPO mix, and IGOA.
| Backbone | Training method | Original ACC | PiP-All ACC | General AVG |
|---|---|---|---|---|
| 8B | Original-only DAPO | 58.73% | 45.07% | 63.05% |
| 8B | DAPO mix | 58.40% | 46.87% | 64.19% |
| 8B | IGOA token | 59.13% | 54.79% | 68.34% |
| 30B-A3B | Original-only DAPO | 59.33% | 47.13% | 64.40% |
| 30B-A3B | DAPO mix | 59.00% | 47.64% | 65.42% |
| 30B-A3B | IGOA token | 59.73% | 53.54% | 70.15% |
Canvas exposure alone yields modest PiP improvements of 1.80 and 0.51 points on 8B and 30B-A3B, respectively. In contrast, IGOA further improves PiP ACC over DAPO mix by 7.92 and 5.90 points, alongside General AVG gains of 4.15 and 4.73 points. These consistent gains suggest that online coordination contributes substantially beyond canvas exposure alone. Further details are provided in Sections 2.1.1 and 2.1.3 and Appendices A.3–A.4.
These results demonstrate IGOA's effectiveness on natural infographic layouts and its additional benefits beyond Unified-Canvas exposure alone. We hope they support a favorable reassessment.
Follow-up on Our Rebuttal
Dear Reviewer EHo9,
Thank you again for your valuable comments on our submission.
As the discussion period draws to a close, we would greatly appreciate your feedback on whether our rebuttal has addressed your concerns. We hope our responses will also be taken into account in your final evaluation.
Thank you very much for your time and consideration.
Best regards,
The Authors
Response to Reviewer EHo9 (Part 1/2)
Thank you for recognizing our existing controls and robustness evidence. We provide additional experiments to clarify the underlying mechanism, assess transfer to natural layouts, and isolate the contribution of Unified-Canvas exposure. We hope these results support a reassessment of the score.
W1 / Q1 · The paper controls several possible factors, but the underlying mechanism remains insufficiently explained. It is still unclear which factors play the main role. Is there further evidence identifying the main contributors?
A1. The controlled results identify region-reference mapping as the most consistent tested contributor. In paper Table 5, query-text changes affect accuracy by only 0.12–0.60 points, while renderer-side scaling changes it by at most 1.53 points. The localization probe in Table 4 also shows that coarse sub-image boundaries are usually recovered. Since these effects cannot explain the full PiP Gap, we conducted a matched intervention study across three scales on the same 500 Qwen3-VL-8B-Instruct questions, holding other conditions fixed. The table reports changes relative to standard PiP-All at each scale.
The standard PiP-All accuracy at the native scale is 59.20%, and the three scale settings give 59.80%, 59.20%, and 60.40%, respectively. Region-reference mapping produces a more stable improvement than query-text assistance at all three scales, while the individual changes in scale, query text, and region reference are all much smaller than the 5.40-point gap under total visual-budget control. After per-source detail alignment, Original and PiP-Visual still differ by 3.60 points.
Overall, region-reference mapping provides the most consistent improvement, but its gain remains far below the full PiP Gap. This suggests that the remaining difficulty arises from broader challenges in unified-canvas reasoning rather than any single tested factor. Relevant definitions and controls are detailed in Section 2.1.1.
W2 / Q2 · The paper is motivated by documents, diagrams, and densely interleaved visual content, but lacks targeted evaluations on realistic dense layouts. The PiP and standard-task results do not establish effectiveness in these real-world scenarios. Can you provide results on representative natural-layout benchmarks?
A2. Thank you for the suggestion. We ran the same Qwen3-VL-8B-Instruct backbone on the complete official InfographicVQA validation set, which contains 2,801 questions and 500 infographics. We used identical evaluation settings for all three checkpoints and reported results using the official ANLS metric.
IGOA improves over the base checkpoint by 0.03 and over DAPO mix by 0.02. This result shows that IGOA retains an advantage on natural infographic layouts.