- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
MVGraph is a benchmark evaluating LVLMs on integrating and reasoning over evidence distributed across multiple visual graph views, addressing the gap that prior evaluations focus on single graphs; it offers 72,000 verified instances across partial and temporal scenarios and 37 tasks, decomposing reasoning into Perception, Alignment, Composition, and Reasoning, and shows across 15 LVLMs that cross-view edge alignment and relation-level composition are key bottlenecks, with temporal reasoning remaining especially hard.
Critical strengths
- Extends evaluation from reasoning over a single complete graph to integrating evidence across multiple views, addressing a gap in existing benchmarks.
- Uses targeted diagnostic tasks and gold intermediate structures to investigate model failures beyond final-answer accuracy.
- Covers both partial and temporal views, with programmatic verification of answers and cross-view evidence dependencies.
Critical weaknesses
- Alignment and composition tasks still depend on visual perception, making it difficult to distinguish perceptual errors from alignment or composition failures.
- Missing matched comparisons between multi-view images, merged single-view images, and separate textual graph views.
- The view-ablation study is conducted in the no-thinking setting, where most models score very low, leaving a narrow range for distinguishing capability differences.
Questions for authors
Listed in the "Weaknesses" section.
4 comments
Response to Reviewer GZEs (2/2)
W3: View-Ablation Evaluation in Thinking Mode.
R3: We thank the reviewer for this valuable suggestion. We further conduct the view-ablation study in thinking mode, comparing three input conditions following our paper: Full (all views), Single View (one randomly selected view), and Leave-One-Out (all views except the selected one).
The results in Table 4 demonstrate that access to the complete set of graph views consistently yields the highest accuracy across all evaluated models, while removing even a single view leads to substantial performance degradation. Moreover, model performance becomes much more similar under reduced-view conditions, despite clear differences under Full. This suggests that models' ability to exploit distributed evidence becomes most apparent when all relevant views are available. These findings further strengthen the evidence for multi-view dependence of our MVGraph benchmark under the stronger thinking setting.
Table 4: Accuracy under different view-availability conditions in thinking mode.
| Models | Full | Single View | Leave-One-Out |
|---|---|---|---|
| Qwen3.5-9B | 83.57 | 18.68 | 36.64 |
| Gemma-4-12B-it | 83.11 | 21.32 | 38.07 |
| GLM-4.6V-Flash | 69.89 | 17.64 | 34.36 |
| Ovis2.5-9B | 66.89 | 17.64 | 33.46 |
Action taken: We have added text-input alignment and composition experiments, summarized in Section 5.2 and reported in full in Appendix D (Tables 18 and 19). We have also added matched input-representation comparisons in Section 5.3 (Table 7) and thinking-mode view-ablation results in Section 5.5 (Figure 6).
Request for Feedback
Dear Reviewer GZEs,
I hope this message finds you well. Thank you very much for your valuable comments! We have carefully addressed each of your concerns in our rebuttal and hope that our additional experiments and clarifications have addressed your questions.
In our previous responses, we specifically addressed your concerns regarding the distinction between visual perception and alignment/composition failures (W1), supported by additional text-input diagnostic experiments; the missing comparisons across different graph representations (W2), including multi-view images, single-view images, per-view text, and composed text; and the view-ablation evaluation in thinking mode (W3), with new results comparing Full, Single View, and Leave-One-Out settings.
We have also incorporated these additional experiments and clarifications into the revised manuscript, with all changes highlighted in blue.
We sincerely hope that our responses have adequately addressed your concerns and would be very grateful to hear your feedback. If any questions or concerns remain, we would be happy to discuss them further and provide additional clarifications.
Thank you again for your time and constructive suggestions!
Best regards,
All Authors
Dear Reviewer GZEs,
Thank you again for your valuable comments and constructive suggestions. We have carefully addressed your concerns regarding the distinction between perception and alignment/composition failures, controlled input-representation comparisons, and thinking-mode view ablations, with additional experiments and analyses.
As less than 24 hours remain in the discussion period, we would be sincerely grateful for your feedback on whether our responses have addressed your concerns. We would be happy to provide further clarification if needed.
Best regards,
All Authors
Response to Reviewer GZEs (1/2)
Thank you for your insightful comments on the diagnostic design and evaluation protocol. We respond to each point below and describe the corresponding additions to the manuscript. We have added text-input alignment and composition diagnostics in Section 5.2 and Appendix D, introduced matched comparisons across multi-view images, complete single-view images, per-view text, and composed text in Section 5.3, and extended the view-ablation evaluation to thinking mode in Section 5.5. All revisions to the manuscript are highlighted in blue for ease of reference. We hope these revisions and responses address your concerns, and we welcome any further questions or suggestions.
R1: We thank the reviewer for this insightful comment. To further distinguish perceptual errors from alignment and composition failures, we conduct additional experiments using ground-truth per-view textual graph triples instead of images, while keeping the tasks and underlying questions unchanged.
The results in Tables 1 and 2 show that textual graph inputs improve performance on several tasks by reducing visual perception errors. However, many tasks remain challenging even with accurate graph structures, particularly edge-level alignment (e.g., A-VSE and A-RE) and evidence composition (e.g., C-UE and C-TPC). These findings highlight that cross-view alignment and composition remain significant challenges beyond visual perception alone.
We have incorporated these additional results into the revised manuscript to provide a clearer distinction between perceptual and cross-view processing difficulties.
Table 1: Accuracy on alignment tasks with textual graph inputs.
Table 2: Accuracy on composition tasks with textual graph inputs.
R2: We thank the reviewer for this valuable suggestion. We conduct additional experiments on four partial-view graph-theoretic tasks (TS, MF, BM, and HP), comparing four input conditions: Multi-view Images (original graph images), Single-view Images (all partial views combined into one complete graph image), Per-view Text (separate ground-truth graph triples), and Composed Text (graph triples after cross-view alignment and composition).
The results in Table 3 show that all four models achieve higher accuracy with Single-view Images than with Multi-view Images, highlighting the additional difficulty of processing fragmented visual evidence. Moreover, most models benefit substantially from textual graph representations, indicating that visual graph perception and cross-view evidence processing remain important challenges. The slight differences between Per-view Text and Composed Text further suggest that cross-view alignment and composition introduce additional challenges even when accurate textual structures are provided.
Table 3: Average accuracy on four partial-view graph-theoretic tasks under different input representations.
We have incorporated these results into the revised manuscript to strengthen the analysis of multi-view visual graph reasoning.