acceptodds
Under review as a conference paper at ICLR 2027

MVGraph: Benchmarking LVLMs on Evidence Composition across Multi-View Visual Graph

Abstract

We introduce MVGraph, a benchmark for evaluating large vision-language models (LVLMs) on multi-graph reasoning where the evidence required by a query is distributed across multiple interrelated visual graph views. Although such evidence commonly resides in source-specific graphs, temporal snapshots, or local observations, existing graph reasoning benchmarks remain confined to a single self-contained structure, assessing only graph perception and reasoning while leaving the alignment and composition of evidence across views underexplored. MVGraph comprises 8k instances spanning 8 graph-theoretic tasks, abstracted from two multi-graph scenarios (partial and temporal graph views). Each problem is programmatically constructed and verified to be unsolvable from a single view. Beyond end-to-end evaluation, we decompose multi-graph reasoning into four complementary capabilities—*Perception*, *Alignment*, *Composition*, and *Reasoning*—and design 29 diagnostic tasks with 64k instances to probe each one, along with golden intermediate representations for evaluating the intermediate process. Across 15 LVLMs, we reveal the difficulty of multi-graph reasoning and identify cross-view edge alignment and relation-level composition as prominent bottlenecks through stage-oriented diagnostics. Meanwhile, temporal constraints and increasing view counts expose further capability boundaries. Code and data are available at https://anonymous.4open.science/r/MVGraph-8C37.

Primary area:
datasets and benchmarks
Keywords:
multi-view visual graph, large vision-language models, benchmarking

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_GZEsOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

MVGraph is a benchmark evaluating LVLMs on integrating and reasoning over evidence distributed across multiple visual graph views, addressing the gap that prior evaluations focus on single graphs; it offers 72,000 verified instances across partial and temporal scenarios and 37 tasks, decomposing reasoning into Perception, Alignment, Composition, and Reasoning, and shows across 15 LVLMs that cross-view edge alignment and relation-level composition are key bottlenecks, with temporal reasoning remaining especially hard.

Critical strengths
  1. Extends evaluation from reasoning over a single complete graph to integrating evidence across multiple views, addressing a gap in existing benchmarks.
  2. Uses targeted diagnostic tasks and gold intermediate structures to investigate model failures beyond final-answer accuracy.
  3. Covers both partial and temporal views, with programmatic verification of answers and cross-view evidence dependencies.
Critical weaknesses
  1. Alignment and composition tasks still depend on visual perception, making it difficult to distinguish perceptual errors from alignment or composition failures.
  2. Missing matched comparisons between multi-view images, merged single-view images, and separate textual graph views.
  3. The view-ablation study is conducted in the no-thinking setting, where most models score very low, leaving a narrow range for distinguishing capability differences.
Questions for authors

Listed in the "Weaknesses" section.

4 comments
AuthorsOpenReview ↗

Response to Reviewer GZEs (1/2)

Thank you for your insightful comments on the diagnostic design and evaluation protocol. We respond to each point below and describe the corresponding additions to the manuscript. We have added text-input alignment and composition diagnostics in Section 5.2 and Appendix D, introduced matched comparisons across multi-view images, complete single-view images, per-view text, and composed text in Section 5.3, and extended the view-ablation evaluation to thinking mode in Section 5.5. All revisions to the manuscript are highlighted in blue for ease of reference. We hope these revisions and responses address your concerns, and we welcome any further questions or suggestions.

W1: Distinguishing Visual Perception from Alignment and Composition

R1: We thank the reviewer for this insightful comment. To further distinguish perceptual errors from alignment and composition failures, we conduct additional experiments using ground-truth per-view textual graph triples instead of images, while keeping the tasks and underlying questions unchanged.

The results in Tables 1 and 2 show that textual graph inputs improve performance on several tasks by reducing visual perception errors. However, many tasks remain challenging even with accurate graph structures, particularly edge-level alignment (e.g., A-VSE and A-RE) and evidence composition (e.g., C-UE and C-TPC). These findings highlight that cross-view alignment and composition remain significant challenges beyond visual perception alone.

We have incorporated these additional results into the revised manuscript to provide a clearer distinction between perceptual and cross-view processing difficulties.

Table 1: Accuracy on alignment tasks with textual graph inputs.

ModelsA-SNA-VSNA-SEA-VSEA-ANA-RNA-AEA-REA-PNA-PE
Qwen3.5-9B80.6798.6778.6788.67100.0095.3398.6759.3396.67100.00
Gemma-4-12B-it88.6798.6777.3380.00100.00100.0091.3386.00100.0097.33
GLM-4.6V-Flash87.3346.0078.6728.0094.0060.6716.006.0086.6783.33
Ovis2.5-9B79.3364.6774.0011.3393.3377.3368.0016.6787.3396.67

Table 2: Accuracy on composition tasks with textual graph inputs.

ModelsC-UNC-UEC-PAC-CDC-MVSC-EVC-PPC-TPC
Qwen3.5-9B96.0069.33100.0096.67100.0099.33100.0090.67
Gemma-4-12B-it97.3364.0097.3394.6799.33100.00100.0077.33
GLM-4.6V-Flash51.3321.3350.6716.6784.0078.0090.000.00
Ovis2.5-9B16.6734.0071.3381.3381.3376.6799.330.00

W2: Missing matched comparisons between multi-view images, merged single-view images, and separate textual graph views.

R2: We thank the reviewer for this valuable suggestion. We conduct additional experiments on four partial-view graph-theoretic tasks (TS, MF, BM, and HP), comparing four input conditions: Multi-view Images (original graph images), Single-view Images (all partial views combined into one complete graph image), Per-view Text (separate ground-truth graph triples), and Composed Text (graph triples after cross-view alignment and composition).

The results in Table 3 show that all four models achieve higher accuracy with Single-view Images than with Multi-view Images, highlighting the additional difficulty of processing fragmented visual evidence. Moreover, most models benefit substantially from textual graph representations, indicating that visual graph perception and cross-view evidence processing remain important challenges. The slight differences between Per-view Text and Composed Text further suggest that cross-view alignment and composition introduce additional challenges even when accurate textual structures are provided.

Table 3: Average accuracy on four partial-view graph-theoretic tasks under different input representations.

ModelsMulti-view ImagesSingle-view ImagesPer-view TextComposed Text
Qwen3.5-9B18.7536.6074.0075.00
Gemma-4-12B-it28.5037.7071.5073.00
GLM-4.6V-Flash3.0010.901.253.50
Ovis2.5-9B5.2511.8016.2511.50

We have incorporated these results into the revised manuscript to strengthen the analysis of multi-view visual graph reasoning.

AuthorsOpenReview ↗

Response to Reviewer GZEs (2/2)

W3: View-Ablation Evaluation in Thinking Mode.

R3: We thank the reviewer for this valuable suggestion. We further conduct the view-ablation study in thinking mode, comparing three input conditions following our paper: Full (all views), Single View (one randomly selected view), and Leave-One-Out (all views except the selected one).

The results in Table 4 demonstrate that access to the complete set of graph views consistently yields the highest accuracy across all evaluated models, while removing even a single view leads to substantial performance degradation. Moreover, model performance becomes much more similar under reduced-view conditions, despite clear differences under Full. This suggests that models' ability to exploit distributed evidence becomes most apparent when all relevant views are available. These findings further strengthen the evidence for multi-view dependence of our MVGraph benchmark under the stronger thinking setting.

Table 4: Accuracy under different view-availability conditions in thinking mode.

ModelsFullSingle ViewLeave-One-Out
Qwen3.5-9B83.5718.6836.64
Gemma-4-12B-it83.1121.3238.07
GLM-4.6V-Flash69.8917.6434.36
Ovis2.5-9B66.8917.6433.46

Action taken: We have added text-input alignment and composition experiments, summarized in Section 5.2 and reported in full in Appendix D (Tables 18 and 19). We have also added matched input-representation comparisons in Section 5.3 (Table 7) and thinking-mode view-ablation results in Section 5.5 (Figure 6).

AuthorsOpenReview ↗

Request for Feedback

Dear Reviewer GZEs,

I hope this message finds you well. Thank you very much for your valuable comments! We have carefully addressed each of your concerns in our rebuttal and hope that our additional experiments and clarifications have addressed your questions.

In our previous responses, we specifically addressed your concerns regarding the distinction between visual perception and alignment/composition failures (W1), supported by additional text-input diagnostic experiments; the missing comparisons across different graph representations (W2), including multi-view images, single-view images, per-view text, and composed text; and the view-ablation evaluation in thinking mode (W3), with new results comparing Full, Single View, and Leave-One-Out settings.

We have also incorporated these additional experiments and clarifications into the revised manuscript, with all changes highlighted in blue.

We sincerely hope that our responses have adequately addressed your concerns and would be very grateful to hear your feedback. If any questions or concerns remain, we would be happy to discuss them further and provide additional clarifications.

Thank you again for your time and constructive suggestions!

Best regards,
All Authors

Dear Reviewer GZEs,

Thank you again for your valuable comments and constructive suggestions. We have carefully addressed your concerns regarding the distinction between perception and alignment/composition failures, controlled input-representation comparisons, and thinking-mode view ablations, with additional experiments and analyses.

As less than 24 hours remain in the discussion period, we would be sincerely grateful for your feedback on whether our responses have addressed your concerns. We would be happy to provide further clarification if needed.

Best regards,

All Authors

Reviewer_bpcXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces MVGRAPH, a benchmark for evaluating large vision-language models on graph-theoretic reasoning when the required evidence is distributed across multiple visual graph views. MVGRAPH contains 8,000 end-to-end instances across eight graph-theoretic tasks and 64,000 diagnostic records targeting perception, alignment, composition, and reasoning. It covers partial graph views and temporally ordered graph snapshots. The authors evaluate 15 LVLMs and report that current models struggle particularly with edge-level alignment, relation-level composition, and temporally coupled reasoning.

Critical strengths
  1. Relevant and underexplored problem. The paper targets graph reasoning over complementary visual views, which is not adequately covered by existing single-graph or generic multi-image benchmarks.
  2. Broad task coverage. MVGRAPH includes four graph-theoretic families, partial and temporal settings, and 29 diagnostic tasks. The distinction between evidence extraction, cross-view processing, and downstream graph reasoning is conceptually useful.
  3. Transparent appendix. The paper documents task definitions, prompts, graph construction, answer formats, and per-task statistics in considerable detail.
Critical weaknesses
  1. Unclear provenance and limited realism of the graph data. It is unclear how much of MVGRAPH reflects graph structures from real-world domains such as biology, chemistry, social networks, or transportation. According to Appendix A.2, the end-to-end graph-theoretic tasks are generated using randomized, task-specific procedures. Some diagnostic instances use topologies sampled from TKGL-SmallPedia, but their original semantics are removed, and the resulting graphs are anonymized and rendered in the same synthetic style. The paper should report the exact proportion of synthetic versus real-derived instances for every task group and discuss how well these structures represent real-world graph distributions. Without such analysis, the benchmark may primarily measure performance on procedurally generated graph puzzles rather than general multi-view graph understanding.
  2. The source of the near-zero end-to-end performance is not cleanly isolated. Table 2 reports 0 or near 0 accuracy for many models, but an end-to-end error may arise from at least three sources: inaccurate visual graph extraction, failure to align and compose evidence across views, or insufficient graph-theoretic reasoning. The authors provide structure-prepared reasoning results in Table 5, which is useful, but this condition supplies textualized and already composed graph structures for partial-view tasks. It therefore changes both the input modality and the evidence-composition requirement. A more informative control would serialize each original view separately as text while preserving the same multi-view fragmentation, question, and underlying instance.
  3. The dataset scale is not contextualized against prior visual-graph benchmarks. The paper emphasizes that MVGRAPH contains 72,000 records, including 8,000 end-to-end instances and 64,000 diagnostic records, but does not quantitatively compare this scale with previous single-graph visual benchmarks such as VisionGraph, VGCURE, VisGraphVar, Visual Graph Arena, or GraphVerse.
  4. Evaluation on frontier proprietary models is encouraged. Although the paper includes API-accessed models from Qwen, GLM, and DeepSeek, it does not evaluate widely used frontier multimodal systems such as GPT, Claude, or Gemini. Since the strongest evaluated models already achieve approximately 50% average end-to-end accuracy and over 90% on several diagnostic tasks, testing stronger proprietary LVLMs is important for determining whether MVGRAPH remains challenging and discriminative at the current capability frontier.
Questions for authors

My questions and concerns have already been presented in the Critical Weaknesses section.

5 comments
AuthorsOpenReview ↗

Response to Reviewer bpcX (1/2)

We appreciate your careful review and constructive suggestions. Our point-by-point responses below explain the corresponding clarifications and additional experiments. We have clarified data provenance in Section 3.3 and Appendix A.7, added structural distribution comparisons in Appendix A.7, and expanded the benchmark positioning comparison in Section 2. We have also strengthened the evaluation through controlled input-representation comparisons in Section 5.3 and GPT-6-Luna and Gemini-3.8-Flash results in Section 5.1. All revisions to the manuscript are highlighted in blue for ease of reference. We hope these revisions and responses address your concerns, and we welcome any further questions or suggestions.

W1: Unclear provenance and limited realism of the graph data.

R1: We thank the reviewer for raising this important concern. We clarify the provenance of MVGraph and provide additional structural distribution analysis.

(1) Data provenance. MVGraph contains 6,498 SmallPedia-derived records (9.03%) originating from real-world entity relations in Wikidata. The detailed statistics are shown in Table 1. SmallPedia is derived from Wikidata facts associated with Wikipedia entities, capturing real-world relational structures across diverse domains. We note that procedural generation help to focus the evaluation on recovering and processing distributed structural evidence while reducing reliance on domain knowledge.

Table 1: Distribution of synthetic and SmallPedia-derived records across task groups.

Task GroupTotalSyntheticSmallPedia-derivedRatio (%)
Graph-Theoretic8,0008,00000.00
Perception36,00034,1111,8895.25
Alignment10,0007,5002,50025.00
Composition8,0006,0002,00025.00
Reasoning10,0009,8911091.09
Total72,00065,5026,4989.03

(2) Structural distribution analysis. We further compare synthetic and SmallPedia-derived instances in terms of shortest-path length, graph size, isolated-node fraction, and connected components per view, as shown in Figure 7 of the revised manuscript. The results reveal strong structural similarities between the two sources. Specifically, 96.34% and 97.26% of shortest paths fall within 1–4 hops for synthetic and SmallPedia-derived graphs, respectively, with an 81.91% distribution overlap. Regarding graph size, 99.04% of synthetic instances contain 7–16 nodes per view, a range also covered by SmallPedia-derived instances. Furthermore, we additionally analyze the node degree distributions, finding that 97.46% and 97.52% of nodes have degrees between 0 and 4 in synthetic and SmallPedia-derived graphs, respectively.

These comparisons provide evidence that the synthetic graphs share several local structural properties with the sampled, transformed SmallPedia subgraphs. We have incorporated these statistics and distribution comparisons into the revised manuscript.

W2: The source of the near-zero end-to-end performance is not cleanly isolated.

R2: We thank the reviewer for this valuable suggestion. To better isolate the sources of end-to-end errors, we conduct an additional controlled experiment on the same instances across four partial-view graph-theoretic tasks (TS, MF, BM, and HP), comparing three input conditions: Multi-view Images (original graph images), Per-view Text (ground-truth triples provided separately for each view), and Composed Text (ground-truth evidence triples after cross-view alignment and composition).

Table 2: Average accuracy on four partial-view graph-theoretic tasks.

ModelsMulti-view ImagesPer-view TextComposed Text
Qwen3.5-9B18.7574.0075.00
Gemma-4-12B-it28.5071.5073.00
GLM-4.6V-Flash3.001.253.50
Ovis2.5-9B5.2516.2511.50

The results in Table 2 show that most models perform substantially better with Per-view Text and Composed Text than with Multi-view Images, highlighting the difficulty of processing distributed visual graph evidence. Moreover, Per-view Text generally performs slightly worse than Composed Text, suggesting that explicitly composing the evidence can further facilitate downstream reasoning even when accurate textual structures are provided. Ovis2.5-9B is an exception, performing better with Per-view Text, with the difference mainly driven by BM. These findings help distinguish visual extraction difficulties from cross-view evidence processing and downstream graph-theoretic reasoning.

AuthorsOpenReview ↗

Response to Reviewer bpcX (2/2)

W3: The dataset scale is not contextualized against prior visual-graph benchmarks.

R3: We thank the reviewer for this valuable suggestion. To better contextualize MVGraph, we extend the comparison to seven representative visual graph benchmarks, considering four dimensions: evaluation focus, graph setting, temporal reasoning support, and benchmark scale.

Table 3: Positioning of MVGraph among existing visual graph benchmarks.

BenchmarkEvaluation FocusGraph SettingTemporal# Tasks# Scale
VisionGraphPerception and graph theorySingle graph✗103,000
GVLQAGraph-theoretic reasoningSingle graph✗7157,896
VGCureFundamental perception and reasoningSingle graph✗22223,646
VisGraphVarGraph analysis and visual robustnessSingle / paired graphs✗7990
Visual Graph ArenaIsomorphism, paths, and cyclesSingle / paired graphs✗689,440
DiGraphHal-BenchStructural, visual, semantic, and integrative QASingle graph✗1269,114
GraphVersePerception, reasoning, and paired comparisonSingle / paired graphs✗1111,000
MVGraphMulti-view visual graph reasoning and diagnosticsMulti-view graphs✓3772,000

As shown in Table 3, existing benchmarks primarily focus on single-graph understanding or paired-graph comparison, whereas MVGraph explicitly targets multi-view visual graph reasoning, including both partial-view and temporal-view scenarios. In particular, MVGraph supports temporal graph reasoning, which is not covered by the compared benchmarks. Furthermore, MVGraph provides 37 tasks across four diagnostic capabilities and end-to-end graph-theoretic reasoning, comprising 8,000 end-to-end instances and 64,000 diagnostic records. This offers more comprehensive capability-oriented evaluation than existing benchmarks. In terms of scale, MVGraph contains 72K test instances, exceeding several representative benchmarks.

Moreover, MVGraph is constructed through programmatic graph generation, enabling scalable instance generation with controllable graph structures and task constraints. We will publicly release the data generation scripts to facilitate reproducibility and further expansion of the benchmark.

W4: Evaluation on frontier proprietary models is encouraged.

R4: We thank the reviewer for this valuable suggestion. To address this concern, we further evaluate two representative frontier proprietary LVLMs, GPT-6-Luna and Gemini-3.8-Flash, using 100 randomly sampled instances from each of the eight end-to-end tasks.

The results in Table 4 show that these frontier models substantially outperform the previously evaluated models, achieving average accuracies of 67.4% and 74.0%, respectively. Nevertheless, their performance remains highly uneven across tasks. For instance, although Gemini-3.8-Flash achieves 99% on BM and 97% on HP, its accuracy on STS is only 27%. Similarly, GPT-6-Luna achieves 46% on STS but only 24% on SBM. These results indicate that even highly capable frontier LVLMs still struggle with certain structural and temporal graph reasoning tasks.

Moreover, the two models exhibit different capability profiles. For example, Gemini-3.8-Flash performs better on MF, BM, and SBM, whereas GPT-6-Luna achieves higher accuracy on TS and STS. This further demonstrates the discriminative power of MVGraph in revealing model-specific strengths and weaknesses.

Table 4: Accuracy of frontier LVLMs on the eight end-to-end graph-theoretic tasks of MVGraph in thinking mode.

ModelsTSMFBMHPSTSMFCSBMTHPAvg.
GPT-6-Luna53.061.094.092.046.078.024.091.067.4
Gemini-3.8-Flash37.073.099.097.027.098.070.091.074.0

We have incorporated these additional results into the revised manuscript to provide a more comprehensive evaluation of frontier LVLMs.

Action taken: We have clarified data provenance in Section 3.3 and Appendix A.7 (Table 16), added structural distribution analysis in Appendix A.7 (Figure 7), and expanded the benchmark comparison in Section 2 (Table 1). We have also added controlled input-representation comparisons in Section 5.3 (Table 7) and frontier-model results in Section 5.1 (Table 3) and Appendix C.1 (Table 17).

AuthorsOpenReview ↗

Request for Feedback

Dear Reviewer bpcX,

I hope this message finds you well. Thank you very much for your valuable comments! We have carefully addressed each of your concerns in our rebuttal and hope that our additional experiments and clarifications have addressed your questions.

In our previous responses, we specifically addressed your concerns regarding the provenance and realism of graph data (W1), including SmallPedia-derived data statistics and structural distribution analysis; the sources of end-to-end reasoning failures (W2), supported by controlled comparisons across different input representations; the quantitative comparison with existing visual-graph benchmarks (W3); and the evaluation on frontier proprietary LVLMs (W4), including new results on GPT-6-Luna and Gemini-3.8-Flash.

We have also incorporated these additional experiments and clarifications into the revised manuscript, with all changes highlighted in blue.

We sincerely hope that our responses have adequately addressed your concerns and would be very grateful to hear your feedback. If any questions or concerns remain, we would be happy to discuss them further and provide additional clarifications.

Thank you again for your time and constructive suggestions!

Best regards,
All Authors

Reviewer_kd2qOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

MVGRAPH is a benchmark for evaluating LVLMs on graph-theoretic questions whose required evidence is distributed across partial or temporal visual graph views. It provides eight end-to-end tasks and 29 diagnostics for perception, alignment, composition, and reasoning, with programmatic validation and gold intermediate structures. The paper evaluates 15 LVLMs end-to-end and seven models on the diagnostic suite.

Critical strengths
  1. The benchmark construction is concrete and well validated.

  2. The diagnostic decomposition produces useful evidence beyond end-to-end accuracy.

  3. The introduction of temporal settings is novel.

Critical weaknesses
  1. My major concern is that the evaluation is not sufficiently comprehensive. Most locally evaluated models are relatively small (e.g., 9B–12B, flash model), with limited coverage of stronger frontier LVLMs. This raises concerns about whether the identified limitations remain significant for more capable models. Moreover, some tasks appear relatively easy, with Bipartite Matching (BM) reaching 95–97% accuracy on the evaluated small models, suggesting limited room for further improvement. A broader evaluation of stronger models would help establish the benchmark's difficulty, discriminative power, and relevance to current LVLM capabilities.

  2. With 72K instances across 37 tasks, the full benchmark incurs substantial evaluation costs, particularly for stronger frontier LVLMs accessed through APIs. To facilitate broader adoption and more comprehensive model comparisons, the authors could consider introducing a lightweight benchmark subset containing carefully selected, representative, and challenging instances. Such a version would reduce evaluation costs while preserving the benchmark's diagnostic value and discriminative power.

  3. The paper decomposes multi-graph reasoning into four capabilities: Perception, Alignment, Composition, and Reasoning. However, these stages involve different tasks and input conditions, with the Reasoning stage directly receiving ground-truth textual structures. Consequently, performance differences across stages may not reliably identify the primary bottlenecks in end-to-end reasoning. More systematic oracle interventions on the same instances would help strengthen the diagnostic conclusions.

  4. Although the paper demonstrates the benefits of multiple graph views, it does not sufficiently disentangle the difficulty of multi-view evidence composition from graph-theoretic reasoning itself. For example, Bipartite Matching achieves up to 97% accuracy, whereas Stable Bipartite Matching reaches only 16%. However, the latter introduces additional temporal consistency constraints, making it difficult to attribute the performance gap specifically to multi-view or temporal reasoning. A controlled comparison using the same graph and question under single complete-view and multiple partial-view settings would provide stronger evidence.

Questions for authors
  1. How representative is the current model selection of stronger frontier LVLMs, particularly given the high accuracy observed on certain tasks?

  2. How should the performance differences across diagnostic stages be interpreted given their different tasks and input conditions?

  3. To what extent can the performance gaps between partial and temporal tasks be attributed to additional task constraints rather than multi-view reasoning itself?

9 comments
AuthorsOpenReview ↗

Response to Reviewer kd2q (1/3)

We sincerely thank you for your thoughtful and constructive feedback. Please find our point-by-point responses below. We have expanded the evaluation with GPT-6-Luna and Gemini-3.8-Flash in Section 5.1, added matched comparisons across four input representations in Section 5.3, and clarified the different constraints of partial-view and temporal-view tasks in Section 5.1 and Appendix C.1. We also describe MVGraph-Lite and its sampling strategy below to address your concern about evaluation cost. All revisions to the manuscript are highlighted in blue for ease of reference. We hope these revisions and responses address your concerns, and we welcome any further questions or suggestions.

W1&Q1: The evaluation is not sufficiently comprehensive.

R1: We thank the reviewer for this valuable suggestion. Beyond the locally deployed 9B–12B models, our original evaluation also includes three large-scale API-accessed models: Qwen3.8-Flash (125B parameters), GLM-5.3-Flash (320B), and DeepSeek-Flash (552B), which represent substantially stronger LVLMs.

According to your suggestion, we expanded the evaluation to include GPT-6-Luna and Gemini-3.8-Flash, using 100 randomly sampled instances from each of the eight end-to-end tasks. As shown in Table 1, they achieve average accuracies of 67.4% and 74.0%, respectively, while leaving substantial room for improvement on several tasks.

Table 1: Accuracy of frontier LVLMs on the eight end-to-end graph-theoretic tasks of MVGraph in thinking mode.

ModelsTSMFBMHPSTSMFCSBMTHPAvg.
GPT-6-Luna53.061.094.092.046.078.024.091.067.4
Gemini-3.8-Flash37.073.099.097.027.098.070.091.074.0

Regarding Bipartite Matching (BM), the original 95–97% scores were obtained by the API-accessed Qwen3.8-Flash and GLM-5.3-Flash, rather than the smaller local models. While BM approaches saturation for these models, other tasks remain considerably more challenging. For example, Qwen3.8-Flash and GLM-5.3-Flash achieve only 8–12% on Stable Topological Sorting (STS) and 8–16% on Stable Bipartite Matching (SBM), with overall accuracies of 50.4% and 52.4%, respectively.

These findings show that MVGraph is not uniformly difficult, but retains unsaturated tasks and distinguishes task-level performance among the evaluated frontier models.

W2: Evaluation Cost and Lightweight Benchmark.

R2: We thank the reviewer for this practical suggestion. Following your suggestion, we have constructed MVGraph-Lite, a lightweight subset containing 100 representative instances for each of the 37 tasks, resulting in 3,700 instances in total. It covers all eight end-to-end graph-theoretic tasks and 29 diagnostic tasks across the four capabilities. Compared with the full benchmark of 72K records, MVGraph-Lite reduces the evaluation scale by 94.86%, substantially lowering computational and API evaluation costs.

To preserve both representativeness and difficulty, we adopt a stratified, difficulty-aware sampling strategy. Specifically, we first balance the distribution of graph view counts within each task. We then select challenging instances based on two complementary factors: (1) model performance, measured by instance-level errors across four locally evaluated LVLMs; and (2) structural complexity, characterized by graph sizes, cross-view edge overlaps or temporal changes, and answer structure complexity. This strategy aims to preserve diverse task characteristics while retaining challenging instances. MVGraph-Lite and its sampling protocol will be released alongside the complete benchmark after acceptance.

AuthorsOpenReview ↗

Response to Reviewer kd2q (2/3)

W3&Q2: Interpretation of Stage-Oriented Diagnostics

R3: We thank the reviewer for this insightful comment. We clarify our diagnostic design and provide additional evidence.

(1) Diagnostic design. Our diagnostic suite is designed to reveal specific failure patterns within multi-view graph processing, rather than rank the four capabilities by their raw accuracies. Since alignment and composition are difficult to isolate in complex end-to-end tasks, we design targeted diagnostics (e.g., Shared Edges, Union Edge Count, and Path Assembly) to explicitly evaluate these capabilities while minimizing interference from other factors.

(2) Existing comparison. Our eight reasoning diagnostics reuse the same instances as the end-to-end tasks but provide ground-truth textual structures with pre-composed evidence instead of graph images. For example, GLM-5.3-Flash improves from 33% to 95% on TS and from 41% to 87% on MF, while R-STS and R-SBM remain challenging, revealing persistent downstream reasoning difficulties.

(3) Additional experiments. We further compare three input conditions on the same instances across four partial-view tasks (TS, MF, BM, HP), including Multi-view Images, Per-view Text (separate ground-truth triples), and Composed Text (pre-composed evidence triples). The results in Table 2 show that most models perform substantially better with Per-view Text and Composed Text than with Multi-view Images, highlighting the difficulty of processing distributed visual graph evidence. Moreover, Per-view Text generally performs slightly worse than Composed Text, suggesting that explicitly composing the evidence can further facilitate downstream reasoning even when accurate textual structures are provided. Ovis2.5-9B is an exception, performing better with Per-view Text, with the difference mainly driven by BM. Together, these results highlight the importance of examining how models obtain and use distributed graph evidence, rather than evaluating graph-theoretic reasoning through final-answer accuracy alone. These findings further motivate our dedicated diagnostics.

Table 2: Average accuracy on four partial-view graph-theoretic tasks.

ModelsMulti-view ImagesPer-view TextComposed Text
Qwen3.5-9B18.7574.0075.00
Gemma-4-12B-it28.5071.5073.00
GLM-4.6V-Flash3.001.253.50
Ovis2.5-9B5.2516.2511.50

W4: Insufficient Disentanglement of Multi-View Composition and Graph-Theoretic Reasoning.

R4: We thank the reviewer for this valuable suggestion. We would like to clarify that our goal is to evaluate two complementary multi-view scenarios (partial and temporal views), rather than construct strictly difficulty-matched task pairs. Temporal tasks naturally introduce additional constraints, such as minimizing changes across snapshots in STS and SBM, which are inherent to temporal multi-view graph reasoning rather than arbitrary increases in task difficulty. Both scenarios are designed to evaluate multi-graph reasoning, requiring models to integrate complementary structural information across partial graph views or track and reason about structural changes across temporal snapshots.

To further disentangle multi-view evidence composition from graph-theoretic reasoning, we conduct an additional controlled experiment on the four partial-view tasks (TS, MF, BM, and HP). Specifically, we combine all partial views into a single complete graph image while preserving the same underlying graph, question, and answer.

Table 3: Accuracy on four graph-theoretic tasks under the single complete-view setting.

ModelsTSMFBMHPAvg.
Qwen3.5-9B17.2043.6053.6032.0036.60
Gemma-4-12B-it15.2049.2058.4028.0037.70
GLM-4.6V-Flash0.0025.2014.004.4010.90
Ovis2.5-9B0.0020.4022.004.8011.80

The results in Table 3 show that all four models achieve higher average accuracy with a single complete graph, improving by 6.55%–17.85% over the corresponding multi-view image setting. Since the graph structures and questions remain unchanged, these findings provide evidence that distributing structural information across multiple visual views introduces additional challenges beyond graph-theoretic reasoning itself. We have incorporated these results and clarified the distinction between partial-view and temporal-view task difficulties in the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer kd2q (3/3)

Q3: To what extent can the performance gaps between partial and temporal tasks be attributed to additional task constraints rather than multi-view reasoning itself?

R5: Thank you for your comment. We agree that the performance gaps between partial and temporal tasks reflect, in part, the additional constraints introduced by the temporal tasks. In particular, STS and SBM require not only valid topological orders or maximum-cardinality matchings at each snapshot, but also consistency across snapshots through minimizing ordering inversions or assignment changes. These constraints capture an important aspect of temporal graph reasoning: coordinating solutions across evolving graph states, rather than solving each snapshot independently. Our goal is therefore to evaluate two complementary multi-view scenarios with distinct reasoning requirements. Accordingly, the partial–temporal comparisons characterize performance under these different requirements, rather than isolate the effect of temporal evidence processing alone.

Action taken: We have expanded the frontier-model evaluation in Section 5.1 (Table 3) and Appendix C.1 (Table 17), added matched input-representation comparisons in Section 5.3 (Table 7), and clarified the interpretation of partial–temporal performance differences in Section 5.1 and Appendix C.1.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.