acceptodds
Under review as a conference paper at ICLR 2027

Knossos and Ariadne: Benchmarking and Learning Complete Diagram Topology Extraction with Vision–Language Models

Abstract

Structural diagrams are widely used to represent complex systems and relational information across scientific, engineering, procedural, and spatial domains. Recent vision–language models (VLMs) have become increasingly capable of recognizing diagram elements and reasoning about their content, while complete diagram topology extraction remains comparatively underexplored. In this paper, we study **diagram-to-graph topology extraction**: extracting all diagram entities and the complete relations among them. To enable large-scale supervised training and systematic evaluation of this task, we introduce **Knossos**, a benchmark of 19,200 diagrams across six diverse domains, with 245,179 nodes and 439,740 edges. Its symbolic generation process provides exact alignment between rendered diagrams and annotations of complete topology, relation types, and connector geometry. To address the modeling challenge of complete topology extraction, we also present **Ariadne**, a structured framework that decomposes the task into node inventory extraction and source-conditioned edge prediction. Extensive experiments show that training on Knossos substantially improves complete topology extraction in smaller open-source VLMs. Ariadne further improves over one-step extraction under matched supervision, demonstrating the additional benefit of structured decomposition. It achieves the highest average Edge F1 among the evaluated methods on Knossos, while both backbone variants also improve over their unadapted counterparts on the real-world external benchmark. The benchmark and code are available on [anonymous repository](https://anonymous.4open.science/r/knossos-ariadne-2C4C/).

Primary area:
datasets and benchmarks
Keywords:
Structured Visual Understanding, Vision-Language Models, Multimodal Reasoning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 4 reviews, above 85% of the others.
Reviewer_usKKOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces Knossos, a synthetic benchmark containing 19,200 diagrams across six domains for complete diagram-to-graph topology extraction. It also proposes Ariadne, a two-stage framework that first identifies diagram nodes and then predicts edges through source-conditioned grouping. Experiments demonstrate substantial improvements over direct graph extraction

Critical strengths

1: Large-scale benchmark: Knossos provides complete node, edge, relation-type, and geometric annotations across six diagram domains, offering valuable supervision for structured diagram understanding.

2: Strong experimental results: Ariadne significantly outperforms existing baselines on the synthetic benchmark, with ablations demonstrating the effectiveness of staged extraction.

3: Comprehensive evaluation: The paper compares diverse open- and closed-source VLMs, includes external generalization experiments, and analyzes structural complexity and geometric grounding.

Critical weaknesses

1: Limited methodological novelty: The proposed framework mainly relies on decomposing graph extraction into node identification and source-group edge prediction. The technical contribution beyond structured prompting and task-specific fine-tuning appears relatively incremental.

2: Limited real-world generalization: The benchmark is entirely synthetic, relying on predefined graph grammars and limited entity vocabularies. Performance drops substantially on the external TopoBench benchmark, raising concerns about generalization to more diverse real-world diagrams.

3: Efficiency concerns: The staged framework requires multiple inference calls, introducing considerable computational overhead. The accuracy–efficiency trade-off deserves further consideration.

4: Incomplete evaluation of graph completeness: Although the paper emphasizes complete topology extraction, the primary evaluation relies on Node F1 and Edge F1. Graph-level exact-match accuracy would provide stronger evidence of whether the entire topology is correctly recovered.

Questions for authors

See weakness.

6 comments

1. Response to Question/Weakness 1

Regarding the reviewer's concern about novelty, we would like to clarify two points.

First, our work makes contributions in both benchmark construction and methodology: Knossos provides complete topology and geometry supervision across six domains, while Ariadne offers an effective framework for learning complete diagram topology extraction. Its central objective is to improve VLMs’ visual perception of diagram entities and their connections through structured supervision. We therefore believe that describing Ariadne primarily as structured prompting understates its focus on learning visual topology perception.

Second, Ariadne's simplicity is motivated by human annotation and supported by empirical exploration. Ariadne is inspired by an intuitive human annotation strategy: first identify the nodes, then focus on one node and annotate its connections before moving to the next. This provides an orderly way to track which parts of the diagram have been covered. During development, we investigated geometry-assisted approaches, including joint connector-trace and edge prediction, as well as reinforcement learning with typed-edge rewards. These more elaborate alternatives did not provide consistent improvements, leading us to retain the simpler topology-first approach as the most effective design among the alternatives we evaluated.

We believe that conceptual simplicity does not diminish a method's contribution. Methods such as ReAct and Chain-of-Thought likewise draw on intuitive human problem-solving strategies. Ariadne's value lies in a logically motivated, effective solution to complete topology extraction, together with the benchmark that enables its systematic study.

2. Response to Question/Weakness 2

We would like to clarify why synthetic generation is appropriate for this task and how we assess its real-world value.

First, synthetic image generation is a common approach to constructing data for diagram-oriented VLM learning and evaluation. For example, PlotQA programmatically renders charts from real-world numerical data, while FlowVQA generates flowchart images by rendering Mermaid code derived from procedural content. These benchmarks illustrate how generated visual data can support structured visual understanding at scale. As models demand more supervision, this approach helps overcome the limited availability and annotation cost of real-world diagram data.

Second, synthesis is particularly well suited to diagrams. Diagrams are deliberately constructed representations of entities and relations, often created with drawing software. Programmatically composing nodes, labels, and connectors reflects this construction process. Generating images and annotations from the same symbolic representation also provides scalable, internally consistent supervision for every node, relation, and connector path, alleviating the substantial cost of complete manual annotation.

Third, to our knowledge, Knossos is the first large-scale benchmark for complete diagram topology extraction, jointly providing typed topology and connector geometry annotations. Its value extends beyond synthetic evaluation: on 162 real-world TopoBench images, both Ariadne variants improve over their corresponding unadapted backbones on both external subsets (Table 1), with gains of 18.2 and 24.5 percentage points in edge F1 on AI2D/Web for GLM and Qwen, respectively. These results support the practical value of our complementary contributions: Knossos provides scalable training supervision, and Ariadne turns that supervision into improved topology extraction on real-world diagrams.

In the newly added Table 2c, Ariadne-Qwen also improves average Edge F1 over the Knossos-trained one-step baseline from 54.8% to 60.9%, with gains on both external subsets. This supports the effectiveness of staged extraction on the evaluated real-world diagrams.

3. Response to Question/Weakness 3

Thank you for raising this concern. The manuscript already examines the accuracy–efficiency trade-off: Section 4.4 discusses the model-dependent benefits, while Appendix D.4 (Table 9) quantifies the protocol overhead in calls, tokens, cost, and latency. Staged extraction benefits smaller, task-adapted VLMs but does not improve GPT-5.6, so its additional cost is not justified for every backbone. To better address the reviewer's concern, we have further expanded Limitations and outlook to clarify Ariadne's intended deployment in offline diagram digitization with task-adapted open-source VLMs, where accuracy and completeness can justify additional inference cost. We also discuss investigating whether decomposition can benefit larger VLMs on more demanding perception tasks as a future research direction.

4. Response to Question/Weakness 4

Thank you for this valuable suggestion, which strengthens our assessment of complete topology extraction. We have computed Graph Exact Match (Graph EM), requiring a complete one-to-one node match and an identical directed edge multiset, with no missing or extra nodes or edges. This metric evaluates topology only, excluding relation types and connector geometry. We report results on Knossos because its difficulty annotations allow us to assess complete graph extraction across different levels of diagram complexity. As shown in the table below, Ariadne-Qwen achieves 45.08% Graph EM, compared with 30.33% for GPT-5.6, with higher accuracy across all three difficulty levels. Graph EM consistently decreases as difficulty increases across the evaluated models, supporting the relevance of Knossos's difficulty stratification and highlighting the growing challenge of extracting an entire graph correctly.

We have added the metric definition, full results, and accompanying discussion to Appendix D.5, “Graph-level completeness across difficulty levels.” We have also expanded Section 4.4, “What diagram structures remain challenging?”, to discuss this trend and direct readers to the appendix.

ModelSimple (164)Medium (571)Difficult (465)Overall (1,200)
Qwen3-VL-8B11.59%1.93%0.00%2.50% (30/1,200)
GPT-4o12.20%1.93%0.00%2.58% (31/1,200)
GPT-5.675.00%35.55%8.17%30.33% (364/1,200)
Ariadne-Qwen84.76%58.49%14.62%45.08% (541/1,200)

We thank the reviewer for the feedback, particularly the suggestion to evaluate graph completeness. We hope the added results address the concerns raised and strengthen confidence in the work.

Thanks for the reply, which addressed some of my concerns. Regarding the novelty, the author said "During development, we investigated geometry-assisted approaches, including joint connector-trace and edge prediction, as well as reinforcement learning with typed-edge rewards. These more elaborate alternatives did not provide consistent improvements". I would like to learn more about this. Could the author compare a bit more about different methods? I would raise my score if this is properly compared. Also, please clarify the major difference with the paper: "DiagramEval: Evaluating LLM-Generated Diagrams via Graphs". Though it is conducted in SVG format, the methodology behind it is quite similar.

Reviewer_W2aTOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies diagram-to-graph extraction of all visible nodes and their directed, potentially typed relations. Knossos provides 19,200 synthetic diagrams across six domains, generated from symbolic graphs with aligned node, edge, and connector annotations. Ariadne predicts a node inventory, extracts outgoing edges for small source groups, and merges these predictions into a graph. A separate predictor supplies connector geometry without changing topology. Experiments compare pretrained VLMs, task-adapted one-step extraction, and staged extraction on 1,200 Knossos test images and a 162-image TopoBench-180 subset. The reported results include topology accuracy, relation typing, geometric grounding, and an analysis of node errors under distribution shift.

Critical strengths
  1. Knossos provides supervision beyond isolated diagram questions, including parallel typed relations and connector paths derived from the same finalized objects as the rendered image (Section 2.3 and Appendix B.4). This supports training and evaluation of complete extraction rather than selected relationships. Its training value is supported by Table 2a, where task-adapted one-step Qwen improves Edge F1 from 46.1% to 80.0% without adopting staged inference.

  2. The external evaluation provides both transfer evidence and a useful diagnosis of its limits. Table 1 reports improvements over the corresponding unadapted backbones for both Ariadne variants on both external subsets. With the Qwen edge predictor fixed, Table 2b raises average external Edge F1 from 60.9% to 67.3% by replacing adapted node predictions with base-model predictions, while oracle nodes reach 78.2%. This controlled comparison identifies upstream node specialization as a concrete bottleneck and suggests a practical way to improve the pipeline.

Critical weaknesses
  1. Table 2a does not fully isolate the benefit of source-group decomposition. The comparison changes both node/edge separation and source grouping, while the training and decoding budgets of the one-step control remain insufficiently specified. Table 10 reports 1,125 NODE training steps followed by 5,960 EDGE steps initialized from the NODE adapter, but Appendix E.1 does not provide the direct baseline's training duration or maximum output length. Thus, the 80.0% versus 91.6% Edge F1 comparison supports the complete pipeline but leaves its gain attributable to several changes, including possible differences in optimization exposure or output truncation. Clarifying the budgets and separating the effect of node conditioning from source grouping would strengthen the central methodological claim.
Questions for authors
  1. For the trained one-step Qwen control in Table 2a, what initialization, optimizer-step count, maximum generation length, and observed truncation rate were used? Please explain how its training exposure compares with the NODE-plus-EDGE schedule so that the decomposition gain can be interpreted independently of budget differences.

  2. Is there a comparison between grouped edge extraction and a two-stage baseline that predicts nodes first, then all edges in one call, with the same node predictor and sufficient output capacity? This would distinguish the benefit of source grouping from the benefit of providing a separate node inventory.

4 comments

1. Response to Question 1

Thank you for highlighting the importance of this comparison. Another reviewer raised a similar concern, reinforcing the need to clarify the training and generation budgets. This analysis helps clarify how training and generation budgets affect the interpretation of the performance gains. Both the one-step baseline and the NODE predictor initialize from the original pretrained Qwen3-VL-8B; the EDGE predictor initializes from the completed NODE adapter. Training steps are aligned between one-step and NODE training because both use one example per image; staged training additionally includes 5,960 EDGE updates. The total supervised-token counts are comparable (24.26M for one-step versus 26.74M for staged), since both protocols must generate the complete node inventory and all edges, either jointly or across stages. We also allow four times as many output tokens per call for one-step generation, with fewer than 1% of outputs reaching the limit.

We have added the training steps, supervised-text token counts, generation limits, and observed truncation rates for the two Ariadne-Qwen variants to Appendix E.2, with a pointer in Section 4.4.

Model / protocolTraining stepsTraining examplesSupervised-text tokensOutput limit per callObserved truncation
Ariadne-Qwen (one-step)1,12518,00024.26M8,1920.79%
Ariadne-Qwen (staged)7,085 (NODE 1,125 + EDGE 5,960)NODE 18,000; EDGE 95,35826.74M2,0480%

2. Response to Question 2

Thank you for suggesting this important comparison. Several reviewers raised this question, highlighting the importance of distinguishing the benefit of identifying nodes first from the additional benefit of source grouping. We have added an edge-prediction ablation in Table 2b, discussed in Section 4.4. To remove the influence of node prediction errors, both protocols receive the same oracle (ground-truth) node inventory. For Ariadne-Qwen, we train an all-edge variant that predicts all edges in one call and compare it with source-group prediction. The all-edge variant uses a training-step budget matched to one-step and NODE training, together with a fourfold larger per-call output limit than source-group prediction (Appendix E.2). The following results report Edge F1 (%) averaged across the Knossos test set.

ModelAll-edge predictionSource-group prediction
GPT-5.684.282.5
Ariadne-Qwen83.793.3

These results show the same model-dependent pattern as Table 2a. With oracle nodes, both models achieve higher Edge F1 under both protocols. The gap between GPT-5.6's two protocols narrows from 4.0 to 1.7 percentage points, while source grouping retains a substantial advantage for Ariadne-Qwen, improving Edge F1 by 9.6 percentage points. These findings support the effectiveness of source-group edge decomposition for Ariadne-Qwen, beyond the benefit of a correct node inventory. We hope this additional comparison addresses the reviewer's concern.

We thank the reviewer for the positive assessment and careful questions. We hope the revisions strengthen confidence in the work.

Thank you for clarifying the training and decoding settings and adding the oracle-node comparison. The larger one-step output limit and low reported truncation rate reduce my concern about output truncation. Giving both edge protocols the same oracle node inventory also provides a useful control for node-prediction errors. The 83.7% versus 93.3% Edge F1 result strengthens the evidence for the adapted Qwen pipeline under fixed node inputs.

Some uncertainty about the attribution to source grouping remains. The all-edge variant is described as using the training-step budget of one-step/NODE training, whereas the grouped EDGE model receives 5,960 updates after NODE initialization. Comparable supervised-text token counts do not fully control optimization exposure or total training compute. Please clarify whether the all-edge variant also initializes from the completed NODE adapter and state these training differences explicitly when interpreting the oracle-node comparison. The GPT-5.6 result also supports keeping the grouping claim specific to the model and adaptation setting.

The additional comparison and protocol details strengthen my positive assessment of the benchmark and pipeline. Given their value and the remaining uncertainty about the independent contribution of grouping, I maintain my recommendation of 3 (Weak acceptance). Please incorporate these qualifications and configuration details into the revised manuscript.

We thank the reviewer for the constructive follow-up and positive assessment of our benchmark and pipeline. Regarding initialization, both the all-edge and grouped EDGE variants start from the completed NODE adapter. Regarding training exposure, source grouping splits each diagram’s edge annotations into smaller training examples and therefore requires more optimizer updates. This is intended to give both variants approximately one pass over each diagram’s full edge annotations, making their supervision exposure comparable. We will clarify these training settings and discussion in the revised manuscript. We hope the reviewer finds this clarification helpful.

The rebuttal has addressed most of my concerns. I raised my score to 3

Reviewer_KFhKOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies complete diagram-to-graph topology extraction, moving beyond diagram question answering that only requires selected entities or relations. The authors introduce Knossos, a synthetic benchmark of 19,200 diagrams across six domains with complete node, edge, relation-type, and connector annotations, and Ariadne, a two-stage framework that first predicts a node inventory and then extracts edges through source-conditioned groups. On the 1,200-image Knossos test set, Ariadne-Qwen achieves promising performance. The paper also reports transfer results on a 162-image subset of TopoBench-180, where task adaptation improves over the corresponding open-source backbones, although the strongest closed-source models remain competitive or stronger.

Critical strengths
  1. Complete topology extraction is a meaningful problem that is not adequately measured by localized diagram QA: a model may answer selected questions while still omitting or inventing other relations. Knossos provides 19,200 diagrams across six domains and aligned annotations for nodes, directed and typed edges, and connector geometry. Deriving the annotations from the same symbolic objects used for rendering also gives the benchmark precise supervision and supports controlled analysis.

  2. Separating node discovery from source-conditioned edge prediction is intuitive and makes the output easier to validate and assemble. The matched Qwen3-VL-8B comparison is particularly persuasive: staged extraction improves Edge F1 from 80.0% to 91.6%, while Node F1 changes only from 97.1% to 97.8%. This suggests that the gain is primarily in relation extraction, rather than simply better node recognition. Ariadne also achieves the best average Typed-Edge F1 on the three domains with explicit relation semantics.

  3. The paper compares several open- and closed-source VLMs and visual-reasoning methods, evaluates two task-adapted backbones, and reports results by domain and diagram difficulty. The external evaluation and node-source ablation are also useful: they reveal that upstream node recognition is an important factor in transfer performance, rather than presenting the external results as uniformly strong.

Critical weaknesses
  1. The synthetic benchmark does not yet establish generalization to genuinely unseen diagram structures. The train/test split uses disjoint random seeds, but both splits are generated from the same domain-specific grammars and finite canonical vocabularies. Thus, the 1,200-image test set mainly measures generalization to new instances and renderings within the benchmark's generation space. The external evaluation is a useful step, but it covers only 162 images from two TopoBench-180 subsets (54 AI2D/Web and 108 Network diagrams). Moreover, Ariadne's external performance remains below strong closed-source systems on important subsets, and the paper's own node-source ablation shows a substantial gap between predicted and oracle node inventories. The evidence therefore supports transfer to a limited external setting, but not yet broad generalization across unseen diagram grammars, vocabularies, or domains.

  2. The methodological novelty and the source of Ariadne's improvement could be isolated more clearly. The core design—predicting nodes first and then generating edges for small source groups—is reasonable, but relatively straightforward. The theoretical analysis establishes lossless partitioning and derives a potential reduction in position-dependent decoding errors under a stylized assumption; it does not guarantee lower prediction error in general. In addition, the one-step-versus-staged comparison combines several changes: an explicit node inventory, restricted source groups, shorter repeated generations, and additional inference calls. The current ablation demonstrates that the complete staged protocol works, but does not fully identify how much of the gain comes specifically from source grouping rather than the intermediate node representation or the increased decoding budget.

  3. The accuracy improvement comes with a meaningful inference-cost increase. The efficiency analysis reports 5.23 model calls per image for staged inference versus one for one-step generation. On GPT-5.6, token usage increases from 3,843 to 10,747 per image (2.80×), estimated API cost from $0.052 to $0.147, and mean latency from 23.26 to 37.12 seconds. The staged protocol also reduces GPT-5.6 Edge F1 from 81.1% to 77.1% in the reported ablation, suggesting that its benefit is concentrated in smaller task-adapted models rather than being universally effective. The paper reports this trade-off, but its practical implications for deployment and the conditions under which the extra cost is worthwhile could be made more explicit.

Questions for authors
  1. Do the Knossos training and test splits share the same template families and canonical vocabularies, despite using disjoint random seeds? How should the external results on the 162-image TopoBench-180 subset affect the scope of the paper's generalization claim?
  2. Could the authors clarify whether the improvement is mainly due to the explicit node inventory, source-group constraints, or shorter repeated generations? In particular, how would a node-first variant that predicts all edges in one subsequent call compare with the current source-group protocol?
  3. Given the 5.23× increase in model invocations and the 2.80× token usage reported for GPT-5.6, what is the intended deployment regime for Ariadne? Also, Is the main benefit expected specifically for smaller task-adapted open-source VLMs, given that staged extraction does not improve the GPT-5.6 result?
2 comments

1. Response to Question 1

Thank you for raising this point. Yes, the splits share domain-specific template families and canonical vocabularies. Before large-scale generation, researchers inspect preview batches and refine the generation rules to ensure readable, visually coherent, and semantically plausible diagrams. Within these validated rules, seed-controlled sampling varies node combinations, connectivity, layouts, and visual presentation. This supports controlled evaluation within a diverse generation space, but does not establish generalization to unseen templates. We will explicitly acknowledge this part in Limitations and outlook section.

However, our evaluation on TopoBench strengthens the evidence for generalization beyond the shared synthetic templates. On 162 independently collected real-world diagrams, both Ariadne variants improve over their corresponding unadapted backbones on both external subsets (Table 1), demonstrating useful transfer to the evaluated real-world settings.

2. Response to Question 2

Thank you for suggesting this important comparison. Several reviewers raised this question, highlighting the importance of distinguishing the benefit of identifying nodes first from the additional benefit of source grouping. We have added an edge-prediction ablation in Table 2b, discussed in Section 4.4. To remove the influence of node prediction errors, both protocols receive the same oracle (ground-truth) node inventory. For Ariadne-Qwen, we train an all-edge variant initialized from the same completed NODE adapter to predicts all edges in one call and compare it with source-group prediction. The following results report Edge F1 (%) averaged across the Knossos test set.

ModelAll-edge predictionSource-group prediction
GPT-5.684.282.5
Ariadne-Qwen83.793.3

These results show the same model-dependent pattern as Table 2a. With oracle nodes, both models achieve higher Edge F1 under both protocols. The gap between GPT-5.6's two protocols narrows from 4.0 to 1.7 percentage points, while source grouping retains a substantial advantage for Ariadne-Qwen, improving Edge F1 by 9.6 percentage points. These findings support the effectiveness of source-group edge decomposition for Ariadne-Qwen, beyond the benefit of a correct node inventory. We hope this additional comparison addresses the reviewer's concern.

3. Response to Question 3

Thank you for highlighting this trade-off. On the current task, our experiments show that GPT-5.6 performs better with one-step extraction, while smaller task-adapted VLMs benefit from decomposition (Section 4.4). We interpret this pattern as consistent with GPT-5.6's stronger visual perception and long-context processing, and with decomposition reducing these demands on smaller models.

Ariadne is intended for offline diagram digitization and graph extraction with task-adapted open-source models, where improved accuracy and completeness can justify the additional inference cost reported in Appendix D.4 (Table 9). The current results do not rule out benefits from decomposition for larger models such as GPT on much more complex tasks that place greater demands on visual perception and contextual processing. We have expanded the Limitations and outlook discussion to clarify this deployment scope and discuss this possibility for larger models as a direction for future research.

We thank the reviewer for the supportive assessment and helpful feedback. We hope the revisions strengthen confidence in the work.

Thank you again for your constructive feedback. As the discussion period is nearing its end, we would be grateful for any further thoughts on our updated response and whether any points would benefit from additional clarification. We hope the new evidence is helpful as you finalize your assessment. Thank you for your time and consideration!

Reviewer_9D5uOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces Knossos and Ariadne for extracting complete graphs from diagram images. Knossos is a synthetic dataset containing 19,200 diagrams across six domains, with annotations for nodes, connections, relation types, and geometry. Ariadne first identifies all nodes, then predicts outgoing connections for small groups of nodes, and combines the results into a complete graph. The authors train Ariadne on Knossos using two vision-language model backbones and evaluate it on the Knossos test set and a 162-image subset of the real-world TopoBench-180 benchmark.

Critical strengths
  1. Knossos appears to be a useful dataset with detailed annotations. Knossos contains 19,200 diagrams from six domains, with annotations for nodes, edges, relation types, and connector geometry. The diagrams and annotations are generated from the same underlying graph, which helps keep them consistent. This is a valuable contribution because it supports training and testing models on complete graph extraction across several types of diagrams.

  2. The proposed method show a large improvement over a trained baseline. On Qwen3-VL-8B, Ariadne improves Edge F1 from 80.0 to 91.6 compared with a one-step model trained on the same data (Table 2a). Both Ariadne variants also improve over their original backbones on the external benchmark (Table 1). These results support the practical value of the overall training and extraction pipeline.

Critical weaknesses
  1. The specific benefit of source grouping is unclear. Ariadne changes several things at once, including separate training for node and edge prediction and multiple model calls per image. The paper does not compare it with a two-stage model that first predicts the nodes and then predicts all edges in one call. The training budgets and output limits of the trained one-step baseline also need clearer reporting. These gaps make it difficult to tell how much of the improvement comes specifically from source grouping.

  2. The evidence for generalization is limited. The Knossos training and test sets share the same graph generation rules, node vocabulary, and visual asset pools. The high test scores therefore do not show how well the method handles unfamiliar structures or styles. The external evaluation is useful, but it does not include the one-step baseline trained on Knossos. As a result, it remains unclear whether source grouping provides an additional benefit on real diagrams beyond the benefit of training on the new dataset.

Questions for authors
  1. For the trained one-step baseline and Ariadne in Table 2a, what are the total training steps, numbers of supervised tokens, and output token limits? How often were outputs cut off because they reached the token limit? This would help me assess whether differences in training or generation budgets could explain part of the gain.

  2. Additional comparison that could change my assessment: With the same predicted node list and comparable training settings, how does predicting all edges in one call compare with predicting them in source groups? This would separate the benefit of predicting nodes first from the additional benefit of source grouping.

  3. Additional result that could change my assessment: How does the one-step model trained on Knossos perform on the external TopoBench subset? Comparing it directly with Ariadne would help establish whether the proposed decomposition also improves performance on real diagrams.

2 comments

1. Response to Question 1

Thank you for requesting these details. This analysis helps clarify how training and generation budgets affect the interpretation of the performance gains. Training steps are aligned between one-step and NODE training because both use one example per image; staged training additionally includes 5,960 EDGE updates. These additional updates arise from splitting each diagram’s edge annotations into smaller source-conditioned training examples. They correspond to approximately one pass over the resulting edge-training examples, keeping coverage of the edge supervision comparable to one-step training. The total supervised-token counts are comparable (24.26M for one-step versus 26.74M for staged), since both protocols must generate the complete node inventory and all edges, either jointly or across stages. We also allow four times as many output tokens per call for one-step generation, with fewer than 1% of outputs reaching the limit. We have added the training steps, supervised-text token counts, generation limits, and observed truncation rates for the two Ariadne-Qwen variants to Appendix E.2 with a pointer in Section 4.4.

Model / protocolTraining stepsTraining examplesSupervised-text tokensOutput limit per callObserved truncation
Ariadne-Qwen (one-step)1,12518,00024.26M8,1920.79%
Ariadne-Qwen (staged)7,085 (NODE 1,125 + EDGE 5,960)NODE 18,000; EDGE 95,35826.74M2,0480%

2. Response to Question 2

Thank you for suggesting this important comparison. Several reviewers raised this question, highlighting the importance of distinguishing the benefit of identifying nodes first from the additional benefit of source grouping. We have added an edge-prediction ablation in Table 2b, discussed in Section 4.4. To remove the influence of node prediction errors, both protocols receive the same oracle (ground-truth) node inventory. For Ariadne-Qwen, we train an all-edge variant initialized from the same completed NODE adapter that predicts all edges in one call and compare it with source-group prediction. The all-edge variant uses a training-step budget matched to one-step and NODE training, together with a fourfold larger per-call output limit than source-group prediction (Appendix E.2). The following results report Edge F1 (%) averaged across the Knossos test set.

ModelAll-edge predictionSource-group prediction
GPT-5.684.282.5
Ariadne-Qwen83.793.3

These results show the same model-dependent pattern as Table 2a. With oracle nodes, both models achieve higher Edge F1 under both protocols. The gap between GPT-5.6's two protocols narrows from 4.0 to 1.7 percentage points, while source grouping retains a substantial advantage for Ariadne-Qwen, improving Edge F1 by 9.6 percentage points. These findings support the effectiveness of source-group edge decomposition for Ariadne-Qwen, beyond the benefit of a correct node inventory.

3. Response to Question 3

Thank you for suggesting this external comparison. We have added the results in Table 2c and discussed them in Section 4.4, “What limits Ariadne’s external generalization?” We evaluate the same Knossos-trained one-step and staged Qwen3-VL-8B models used in Table 2a on the 162-image TopoBench subset, without further adaptation on TopoBench. All scores below are percentages; averages give equal weight to the two external subsets.

ProtocolAverage Node F1 ↑Average Edge F1 ↑Web-style Edge F1 ↑Network-style Edge F1 ↑
One-step81.254.862.846.7
Staged extraction85.860.968.653.2

Staged extraction improves average Edge F1 from 54.8% to 60.9%, a gain of 6.1 percentage points, with improvements on both external subsets. Although substantial room for improvement remains on external diagrams, these gains over the Knossos-trained one-step baseline support the effectiveness of the staged pipeline in the evaluated real-world settings.

We thank the reviewer for the constructive suggestions that helped strengthen our comparisons. We hope the additional evidence addresses the concerns raised and strengthen confidence in the work.

Thank you again for your constructive feedback. We have updated our response with training-budget details, an edge-prediction ablation, and the external baseline comparison you requested. As the discussion period is nearing its end, we would appreciate hearing whether these additions address your concerns and whether any questions remain. If your main concerns have been resolved, we would also be grateful if you could take the additional evidence into account in your updated assessment. Since evaluations on our large test set take considerable time, receiving any further experimental requests soon would help us prioritize and complete them before the deadline. Thank you for your time and consideration!

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary for the Area Chair

We thank the Area Chair and reviewers for their careful assessment. We summarize the main additions and clarifications below.

  1. Reviewers 9D5u, KFhK, and W2aT asked how predicting all edges in one call compares with source-group prediction when both receive the same node inventory. We added a trained all-edge variant and compared the two protocols under the same oracle nodes to remove node prediction errors from the comparison. The results are reported in Table 2b, with accompanying analysis added to Section 4.4.

  2. Reviewers 9D5u and W2aT requested clearer training and generation settings to assess whether budget differences or output truncation could explain the gains. We added training steps, supervised-text token counts, output limits, and truncation rates, together with the all-edge variant's configuration, in Appendix E.2. We also added an analysis of the budget similarities and differences there, with a pointer in Section 4.4.

  3. Reviewer 9D5u requested an external evaluation of the Knossos-trained one-step baseline against Ariadne. We added this comparison on TopoBench using the same trained models as Table 2a, without further adaptation on the external data. The results appear in Table 2c, and we added an analysis of their implications for real-world transfer to Section 4.4.

  4. The remaining questions concern generalization scope and deployment. Our response to KFhK clarifies the shared template families and vocabularies and the limits of the external evidence. Building on the existing efficiency analysis (Appendix D.4), we expanded the Limitations and outlook discussion in Section 5 to explain the intended offline use, the model-dependent trade-off, and possible applications to larger models on more demanding tasks as future work.

  5. Reviewer usKK raised four concerns: methodological novelty, real-world generalization, inference efficiency, and complete-graph evaluation. Our response explains the complementary benchmark and method contributions, the rationale for synthetic supervision, and the external transfer evidence, including the new Table 2c comparison. We also address efficiency through the reported cost analysis and expanded deployment discussion. To assess complete topology recovery, we added Graph Exact Match results by difficulty, together with the metric definition and analysis in Appendix D.5. We also added a discussion of the difficulty trend and a pointer to these results in Section 4.4.

We hope that these additional experiments, analyses, and clarifications address all of the reviewers’ concerns. We thank the reviewers and the Area Chair for their constructive feedback, which has helped us improve the quality and clarity of the paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.