acceptodds
Under review as a conference paper at ICLR 2027

Parsing and Binding: Chart Understanding as Queryable Instance Graph Recovery

Abstract

Chart understanding is commonly scored by answers or recovered tables. These metrics can look strong even when marks are merged or legend bindings are wrong, because they are blind to the underlying instance structure. We propose a structural view: recovery of a queryable instance graph, in which each claim remains grounded in masks and "what is seen” is scored separately from "how instances are bound.” We operationalize this in ChartLogic, an ontology-guided Parse-and-Connect model over shared instance anchors. Parse lifts pixels to instances with category, grouping, and identity; Connect binds them into relations among marks, axes, legends, and data structures. A shared generation source yields aligned images, hierarchical label maps, and ontology-typed ABox graphs, providing supervision for both stages. At evaluation, task-conditioned SPARQL templates are run on a graph built only from predicted instances, with a ground-truth skeleton as a diagnostic. Experiments on synthetic charts and on verify-120 (ChartQA-Redraw-120)—a human-calibrated, re-rendered ChartQA validation subset—yield three observations. First, ontology-aligned hierarchical supervision recovers the structured instance identity that enables downstream binding. Second, GT-skeleton scores remain substantially higher than open recovery even when many query subjects are missing from the predicted graph; open recovery failures decompose into unmatched instances and wrong bindings among matched ones, so Parse and Connect must be attributed separately. Third, under transfer, category accuracy stays high while geometric recall falls first, a category-vs-geometry gap that also appears on verify-120. Together, these observations suggest that content-level and GT-skeleton scores can mask an instance-topology failure that only open recovery reveals.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
chart understanding, queryable instance graphs, instance segmentation, relational grounding

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_CngvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces ChartLogic, a Parse-and-Connect framework that maps chart images to ontology-typed instance graphs (ABoxes). The Parse stage recovers visual instances and hierarchical identities, while the Connect stage predicts relations such as axis membership and legend–mark correspondence. The recovered graph is evaluated through structural SPARQL queries rather than only answer- or table-level metrics. The authors train on a synthetic dataset with aligned images, instance masks, and ABoxes, and evaluate on both a held-out synthetic set and a re-rendered ChartQA-derived subset. The experiments focus on series identity recovery, open-recovery versus ground-truth-skeleton evaluation, and attribution of failures to parsing or relation binding.

Critical strengths
  1. Fine-grained structural evaluation. The separation of parsing and relational binding, especially the comparison between open recovery and ground-truth-skeleton evaluation, provides a useful diagnostic framework for distinguishing perception failures from relation-prediction failures. This is a meaningful perspective beyond answer-level chart evaluation.

  2. Well-aligned synthetic supervision. The shared generation pipeline provides exact alignment among chart images, instance-level masks, and symbolic ABox annotations. This makes it possible to study instance identity and structural bindings under controlled conditions without requiring expensive manual graph annotations.

Critical weaknesses
  1. The proposed structural task is not sufficiently validated as a better measure of chart understanding.
    The paper motivates instance-graph recovery by arguing that answer- or table-level metrics may hide structural errors, but it does not demonstrate that the proposed task is in fact a better proxy for chart understanding. The evaluation is limited to structural queries defined within the proposed ABox representation, such as axis membership, series grouping, and legend–mark correspondence, without showing whether better structural recovery predicts or improves performance on downstream tasks such as ChartQA. The paper also does not evaluate a representative set of existing chart models to show that conventional benchmarks fail to distinguish models while the proposed task reveals meaningful differences. As a result, the experiments show that the proposed metrics can detect structural errors, but not that these errors are consequential for broader chart understanding.

  2. The method contribution of ChartLogic is difficult to assess due to the lack of competitive baselines and limited external validation.
    ChartLogic is evaluated mainly through internal ablations and comparisons against heuristic or ground-truth-skeleton variants, rather than against representative prior chart parsing, derendering, or structured scene-graph methods. This makes it difficult to determine whether ChartLogic itself provides an advantage over existing approaches or whether the reported performance mainly reflects the proposed supervision and evaluation setup. In addition, the main experiments use charts generated by the authors’ own renderer, while verify-120 is also re-rendered through the same pipeline and is evaluated only at the Parse level rather than with full open recovery. Therefore, both the comparative effectiveness and real-world generalization of the proposed method remain unclear.

Questions for authors

Clarification requests

  1. What do the authors consider the primary contribution of the paper: a new chart-understanding task/evaluation protocol, a new dataset/benchmark, or the ChartLogic model itself? The current presentation appears to combine all three, and clarifying this would affect how I assess the required empirical validation.

  2. Are there existing chart parsing, derendering, scene-graph, or multimodal models that can be adapted to the proposed ABox recovery task? If so, could the authors clarify why no such external model baselines are included in the evaluation?

Assessment-critical empirical questions

  1. Can the authors provide evidence that the proposed structural-recovery task captures chart understanding beyond what existing answer- or table-level benchmarks measure? For example, do different existing chart models exhibit similar ChartQA/table-recovery performance but substantially different structural-recovery performance, and are these differences associated with meaningful downstream failures?

  2. Is there any evidence that better ABox recovery translates into better downstream chart reasoning, such as ChartQA performance? The current evaluation demonstrates that the proposed metrics detect structural errors, but it is unclear whether these errors are consequential for downstream chart understanding.

  3. Do the authors have results for full open recovery on charts outside their own rendering pipeline, rather than only on synthetic charts and the re-rendered verify-120 subset? Such evidence would materially affect my assessment of the external validity of the proposed task and method.

1 comment
AuthorsOpenReview ↗

Response to Cngv: what the paper claims, which baselines we ran, and whether the errors matter

Dear Reviewer Cngv,

Thank you — your review is the one that forced us to say what the paper claims. You asked for one contribution, for a representative set of existing models, and for evidence that structural errors matter; we answer in that order.

Q1 — Task, benchmark, or model?

One contribution: a measurement protocol, with the model as its instrument. The protocol scores a recovered instance graph so that misses, wrong bindings and unsupported assertions are separated and priced differently. Where the abstract now reads as three co-equal contributions, the revision will state one contribution and one set of comparisons (Abstract and §5).

Q2 — A representative set of existing models

Four families, run on the same inputs. Two can be placed on our predicates, two cannot:

familysystemoutputon our predicates
oursChartLogicannotated instance graph0.922 / 0.490 / 0.462
detection parserChartOCR [2]marks, roles, seriesaxis 0.000, series 0.092
derenderingDePlot [3]tableno bindings
chart-pretrained VLMUniChart [4]text and table answersno bindings
general MLLMQwen3-VL-8B; Qwen2.5-VL-7B; Llama-3.2-VJSON graph on request0.649/0.415/0.681; 0.511/0.496/0.539; 0.222/0.123/0.082

The last column mixes two protocols, and we do not compare across it: ChartOCR and our model are scored on mask-aligned binding, the MLLMs on a semantic probe that credits a mapping stated in text without anchoring it to a handle.

LineFormer and LineEX emit masks or keypoints without a binding layer; structured scene-graph models such as RelTR have no chart checkpoint, and re-training one on charts would replace its supervision with ours. Thus the absence you note is partly a property of the field: most chart systems do not produce a graph on which binding can be scored. This table goes into §4.4.

Q3 / W1 — Is the structural task a better measure?

At the metric level and, in a controlled setting, at the consequence level; not at the model level. Same 400400 charts, same predictions:

chart typetable-level proxyinstance-level binding
line0.8660.031
scatter0.8420.147
pie0.8450.667
histogram0.8020.924

Thus a table-level score cannot tell a chart type that is nearly solved from one that is not. For consequence, we corrupt the reference graph and hold the tuple total fixed: damaging 5%5\% of belongsToAxis drops axis queries from 1.0001.000 to 0.4450.445 while counting stays at 1.0001.000; damaging 5%5\% of usesDataStructure drops counting from 1.0001.000 to 0.3800.380. Structural error is not uniform, and counting the wrong tuples hides which query breaks.

Q4 — Downstream ChartQA

In part, on a controlled version of the question. We answer 646646 templated questions with a deterministic program reasoner — no QA head — from the gold calibrated table and ABox of verify-120, and from our predicted recovery. Accuracy is 0.5800.580 against 1.0001.000 from gold: a drop of 0.4200.420, with 0.1100.110 unanswerable against none. The gold arm is 11 by construction, so it measures no difficulty; what it measures is what recovery error costs, and it costs two answers in five. Nor is it the tuple error restated: 0.3740.374 of reference tuples are unmatched, yet answers fall by 0.4200.420. Thus within this probe the errors are consequential. It remains a probe, not a ChartQA result: templates are not native ChartQA language, the reasoner is a program not a trained head, and scoring is exact or tolerance. The revision will report this at §4.5 and keep a trained QA head as a limitation at §5.

Q5 / W2 — The supervision and evaluation setup as the main factor

In part, and the comparison least exposed to it can be named. Every synthetic result uses our generator and supervision; shuffled versus real series supervision moves Acccs\mathrm{Acc}_{cs} from 0.0390.039 to 0.9590.959, more than any architectural effect, so we no longer present the model as an architecture contribution. The comparison that does not rest on our annotations is the CHART-Info set [1]: image only, organizers' script, 396396 charts, no system sees the scored ground truth. Extraction there is level (−0.4-0.4). Thus our claim is about what the protocol reveals, not about beating a parser on its own task. The revision will report both at §4.4 and §4.6, and state one contribution instead of claiming a model advantage.

  • [1] Davila et al. ICPR 2020 competition on harvesting raw tables from infographics. ICPR, 2021.
  • [2] Luo et al. ChartOCR: Data extraction from charts images via a deep hybrid framework. WACV, 2021.
  • [3] Liu et al. DePlot: One-shot visual language reasoning by plot-to-table translation. Findings of ACL, 2023.
  • [4] Masry et al. UniChart: A universal vision-language pretrained model for chart comprehension and reasoning. EMNLP, 2023.
Reviewer_4o4HOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ChartLogic, an ontology-guided framework that formulates chart understanding as the recovery of a queryable instance graph rather than direct question answering or table extraction. The framework consists of two stages. Parse uses hierarchical panoptic segmentation to recover chart elements with category, series, and instance identities. Connect predicts organizational and visual relationships among the recovered instances, allowing the resulting ABox graph to be queried using task-conditioned SPARQL templates. To provide aligned supervision, the authors construct a synthetic chart-generation pipeline that jointly produces images, hierarchical instance maps, and ontology-typed ABox representations. The framework is evaluated using both a ground-truth-skeleton diagnostic and an open-recovery protocol based on predicted instances. Experiments on 400 held-out synthetic charts demonstrate that aligned series supervision substantially improves series identity recovery. The authors also show that strong relational performance on a ground-truth skeleton does not necessarily translate into successful open recovery because instances may be missing or incorrectly connected. A further evaluation on verify-120, a ChartQA-derived re-rendered dataset, reveals a substantial decrease in geometric recall despite relatively high conditional category accuracy. Overall, the paper introduces a useful structural evaluation perspective for chart understanding. However, important concerns remain regarding the completeness of the proposed query-based evaluation, the generalizability of the findings beyond synthetic rendering, and the empirical advantage of the complete Parse–Connect architecture over alternative structural approaches.

Critical strengths

S1. A meaningful formulation of chart understanding as grounded instance and relation recovery. The paper identifies a legitimate limitation of answer-level and table-recovery evaluation: a correct answer does not necessarily imply that the underlying chart elements and their relations have been correctly identified. By introducing shared instance anchors and ontology-typed ABox representations, the proposed framework provides a structured intermediate representation connecting visual elements to semantic relations. The separation of Parse and Connect is valuable because it supports more detailed attribution of structural errors. In particular, the comparison between ground-truth-skeleton and open-recovery protocols highlights the importance of accounting for missing visual instances before drawing conclusions about relational reasoning quality. This formulation provides a useful direction for developing more interpretable and auditable chart-understanding systems. S2. Carefully designed diagnostic experiments reveal distinct instance-identity and relational failure modes. The paper provides convincing controlled evidence that conventional panoptic metrics do not capture all relevant aspects of chart-instance recovery. In Table 1, aligned series supervision improves ID-strict category–series accuracy from 0.039 under shuffled supervision to 0.959 under real series supervision, while panoptic quality changes only modestly. Table 2 further distinguishes failed series partitioning from simple label permutation through chart-local remapping accuracy and adjusted Rand index. The open-recovery experiments also provide useful diagnostic evidence. In Table 4, legend–mark correspondence achieves 0.474 recall with the learned approach, compared with 0.319 for the heuristic baseline. Table 5 further shows that poor series binding in line and scatter charts is largely associated with missing detected marks, whereas legend–mark errors are more frequently caused by incorrect bindings among matched instances. These findings substantiate the value of separating perception failures from relational failures and provide more informative diagnostic outcomes than a single aggregate score.

Critical weaknesses

W1. The query-based evaluation is too narrow and recall-focused to establish the semantic correctness of the recovered instance graphs. Evidence: Sections 3.1–3.2, Section 4.1, and Tables 3–5. This is my primary concern. The central claim is that chart understanding should be evaluated through recovery of a queryable instance graph, where visual objects, literal attributes, and semantic relations are grounded in the source image. However, the main open-recovery evaluation is restricted to three structural query categories: axis membership, mark–series grouping, and legend–mark correspondence. These are useful diagnostic tasks, but they cover only a limited subset of the ontology and ABox representation described in Sections 3.1 and 3.2. The representation includes literal attributes such as text, color, and numerical quantities, as well as organizational and visual object properties. The evaluation does not establish end-to-end correctness for the broader range of information encoded in the recovered graph. More importantly, the primary binding metric is recall. Recall measures whether reference query bindings are recovered, but it does not sufficiently penalize additional incorrect bindings produced by the predicted graph. For example, a system that generates the correct legend–mark correspondence together with several spurious correspondences could retain high recall despite producing an unreliable semantic graph. An auditable graph should not only recover correct relationships but also avoid unsupported assertions. The existing matched-wrong and unmatched-subject diagnostics are useful, but they primarily characterize failures relative to expected reference bindings. They do not replace precision-oriented evaluation of additional predicted triples and query answers. There is also an interpretability issue in the reported error decomposition: Section 4.1 specifies that recall is macro-averaged across charts, whereas unmatched and matched-wrong rates are computed as tuple-level aggregates. These values do not necessarily form an additive decomposition on the same aggregation basis. For example, the Table 4 mark–series values 0.484, 0.197, and 0.157 should not be interpreted as complementary probabilities summing to one. The differing conventions are disclosed, so this is not necessarily a numerical error. I recommend reporting binding precision, recall, and F1; evaluating spurious graph assertions; and extending the query suite to literal attributes, multi-relation compositions, and graph-consistency checks. A strict tuple-level decomposition under a common denominator, alongside chart-macro metrics, would make attribution easier to interpret. Without these experiments, the paper demonstrates meaningful recovery of selected graph relationships but does not yet establish comprehensive semantic correctness of the predicted instance graphs. W2. The evaluation provides insufficient evidence of real-world generalization or a clear advantage of the full architecture over stronger structural alternatives. Evidence: Sections 3.3–3.4, Sections 4.1 and 4.4–4.5, and Tables 1, 4, and 6. The principal dataset contains 3,200 training, 400 validation, and 400 test charts, all produced using the same controlled chart-generation pipeline. The shared generator enables aligned visual and symbolic supervision, but results from this setting need not transfer to charts with independent rendering styles, complex layouts, occlusions, or unconventional visual elements. The external verify-120 evaluation only partially addresses this concern. Section 4.1 explicitly states that verify-120 consists of ChartQA-sourced charts reconstructed using the authors' rendering pipeline, not the original ChartQA pixels. Moreover, it is used only for Parse-level evaluation. Geometric recall falls from 0.678 on the synthetic test to 0.505 on verify-120, while conditional category accuracy remains high. This supports a category-versus-geometry gap under the evaluated content-source shift, but does not establish reliable graph recovery on genuinely external visual distributions. The open-recovery results on the synthetic benchmark are also limited. Table 4 shows that the learned method clearly improves over same-instance heuristics only for legend–mark correspondence (0.474 vs. 0.319). Heuristics are slightly better for axis membership (0.932 vs. 0.920) and mark–series grouping (0.504 vs. 0.484). Table 5 reports very low mark–series binding recall for line and scatter charts, at 0.031 and 0.147. These findings are useful diagnostics but indicate that the system is not yet reliably recovering several essential structures. In addition, the experiments primarily compare the proposed variants with shuffled supervision and simple relation heuristics. Such ablations support the role of aligned supervision, but do not isolate the ontology-guided architecture's contribution relative to competitive graph-based or modular parsing-and-relation-prediction alternatives. Since the TBox serves as controlled vocabulary and query interface rather than a training constraint, the added value of ontology typing over an equivalent non-ontological graph representation remains unclear. I would expect full open-recovery evaluation on independently rendered charts, stronger structural baselines, and an ablation isolating the ontology representation. These comparisons are needed to substantiate broader practical significance beyond the controlled synthetic setting.

Questions for authors

Q1. How does the proposed evaluation penalize spurious graph assertions and incorrect extra query results? The principal metric is binding-tuple recall over three selected predicates. Can the authors report binding precision and F1, as well as graph-level triple precision and recall? How would the current evaluation score a predicted graph containing all reference relationships together with numerous additional incorrect relationships? A controlled false-edge injection experiment would help establish whether the protocol measures semantic correctness. Q2. Can the authors demonstrate complete queryability beyond the three evaluated structural predicates? The ABox includes literal attributes, organizational relations, and visual bindings, but the main evaluation addresses only axis membership, mark–series grouping, and legend–mark correspondence. Can the authors evaluate text and numerical attribute recovery, multi-hop relations, and compositional queries that combine instances, relations, and literal values? Such experiments would clarify whether the graph supports chart understanding beyond a restricted set of structural tasks. Q3. How well does the complete Parse–Connect system perform on independently rendered real chart images? verify-120 uses the authors' renderer and is evaluated only at the Parse level. Could the authors additionally test the system on original ChartQA images or another independently produced chart dataset with suitable structural annotations? In particular, can they report full open-recovery binding precision, recall, and F1 on this external distribution? Q4. What is the incremental contribution of the ontology representation and learned Connect module over alternative structural pipelines? Table 4 indicates that learned relations improve only legend–mark correspondence relative to heuristics, while other tasks are comparable or slightly worse. Could the authors compare against a non-ontological graph representation with equivalent nodes and relation predictors, as well as a stronger independently trained relation-classification baseline? Are ontology constraints actively checked or enforced, and what measurable consistency benefit results? Q5. Can the error decomposition be reported with a consistent aggregation convention? Section 4.1 uses chart-macro averaging for recall and tuple-level aggregation for unmatched and matched-wrong rates. Could the authors additionally report all three quantities under a common tuple-level denominator, together with per-chart-type confidence intervals? This would clarify error attribution independently of averaging conventions.

1 comment
AuthorsOpenReview ↗

Response to 4o4H: spurious assertions, query scope, aggregation, independent images, and the ontology

Dear Reviewer 4o4H,

Thank you for an unusually specific review; several experiments below exist because of it. Your central point — that an auditable graph should also avoid unsupported assertions — is the one we build on, and we begin where our own reporting fell short of it: Table 4 is captioned "All are binding-tuple recall on the same scale." They are not — on series the three columns decompose to 0.8380.838, on legend to 1.0541.054 — and we will rewrite that caption.

Q1 / W1 — Would the metric penalise spurious assertions?

Yes — and the evidence was already in our numbers; we had printed only recall. Bootstrap 95% CI over 400400 charts: axis P/R/F1\mathrm{P}/\mathrm{R}/\mathrm{F}_1 0.925/0.920/0.922 [0.913,0.931]0.925/0.920/0.922\,[0.913,0.931], mark-series 0.499/0.484/0.490 [0.449,0.526]0.499/0.484/0.490\,[0.449,0.526], legend-mark 0.463/0.474/0.462 [0.422,0.501]0.463/0.474/0.462\,[0.422,0.501]. For your endpoint we injected false edges into a perfect graph:

InjectedPRF1
none1.0001.0001.000
10%0.9151.0000.956
50%0.6681.0000.801
100%0.5001.0000.667

Recall never moves — your point exactly: that case scores 0.6670.667, not 1.0001.000. We will report both views at §4.3 and plot the injection curve beside the triple scores.

Q2 / W1 — Queryability beyond the three predicates

In part: the suite is wider. Literal attributes answer non-empty at 1.0001.000, multi-hop at 0.890.89; both are Parse outputs, not Connect predictions, so they count as coverage. Two consistency checks show no violation on reference graphs. Compositional counting is unscored: no aggregation layer. Thus two of your three classes are scored and the third is bounded. In the revision this goes into §4.3, with the missing layer stated as a limitation at §5.

Q5 / W1 — Consistent aggregation

You are right — a real inconsistency. With one denominator every tuple is a hit, a miss or a match-wrong, and the three columns sum to 1.0001.000:

Taskhitmissmatched-wrongchart-macro recall
axis membership0.9180.0690.0130.920
mark-series0.6460.1970.1570.484
legend-mark0.4200.0410.5390.474

Thus the 0.6460.646 vs 0.4840.484 gap on mark-series is why the two conventions cannot be interchanged. The revised caption will name the three denominators and mark the last column as a chart-macro average.

Q3 / W2 — Generalization to independently rendered images

In part: we can score a corpus we did not render. verify-120, full open recovery, tuple-level:

TaskPRF1missmatched-wrong
overall0.5400.5370.5380.3740.089
axis membership0.6850.6860.6860.3050.009
legend-mark0.0530.1250.0740.2270.648

Legend is where we over-generate: 2.252.25 handles per reference. On the official CHART-Info set [1] (UB-PMC split_5, 396396 charts, image only):

SystemLF-6b
ours (zero-shot)0.339
ChartOCR [2]0.343
DePlot [3]0.188
UniChart [4]0.210

Extraction is level (−0.4-0.4). Thus extraction and binding fail in different places: on this corpus horizontal bars are the weakest type (0.0740.074, against 0.240.24–0.460.46 for the other three). Original ChartQA pixels stay unscored: a mask-aligned score needs instance-level ground truth. The revision will also delete the §4.1 limitation, report these numbers at §4.5, and state the missing real-distribution score at §5.

Q4 / W2 — The ontology and the learned Connect

The ontology now does measurable work; one gap stays open. Against the TBox axioms 4.34.3 percent of reference tuples violate domain, range or functionality. Injected violations are detected; in a controlled test on 6969 charts we inject a duplicate handle binding, enforce functionality, and 3737 of them return from wrong to right. The figures you cite are the recall column; on F1\mathrm{F}_1 the two overlap. Axis and series lie inside overlapping intervals (0.9220.922 vs 0.9340.934; 0.4900.490 vs 0.5090.509); legend is the one disjoint gap (0.4620.462 vs 0.3070.307). Thus the rule and the learned binder are indistinguishable on two predicates and separated on the third. We built no untyped twin, so we claim no architecture increment — only that typing supports checks an untyped graph cannot. §3.1 will state what the TBox buys, the checker and repair test will be reported as measurements, and §5 will state that the TBox is not a training constraint.

  • [1] Davila et al. ICPR 2020 competition on harvesting raw tables from infographics. ICPR, 2021.
  • [2] Luo et al. ChartOCR: Data extraction from charts images via a deep hybrid framework. WACV, 2021.
  • [3] Liu et al. DePlot: One-shot visual language reasoning by plot-to-table translation. Findings of ACL, 2023.
  • [4] Masry et al. UniChart: A universal vision-language pretrained model for chart comprehension and reasoning. EMNLP, 2023.
Reviewer_UAAEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ChartLogic, a framework that formulates chart understanding as queryable instance-graph recovery. It employs a two-stage Parse-and-Connect pipeline to recover chart elements and their semantic relationships, evaluated through SPARQL queries. The authors also construct two datasets: a synthetic chart dataset with aligned instance-level and graph annotations, and ChartQA-Redraw-120 for cross-source transfer evaluation. Experiments show that conventional PQ metrics overlook series identity errors, GT-Skeleton evaluation overestimates open-recovery performance, and instance localization degrades under cross-source transfer.

Critical strengths
  1. The paper proposes ChartLogic, a Parse-and-Connect framework that unifies visual instance parsing and relational grounding to enable queryable instance graph recovery.
  2. The authors construct two datasets: a synthetic chart dataset with aligned hierarchical label maps and ontology-typed ABox graphs, and ChartQA-Redraw-120 for cross-source transfer evaluation, providing structured supervision and evaluation resources for chart understanding.
Critical weaknesses
  1. The review of related work is insufficient. Several existing multimodal large language models (MLLMs)—both open-source and proprietary, such as Qwen3-VL and the GPT series—have demonstrated strong capabilities in chart comprehension and can output knowledge graphs in structured formats; however, the paper’s discussion of these approaches is limited.
  2. There is a lack of comprehensive comparison with existing methods. Although the authors note in the Conclusion that traditional ChartQA methods cannot be directly evaluated for structured relationship extraction due to differences in output formats, this does not justify the absence of other chart parsing or structured comprehension methods as baselines.
  3. Cross-domain evaluation is incomplete. The paper evaluates the "Parse" component only on the "verify-120" dataset and fails to report performance on relationship binding for "Open Recovery." While the authors argue that a low instance recall rate would cause query results to primarily reflect parsing errors, the failure to detect instances is, in itself, a critical aspect of end-to-end chart recovery performance.
Questions for authors
  1. It is recommended that the authors incorporate references to and analyses of recent, representative work, and further clarify the advantages of the proposed method over these existing MLLMs.
  2. The current experiments rely primarily on internal ablation studies and comparisons against heuristic-based rules, which are insufficient to fully demonstrate the advantages of ChartLogic over existing approaches.
  3. It is recommended that the authors include "Binding Recall" metrics under cross-domain conditions to provide a more thorough assessment of the framework's generalization capabilities.
1 comment
AuthorsOpenReview ↗

Response to UAAE: external baselines, precision, and cross-domain binding

Dear Reviewer UAAE,

Thank you for your careful review. Below we answer your three questions in order, with the two weaknesses they overlap folded into them. Each answer ends with the section where the change lands in the revision.

Q1 / W1 — Recent work, and what these models actually produce

You are right on both counts: our related work stopped short, and what was owed is a comparison rather than a citation list. What we can measure now: asked for a JSON instance graph, all three open VLMs return parseable JSON on all 400400 charts — once we fixed our own prompt and parser; the earlier parse failures were ours, not theirs. What they produce is a different matter:

Systemparsedaxismark-serieslegend-markscoring
ours (ChartLogic)—0.9220.4900.462instance-anchored
Qwen3-VL-8B400/4000.6490.4150.681semantic probe
Qwen2.5-VL-7B400/4000.5110.4960.539semantic probe
Llama-3.2-V-11B400/4000.2220.1230.082semantic probe

Two things follow. The 6464-chart subset we began with reads 0.9460.946 for Qwen3-VL against 0.6490.649 on all 400400, so we report the full set only. And the comparison does not flatter them: mark-series precision is 0.4520.452 against recall 0.7720.772 — the models over-generate bindings, which a recall-only metric would have recorded as success. On line and scatter charts, binding falls to 0.120.12 and 0.070.07 even with a legal format. Thus emitting a graph and recovering one are different capabilities, and the second is what our protocol measures.

The last column is the caveat, and we keep our own row in the same table rather than beside it. The probe credits a mapping stated in text without anchoring it to a detected handle and without charging for extra candidates, which flatters legend in particular; our row is scored on anchored matches, and we compare across that column nowhere.

Q2 / W2 — Comparison with existing methods

We replaced the argument with a measurement. Under our mask-aligned protocol ChartOCR scores axis 0.0000.000, mark-series 0.0920.092, legend-mark 0.0720.072 — not a coding error: its masks do not resolve to our instance anchors, which is the point. On a corpus we did not render — the official CHART-Info set [1], image only, 396396 charts — extraction is level with it, and our binding protocol reproduces the same diagnosis:

Systemextraction (LF-6b)
ours (zero-shot)0.339
ChartOCR [2]0.343
DePlot [3]0.188
UniChart [4]0.210

On that corpus the binding task gives axis macro-F1\mathrm{F}_1 0.4250.425 and mark-series 0.0530.053, with 0.960.96 of mark-series reference tuples unmatched rather than wrongly bound: Parse coverage again, not Connect. We also re-derived the precision side, and the same-mask heuristics are not better than the learned binder once intervals are reported — axis 0.9220.922 [0.913,0.931][0.913,0.931] against 0.9340.934 [0.925,0.942][0.925,0.942], mark-series 0.4900.490 [0.449,0.526][0.449,0.526] against 0.5090.509 [0.467,0.554][0.467,0.554], legend-mark 0.4620.462 [0.422,0.501][0.422,0.501] against 0.3070.307 [0.266,0.349][0.266,0.349]. Thus where we differ from prior systems is not extraction but whether a binding is recoverable at all. The revision will add the protocol table at §4.4 and the external corpus at §4.6.

Q3 / W3 — Cross-domain evaluation, binding as well as Parse

Done. Overall tuple-level P/R/F1\mathrm{P}/\mathrm{R}/\mathrm{F}_1 on verify-120 is 0.540/0.537/0.5380.540/0.537/0.538. 0.3740.374 of reference tuples are unmatched instances, so Parse misses do dominate. But legend–mark shows the opposite failure: 2.252.25 predicted handles per reference handle and 0.6480.648 of tuples matched to the wrong object. A Parse-only cross-domain table cannot see that, because it is Connect-side and over-generated rather than missed. Thus the evaluation we declined would have surfaced a Connect-side failure that Parse-only metrics cannot see. We will delete that sentence and report these numbers at §4.5.

  • [1] Davila et al. ICPR 2020 competition on harvesting raw tables from infographics. ICPR, 2021.
  • [2] Luo et al. ChartOCR: Data extraction from charts images via a deep hybrid framework. WACV, 2021.
  • [3] Liu et al. DePlot: One-shot visual language reasoning by plot-to-table translation. Findings of ACL, 2023.
  • [4] Masry et al. UniChart: A universal vision-language pretrained model for chart comprehension and reasoning. EMNLP, 2023.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.