Parsing and Binding: Chart Understanding as Queryable Instance Graph Recovery

Chart understanding is commonly scored by answers or recovered tables. These metrics can look strong even when marks are merged or legend bindings are wrong, because they are blind to the underlying instance structure. We propose a structural view: recovery of a queryable instance graph, in which each claim remains grounded in masks and "what is seen” is scored separately from "how instances are bound.” We operationalize this in ChartLogic, an ontology-guided Parse-and-Connect model over shared instance anchors. Parse lifts pixels to instances with category, grouping, and identity; Connect binds them into relations among marks, axes, legends, and data structures. A shared generation source yields aligned images, hierarchical label maps, and ontology-typed ABox graphs, providing supervision for both stages. At evaluation, task-conditioned SPARQL templates are run on a graph built only from predicted instances, with a ground-truth skeleton as a diagnostic. Experiments on synthetic charts and on verify-120 (ChartQA-Redraw-120)—a human-calibrated, re-rendered ChartQA validation subset—yield three observations. First, ontology-aligned hierarchical supervision recovers the structured instance identity that enables downstream binding. Second, GT-skeleton scores remain substantially higher than open recovery even when many query subjects are missing from the predicted graph; open recovery failures decompose into unmatched instances and wrong bindings among matched ones, so Parse and Connect must be attributed separately. Third, under transfer, category accuracy stays high while geometric recall falls first, a category-vs-geometry gap that also appears on verify-120. Together, these observations suggest that content-level and GT-skeleton scores can mask an instance-topology failure that only open recovery reveals.