acceptodds
Under review as a conference paper at ICLR 2027

formal300: Evaluating Local-to-Global Procedure Reconstruction from FAA Approach Charts

Abstract

Professional charts and machine records encode the same procedure at different representational levels: charts distribute intent across text, symbols, and spatial geometry, whereas machine records require ordered and normalized fields. We introduce formal300, a benchmark for recovering canonical missed-approach procedure structure from 300 FAA approach charts with 964 normalized legs. Rather than treating extraction as independent field prediction, formal300 evaluates a hierarchy from atom correctness to exact legs and exact procedures under a frozen canonicalization and scoring contract. A chart-blind majority prior attains 58.12% atom accuracy but recovers no exact leg or procedure, showing that correct individual scoring atoms need not form a correct ordered procedure. A layout-aware domain-rule pipeline uses OCR layout and public procedure rules to assemble ordered legs, while an auxiliary-trained 2B-parameter vision-language system predicts canonical structures from chart images. Their errors on individual scoring atoms overlap only partially, and both achieve exact recovery for only a small fraction of complete procedures. Reference recoverability is supported by a blind 200-atom re-audit and a separate blind trained-reader calibration yielding 59/60 exact submissions on 30 charts. formal300 provides an auditable, controlled testbed for studying how distributed professional-chart evidence is converted into canonical ordered structure.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.