Learning to Understand Symbolic Drawings with Structure-Guided On-Policy Distillation

Vision language models (VLMs) perform well on natural images and documents but often struggle with complex symbolic drawings, such as mechanical CAD drawings and mathematical geometry diagrams. These drawings encode information through sparse geometry, numerical annotations, multiple views, and formal relations, requiring accurate object identification and annotation binding. We first study structure-assisted reasoning, where a question-independent structured description is provided alongside the image to explicitly represent geometric elements, dimensions, cross-view correspondences, and constraints. Experiments show that such information consistently improves question-answering performance, suggesting that many failures arise from unstable visual-to-structural interpretation. We further propose Structure-Guided On-Policy Distillation (SG-OPD) to transfer this benefit into an image-only model. During training, a structure-assisted teacher provides token-level supervision on trajectories generated by the image-only student. Experiments on mechanical CAD drawings and mathematical geometry diagrams show that SG-OPD consistently improves image-only baselines without requiring structured inputs at inference time. These results demonstrate that explicit structure can facilitate symbolic visual reasoning and be effectively internalized through on-policy distillation.