OptoFig: Can Vision-Language Models Read Optoelectronics?

Vision–language models (VLMs) offer new opportunities for automating scientific research, including the design of optoelectronic devices. Realizing this potential requires understanding the figures through which such devices are designed and evaluated, from circuit schematics and device structures to the related performance, such as spectra, mode profiles and timing diagrams. However, existing benchmarks do not capture this capability. We introduce OptoFig, a benchmark of 1,092 questions over 799 figures of optoelectronic devices, covering both how the devices are built and their corresponding performance characterization. OptoFig is simulation-grounded and rigorously validated: every answer is measured on the figure itself, read from the simulator or drawing-script record, or written by a domain expert, and passes automated checks and multi-stage expert review. Its 39 task types span three capability categories: perception and recognition, quantitative and comparative, and physical and functional reasoning, with difficulty defined by construction parameters. Experimental results across seven state-of-the-art VLMs reveal substantial room for improvement: overall accuracy ranges from 54.2% to 74.5%, no single model leads across capabilities, and average accuracy is 10.2 points lower on hard items than on easy ones. The experiments further reveal two findings. First, models perceive figures with limited precision: they capture coarse visual patterns but miss the exact quantities a question depends on. Second, models lack the reasoning needed to interpret what they perceive: on the physics questions, even when a model reads a figure correctly, it draws the wrong physical conclusion about one time in three (67.3% correct on average). VLMs' limitations on these device figures thus lie both in the precision of perception and in reasoning from what they see to the underlying physics.