SGAnalog: An End-to-End Circuit Benchmark from Open-Source Silicon Tapeouts
Abstract
Existing analog integrated circuit design benchmarks make two questions hard to answer: whether a model has learned transferable circuit skills rather than recalled familiar examples, and whether its output works under defined process and test conditions. We introduce SGAnalog, a benchmark built from human-designed, open-source circuits associated with Tiny Tapeout manufacturing shuttles. The collection holds 425 top-level designs spanning 273 distinct circuit topologies; the fixed evaluation sets hold 66 transcription tasks and 17 sizing tasks. Every source is retrieved at the revision recorded for its shuttle submission and processed in a fixed containerized environment. The pipeline exports each schematic image and its SPICE netlist from the same source file, giving transcription an exact structural reference. Commit dates support model-specific trainingcutoff analysis, while author testbenches provide the simulation context for sizing. The benchmark evaluates schematic-to-netlist transcription and device sizing. Across seven models on the fixed transcription set, the strongest model reaches 56.1% exact graph isomorphism, and six of seven models drop sharply from the small to the medium tier. Removing author-chosen labels lowers exact matches for every model above floor, with larger drops for stronger models, while parameter accuracy holds, indicating that labels aid connectivity tracing. On the sizing set a different model leads. It converges on every proposal and reaches 91.2 out of 100 against the human reference (85.1 on the ten tasks with a scalar metric), while the two newest Claude models refuse 4 and 11 of the same prompts they transcribe without objection; a proposal without sizes scores zero. The two tasks produce different model rankings, exposing distinct visual and design capabilities and, in one family, a policy rather than capability limit.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.