FULL-FORMULATION DC-OPF SYNTHESIS: A probe into how LLMs fail in physical modeling

We introduce full-formulation direct-current optimal power flow (DC-OPF) synthesis as a probe of how reliably large language models (LLMs) construct mathematical models of physical systems. The task requires an LLM to construct a complete executable formulation from a natural-language dispatch instruction and an explicit modeling contract, without a supplied OPF backbone. We construct a benchmark of 118 instructions over three PGLib systems and develop a certificate-based evaluator for instance-level formulation equivalence. Its bidirectional violation linear programs detect discrepancies in hard feasible regions even when candidate and reference optimal values agree. We precisely specify the mathematical objects assessed by the evaluator and the scope of certification. Across nine LLMs, 244 of 1,062 outputs fail the certificate checks. A source-level fidelity review of all 244 failed outputs identifies substantive modeling errors in every evaluated model. These findings motivate a different allocation of modeling responsibilities: the LLM interprets the operational meaning of a natural-language requirement, while deterministic software binds it to concrete network entities and instance-dependent data and constructs the complete DC-OPF. A protocolized workflow implementing this allocation raises aggregate certificate passes from 818/1,062 to 1,054/1,062.