CONSTRAINT COVERAGE GUIDED HIDDEN ERRORS EXPOSURE FOR AUTOMATED OPTIMIZATION MODELLING

Optimization modeling translates real-world requirements into mathematical models and executable solver code, and large language models (LLMs) have shown promise in automating this process from natural-language descriptions. However, existing evaluations typically test a generated optimization model only on the original numerical instance and judge correctness by its optimal objective value. From a software-testing perspective, this resembles executing a program on a single input and checking only its final output: constraints irrelevant to the original optimum may remain untested. We introduce a constraint-coverage-guided validation paradigm for automated linear programming modeling, which assesses whether each target inequality constraint contributes to establishing optimality in at least one test instance. Inspired by coverage-guided software testing and inverse optimization, our framework treats constraints as test targets and searches prescribed parameter domains for problem variants that exercise previously uncovered constraints. It then reconstructs the generated variants as complete natural-language problems, producing an augmented test suite of paired problem descriptions and reference models. Experiments on public optimization-modeling benchmarks show that the augmented test suites substantially increase constraint coverage and expose hidden constraint-level modeling errors in optimization models produced by automated optimization modeling systems, even when these formulations yield correct objective values on the original instances. These results demonstrate that constraint coverage provides a more diagnostic complement to conventional answer-based evaluation.