- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes OptGraph for translating natural-language optimization problems into mathematical formulations and executable solver code. It stores reusable knowledge about problem types, modeling variants, formulations, code snippets, common errors, and repair hints in a heterogeneous graph. For each problem, it retrieves relevant graph nodes to guide formulation and code generation, executes the generated code, uses a separate LLM to validate the result, and performs up to three rounds of repair when needed. The framework can also update the graph using execution traces and validation feedback. The experiments across 6 optimization-modeling datasets demonstrated the effectiveness of OptGraph.
Critical strengths
- OptGraph addresses several connected stages of optimization modeling: interpreting a natural-language problem, constructing a mathematical formulation, generating executable solver code, and repairing errors. Its graph representation brings formulations, implementations, and common failure patterns into one workflow.
- The paper evaluates the method on six datasets with different problem structures and difficulty levels. It reports gains over the listed baselines, while the ablation study provides initial evidence that both graph-based retrieval and validation contribute to performance.
- The paper examines retrieved knowledge, repair outcomes, and the use of GraphRAG within other modeling frameworks. It also reports token consumption and runtime, giving readers information about the computational cost associated with the reported gains.
Critical weaknesses
- Section 3.1 permits benchmark instances, formulations, and code as graph resources, while Algorithm 1 updates the graph after each evaluated problem. The paper does not specify source splits, overlap checks, test order, or frozen-graph results. This is an unresolved leakage and evaluation-protocol risk, not proof that leakage occurred.
- Equation (9) does not explain how the Gemini-based verifier detects semantic or constraint errors without an oracle. Executability or matching an optimal value does not establish formulation correctness. False-accept and false-reject rates are needed.
- The relevance score in Equation (3) is unspecified, and Equation (5) retrieves one-hop neighbors. Table 2 lacks content- and token-matched flat-retrieval and no-update controls.
- The backbone and inference budgets of the competing systems are not specified. OptGraph uses about 6.5 times the tokens and 5.5 times the latency of Lean-LLM-OPT in Table 3. The comparison with OptiTree (NeurIPS 2025) needs matched conditions; OR-R1 (AAAI 2026) is discussed but not evaluated, so broad state-of-the-art claims should be qualified.
- Several reporting details need correction. The 8.9-point gain is described as “8.9%”; NL4Opt exact accuracy ties OptiMUS rather than exceeding it; and the unusually large 67.8%/92.5% exact/tolerance gap in one Table 2 cell warrants an audit. Figure 3 should also label its CVRP formulation as partial or include the missing routing constraints.
- Figure 2 has rendering issues. Several mathematical symbols in the “Model Formulation” panel appear as garbled characters, making the example difficult to read. The authors should correct the font or export settings and verify the figure in the final PDF.
Questions for authors
See weakness.