acceptodds
Under review as a conference paper at ICLR 2027

OptGraph: Large Language Models-Enhanced Automated Optimization Modeling via Graph Retrieval-Augmented Generation

Abstract

Large language models (LLMs) have emerged as a powerful tool for automated optimization modeling, but existing methods remain limited in pattern reuse, error-aware refinement, and retrieval robustness across diverse tasks. To address these limitations, we propose OptGraph, the first optimization agentic workflow that integrates graph retrieval-augmented generation (GraphRAG). Specifically, OptGraph first constructs reusable experience as a typed graph, capturing the relationships among modeling patterns, problem formalization, implementation details, and error corrections. In the inference stage, OptGraph leverages graph neighborhood information to enrich retrieved knowledge, providing structured context to improve modeling, verification, and iterative refinement. Moreover, OptGraph supports adaptive knowledge updates, enabling the distillation of execution traces and verification feedback into reusable graph knowledge without undertaking LLM parameter tuning. Experiments on benchmark datasets show that our OptGraph improves average exact accuracy by 8.9% over state-of-the-art prompt-based automated optimization modeling frameworks.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Automated optimization modeling, large language models, graph retrieval-augmented generation, agentic workflow

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_RUF2OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes OptGraph for translating natural-language optimization problems into mathematical formulations and executable solver code. It stores reusable knowledge about problem types, modeling variants, formulations, code snippets, common errors, and repair hints in a heterogeneous graph. For each problem, it retrieves relevant graph nodes to guide formulation and code generation, executes the generated code, uses a separate LLM to validate the result, and performs up to three rounds of repair when needed. The framework can also update the graph using execution traces and validation feedback. The experiments across 6 optimization-modeling datasets demonstrated the effectiveness of OptGraph.

Critical strengths
  1. OptGraph addresses several connected stages of optimization modeling: interpreting a natural-language problem, constructing a mathematical formulation, generating executable solver code, and repairing errors. Its graph representation brings formulations, implementations, and common failure patterns into one workflow.
  2. The paper evaluates the method on six datasets with different problem structures and difficulty levels. It reports gains over the listed baselines, while the ablation study provides initial evidence that both graph-based retrieval and validation contribute to performance.
  3. The paper examines retrieved knowledge, repair outcomes, and the use of GraphRAG within other modeling frameworks. It also reports token consumption and runtime, giving readers information about the computational cost associated with the reported gains.
Critical weaknesses
  1. Section 3.1 permits benchmark instances, formulations, and code as graph resources, while Algorithm 1 updates the graph after each evaluated problem. The paper does not specify source splits, overlap checks, test order, or frozen-graph results. This is an unresolved leakage and evaluation-protocol risk, not proof that leakage occurred.
  2. Equation (9) does not explain how the Gemini-based verifier detects semantic or constraint errors without an oracle. Executability or matching an optimal value does not establish formulation correctness. False-accept and false-reject rates are needed.
  3. The relevance score in Equation (3) is unspecified, and Equation (5) retrieves one-hop neighbors. Table 2 lacks content- and token-matched flat-retrieval and no-update controls.
  4. The backbone and inference budgets of the competing systems are not specified. OptGraph uses about 6.5 times the tokens and 5.5 times the latency of Lean-LLM-OPT in Table 3. The comparison with OptiTree (NeurIPS 2025) needs matched conditions; OR-R1 (AAAI 2026) is discussed but not evaluated, so broad state-of-the-art claims should be qualified.
  5. Several reporting details need correction. The 8.9-point gain is described as “8.9%”; NL4Opt exact accuracy ties OptiMUS rather than exceeding it; and the unusually large 67.8%/92.5% exact/tolerance gap in one Table 2 cell warrants an audit. Figure 3 should also label its CVRP formulation as partial or include the missing routing constraints.
  6. Figure 2 has rendering issues. Several mathematical symbols in the “Model Formulation” panel appear as garbled characters, making the example difficult to read. The authors should correct the font or export settings and verify the figure in the final PDF.
Questions for authors

See weakness.

Reviewer_mrRbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

OptGraph is a training-free workflow for optimization modeling. It stores modeling knowledge in a typed graph (problem types, formulations, code, typical errors, fix hints), retrieves relevant subgraphs to generate a model and solver code, checks the result with a Gemini judge for up to three repair rounds, and writes execution feedback back into the graph. Across six benchmarks it reports 75.4% / 83.2% average exact / 5%-tolerance accuracy, 8.9 points above the best prompt-based baseline.

Critical strengths
  1. The graph schema follows the stages and failure modes of OR modeling, which carries more structure than isolated few-shot examples.
  2. The reported results are strong, with the largest margins on the harder IndustryOR, MAMO-Complex and OptMATH.
Critical weaknesses
  1. As released, the code does not appear to run GraphRAG. WorkFlow.py imports the retriever from a non-existent GraphRAG/ directory, silently catches the error and uses an empty context. As far as I can tell, no graph retrieval would happen in this setup, and the adaptive update exists only as an offline script.
  2. Benchmark information may be in the graph. The released graph JSON says all six evaluation sets were inspected and contains a MAMO-Easy-specific variant and adaptive nodes citing NL4Opt evidence_problem_ids. The paper gives no protocol separating graph construction from test data.
Questions for authors
  1. Can you rerun all six benchmarks with a graph that provably excludes test problems, labels and traces?
  2. What code produced Tables 1-3, and was GraphRAG actually loaded? What is the accuracy under Algorithm 1 with ID-based scoring?
Reviewer_3n6GOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes OptGraph, an agentic workflow for automated optimization modeling that integrates GraphRAG. The core idea is to represent reusable modeling experience as a heterogeneous graph connecting problem types, variants, formulations, code snippets, typical errors, and fix hints, retrieve connected subgraphs to guide formulation/code generation, and use a validation-guided repair loop to iteratively correct failures. Execution traces and validation feedback are distilled into new graph nodes/edges to enable non-parametric, adaptive knowledge accumulation. Across six benchmarks, the authors report substantial gains over strong prompt-based systems, with especially large improvements on complex datasets.-

Critical strengths
  1. Broad evaluation on six diverse datasets (NL4OPT, IndustryOR, OptiBench, MAMO- E/MAMO-C, OptMATH), comparing to multiple strong agentic baselines and large standalone LLMs.

  2. Ablation studies tease apart GraphRAG and validation contributions; repair success and trigger rates are reported; plug-and-play tests show GraphRAG benefits other frameworks.

Critical weaknesses
  1. The retrieval scoring function S(Piv)S(Piv), edge types, and traversal details are underspecified, including embedding models, ranking metrics, and neighborhood expansion constraints. This makes it difficult to reproduce the method or assess its robustness.

  2. The validation function V(⋅)V(\cdot) claims to check objective consistency and constraint completeness, but does not concretely define how these checks are implemented for arbitrary tasks. The paper should clarify how much of the validation relies on solver-derived checks versus LLM judgments.

  3. The evaluation protocol for graph updates is unclear. The method updates the graph using execution traces, but it is not clear whether the results in Table 1 use a frozen graph or incorporate traces from earlier test instances. This complicates the comparison with baselines that do not adapt online.

Questions for authors
  1. How is the retrieval score $S(Piv) $ computed, and what values are used for the retrieval size kk and neighborhood expansion depth?
  2. How are objective consistency and constraint completeness checked in practice? Could you provide one example showing the validation feedback and the resulting repair?
  3. Were the results in Table 1 obtained with a frozen graph or with online updates across test instances? Please clarify the data used to initialize the graph and whether it is reset between datasets.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.