- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper studies the efficiency of LLM-based optimization modeling (converting optimization problems given in natural language into math models and corresponding solver code). Rather than using conventional autoregressive language models, they propose to use diffusion language models. To achieve this, they design an intermediate representation to handle variable semantics, relation structure, and numerical grounding. On this basis, the masked diffusion can reconstruct multiple decisions at the same time and revise selected parts through remasking, and thus can generate contents in a faster way. They also conduct experiments to show the effectiveness of their method, and conduct ablation studies to show the necessity of each design component.
Critical strengths
-
This paper is very easy to follow: the writing is smooth and the sections are well organised.
-
The topic (efficiency of optimization modeling) is very interesting, but this topic is under-explored in the optimization modeling community.
Critical weaknesses
-
[Design Choice]: "we design an intermediate representation (IR) that separates them while preserving their dependencies through shared semantic declarations." Is this design necessary? This kind of design (namely explicitly modeling intermediate representatio/IR) appeared at the early stage of LLM-based automated optimization modeling (say 2022 or 2023). However, few researchers employ this kind of design as they found that: using IR is not necessary and it may degrade the performance of LLM in optimization modeling (I did not remember which paper(s) exactly. But from the trend of previously published papers in this direction, we know that few studies employs IR to help optimization modeling anymore).
-
[Selection of Baselines]: The selection of baselines is confusing. For commercial/open LLMs with APIs, they only compare it with GPT-4o (which is a very old model in the year of 2024) and Qwen3-3B. Why not more recent models such as GPT5.4, 5.5, 5.6, 6? Why not Claude families? Why not Gemini families (especially the Gemini Flash Series)? Why not DeepSeek families (DeepSeek Flash Models:)? Besides, for a given model such as GPT-5.6, how to set the thinking mode (low, medium, high, ultra) as we know that more thinking need more generation time? Besides, why SIRL and OptMATH (both are open-sourced) were omitted for comparison?
-
[Considering only Generation Time is Not Enough]: The efficiency of optimization modeling should be measured in at least two dimensions (see SAGE:https://arxiv.org/pdf/2605.02545): 1) the time to generate solver-executable code and 2)the time to execute it with Gurobi/COPT. However, this paper only considers/compares the time of generation (without consideirng the time of generation), which can be misleading. For example, if an LLM (say GPT-5.6-Sol-Ultra) takes 5 minutes to generate code and the generated solver code takes 1 minute to solve; On the contrary, OptDiff takes 1 seconds to generate the solver code, but takes 1 hour to run the solver code. I think most users will perfer GPT-5.6-Sol-Ultra in this case.
-
[Insufficient Benchmarks]: Most papers in optimization modeling in the year of 2026 perform experiments on at least 7/8 benchmarks (see StepORLM (https://arxiv.org/pdf/2509.22558), SAGE(https://arxiv.org/pdf/2605.02545), optiTree(https://arxiv.org/pdf/2510.22192)). The authors only conducted experiments on 4 benchmarks. Therefore, I do not trust the generalibility of their conclusions.
-
[How to Handle Large-Scale Optimization Problem?] 1) In large-scale optimization modeling scenario, the data and problem description are usually stored separately: How can the diffusion LLMs handle it? (Adding the data into the problem description will lead to a context length of millions tokens or even more); 2) in large scale optimization modeling, the number of variables and/or constraints can be millions or even more, how to handle this? (This weak point is not a specific weak point to this paper, but to almost all optimization modeling papers. So I did not consider this point when I gave the ratings. However, considering that this paper is tackling the problem of efficiency, I think this is closely related).
I am willing to raise my ratings if these concerns are (partially) addressed.
Questions for authors
Please check the weak points.
2 comments
Response to Reviewer 3GUP — Part 2/2
CW3: "Considering only Generation Time Is Not Enough"; "the time to generate solver-executable code" and "the time to execute it with Gurobi/COPT".
Compilation and complete execution time.
Thank you for this helpful suggestion. We report supplementary end-to-end timing results in Table 3.3 below, showing that OptDiff retains its speed advantage after compilation and solving are included. Specifically, we conducted additional timing experiments on NL4Opt and MAMO-ComplexLP, comparing OptDiff with five baselines under matched solver settings, CPU resources, and time limits. OptDiff's deterministic compiler converts the generated IR into executable solver code (Sec. 3.5). Process Overhead covers process management, library loading, file/result handling, and coordination between stages.
End-to-end results and accounting.
OptDiff achieves the lowest end-to-end latency among the six methods: 2.852 s on NL4Opt and 9.050 s on MAMO-ComplexLP, corresponding to 1.95× and 5.94× speedups over Qwen3.5-9B + SFT. Its mean solving time is 0.005 s on each dataset, indicating that the generation savings are not offset by solver execution. End-to-end latency is measured directly through completion or termination, with failures and timeouts included in the averages. Repair-related operations are assigned to their respective stages without double counting. We have supplemented Appendix B with this end-to-end table and a short explanation of the stage definitions and timing scope.
Table 3.3. End-to-end latency and stage-wise breakdown. All times are in seconds.
NL4Opt (N = 213)
| Method | Gen. | Extract & Check | Compile & Build | Solve | Process Overhead | End-to-end |
|---|---|---|---|---|---|---|
| GPT-5.6-Sol (Medium) | 28.821 | 0.001 | 0.024 | 0.005 | 0.025 | 28.876 |
| StepORLM (8B) | 21.541 | 0.002 | 0.036 | 0.005 | 0.028 | 21.612 |
| SIRL-Qwen2.5 (7B) | 13.175 | 0.001 | 0.046 | 0.006 | 0.033 | 13.261 |
| OptMATH-Qwen2.5 (7B) | 29.233 | 0.001 | 0.041 | 0.004 | 0.032 | 29.311 |
| Qwen3.5-9B + SFT | 5.508 | 0.002 | 0.005 | 0.006 | 0.030 | 5.551 |
| OptDiff (8B) | 2.806 | 0.003 | 0.006 | 0.005 | 0.032 | 2.852 |
MAMO-ComplexLP (N = 111)
| Method | Gen. | Extract & Check | Compile & Build | Solve | Process Overhead | End-to-end |
|---|---|---|---|---|---|---|
| GPT-5.6-Sol (Medium) | 72.744 | 0.002 | 0.025 | 0.004 | 0.034 | 72.809 |
| StepORLM (8B) | 30.315 | 0.008 | 0.034 | 0.005 | 0.033 | 30.395 |
| SIRL-Qwen2.5 (7B) | 29.620 | 0.002 | 0.045 | 0.005 | 0.035 | 29.707 |
| OptMATH-Qwen2.5 (7B) | 26.881 | 0.002 | 0.041 | 0.003 | 0.032 | 26.959 |
| Qwen3.5-9B + SFT | 53.709 | 0.006 | 0.008 | 0.003 | 0.034 | 53.760 |
| OptDiff (8B) | 8.998 | 0.004 | 0.008 | 0.005 | 0.035 | 9.050 |
CW4: "The authors only conducted experiments on 4 benchmarks."
Broader benchmark coverage.
Thank you for raising this concern. We have expanded the evaluation from four to eight benchmarks, adding ComplexOR, IndustryOR, ReSocratic, and OptMATH Bench. The additional results are reported in Table 3.1 below; we will include the full comparison and analysis in the appendix.
Table 3.1 (continued). Additional four benchmarks.
| Method | ComplexOR N=18 | IndustryOR N=100 | ReSocratic N=403 | OptMATH Bench N=166 | ||||
|---|---|---|---|---|---|---|---|---|
| Acc. (%) | Gen. (s) | Acc. (%) | Gen. (s) | Acc. (%) | Gen. (s) | Acc. (%) | Gen. (s) | |
| Zero-shot LLMs | ||||||||
| GPT-4o (closed) | 44.4 | 24.98 | 38.0 | 21.45 | 42.4 | 19.91 | 17.5 | 28.63 |
| GPT-5.6-Sol (Medium) | 50.0 | 64.17 | 50.0 | 60.67 | 87.3 | 31.71 | 28.9 | 59.99 |
| GPT-6-Sol (Medium) | 72.2 | 31.77 | 62.0 | 39.98 | 87.6 | 28.21 | 30.7 | 64.82 |
| Qwen3-32B | 27.8 | 66.74 | 35.0 | 80.07 | 68.0 | 54.39 | 14.5 | 87.56 |
| Fine-tuned LLMs | ||||||||
| ORLM-Llama3 (8B) | 50.0 | 10.62 | 35.0 | 16.98 | 75.4 | 13.07 | 3.6 | 26.29 |
| StepORLM (8B) | 38.9 | 23.30 | 42.0 | 38.76 | 82.4 | 22.70 | 12.7 | 75.48 |
| LLMOPT-Qwen2.5 (14B) | 27.8 | 29.93 | 31.0 | 34.23 | 80.1 | 24.10 | 5.4 | 54.84 |
| SIRL-Qwen2.5 (7B) | 55.6 | 18.33 | 39.0 | 25.37 | 81.4 | 14.85 | 25.3 | 53.09 |
| OptMATH-Qwen2.5 (7B) | 44.4 | 25.11 | 26.0 | 27.82 | 69.0 | 34.27 | 22.9 | 38.97 |
| Prompting and Agentic Methods | ||||||||
| Chain-of-Experts (GPT-4o) | 55.6 | 85.15 | 33.0 | 72.11 | 72.2 | 65.30 | 18.7 | 75.50 |
| OptiMUS-v0.3 (GPT-4o) | 50.0 | 82.73 | 44.0 | 79.02 | 87.6 | 70.22 | 22.3 | 81.54 |
| CAFA (GPT-4o) | 44.4 | 9.81 | 41.0 | 8.83 | 62.0 | 7.88 | 12.0 | 9.08 |
| Controlled Comparison | ||||||||
| Qwen3.5-9B + SFT | 38.9 | 7.75 | 36.0 | 18.88 | 68.7 | 6.03 | 5.4 | 83.81 |
| OptDiff (8B) | 50.0 | 3.83 | 40.0 | 4.51 | 80.1 | 2.21 | 21.7 | 6.60 |
CW5: "the data and problem description are usually stored separately. How can the diffusion LLMs handle it?" "the number of variables and/or constraints can be millions or even more, how to handle this?"
Large-scale data and model construction.
Thank you for highlighting this broader setting. Our evaluation covers text-specified benchmark problems; external-data interfaces and scalable model construction are future directions. This setting requires external-data integration, long-range dependency management, and memory-efficient construction of large variable and constraint systems. We will state this limitation and these directions in the Conclusion (Sec. 5).
Response to Reviewer 3GUP — Part 1/2
CW1: "Is this design necessary?"
Role of IR in diffusion.
Thank you for raising this design question. We acknowledge that an IR is not generally necessary for autoregressive optimization modeling. In contrast to AR approaches, our diffusion-based framework uses the IR to preserve dependencies among modeling decisions while enabling parallel generation and localized reconstruction, making it necessary for our proposed generation and repair workflow. When generating solver code directly, variable declarations and their dependent relations can be reconstructed simultaneously before the declarations are settled, making consistency harder to maintain. Our IR establishes shared variable declarations first, supports parallel objective and constraint generation under this shared context, and then binds numerical values. It also allows selected fields to be remasked and corrected while unaffected content remains fixed.
Evidence from the ablations.
Table 2 of the manuscript supports this design within the same iLLaDA backbone. After task SFT and adaptive denoising, cumulatively adding shared semantics, branch decoupling, and separate numerical binding raises NL4Opt accuracy from 69.0% to 89.7% and reduces generation time from 4.51 s to 2.44 s, before local repair. We will clarify this role of IR in Sec. 3.1.
CW2: "Why not more recent models such as GPT5.4, 5.5, 5.6, 6?"
Expanded comparisons and reasoning budgets.
Thank you for highlighting these gaps. We have added SIRL, OptMATH, and recent GPT baselines; the results show that OptDiff retains a generation-speed advantage across the evaluated methods and reasoning settings. We evaluate GPT-5.6-Sol and GPT-6-Sol at Medium reasoning effort in Table 3.1 and report Low, Medium, and High reasoning-effort results on the four original benchmarks in Table 3.2.
Comparison scope.
We compare our 8B diffusion model with similarly sized open models, including ORLM, StepORLM, and Qwen3.5-9B + SFT. GPT models add practical accuracy–latency reference points. Since their capacity, internal architecture, and serving resources cannot be matched to our 8B setup, we interpret their scores as system-level comparisons that complement the open-model results.
Accuracy and generation efficiency.
Across all eight benchmarks, OptDiff is both faster and more accurate than Qwen3.5-9B + SFT. Its speed advantage also holds against both GPT models on all eight benchmarks: on NL4Opt, for example, OptDiff takes 2.81 s, compared with 28.82 s and 26.28 s for GPT-5.6-Sol and GPT-6-Sol at Medium effort. We will add Tables 3.1 and 3.2 with this analysis to the appendix.
Table 3.1. Expanded baseline comparison and additional benchmark evaluation.
Original four benchmarks
Table 3.2. GPT models across reasoning effort levels.