acceptodds
Under review as a conference paper at ICLR 2027

LLMs as Adaptive Meta-Solvers: Strategy-Diverse RL for Industrial-Scale Optimization

Abstract

Scaling LLM-based optimization from textbook-scale instances to real-world, industrial tasks remains a critical open challenge. Existing approaches are predominantly evaluated on small, self-contained textual problems and often commit to a solver-integrated paradigm, limiting their ability to handle the scale and structural diversity of practical optimization workloads. In this work, we propose a practical framework for training open-source LLMs to tackle real-world, industrial-scale optimization. We first show empirically that solver-integrated reasoning, exact combinatorial algorithm, and heuristic search exhibit complementary strengths across different problem structures and scales. Motivated by this, we introduce Strategy-Diverse Reinforcement Learning (SDRL), which trains LLMs as adaptive optimization meta-solvers. SDRL leverages this complementarity through a correctness-gated hierarchical diversity reward that promotes robust exploration across varying strategies and within each strategy, effectively preventing premature strategy collapse. We further introduce a mixed-format training scheme that jointly supports both self-contained textual problems and file-grounded instances. Across comprehensive evaluations, our framework outperforms existing fine-tuned methods and frontier models including DeepSeek-V4-Pro and GPT-5.5, both on average across benchmarks and on industrial-scale optimization tasks.

Primary area:
optimization
Keywords:
Optimization Modeling, Large Language Models, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_LoAkOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
3(+1)
Presentation
2
Soundness
3(+1)
Read review
Paper summary

The paper proposes Strategy-Diverse Reinforcement Learning, combining correctness-gated rewards for strategy rarity and runtime-mode diversity with GRPO. Mixed textual and file-grounded training improves optimization performance across seven benchmarks.

Critical strengths
  • Strategy complementarity provides a clear motivation for adaptive optimization meta-solvers.
  • Reward ablations, a same-backbone SIR-only RL baseline, and format-controlled comparisons provide meaningful evidence for the proposed components.
  • Improvements over the base models on file-grounded MIPLIB-NL are substantial, and training details and prompts are documented extensively.
Critical weaknesses
  • The stated correctness criterion checks execution and objective-value agreement, but does not describe independent validation of returned assignments against the original constraints. Objective agreement alone does not establish primal feasibility.
  • Runtime clusters and keyword-based strategy checks are imperfect proxies for algorithmic diversity, without a reported reliability audit. The runtime analysis compares different sets of successful programs, limiting its ability to isolate the micro-reward’s effect.
  • Training-seed variation and confidence intervals are absent. This particularly weakens the claimed advantage over GPT-5.5: 0.1 percentage points on average and 0.9 points on MIPLIB-NL. Common execution limits for the main comparisons also need clearer specification.
Questions for authors
  • Are returned assignments independently checked for feasibility and objective consistency? What execution limits and solver settings are shared across models?
  • Can repeated runs and an audit of strategy labels/runtime clusters establish that the gains reflect meaningful computational diversity?
5 comments
AuthorsOpenReview ↗

Response to Reviewer LoAk - Part 1

Rebuttal:

W1 & Q1: Feasibility Verification and Execution Settings

We sincerely thank the reviewer for this insightful comment and acknowledge that explicit feasibility verification is currently absent from our training framework.

In our experimental setup, we follow the standard convention in LLM-based optimization modeling: both training and evaluation verify predicted objective values within a relative tolerance of 10−610^{-6}, which is consistent with prior work and is substantially stricter than recent baselines such as StepORLM (0.050.05). Execution timeouts are set to 150s for training, and to 300s (self-contained benchmarks) or 1,800s (MIPLIB-NL) for evaluation, with a memory limit of 8,192 MB per program. For solver-based programs, we use Gurobi via gurobipy 13.0.2, restricted to a single thread (Threads=1).

Implementing feasibility verification across heterogeneous optimization tasks is highly problem-class dependent. For example, verifying a TSP solution requires parsing raw instance data (e.g., distance matrices) alongside domain-specific constraint validators for routes, or subtour. Because our training data is adapted from public text-based corpora (SIRL and OptMATH) that lack standardized executable verifiers and raw instance parsers, objective-value matching remains the most scalable reward signal for general-purpose RLVR.

To address the reviewer's concern regarding feasibility checks during evaluation, we report the evaluation results on the FrontierOR benchmark [1], which explicitly verifies constraint satisfaction on the predicted decision variables and jointly evaluates solution feasibility, quality, and efficiency (QTE):

ModelFull ExecutionFull FeasibilityFull Solution qualityFull QTEHard ExecutionHard FeasibilityHard Solution qualityHard QTE
Claude Opus 4.60.930.620.480.310.940.600.440.32
GPT-5.3-Codex0.980.600.480.260.980.490.300.18
Gemini 3.1 Pro0.930.610.520.251.000.640.440.22
DeepSeek-R10.740.420.310.170.820.370.200.11
Grok-4.20-beta0.740.280.220.130.760.200.140.06
Qwen3-Coder-Plus0.600.260.200.090.520.210.120.07
LLaMA-4-Maverick0.470.180.130.060.520.130.070.02
SIR-RL-Qwen3-32B0.860.220.150.090.820.150.120.10
SDRL-Qwen3-32B0.900.330.260.180.880.290.220.22

Baseline results are taken directly from FrontierOR.

As shown above, even though our framework was not trained under the FrontierOR protocol, it generalizes strongly to decision-variable feasibility evaluation. With identical training data, SDRL-Qwen3-32B improves feasibility over SIR-RL-Qwen3-32B from 0.22 to 0.33 on the Full split and from 0.15 to 0.29 on the Hard split, and roughly doubles QTE on both splits (0.09 → 0.18 and 0.10 → 0.22). It also outperforms Grok-4.20-beta and LLaMA-4-Maverick on all metrics, and its QTE matches or exceeds that of DeepSeek-R1 (0.18 vs. 0.17 on Full; 0.22 vs. 0.11 on Hard).

AuthorsOpenReview ↗

Response to Reviewer LoAk - Part 2

W2 & Q2: Reliability of Diversity Measurements

First, we acknowledge that runtime clusters serve as an imperfect yet highly efficient proxy for computational diversity. Execution time is collected for free during correctness verification, whereas invoking a mainstream LLM judge during training would add prohibitive token cost and inference latency.

To address the reviewer's concern, for the macro-reward reliability, we apply LLMs-as-judge verification checks. An offline GPT-6-Astra audit of randomly sampled training rollouts showed 99.5% label agreement among successful programs with comparable labels. For example, in one TSP rollout, the model declared Exact Combinatorial Algorithm in its reasoning tag but actually invoked Gurobi with Miller–Tucker–Zemlin (MTZ) constraints in code. GPT-6-Astra flagged it as solver-based, and our rule-based verifier (Appendix C.3) similarly rejected the mismatched tag as unknown, assigning zero diversity reward despite the program achieving the correct objective value.

For the micro-reward efficacy: We can directly assess the effectiveness of the runtime micro-reward from our ablation study (Appendix C.6), where entries report Avg@8 (%) ± standard deviation:

MethodMAMO-CIndustryOROptMathMIPLIB-NL
Base model42.6 ± 2.738.3 ± 3.216.1 ± 1.86.0 ± 1.1
SDRL w/o diversity reward71.2 ± 1.851.8 ± 1.834.6 ± 1.912.3 ± 1.5
SDRL w/o micro reward72.6 ± 2.550.1 ± 1.835.8 ± 2.313.4 ± 1.0
SDRL w/o macro reward75.6 ± 1.250.8 ± 2.034.8 ± 1.613.6 ± 1.3
Full SDRL75.7 ± 1.252.6 ± 2.035.6 ± 2.115.2 ± 1.1

As shown in the table, Full SDRL outperforms the micro-reward ablation on three of the four benchmarks (including a +3.1 point gain on MAMO-C and +1.8 points on MIPLIB-NL). Furthermore, when tackling large-scale problems like MIPLIB-NL, pairing high-level strategy diversity with runtime efficiency is critical: even within the same broad algorithmic strategy, different formulations or implementations can incur vastly different computational overhead (15.2 vs. 12.3).


W3: Variance and Comparison with GPT-5.5

Here we expand the comparison using eight samples per instance, reporting Avg@8 (%) ± the standard deviation of accuracy across the eight sampling runs (percentage points), with a relative-error tolerance of 10−610^{-6}.

ModelNL4OptMAMO-EMAMO-CInd-OROptMathOptiBenchMIPLIB-NLAvg.
SDRL-Qwen3-4B91.0 ± 1.491.1 ± 0.679.6 ± 1.653.8 ± 2.840.0 ± 1.367.9 ± 0.822.2 ± 1.863.7
SDRL-Qwen3-32B95.7 ± 0.696.0 ± 0.280.7 ± 1.356.0 ± 2.557.9 ± 1.568.1 ± 0.728.9 ± 0.969.0
GPT-5.594.4 ± 1.089.8 ± 0.988.1 ± 1.365.8 ± 2.342.9 ± 1.367.9 ± 0.428.2 ± 1.668.2

SDRL-Qwen3-32B achieves a higher average accuracy than GPT-5.5 (69.0% vs. 68.2%), with higher mean accuracy on five of the seven benchmarks.


Reference

[1] Minwei Kong, Chonghe Jiang, Ao Qu, Wenbin Ouyang, Zhaoming Zeng, Xiaotong Guo, Zhekai Li, Junyi Li, Yi Fan, Xinshou Zheng, et al. (2026). FrontierOR: Benchmarking LLMs' Capacity for Efficient Algorithm Design in Large-Scale Optimization.


We sincerely appreciate the reviewer's constructive feedback and hope that we have addressed all of the comments above. We have made the training dataset, benchmark data, and evaluation code available in our anonymized repository: https://anonymous.4open.science/r/optimization-review-42D7/.

I thank the authors for their detailed responses, which have largely addressed my concerns. I remain positive about the paper and maintain my original score.

Reviewer_HEgJOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper trains LLMs as optimization "meta-solvers". For each problem, the model chooses one of three strategies (solver-based modeling, an exact algorithm, or heuristic search) and writes executable code, which is scored against a reference optimum. The method, SDRL, adds a correctness-gated diversity bonus to GRPO. The bonus rewards rare strategies across rollouts and rare runtime modes within a strategy. The paper also mixes textual and file-grounded training data. On seven benchmarks, including the industrial-scale MIPLIB-NL, SDRL beats prior fine-tuned models, and its 32B model is comparable to frontier LLMs.

Critical strengths

Practical file-grounded setting. The paper moves beyond prompt-embedded textbook problems to instances whose data live in external files. On MIPLIB-NL, SDRL-Qwen3-4B reaches 23.6 Pass@1, against at most 5.5 for prior fine-tuned models, and SDRL-Qwen3-32B (30.0) is comparable to frontier models (Table 1).

Critical weaknesses

Weaknesses

W1. Incremental contribution. LLM-based NL-to-solver-code generation (ORLM, OptMATH) and execution-verified RL fine-tuning (SIRL, StepORLM) are established. The new parts, the hierarchical diversity reward and mixed-format training, are reasonable but of limited novelty, and the gains are modest. Against SIR-RL, SDRL gains only +1.6 Pass@1 and +3.6 Pass@8 (Table 9). The diversity reward is heuristic: a self-information bonus plus DBSCAN on runtimes, with hand-set hyperparameters. All ablations use one backbone (Qwen3-4B) and report no seeds or variances, and Pass@1 is non-monotonic in λ (Table 8). It is unclear whether the method is robust across models.

W2. Absolute-error correctness with unreliable labels. Correctness uses |ô − o⋆|/|o⋆| with no direction and no feasibility check. This is only safe if o⋆ is truly optimal. OptiMind (arXiv:2509.22979) reports that 577 of 2,600 OptMATH training labels differ on re-derivation, and that 30–50% of OptMATH test instances have errors, including wrong optima. In that case, a solution better than a suboptimal label is scored as wrong, which penalizes exactly what CO should reward. A feasibility check on the decision variables plus a one-sided or gap-based score, and results on cleaned benchmarks, would address this.

Questions for authors

Q1. Reproducible code. Code and data are promised only "upon publication". To verify the experiments rigorously, could the authors share anonymized, runnable code during review? This should include the reward (with the consistency checks and DBSCAN), the sandbox and evaluation scripts, the exact training subset and benchmark versions, and the seeds and checkpoints for the Table 2 and Table 9 runs.

2 comments
AuthorsOpenReview ↗

Response to Reviewer HEgJ - Part 1

W1.1: Clarification of Contributions

We appreciate the reviewer's insightful comments and thanks for the opportunity to clarify our contribution. We fully agree that LLM-based optimization in the traditional self-contained setting, where all infos are embedded within the natural-language description and involving fewer than 50 parameters, has been largely addressed. The core challenge is exact mathematical formulation, and solver-informed methods can solve almost all instances within two minutes of execution time. Meanwhile, these benchmarks are nearing saturation.

However, when extending to large-scale industrial settings, such as MIPLIB-NL, which involves thousands to millions of decision variables. At this scale, the computational strategy and algorithm become the primary bottleneck: obtaining a feasible, high-quality solution within a reasonable runtime is far more valuable than relying on an exact solver that becomes bottlenecked or times out. For example, a 50-node TSP can be solved quickly with Gurobi in about two minutes, whereas with thousands of nodes an efficient heuristic such as LKH is preferable to an exact solver.

Notably, during the rebuttal period, Anthropic reported that Claude agents improved the best-known solutions on 72 of 203 open MIPLIB instances and proved optimality for 45, underscoring the strong potential of agentic LLMs in tackling challenging optimization problems. This milestone further validates MIPLIB-NL as a timely and valuable benchmark for evaluating agentic, LLM-based optimization. At the same time, it highlights the exact gap SDRL is designed to address: moving beyond passive, solver-dependent mathematical modeling toward adaptive, strategy-diverse optimization for industrial-scale tasks.


Q1: Reproducible Code

We sincerely thank the reviewer for emphasizing reproducibility. We have made the training dataset, benchmark data, and evaluation code available in our anonymized repository: https://anonymous.4open.science/r/optimization-review-42D7/. And we commit to publicly releasing the training code and model checkpoints upon publication.


W1.2 & W2.2 (Part 1) : Comprehensive Comparison and Feasibility Verification

We thank the reviewer for raising this point, as we honestly report a strong SIR-RL baseline to demonstrate its competitiveness.

First, as noted in the Limitations section, our primary evaluation enforces a strict objective-value tolerance of 10−610^{-6}, which inherently favors the solver-integrated paradigm where external solvers can easily find exact optima, while potentially undervaluing the near-optimal solutions produced by combinatorial algorithms and heuristic search.

Second, under this strict criterion, scaling from Pass@1 to Pass@8 widens SDRL’s average advantage over SIR-RL from +1.6% to +3.6% (with +5% on MIPLIB-NL), reflecting stronger solution diversity, which is also presented in Figure 6.

MethodCriterionNL4OptMAMO-EMAMO-CIndustryOROptMathOptiBenchMIPLIB-NLAverage
SIR-RL-4BPass@194.792.870.448.036.166.814.160.4
SDRL-4BPass@193.992.275.953.036.765.816.862.0
GapPass@1−0.8−0.6+5.5+5.0+0.6−1.0+2.7+1.6
SIR-RL-4BPass@895.996.685.765.048.269.628.269.9
SDRL-4BPass@895.597.490.271.054.872.633.273.5
GapPass@8−0.4+0.8+4.5+6.0+6.6+3.0+5.0+3.6

Regarding W2.2, we fully agree that verifying decision-variable feasibility and moving beyond strict objective-value matching provide a much more rigorous evaluation. To address this, we extended our experiments on both 32B models to FrontierOR [1], a benchmark that explicitly checks constraint satisfaction on decision variables and jointly evaluates solution feasibility, quality, and efficiency (QTE):

ModelFull ExecutionFull FeasibilityFull Solution qualityFull QTEHard ExecutionHard FeasibilityHard Solution qualityHard QTE
Claude Opus 4.60.930.620.480.310.940.600.440.32
GPT-5.3-Codex0.980.600.480.260.980.490.300.18
Gemini 3.1 Pro0.930.610.520.251.000.640.440.22
DeepSeek-R10.740.420.310.170.820.370.200.11
Grok-4.20-beta0.740.280.220.130.760.200.140.06
Qwen3-Coder-Plus0.600.260.200.090.520.210.120.07
LLaMA-4-Maverick0.470.180.130.060.520.130.070.02
SIR-RL-Qwen3-32B0.860.220.150.090.820.150.120.10
SDRL-Qwen3-32B0.900.330.260.180.880.290.220.22

Baseline results are taken directly from FrontierOR.

AuthorsOpenReview ↗

Response to Reviewer HEgJ - Part 2

W1.2 & W2.2 (Part 2): Comprehensive Baseline Comparison and Feasibility Verification

As we can see, when accounting for feasibility and efficiency, SDRL-Qwen3-32B roughly doubles the QTE of SIR-RL-Qwen3-32B (0.09 → 0.18 on Full; 0.10 → 0.22 on Hard) and substantially improves feasibility. It also outperforms much larger models such as LLaMA-4-Maverick and Grok-4.20-beta across all metrics, and doubles the Hard QTE of DeepSeek-R1 (0.22 vs. 0.11).

Meanwhile, we also conducted experiments on the cleaned benchmark subsets used in prior work such as StepORLM, where erroneous instances are filtered out (e.g., retaining 42 instances for IndustryOR). We did not adopt these filtered subsets as our primary benchmarks for two reasons: (1) removing challenging edge cases makes them substantially easier and largely saturated, reducing their discriminative power; and (2) for IndustryOR specifically, the original authors have already released an officially revised version of the dataset.

Table: Pass@1 accuracy (%) on the cleaned benchmarks used by StepORLM

ModelNL4OPTMAMO EasyLPMAMO ComplexLPNLP4LPCompORIndORReSocraticAvg.
Qwen3-4B-Instruct70.968.342.380.344.454.876.762.5
Qwen3-32B84.093.878.493.350.054.883.676.8
StepORLM-Qwen3-8B†96.797.6‾\underline{\mathsf{97.6}}77.597.250.052.481.979.0
OptMath-Qwen2.5-32B†95.9‾\underline{\mathsf{95.9}}*89.9*54.1*—————
SDRL-Qwen3-4B87.893.890.197.8‾\underline{\mathsf{97.8}}61.171.494.0‾\underline{\mathsf{94.0}}85.1‾\underline{\mathsf{85.1}}
SDRL-Qwen3-32B91.598.789.2‾\underline{\mathsf{89.2}}98.355.6‾\underline{\mathsf{55.6}}69.0‾\underline{\mathsf{69.0}}96.585.6

Note: † denotes results from the StepORLM paper (* from original publications); other results are from our evaluation. Best scores are in bold, and second-best scores are underlined; — denotes results not reported. Avg. is the mean over the seven benchmarks.

On these cleaned subsets, both SDRL-Qwen3-4B (85.1%) and SDRL-Qwen3-32B (85.6%) substantially outperform StepORLM’s reported average Pass@1 (79.0%).

All evaluation code and inference scripts are now available in our updated anonymous GitHub repository.


W2.1: Label Reliability and Feasibility check of training data

We fully agree that label noise in training datasets (such as OptMATH) and purely symmetric objective-value matching present real limitations—both for reward reliability during training and evaluation.

To mitigate label errors before RLVR training, we enforce strict multi-stage filtering pipelines across our training data: (1) for Self-contained data, we perform 5-rollout self-consistency verification using Qwen3-32B and retain only instances exhibiting moderate consensus (1 to 4 consistent rollouts). This removes both mislabeled/unsolvable instances (0/5 consensus) and overly trivial problems (5/5 consensus). (2) For file-grounded data: We apply a three-stage verification pipeline (Appendix B.1.2): (a) a structural check verifying file existence; (b) a semantic audit by an independent LLM judge to ensure alignment between the problem description and formulation; and (c) solver backtesting to confirm solvability and label consistency.

While no automated pipeline can guarantee perfect labels, this multi-stage curation provides clean, high-signal supervision well suited for RLVR and consistent with current practices in the RLVR community.

Meanwhile, we acknowledge that feasibility checking is currently absent from our training pipeline. Implementing feasibility verification across heterogeneous optimization tasks is highly problem-class dependent. For example, verifying a TSP solution requires parsing raw instance data (e.g., distance matrices) alongside domain-specific constraint validators for routes, or subtour. Because our training data is adapted from public text-based corpora (SIRL and OptMATH) that lack standardized executable verifiers and raw instance parsers, objective-value matching remains the most scalable reward signal for general-purpose RLVR. To address the reviewer's concern on the evaluation side, we have expanded our experiments to include FrontierOR [1], which enforces explicit decision-variable feasibility checks, as detailed in our response to W1.2 & W2.2 above.


Reference

[1] Minwei Kong, Chonghe Jiang, Ao Qu, Wenbin Ouyang, Zhaoming Zeng, Xiaotong Guo, Zhekai Li, Junyi Li, Yi Fan, Xinshou Zheng, et al. (2026). FrontierOR: Benchmarking LLMs' Capacity for Efficient Algorithm Design in Large-Scale Optimization.


We sincerely thank the reviewer for the time and thoughtful feedback on the training and evaluation framework, and hope that our responses and additional experiments have addressed all of your concerns.

Reviewer_toSEOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper trains open models to solve natural-language optimization problems in three stages: analyze the problem, pick one of three paradigms (call Gurobi on a written formulation, write an exact combinatorial algorithm, write a heuristic), then implement it. Training is GRPO with verifiable rewards plus a diversity bonus paid only on correct rollouts, a macro term for rarely declared paradigms and a micro term for rare runtime clusters from 1-D DBSCAN on log runtimes. Text instances are also converted into file-grounded ones and mixed into training. Validation uses a set of NL-to-Opt benchmarks, including an industrial one, and two backbones. Ablations cover each reward level, the mixed data, the macro/micro weight, training-set size, and a single-paradigm RL control.

Critical strengths

The submission covers the full pipeline, routing among three solution paradigms, a reward that pays for correct diversity across and within them, and a data conversion from text to file-grounded instances.

It is evaluated on both self-contained and industrial file-grounded benchmarks at two model scales, with component ablations, weight and training-set sweeps, and sampling-budget curves.

No single piece is new, but the package is coherent and the evidence is broader than most submissions in this area provide.

Critical weaknesses
  1. RL without the diversity term already delivers most of the gain, and the single-paradigm control (SIR-RL) leaves only a slim margin for routing and diversity. No table reports seeds or confidence intervals, and the number of runs is never stated.

  2. Related work also covers AAD only through the LLM program-search line, with no non-LLM AAD comparison and no LLM-based AAD baseline, so it is unclear whether routing beats a simple selector.

  3. The motivation is per-instance complementarity, but there is no best-of-three bound, no agreement rate between the chosen and the best paradigm, and no failure rate for the keyword check that validates labels.

Questions for authors
  1. Is the RL stage full-parameter or PEFT? Please report trainable parameters, GPU hours and instances seen for both backbones.

  2. How many independent runs underlie the main and ablation tables? Please add standard deviations, in particular for the gap to SIR-RL and for the differences between the full model and the single-level variants.

  3. Which tolerance defines the successful set, and do rollouts with only near-optimal solutions enter it? Please also give the fraction of groups in which the micro term is non-zero, and the per-paradigm success rates.

  4. Please report per instance the best-of-three upper bound, the fraction of instances on which the chosen paradigm is the best one, and the pass rate of the rule-based strategy check.

  5. A non-trivial fraction of the converted instances fail at least one verification stage. Were they kept in training? If so, please report results from a model trained on the verified subset only.

  6. Which non-LLM AAD lines do the authors consider closest, and can a feature-based selector over the three paradigms be added as a baseline?

6 comments
AuthorsOpenReview ↗

Response to Reviewer toSE - Part 1

Rebuttal:

W1 & Q2.2: Comparison between SDRL and SIR-RL.

We thank the reviewer for raising this critical point, as we honestly report a strong SIR-RL baseline to demonstrate its competitiveness.

First, as noted in the Limitations section, our primary evaluation enforces a strict objective-value tolerance of 10−610^{-6}, which inherently favors the solver-integrated paradigm where external solvers can easily find exact optima, while potentially undervaluing the near-optimal solutions produced by combinatorial algorithms and heuristic search.

Second, under this strict criterion, scaling from Pass@1 to Pass@8 widens SDRL’s average advantage over SIR-RL from +1.6% to +3.6% (with gains up to +5% on MIPLIB-NL), reflecting stronger solution diversity, which is also presented in Figure 6.

MethodCriterionNL4OptMAMO-EMAMO-CIndustryOROptMathOptiBenchMIPLIB-NLAverage
SIR-RL-4BPass@194.792.870.448.036.166.814.160.4
SDRL-4BPass@193.992.275.953.036.765.816.862.0
GapPass@1−0.8−0.6+5.5+5.0+0.6−1.0+2.7+1.6
SIR-RL-4BPass@895.996.685.765.048.269.628.269.9
SDRL-4BPass@895.597.490.271.054.872.633.273.5
GapPass@8−0.4+0.8+4.5+6.0+6.6+3.0+5.0+3.6

Here is the supplementary comparison of Avg@8 as mean accuracy ± std. Complete Avg@8 results for all main and ablation experiments are provided in Appendix C.6 of the revision.

Supplementary comparison of SIR-RL and SDRL: Avg@8 ± standard deviation (%).

MethodNL4OptMAMO-EMAMO-CIndustryOROptMathOptiBenchMIPLIB-NL
SIR-RL-4B94.0 ± 0.792.6 ± 0.669.1 ± 1.751.0 ± 1.834.5 ± 1.664.0 ± 1.512.4 ± 1.6
SDRL-4B93.5 ± 0.892.5 ± 0.975.7 ± 1.252.6 ± 2.035.6 ± 2.165.1 ± 1.315.2 ± 1.1

Third, to evaluate both 32B models beyond strict objective-value matching, we extend our experiments to FrontierOR [1], which jointly evaluates feasibility, solution quality, and efficiency (QTE):

ModelFull ExecutionFull FeasibilityFull Solution qualityFull QTEHard ExecutionHard FeasibilityHard Solution qualityHard QTE
Claude Opus 4.60.930.620.480.310.940.600.440.32
GPT-5.3-Codex0.980.600.480.260.980.490.300.18
Gemini 3.1 Pro0.930.610.520.251.000.640.440.22
DeepSeek-R10.740.420.310.170.820.370.200.11
Grok-4.20-beta0.740.280.220.130.760.200.140.06
Qwen3-Coder-Plus0.600.260.200.090.520.210.120.07
LLaMA-4-Maverick0.470.180.130.060.520.130.070.02
SIR-RL-Qwen3-32B0.860.220.150.090.820.150.120.10
SDRL-Qwen3-32B0.900.330.260.180.880.290.220.22

Baseline results are taken directly from FrontierOR.

When accounting for feasibility and efficiency, SDRL-Qwen3-32B roughly doubles the QTE of SIR-RL-Qwen3-32B (0.09 → 0.18 on Full; 0.10 → 0.22 on Hard) and substantially improves feasibility. It also outperforms much larger models such as LLaMA-4-Maverick and Grok-4.20-beta across all metrics, and doubles the Hard QTE of DeepSeek-R1 (0.22 vs. 0.11).

AuthorsOpenReview ↗

Response to Reviewer toSE - Part 2

W2.1 & Q6.1: Comparison with LLM-Based AAD Methods.

While both LLM-based AAD and LLM-based optimization modeling generate executable optimization code, they target different settings: LLM-based AAD methods (e.g., AlphaEvolve, EoH [2], ReEvo) rely on test-time evolutionary search over hundreds of evaluator calls to refine a specific heuristic for a single, pre-formalized problem family. In contrast, optimization modeling (our focus) operates in a one-shot, cross-domain optimization class, synthesizing end-to-end solver code in a single pass.

To compare them directly, we converted EoH’s online bin packing, TSP, and flow shop scheduling instances into our file-grounded format and evaluated our SDRL-Qwen3-32B via single-pass inference under matching budget constraints:

Table 1. Online bin packing: excess bins over the lower bound (%). Lower is better.

Method1k_C1005k_C10010k_C1001k_C5005k_C50010k_C500
First Fit5.324.404.444.974.274.28
Best Fit4.874.084.094.503.913.95
FunSearch3.780.800.336.751.470.74
EoH2.240.800.612.130.780.61
SDRL0.710.430.310.000.000.00

Table 2. Traveling salesman problem: distance gap to the best-known solution (%). Lower is better.

Methodrd100pr124bier127kroA150u159kroB200
NI19.9115.5023.2118.1723.5924.10
FI9.384.438.048.5411.157.54
OR-Tools0.010.550.660.021.752.57
AM3.413.685.913.787.557.11
POMO0.010.6013.720.700.951.58
LEHD0.011.114.761.401.130.64
EoH0.010.000.420.000.000.20
SDRL0.002.371.080.980.003.74

Table 3. Flow shop scheduling: relative makespan gap (%). Lower is better.

Methodn20m10n20m20n50m10n50m20n100m10n100m20
GUPTA23.4221.7920.1122.7815.0321.00
CDS12.8710.3512.7215.039.3613.55
NEH4.053.063.475.482.073.58
NEHFF4.152.723.625.101.883.73
PFSPNet14.7814.6911.9516.958.2116.47
PFSPNet-NEH4.042.963.485.051.723.56
EoH0.300.100.190.600.140.41
SDRL2.281.707.044.361.324.06

Note: All baseline results are taken directly from EoH [2].

As we can see, despite operating in a one-shot, cross-domain setting without iterative evolutionary search, SDRL outperforms EoH on online bin packing and remains competitive on TSP and flow shop scheduling, offering a strong single-pass alternative to AAD.

AuthorsOpenReview ↗

Response to Reviewer toSE - Part 3

W2.2 & W3 & Q4 & Q6.2: Routing Strategy Comparison and Diagnostics

To provide a comprehensive and transparent comparison, we evaluate three single-paradigm baselines, a meta-paradigm baseline, a feature-based strategy selector, and an oracle Best-of-Three upper bound.

For the feature-based selector, we consider Jev [3], which maps each input problem to one of the three computational paradigms.

All methods use the same SDRL backbone, with four corresponding prompt templates: Solver-Integrated Reasoning, Exact Combinatorial Algorithms, Heuristic Search, and Meta-Solving. The single-paradigm baselines use their respective strategy-specific prompts, while the meta-paradigm baseline allows the model to select a strategy adaptively.

Finally, the Oracle Best-of-Three generates one candidate solution under each of the three paradigms per instance and marks the instance as solved if at least one succeeds.

SDRL-Qwen3-4B.

MethodNL4OptMAMO-EMAMO-CIndustryOROptMathOptiBenchMIPLIB-NLAvg.
SDRL + SIR prompt93.9‾\underline{\mathsf{93.9}}92.255.255.0‾\underline{\mathsf{55.0}}28.364.822.358.8
SDRL + Algorithm prompt86.188.056.253.038.064.69.156.4
SDRL + Heuristic prompt66.158.735.036.025.345.85.939.0
SDRL + Jev selector94.792.5‾\underline{\mathsf{92.5}}62.655.0‾\underline{\mathsf{55.0}}30.165.823.260.6
SDRL + Meta prompt93.692.479.3‾\underline{\mathsf{79.3}}55.0‾\underline{\mathsf{55.0}}41.0‾\underline{\mathsf{41.0}}67.1‾\underline{\mathsf{67.1}}23.6‾\underline{\mathsf{23.6}}64.6‾\underline{\mathsf{64.6}}
SDRL + Best-of-Three94.795.382.864.050.670.727.369.3
Routing agreement (%)98.897.095.885.981.094.986.491.4

Best results are bold; second-best results are underlined. Routing agreement is reported separately in the last row.

SDRL-Qwen3-32B.

MethodNL4OptMAMO-EMAMO-CIndustryOROptMathOptiBenchMIPLIB-NLAvg.
SDRL + SIR prompt94.794.573.954.054.267.128.666.7
SDRL + Algorithm prompt85.377.657.641.033.761.37.352.0
SDRL + Heuristic prompt67.367.147.326.027.742.86.840.7
SDRL + Jev selector94.394.980.858.0‾\underline{\mathsf{58.0}}55.467.327.768.3
SDRL + Meta prompt96.3‾\underline{\mathsf{96.3}}96.0‾\underline{\mathsf{96.0}}81.8‾\underline{\mathsf{81.8}}56.059.0‾\underline{\mathsf{59.0}}69.1‾\underline{\mathsf{69.1}}30.0‾\underline{\mathsf{30.0}}69.7‾\underline{\mathsf{69.7}}
SDRL + Best-of-Three96.796.484.260.063.371.934.172.4
Routing agreement (%)99.699.697.193.393.296.188.094.8

Best results are bold; second-best results are underlined. Routing agreement is reported separately in the last row.

As we can see, first the Oracle Best-of-Three outperforms the strongest single paradigm (SIR) by 10.5 (4B) and 5.7 (32B) percentage points, confirming the value of strategy diversity. Second, while Jev improves over fixed SIR, Meta-routing further surpasses Jev by 4.0 (4B) and 1.4 (32B) points—most notably on MAMO-C (79.3% vs. 62.6% on 4B). Consequently, single-pass Meta-routing closes 55.2% (4B) and 52.6% (32B) of the gap between SIR and the Oracle upper bound, with the 32B Meta prompt outperforming every isolated single-paradigm prompt across all seven benchmarks. We thank the reviewer for this insightful question, which helped strengthen our empirical evaluation and highlight the effectiveness of our meta-solver.

Routing agreement. The last row reports the fraction of instances on which the paradigm chosen is the best one, we evaluate whether the paradigm autonomously chosen under the Meta prompt belongs to that instance's set of successful paradigms (counting any successful paradigm as optimal when multiple succeed).


Q1: Training Setup

We use full-parameter fine-tuning (rather than PEFT) for both the 4B and 32B models. Both models are trained for 400 RL steps with a global batch size of 64 on a 20,000-instance dataset, requiring 230 and 560 H200 GPU-hours, respectively. A more detailed training configuration is provided in Appendix C.1, and we promise that we will publicly release the complete training recipe and code to facilitate future research.

Comments on the paper

3 comments

I thank the authors for the second response. I keep my rating at weak reject.

Novelty. Prior solver-informed methods use Gurobi only. This paper adds more solving options and a diversity term. Both are reasonable choices. They are not enough for a top venue like ICLR. The diversity term is one of many ways to encourage general exploration. I do not see a new design, or one tailored to this setting.

Size of the gain. The Pass@1 gain over SIR-RL is small (+1.5). The larger gains appear only under repeated sampling. The results do not show an improvement that goes beyond what the wider option set and standard exploration would give.

Final assessment. I do not expect further experiments to change my recommendation.

R1: Re-clarification of Novelty and Contributions

We thank the reviewer for acknowledging that our design choices are reasonable and effective. To clarify our core contribution: SDRL is the first reinforcement learning framework to train open-source LLMs as optimization meta-solver that jointly learns computational paradigm selection and problem-specific executable solution generation for real-world, industrial-scale Operations Research (OR), with a natural extension to the LLMs-based AAD tasks.

For comparison, here we illustrate some previous works and their contributions:

  1. SIRL (NeurIPS 2025): Introduces solver-informed reinforcement learning with verifiable rewards to improve the correctness of mathematical optimization formulations and executable solver code. Its central contribution is grounding optimization modeling through external solver feedback.
  2. StepORLM (ICLR 2026): Introduces a self-evolving framework that combines generative process supervision with solver-based outcome verification.
  3. STORM (ICML 2026): Proposes corrective adaptation through lightweight interventions in the model's reasoning trajectories, followed by supervised fine-tuning and reinforcement learning. Its primary contribution is preserving and improving native optimization-modeling reasoning rather than replacing entire solutions through distillation.

While these works make strong progress, all three focus on small-scale, self-contained problems under a single-solver-integrated paradigm,where all parameters fit inside the prompt and the LLM acts solely as a translator for a single external solver (e.g., Gurobi). In contrast, our work trains an agentic meta-solver capable of handling end-to-end industrial OR workflows. Empirically, this paradigm shift yields SOTA results on conventional self-contained benchmarks while achieving performance comparable to frontier models on complex, industrial-scale benchmarks, including AAD, FrontierOR, and MIPLIB-NL. More importantly, our framework establishes a novel intersection between LLM-based AAD and LLMs for optimization modeling , opening promising directions for future research for both communities.


R2: Summary of Comparative Results: SDRL vs. SIR-RL

As noted in our previous discussion, we have already provided Pass@1 and pass@8 on your mentioned self-contained benchmarks, alongside results of frontier benchmarks including MIPLIB-NL, FrontierOR, and Automatic Algorithm Design (AAD). To provide a clearer picture, we summarize SDRL's improvements over SIR-RL across benchmarks of increasing computational scale and algorithmic difficulty.

Table: SDRL's gains over SIR-RL (Qwen3-32B) grow with problem scale and algorithmic difficulty.

Benchmark TierBenchmark Nature & ScaleSDRL Gain over SIR-RL
Classical NL-to-Opt benchmarksSelf-contained; parameters <100 variables; near saturation+4.5% / +3.7% relative<br>(+3.3 / +3.0 points, Pass@1 / Pass@8)
MIPLIB-NLIndustrial scales; >=103>= 10^3 variables+40.2% / +26.4% relative(+8.6 / +9.4 points, Pass@1 / Pass@8)
AAD(EoH tasks)Combinatorial problems: bin packing with 1k–10k items, TSP with 100–200 cities, flow shop with 20–100 jobsSolves 6/6 bin packing groups (vs. 2/6 due to OOM); 84.0% gap reduction on flow shop; 11.1% on TSP
FrontierOR10210^2–10710^7 variables and constraints+100% / +120% QTE (Full / Hard);<br>feasibility +50% / +93%

These results reveal a clear trend: SDRL's advantages become substantially more pronounced as optimization problems demand greater computational flexibility and scalability. Beyond the modest gains on conventional benchmarks, SDRL achieves considerably larger improvements on industrial-scale MIPLIB-NL, successfully handles large bin packing instances where SIR-RL encounters out-of-memory failures, and substantially improves feasibility and computational efficiency on FrontierOR. It also provide stronger evidence of SDRL's practical significance for challenging, real-world Operations Research tasks.


We sincerely thank the reviewer for the constructive and timely feedback. We would like to emphasize that the experimental evidence has already been presented in our earlier rebuttal responses. Could you please let us know if those results fully resolve your concerns?

Dear Reviewer HEgJ,

Thank you again for your time and for the constructive comments on our submission. We have posted a detailed response addressing each of the points you raised. As the author–reviewer discussion period will close soon, we would like to kindly ask whether our response has adequately addressed your concerns. We would be happy to provide any further clarification or additional results that you may find helpful.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.