- Rating
- 4(+1)
- Confidence
- 3
- Informativeness
- 3(+1)
- Presentation
- 2
- Soundness
- 3(+1)
Read review
Paper summary
The paper proposes Strategy-Diverse Reinforcement Learning, combining correctness-gated rewards for strategy rarity and runtime-mode diversity with GRPO. Mixed textual and file-grounded training improves optimization performance across seven benchmarks.
Critical strengths
- Strategy complementarity provides a clear motivation for adaptive optimization meta-solvers.
- Reward ablations, a same-backbone SIR-only RL baseline, and format-controlled comparisons provide meaningful evidence for the proposed components.
- Improvements over the base models on file-grounded MIPLIB-NL are substantial, and training details and prompts are documented extensively.
Critical weaknesses
- The stated correctness criterion checks execution and objective-value agreement, but does not describe independent validation of returned assignments against the original constraints. Objective agreement alone does not establish primal feasibility.
- Runtime clusters and keyword-based strategy checks are imperfect proxies for algorithmic diversity, without a reported reliability audit. The runtime analysis compares different sets of successful programs, limiting its ability to isolate the micro-reward’s effect.
- Training-seed variation and confidence intervals are absent. This particularly weakens the claimed advantage over GPT-5.5: 0.1 percentage points on average and 0.9 points on MIPLIB-NL. Common execution limits for the main comparisons also need clearer specification.
Questions for authors
- Are returned assignments independently checked for feasibility and objective consistency? What execution limits and solver settings are shared across models?
- Can repeated runs and an audit of strategy labels/runtime clusters establish that the gains reflect meaningful computational diversity?
5 comments
Response to Reviewer LoAk - Part 2
W2 & Q2: Reliability of Diversity Measurements
First, we acknowledge that runtime clusters serve as an imperfect yet highly efficient proxy for computational diversity. Execution time is collected for free during correctness verification, whereas invoking a mainstream LLM judge during training would add prohibitive token cost and inference latency.
To address the reviewer's concern, for the macro-reward reliability, we apply LLMs-as-judge verification checks. An offline GPT-6-Astra audit of randomly sampled training rollouts showed 99.5% label agreement among successful programs with comparable labels. For example, in one TSP rollout, the model declared Exact Combinatorial Algorithm in its reasoning tag but actually invoked Gurobi with Miller–Tucker–Zemlin (MTZ) constraints in code. GPT-6-Astra flagged it as solver-based, and our rule-based verifier (Appendix C.3) similarly rejected the mismatched tag as unknown, assigning zero diversity reward despite the program achieving the correct objective value.
For the micro-reward efficacy: We can directly assess the effectiveness of the runtime micro-reward from our ablation study (Appendix C.6), where entries report Avg@8 (%) ± standard deviation:
| Method | MAMO-C | IndustryOR | OptMath | MIPLIB-NL |
|---|---|---|---|---|
| Base model | 42.6 ± 2.7 | 38.3 ± 3.2 | 16.1 ± 1.8 | 6.0 ± 1.1 |
| SDRL w/o diversity reward | 71.2 ± 1.8 | 51.8 ± 1.8 | 34.6 ± 1.9 | 12.3 ± 1.5 |
| SDRL w/o micro reward | 72.6 ± 2.5 | 50.1 ± 1.8 | 35.8 ± 2.3 | 13.4 ± 1.0 |
| SDRL w/o macro reward | 75.6 ± 1.2 | 50.8 ± 2.0 | 34.8 ± 1.6 | 13.6 ± 1.3 |
| Full SDRL | 75.7 ± 1.2 | 52.6 ± 2.0 | 35.6 ± 2.1 | 15.2 ± 1.1 |
As shown in the table, Full SDRL outperforms the micro-reward ablation on three of the four benchmarks (including a +3.1 point gain on MAMO-C and +1.8 points on MIPLIB-NL). Furthermore, when tackling large-scale problems like MIPLIB-NL, pairing high-level strategy diversity with runtime efficiency is critical: even within the same broad algorithmic strategy, different formulations or implementations can incur vastly different computational overhead (15.2 vs. 12.3).
W3: Variance and Comparison with GPT-5.5
Here we expand the comparison using eight samples per instance, reporting Avg@8 (%) ± the standard deviation of accuracy across the eight sampling runs (percentage points), with a relative-error tolerance of .
| Model | NL4Opt | MAMO-E | MAMO-C | Ind-OR | OptMath | OptiBench | MIPLIB-NL | Avg. |
|---|---|---|---|---|---|---|---|---|
| SDRL-Qwen3-4B | 91.0 ± 1.4 | 91.1 ± 0.6 | 79.6 ± 1.6 | 53.8 ± 2.8 | 40.0 ± 1.3 | 67.9 ± 0.8 | 22.2 ± 1.8 | 63.7 |
| SDRL-Qwen3-32B | 95.7 ± 0.6 | 96.0 ± 0.2 | 80.7 ± 1.3 | 56.0 ± 2.5 | 57.9 ± 1.5 | 68.1 ± 0.7 | 28.9 ± 0.9 | 69.0 |
| GPT-5.5 | 94.4 ± 1.0 | 89.8 ± 0.9 | 88.1 ± 1.3 | 65.8 ± 2.3 | 42.9 ± 1.3 | 67.9 ± 0.4 | 28.2 ± 1.6 | 68.2 |
SDRL-Qwen3-32B achieves a higher average accuracy than GPT-5.5 (69.0% vs. 68.2%), with higher mean accuracy on five of the seven benchmarks.
Reference
[1] Minwei Kong, Chonghe Jiang, Ao Qu, Wenbin Ouyang, Zhaoming Zeng, Xiaotong Guo, Zhekai Li, Junyi Li, Yi Fan, Xinshou Zheng, et al. (2026). FrontierOR: Benchmarking LLMs' Capacity for Efficient Algorithm Design in Large-Scale Optimization.
We sincerely appreciate the reviewer's constructive feedback and hope that we have addressed all of the comments above. We have made the training dataset, benchmark data, and evaluation code available in our anonymized repository: https://anonymous.4open.science/r/optimization-review-42D7/.
I thank the authors for their detailed responses, which have largely addressed my concerns. I remain positive about the paper and maintain my original score.
Response to Reviewer LoAk - Part 1
Rebuttal:
W1 & Q1: Feasibility Verification and Execution Settings
We sincerely thank the reviewer for this insightful comment and acknowledge that explicit feasibility verification is currently absent from our training framework.
In our experimental setup, we follow the standard convention in LLM-based optimization modeling: both training and evaluation verify predicted objective values within a relative tolerance of 10−6, which is consistent with prior work and is substantially stricter than recent baselines such as StepORLM (0.05). Execution timeouts are set to 150s for training, and to 300s (self-contained benchmarks) or 1,800s (MIPLIB-NL) for evaluation, with a memory limit of 8,192 MB per program. For solver-based programs, we use Gurobi via
gurobipy13.0.2, restricted to a single thread (Threads=1).Implementing feasibility verification across heterogeneous optimization tasks is highly problem-class dependent. For example, verifying a TSP solution requires parsing raw instance data (e.g., distance matrices) alongside domain-specific constraint validators for routes, or subtour. Because our training data is adapted from public text-based corpora (SIRL and OptMATH) that lack standardized executable verifiers and raw instance parsers, objective-value matching remains the most scalable reward signal for general-purpose RLVR.
To address the reviewer's concern regarding feasibility checks during evaluation, we report the evaluation results on the FrontierOR benchmark [1], which explicitly verifies constraint satisfaction on the predicted decision variables and jointly evaluates solution feasibility, quality, and efficiency (QTE):
Baseline results are taken directly from FrontierOR.
As shown above, even though our framework was not trained under the FrontierOR protocol, it generalizes strongly to decision-variable feasibility evaluation. With identical training data, SDRL-Qwen3-32B improves feasibility over SIR-RL-Qwen3-32B from 0.22 to 0.33 on the Full split and from 0.15 to 0.29 on the Hard split, and roughly doubles QTE on both splits (0.09 → 0.18 and 0.10 → 0.22). It also outperforms Grok-4.20-beta and LLaMA-4-Maverick on all metrics, and its QTE matches or exceeds that of DeepSeek-R1 (0.18 vs. 0.17 on Full; 0.22 vs. 0.11 on Hard).