acceptodds
Under review as a conference paper at ICLR 2027

When LLMs Meet Nonlinear Optimization: NED-Tree for End-to-End OR Modeling

Abstract

Automating Operations Research (OR) modeling from natural language to executable solver code remains difficult when problem descriptions contain nonlinear expressions. Existing frameworks can often handle linear problems, but nonlinear terms introduce an additional semantic gap: mathematically plausible expressions must still be converted into solver-compatible expressions. To bridge this gap, we propose a structured LLM modeling framework centered on the Nonlinear Element Decomposition Tree (NED-Tree). By making nonlinear subexpressions and their dependencies explicit, NED-Tree provides a structured construction path from extracted mathematical expressions to solver-compatible code representations.The framework (1) leverages dual-granularity element extraction to build traceable modeling elements from natural language, (2) exploits NED-Tree to decompose complex nonlinear expressions into linear backbones, auxiliary definitions, and atomic operator mappings, and (3) maps the decomposed representation into executable solver code with a repair stage. We also construct NEXTOR, a benchmark designed to evaluate nonlinear OR modeling under long descriptions, redundant information, and diverse nonlinear structures. Experiments on 10 benchmarks show that our framework achieves the best average pass@1 accuracy of 72.51%. On the NEXTOR nonlinear split, the framework reaches 92.11% accuracy and 100.00% pass rate, while also preserving the best accuracy and pass rate on the linear split. The code is released at https://anonymous.4open.science/r/NEXTOR.

Primary area:
optimization
Keywords:
Large language models, nonlinear optimization, operations research, automatic modeling

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_ch2rOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper considers the task of translating NL descriptions of nonlinear optimization problems into executable solver code. The proposed framework first extracts the optimization model using global and sentence-level passes (a step termed dual-granularity extraction) and then applies the NED-tree. NED-tree converts complex non-linear expressions into a so-called 'linear backbone', including auxiliary variable definitions and atomic operations; the underlying idea here is that this decomposition can be directly mapped to solver APIs. Finally, a subsequent coding and repair step produces the executable code. To evaluate the system, the paper introduces the NEXTOR benchmark, and shows sizeable accuracy improvements across multiple benchmarks.

Critical strengths

Thank you for submitting your work to ICLR. The main strength of the paper is in designing an end-to-end solution for the task of NL2Code for non-linear mathematical programs. This problem is more complex than linear programming as the translation to code is less direct; furthermore, the interpretation of the description is inherently more involved. On a related note, there is significantly less training data for non-linear problems, which may also explain the inferior results of existing language models.

Critical weaknesses
  1. Solution robustness (and current limitations). Section 5.3-5.4 report a clear advantage of using NED-Tree. That said, I would have liked to have more insights into the framework’s behavior when decomposition is difficult or unsupported. In particular, the paper can be made stronger with a more systematic analysis of unsupported cases, decomposition failures, and the effect on performance as expression complexity increases.
  2. Flexibility and adaptability of the system. The experiments appear to center on a particular set of operators and solver interfaces. Because optimization solvers continue to expand their native non-linear support (e.g., Gurobi 13, and the upcoming Gurobi 14), I would have liked to see explicit accounting for the 'stability' of NED-Tree - e.g., can it remain as is or does it have to evolve with a new set of solver APIs?

Aside from the above, I found the paper's presentation very hard to follow and below the standard of a top conference like ICLR. The authors used complicated terms and notations that are not intuitive. Furthermore there are no examples until fairly late in the paper. I would suggest the completely revise the paper, and introduce a running-example early on, which can be used to exemplify the different stages of the pipeline.

Questions for authors
  1. Following my comment from above, Sec. 5.3 shows a substantial ~26 percentage-point accuracy drop when NED-Tree is removed, and Sec. 5.4 reports successful preservation and mapping results on six tested expression patterns. At the same time, Algorithm 1 can return UNSUPPORTED, and the conditional equivalence result in Appendix I excludes unsupported operators, cyclic or incomplete definitions, and missing domain enforcement. Can the authors please report the frequently of such cases and provide a taxonomy of unsupported or failed decompositions. If possible, please show how performance varies with expression depth, number of operators, etc.
  2. Can the authors suggest potential future approaches to overcome failures as outlined above?
  3. It seems to me that the prompting-based techniques can be carried over in order to fine-tune models. Can the authors comment on that, and do they anticipate that those guidance techniques will eventually be absorbed in model training?
12 comments
AuthorsOpenReview ↗

Response to Reviewer ch2r(1/7)

Dear Reviewer ch2r,

Thank you for your thoughtful review. We appreciate your recognition of the value of an end-to-end approach to nonlinear optimization modeling. Below, we provide additional explanations and experiments addressing your comments. We have also updated the paper PDF. If you have any further questions, we would be happy to discuss and address them.

AuthorsOpenReview ↗

Response to Reviewer ch2r(2/7)

W1 Q1 Q2: Unsupported or failed decompositions and potential improvements

We analyzed 255 questions individually to study failure frequency, the failure types observed, and the effect of expression complexity. We also compared code generation using fixed mathematical inputs and ran an independent test of first code-generation attempts on pure mathematical models. NEXTOR's nonlinear subset contains 38 questions. We also included 183 questions from OptiBench and 34 from OptMATH.

Table 1. Datasets and Evaluation Scope

Dataset / setNumber of questionsComposition and purpose
NEXTOR nonlinear set38More complex nonlinear tasks; end-to-end testing and paired comparison at the coding stage
OptiBench nonlinear set183External task set; comparison with fixed inputs
OptMATH nonlinear set34External task set; comparison with fixed inputs

(a) Bad-case statistics and analysis

First, we analyzed Auxiliary-stage failure records for GPT-6-sol (medium) on 255 questions. The table classifies the outputs and validation results in the logs. Empty candidates, validation rejections, service errors, and cases with no mathematical input from the upstream stage are listed separately.

Table 2. Auxiliary-Stage Outcomes on 255 Questions

Failure or anomaly categoryNumber / all 255 questionsDetails and handling
Nonempty candidate tree passing structural checks149(58.43%)Construction information proceeds to subsequent coding; passing structural checks does not mean mathematical equivalence has been proven
Inconsistent symbol binding or auxiliary-variable naming32(12.55%)13 cases where symbols/indexed elements were unbound; 19 cases of auxiliary-variable name conflicts across candidates
Beyond current expression/subscript parsing support11(4.31%)8 cases of unsupported subscript forms; 3 cases where expressions or functions failed parsing
Returned field structure did not meet the contract3(1.18%)Output did not meet the field requirements of the construction interface
Decomposition did not correspond to the original expression/auxiliary definitions3(1.18%)2 cases did not correspond to the source expression; 1 linearized form did not reference a declared auxiliary definition
No decomposition candidate returned42(16.47%)Empty candidate, recorded separately and not directly classified as a mathematical error
Auxiliary request failed2(0.78%)API connection failure
No mathematical input generated upstream13(5.10%)Did not enter the Auxiliary stage

Among the 242 questions that reached Auxiliary, 49 failed the structural/interface checks (20.25%). Of these, 32 involved symbols or naming, 11 involved current parsing support, and 6 involved output fields or expression correspondence. The implementation rejects these candidates and falls back to the original mathematical model. Including 2 service errors, there were 51 fallback records, and 42 were ultimately scored as correct. This shows that the workflow can still continue solving with the original mathematical model when decomposition checks fail, providing some robustness.

The 42 questions with no returned decomposition candidate are recorded separately as no_decomposition. Coding uses the original mathematical model and any returned modeling suggestions, without a decomposition candidate. These 42 questions are not included in the 51 fallback records, which consist of 49 validation rejections and 2 service errors. All 42 empty-candidate cases produced answers that could be scored; 30 were correct (30/42, 71.43%).

AuthorsOpenReview ↗

Response to Reviewer ch2r(3/7)

(b) Further checks of the four decomposition issues you identified

To directly address your questions about unsupported operators, cyclic dependencies, incomplete definitions, and domain enforcement, we further reviewed the trajectories of 255 questions. We checked the original candidates and their generated code, rather than classifying cases only by interface errors.

Table 3. Decomposition Checks and Observed Frequencies

Item checkedFrequencyChecks and specific findings
Candidates containing operators unsupported by the current constructor1/198 (0.51%)OptiBench:345 directly uses max⁡(0,x−c)\max(0,x-c) in an auxiliary definition, outside the function set supported by the current candidate parser. This is an implementation support limit; it does not mean that the problem cannot be solved.
Cyclic dependencies in auxiliary definitions0/198 (0.00%)We rebuilt the dependency graph from the actual references on the right-hand sides of definitions and checked for self-references and cycles across candidates. Repeated use of the same auxiliary name was handled separately, rather than directly counted as a cycle.
Incomplete auxiliary definitions6/198 (3.03%)Candidates referenced quantities that were neither original variables/parameters nor defined in the auxiliary definitions. For example, OptiBench:70 defines the numerator NN and denominator DD, but not the objective tt; NEXTOR:37 lacks a definition of the cost quantity cAc_A.
Necessary domain conditions omitted during decomposition1/66 (1.52%)66 questions involve variable denominators, roots, logarithms, or non-integer powers. NEXTOR:2 does not retain the nonzero condition when eliminating the denominator, allowing a zero at which the original fraction is undefined. This accounts for 0.51% of all 198 questions with candidates.
Necessary domain safeguards not enforced in the final code0/43 (0.00%)Among these 66 questions, 43 passed the checks and proceeded to tree-guided coding. We checked variable bounds, associated constraints, and operator implementations in the final retained code and found no such omissions.

The table shows that the issues you raised occur relatively rarely. The main issue is incomplete auxiliary definitions, which may stem from the modeling stage, while the NED-Tree component has almost no bad cases.

(c) Potential future methods for the main failure issues

Based on the results in (a–b), we identified the main failure categories and the following possible approaches to address them:

Table 4. Potential Improvements by Observed Category

Observed problem categoryPotential improvement step
Symbol binding/auxiliary naming issuesEstablish a shared symbol table and unique names; after renaming, check that each reference points to a unique, defined object
Expression/subscript parsing rejectionsFirst normalize finite indices, then add parsing forms with explicit mathematical semantics and domain rules
Field or expression correspondence issuesCheck the correspondence among output fields, source expressions, and candidate definitions; validate along the dependency graph that every intermediate quantity is defined and the graph is acyclic
Domain enforcement and generated-code interface errorsAttach domain conditions to local operations and check that the code enforces them; check function names, input/result-variable order, and outer relations for each backend
Incomplete auxiliary definitionsUse more detailed modeling, establish a traceable modeling process, and try multiple backbones
Reviewer_jFLEOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies end-to-end operations research optimization modeling from natural language descriptions to executable solver code, with a focus on addressing errors in modeling-element extraction and errors in mapping nonlinear expressions to solver interfaces. The proposed framework combines dual-granularity element extraction, NED-Tree decomposition, and execution-feedback repair. NED-Tree organizes nonlinear expressions into a linear backbone that preserves outer-level relations, auxiliary variable definitions, and atomic operator mappings; the complete model remains nonlinear.

Critical strengths

1.The paper distinguishes errors in recovering the mathematical model from errors in translating nonlinear relations into solver code. NED-Tree makes the outer relations, auxiliary definitions, and domain requirements explicit before code generation. Its value lies in organizing the construction information supplied to the coding agent, rather than introducing new solver capabilities. This provides a useful design for improving the reliability of automated optimization modeling.

2.The framework achieves a reported average accuracy of 72.51% across ten benchmarks, compared with 65.07% for OR-LLM-Agent. The ablation also reports a 26.32-percentage-point decrease in NEXTOR nonlinear accuracy when NED-Tree is removed. Although these results do not fully separate all sources of improvement, the combination of benchmark comparisons and component analysis supports the practical utility of the framework.

Critical weaknesses

1.Appendix P identifies a bridge-beam problem whose requested quantity does not match the reference answer and whose statement omits a required material parameter. The authors acknowledge that numerical agreement on this instance does not establish a semantically correct formulation, yet the reported results still use the original evaluation version. I appreciate this disclosure, and the issue does not by itself invalidate the broader multi-benchmark findings. However, its effect on the proposed method and competing methods should be quantified before interpreting the reported nonlinear accuracy as evidence of correct modeling.

Questions for authors
  1. After correcting or excluding the bridge-beam instance in Appendix P, how do the results for the proposed method and the strongest baselines change? Have the remaining NEXTOR instances been checked for consistency between the requested quantities, required constants, and reference answers?
  2. Does the ablation without NED-Tree retain the same domain information, solver API guidance, and repair procedure? Appendix J shows that the native-expression route succeeds on c02 using a top-level result variable; clarifying what additional benefit comes from the full decomposition would help establish the scope of the contribution.
8 comments
AuthorsOpenReview ↗

Response to Reviewer jFLE (1/3)

Dear Reviewer jFLE,

Thank you for your thoughtful review. We appreciate your recognition of our core contributions and experimental evaluation. Below, we provide additional explanations and experiments addressing your comments. We have also updated the paper PDF. If you have any further questions, we would be happy to discuss and address them.

AuthorsOpenReview ↗

Response to Reviewer jFLE (2/3)

W1 Q1: Bridge-Problem Clarification and NEXTOR Verification

(a) Comparison of results after verification

Thank you for noting that the requested quantity and reference answer for the bridge problem in Appendix P needed to be checked against each other. We revised the problem statement to make the requested quantity and required parameters explicit and to show how they match the reference answer. After checking the revised version again, our method returns the correct answer. The revised problem statement, analytical reference derivation, code, and output have been saved separately for inspection. After excluding the original bridge problem, our method answers 35 of the remaining 37 problems correctly.

For each path, we exclude its original bridge-problem output, which was incorrect in every case. The 35/38 NED result for our method comes from the submitted results. All other results come from new experiments in this round using the same GPT-6-Sol batch.

Table 1. Results Before and After Excluding the Bridge Problem

Framework (Native→NED)Original result on 38 problemsResult on 37 problems after excluding each path's original bridge problem
Our method32/38 (84.21%)→35/38 (92.11%)32/37 (86.49%)→35/37 (94.59%)
Standard32/38 (84.21%)→33/38 (86.84%)32/37 (86.49%)→33/37 (89.19%)
CoE29/38 (76.32%)→30/38 (78.95%)29/37 (78.38%)→30/37 (81.08%)
OptiMUS5/38 (13.16%)→7/38 (18.42%)5/37 (13.51%)→7/37 (18.92%)

Target quantity and material parameters in the bridge problem. We confirmed that the requested quantity is the maximum value of F=x1x22F=x_1x_2^2, and we retain the original reference answer, 10,392,304.84. The revised statement replaces “return x1x_1” with an explicit request for the maximum FF. It also explains that, at a fixed allowable stress, this quantity is proportional to the allowable bending moment. The rest of the background, r=300r=300, and the original reference answer are unchanged.

Regarding the material parameter you noted, the problem gives the stress relation σmax⁡=3M4x1x22=3M4F\sigma_{\max}=\frac{3M}{4x_1x_2^2}=\frac{3M}{4F}. Let σallow>0\sigma_{\mathrm{allow}}>0 denote the material's fixed allowable stress. From σmax⁡≤σallow\sigma_{\max}\le\sigma_{\mathrm{allow}}, we get the allowable bending-moment limit Mallow=43σallowFM_{\mathrm{allow}}=\frac{4}{3}\sigma_{\mathrm{allow}}F. To compute the numerical value of the maximum allowable bending moment, the allowable stress must be provided. The revised statement asks for the geometric quantity F∗F^*. For this quantity, the material constant only affects the proportionality factor when converting to a bending moment.

Specifically, the cross-section has width and height 2x12x_1 and 2x22x_2, respectively, and the rectangle lies inside a circular bar of radius rr. Thus, the geometric constraints are x12+x22≤r2x_1^2+x_2^2\le r^2, x1,x2≥0x_1,x_2\ge0. At the optimum, x22=r2−x12x_2^2=r^2-x_1^2, so the objective becomes F=x1(r2−x12)F=x_1(r^2-x_1^2), yielding the analytical solution x1=r/3x_1=r/\sqrt3, x2=r2/3x_2=r\sqrt{2/3}. Substituting r=300r=300 gives x1=300/3x_1=300/\sqrt3, x2=3002/3x_2=300\sqrt{2/3}, and F∗=2⋅3003/(33)≈10,392,304.8454F^*=2\cdot300^3/(3\sqrt3)\approx10,392,304.8454, which corresponds to the retained original reference answer.

(b) Verification workflow and handling for the remaining problems.

We checked the current NEXTOR with Codex GPT-6-Astra / xhigh and Claude Code Opus-5.5 / xhigh. Two researchers with PhDs in operations research also manually reviewed consistency among the problem statements, mathematical models, reference implementations, and answers. So far, apart from the bridge problem revised above, we have found no other unresolved issue of the same kind. We checked:

  1. Problems and versions. For each problem, we compared the statement, requested quantity, reference model, code, and answer, keeping the original and revised versions distinct.
  2. Problem statement and model. We checked whether each reference model fully expresses the statement, including its variables, constants, units, objective, constraints, and domains.
  3. Code and problem statement. We compared the relationships implemented in the code, the returned quantity, numerical scaling, and units with the problem statement. We distinguished numerical closeness from semantic consistency.
  4. Cross-checks and human review. We traced each concern to a problem statement, equation, or code. Human reviewers judged whether clarification or revision was needed, and we checked again after each revision.

As a result, the initial LLM-assisted review covered 37 problems. It flagged 2 as low-confidence; both were referred for human review, and neither was changed. This section also reports the clarified problem statement, the reference derivation, and outputs from 9 paths. It gives the scores after excluding the original bridge problem or replacing it with the revised version.

AuthorsOpenReview ↗

Response to Reviewer jFLE (3/3)

Q2: Ablation Controls and the Contribution of Full Decomposition

Your reading of c02 is correct: the native-expression route (Native) can implement the relation using a valid top-level result variable. Table 4 of the submitted paper reports successful program execution on 9/10 with Native and 10/10 with NED. In c02, Direct AST produces an interface error because it submits the outer nonlinear inequality directly. We believe the full decomposition gives the coding model the local definitions, their dependency order, and the outer relation. These details help it choose valid solver calls and connect the input and result variables.

(a) Control conditions in the original ablation

Table 2. Original Ablation Settings

Control factorRelationship between the full method and the original w/o NED
Mathematical model and domain informationThe same generated mathematical model and domain information are used
Backbone, token limit, solver time limit, repair budgetKept identical
Repair procedure (error feedback, trigger, repair agent, and prompt)Kept identical; the number of repairs and trajectories may differ with the generated output
Decomposition information and dedicated coding promptw/o NED skips Auxiliary by forcing Problem_Type=MILP and switches to the MILP prompt; the nonlinear solver API guidance changes accordingly

Therefore, the original ablation tests the full NED construction path, including changes to the decomposition and the matching coding guidance. We also ran a new comparison to answer what the full decomposition adds when Native already allows a valid result variable. In this comparison, Native can use its native nonlinear interface and add auxiliary variables on its own. Both paths use the same mathematical input, general interface guidance, and repair budget.

(b) The Contribution of Full Decomposition

We ran a diagnostic of first code-generation attempts on 11 pure mathematical models with analytical reference values. We did not use natural-language extraction. Each arm generated code once for each problem, with no repair. We show examples in which Native's first code-generation attempt was incorrect and NED's was correct, together with the intermediate construction steps.

The key difference in these examples is whether the coding model can implement each internal definition and reliably map it to a valid solver interface. Native also decomposed the expressions, but it used a nonexistent Gurobi call for a mathematical function. NED supplied local definitions and operator-mapping guidance. The successful code used them to connect the input and result variables through valid solver calls. The complete diagnostic results are as follows:

Table 3. First-Attempt Accuracy on 11 Mathematical Models

Coding modelNative correctNED correct
GPT-58/1111/11
DeepSeek Flash10/1111/11
GPT-6-Sol9/1111/11
Claude Sonnet 5.58/119/11

In the newly added first-attempt code-generation diagnostic, Native and NED used the same mathematical model for each of the 11 problems. Each path generated code once, without repair. As the table shows, GPT-5 was correct on 8/11 problems with Native and 11/11 with NED. M11 keeps the outer relation between the nonlinear expressions on both sides. M09 connects the final result variable to the objective interface. Two examples are:

Table 4. Native and NED-Tree Code Construction on M09 and M11

Mathematical model and contextActual issue in Native's first answerConstruction completed in NED's first answer
M09: min⁡ln⁡(1+ex)\min\ln(1+e^x), x∈[−2,2]x\in[-2,2]; the nested function is in the objectiveThe auxiliary chain for the exponential, addition of a constant, and logarithm was defined, but the addGenConstrNL(evar, gp.exp(x)) call failedUsed addGenConstrExp and addGenConstrLog to connect the corresponding variables, then used the final result variable as the objective
M11: min⁡z\min z, ex2+ln⁡y≤z+0.5e^{x^2}+\ln y\le\sqrt z+0.5; x∈[−1,1],y∈[1,e],z∈[0,9]x\in[-1,1],y\in[1,e],z\in[0,9]Decomposed the square, exponential, logarithm, and square root and retained the outer inequality, but the addGenConstrNL(exp_x2, gp.exp(x2)) call failedUsed addGenConstrPow/Exp/Log to implement u=x2u=x^2, w=euw=e^u, v=ln⁡yv=\ln y, and s=zs=\sqrt z, then added w+v≤s+0.5w+v\le s+0.5

In the second example, the top-level result variable only specifies where to place the final relation. The full decomposition also specifies how to connect the square, exponential, logarithm, and square-root components. The code generated with NED follows these local definitions and calls the corresponding solver interfaces, obtaining z∗=0.25z^*=0.25. This directly shows how the added construction information helps LLM code generation.

Reviewer_B3m4OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces NED-Tree, a component that decomposes nonlinear expressions into a linear backbone and auxiliary definitions. NED-Tree helps LLMs translate natural-language descriptions of optimization problems into executable solver code. Experiments on 10 benchmarks, including the newly proposed NEXTOR benchmark, demonstrate the framework’s strong overall performance in supporting the solution of nonlinear optimization problems.

Critical strengths

1.The paper is clearly structured and easy to follow. 2.The authors investigate an important yet underexplored challenge in LLM-based optimization modeling: solving nonlinear optimization problems.

Critical weaknesses
  1. NEXTOR uses a 1% relative-error tolerance to determine answer correctness, but provides insufficient justification for this threshold and no sensitivity analysis. This relatively loose criterion may hide some modeling errors. Optimization modeling benchmarks generally use stricter tolerances.
  2. [Lack of ablations on external benchmarks] End-to-end component ablations are conducted only on the authors’ NEXTOR benchmark, leaving it unclear whether the independent benefits of NED-Tree generalize to other datasets. I suggest conducting additional ablations on the nonlinear subsets of external benchmarks such as OptiBench and OptMATH.
  3. [Lack of validation across backbone models] As an intermediate representation for code generation, NED-Tree requires further validation across different backbone models. The current evaluation lacks systematic ablations across multiple backbones under a consistent experimental protocol.
  4. [Lack of cost analysis] Since the method involves multiple stages of model calls and execution-feedback repair, I am concerned that its high accuracy may come at a very high computational cost. I suggest reporting the number of model calls and total token consumption, together with comparisons against other methods.
  5. [Limited size of NEXTOR] The nonlinear subset of NEXTOR contains only 38 problems, raising concerns about whether this benchmark can reliably assess models’ capabilities.
Questions for authors

Please check the weak points.

15 comments
AuthorsOpenReview ↗

Response to Reviewer B3m4 (1/5)

Dear Reviewer B3m4,

Thank you for your thoughtful review. We appreciate your recognition of the paper's clear structure and the importance of the research problem. Below, we provide additional explanations and experiments addressing your comments. We have also updated the paper PDF. If you have any further questions, we would be happy to discuss and address them.

W1: Answer Threshold and Sensitivity Analysis

Our method uses a 1% relative-error tolerance to compare the predicted answer with a nonzero reference answer. Related optimization benchmarks use different answer rules, including relative-error or objective-quality windows at the 1% level. To put this choice in context, we list relative/normalized rules separately from absolute-error rules below. Let Δ=∣z^−z⋆∣\Delta=\lvert\hat z-z^\star\rvert and A=∣z⋆∣A=\lvert z^\star\rvert.

(a) Answer thresholds and settings in prior work

The following table summarizes answer-matching rules in prior work, including relative and absolute error:

Table 1. Answer Thresholds and Evaluation Rules

WorkThreshold typeThresholdNotes
OursRelative1%Exact matching when the reference value is zero
OptiTree (NeurIPS 2025)Relative5%Direct equality is checked first; a zero reference uses an absolute error of 0.05;
ORLM / IndustryORRelative5%The prediction and reference values are rounded before comparison; when the rounded reference is zero, an absolute error of 0.05 is used
FrontierOR (2026)Relative1%Feasibility is checked separately first; objective quality relative to the Gurobi reference is compared and time is reported separately;
CIR / R2C (2026 )Relative1%Relative to the reference optimal objective value; this passage does not specify zero handling or equality details
OptiBench / ReSocraticAbsolute10−410^{-4}Every requested value must pass
ORThoughtAbsolute0.010.01This branch is used for numeric scalars; lists, tuples, and other types are handled by other branches
OR-LLM-Agent (2025)Absolute0.10.1Absolute error for the final numeric answer

A 1% threshold has clear precedent in comparable optimization evaluations and is one reasonable protocol used in prior work. We report what each rule scores and how it handles zero values. We retain NEXTOR's established 1% primary threshold.

(b) Comparisons across multiple answer-threshold scales

We rescore the same saved outputs offline at 0.1% and 0.01%. The table reports the results for the same batch of GPT-6-sol /NEXTOR outputs; we made no new model calls. In this re-scoring, nonzero references use relative error, and zero references use an absolute tolerance of 10−510^{-5} at all three thresholds. No-answer cases and failures remain in the denominator.

Table 2. Accuracy Across Answer Thresholds

Path1% (Native → NED)0.1% (Native → NED)0.01% (Native → NED)
Our method32→33 / 38 (84.21%→86.84%)30→31 / 38 (78.95%→81.58%)29→30 / 38 (76.32%→78.95%)
Standard32→33 / 38 (84.21%→86.84%)31→31 / 38 (81.58%→81.58%)29→30 / 38 (76.32%→78.95%)
CoE29→30 / 38 (76.32%→78.95%)26→29 / 38 (68.42%→76.32%)25→29 / 38 (65.79%→76.32%)
OptiMUS5→7 / 38 (13.16%→18.42%)5→7 / 38 (13.16%→18.42%)5→7 / 38 (13.16%→18.42%)

When we tighten the threshold from 1% to 0.01%, our method consistently gets 1 more problem correct than Native (+2.63 pp). At every tested threshold, NED is no worse than Native for Standard, CoE, and OptiMUS. Standard ties Native at the intermediate threshold. Thus, the advantage in this round of comparisons does not appear only under the 1% window and is not sensitive to 1‰ or 0.1‰.

AuthorsOpenReview ↗

Response to Reviewer B3m4 (2/5)

W2: Evaluation on External Nonlinear Benchmarks

To study failure frequency, the failure types recorded in the logs, and the effect of expression complexity, we analyzed 255 questions individually. We also compared code generation using fixed mathematical inputs and ran an independent test of first code-generation attempts on pure mathematical models. NEXTOR's nonlinear subset contains 38 questions. We also added 183 questions from OptiBench and 34 from OptMATH.

Table 3. Composition of the 255-Question Evaluation Set

Dataset / setNumber of questionsComposition and purpose
NEXTOR nonlinear set38More complex nonlinear tasks; end-to-end testing and paired comparison at the code-generation stage
OptiBench nonlinear set183External task set; fixed-input comparison
OptMATH nonlinear set34External task set; fixed-input comparison

We completed an end-to-end paired comparison of GPT-6-sol Native and NED on the fixed nonlinear subsets from OptiBench and OptMATH. The two paths share the same mathematical inputs. They also use the same backbone, source of domain definitions, solver budget, repair rules, and scoring rules. Failed instances remain in the denominator.

Table 4. Native and NED Results on External Nonlinear Benchmarks

DatasetNativeNEDDifference
OptiBench132/183 (72.13%)137/183 (74.86%)+2.73 pp
OptMATH19/34 (55.88%)20/34 (58.82%)+2.94 pp

This table reports the component comparison on external nonlinear benchmarks that you requested. R3 provides further analysis.

W3: Cross-Backbone Evaluation

We completed fixed-input comparisons across four backbone models, three datasets, and three code-generation frameworks. All code-generation paths use the same mathematical inputs, which were generated by GPT-6-Sol. The NED arm reuses Auxiliary construction information once per question. For each backbone, model parameters, general interface guidance, and repair budgets are held constant between the Native and NED arms. The tables first summarize the number of correctly answered questions for each backbone and framework across the three datasets. They then present a one-round end-to-end comparison of our method on NEXTOR.

Table 5. Native and NED-Tree Coding Results on 255 Questions

BackboneStandard (Native→NED / 255)CoE (Native→NED / 255)OptiMUS (Native→NED / 255)
GPT-5191→194 (+1.18 pp)175→177 (+0.78 pp)80→90 (+3.92 pp)
DeepSeek Flash193→199 (+2.35 pp)182→191 (+3.53 pp)91→95 (+1.57 pp)
GPT-6-Sol187→190 (+1.18 pp)172→175 (+1.18 pp)75→74 (−0.39 pp)
Claude Sonnet 5.5199→202 (+1.18 pp)187→189 (+0.78 pp)93→94 (+0.39 pp)

In the end-to-end comparison on NEXTOR (38 cases), each model uses its own upstream extraction, and Native/NED are paired within each model:

Table 6. Cross-Backbone End-to-End Comparison on NEXTOR

BackboneNativeNEDDifference
GPT-532/38 (84.21%)35/38 (92.11%)+7.89 pp
DeepSeek Flash34/38 (89.47%)35/38 (92.11%)+2.63 pp
GPT-6-Sol32/38 (84.21%)33/38 (86.84%)+2.63 pp
Claude Sonnet 5.533/38 (86.84%)34/38 (89.47%)+2.63 pp

These results support using NED construction information across different backbone models and code-generation frameworks. For each of the three frameworks, the total number of correct answers increases. All four backbones improve in the NEXTOR end-to-end comparison. The results in R3 and R4 also demonstrate NED's transferability across different base models and different baselines, as well as its independence from a particular backbone.

AuthorsOpenReview ↗

Response to Reviewer B3m4 (3/5)

W4: Accuracy and Computational Cost

Across the four coding-cost comparisons on GPT-6-sol/NEXTOR, NED did not increase the number of requests in any case, and accuracy improved in all four cases. The table reports average per-question costs for code generation and repairs within budget. It includes failed questions and input tokens for the tree text. It excludes upstream extraction, mathematical modeling, and Auxiliary calls.

Table 7. Accuracy and Coding-Stage API Costs

FrameworkCorrect (Native→NED)Requests/question (Native→NED)Total tokens/question (Native→NED)Repair-code requests/question (Native→NED)
Ours32→33 / 381.08→1.035427.5→6036.20.053→0.026
Standard32→33 / 381.08→1.003702.6→4438.60.105→0.026
CoE29→30 / 381.00→0.973351.3→4275.00.000→0.000
OptiMUS5→7 / 381.29→1.294025.1→5164.60.026→0.026

For our method, repair-code requests fell from 0.053 to 0.026 per question. Total tokens increased by about 11.2%. Standard also reduced repair-code requests from 0.105 to 0.026 per question. These results show that structural guidance can increase the number of correct answers without increasing the number of code-generation calls. It can also reduce some feedback repairs. The main cost is a longer structured input.

Time is reported separately, all in seconds per question. Cumulative API time is the sum of the durations of all requests; cumulative program time includes startup, modeling, solving, and output:

Table 8. API and Solver Time per Question

Framework/path (Native→NED)Cumulative API seconds/questionCumulative solving seconds/questionSolving time known / executions
Ours62.19→63.940.0121​→0.011137/39→38/38
Standard69.55→63.810.0148​→0.012837/41→37/38
CoE76.00→66.120.1159​→0.027834/37→36/37
OptiMUS20.75→18.690.0033​→0.00206/8→7/8

Overall, NED reaches the Pareto frontier. In these frameworks, Ours increases coding input tokens but reduces the average time per question and improves accuracy under most frameworks. Porting NED to other frameworks also improves accuracy, demonstrating its portability and independence from a particular backbone.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.