acceptodds
Under review as a conference paper at ICLR 2027

AutoHOPT: From Natural Language to Model-Conditioned Metaheuristic Search

Abstract

Existing language-to-optimization systems primarily treat mathematical models as inputs to solvers. Yet the same constraint structure that defines feasibility can also guide how solutions are constructed and improved. We introduce AutoHOpt, a framework that turns natural-language requirements into modelconditioned metaheuristic search. AutoHOpt first recovers a source-reviewed executable model, OPTSPEC, and then uses its decisions and constraints to generate SEARCHIR, a task-specific search program. The accepted model remains fixed while execution feedback refines initialization and neighborhood-repair operators, which run within adaptive large neighborhood search without per-iteration LLM calls. Across ten external benchmarks, AutoHOpt achieves the highest mean zerogap success on eight. On IndustryOR and MIPLIB-NL, it reaches 83.5% and 78.8%, exceeding the strongest baseline by 20.2 and 9.5 percentage points, respectively. Ablations and common-budget controls demonstrate the contributions of modeling feedback and model-conditioned search. These results establish executable models as an effective interface between natural-language problem understanding and automated search design.

Primary area:
optimization
Keywords:
Metaheuristic, AAD

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 5 reviews, above 85% of the others.
Reviewer_BFSVOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3(+1)
Read review
Paper summary

This paper presents AutoHOpt, a framework that connects natural-language optimization modeling with task-specific heuristic search. It first constructs an executable model, OPTSPEC, and checks its interpretation of the source requirements using behavioral probes and retained evidence from previous failures. The accepted model then guides the generation of initialization, destroy, and repair operators in SEARCHIR. These operators run within adaptive large neighborhood search while the model remains fixed. Experiments evaluate model recovery and search performance on ten benchmarks and the new 1,440-task CoPBench, with additional component ablations and fixed-model comparisons.

Critical strengths
  1. A useful connection between model recovery and search design AutoHOpt gives the recovered model a concrete role in search design. OPTSPEC supplies the objective and feasibility evaluator together with the decision structure used to construct search operators. Constraints that couple decisions can therefore guide which variables are released together and how partial solutions are reconstructed. This makes the recovered mathematical structure directly useful in designing the search procedure.
  2. Gains in feasible coverage and solution quality The main benchmark comparisons are based on 30 independent runs of the complete pipeline. On CoPBench, AutoHOpt achieves 87.4% full-suite feasibility which is higher than 79.9% for NEMO. On each run's common feasible intersection, its mean gap to the best feasible objective observed across the compared methods is 2.4% , a substantial improvement over NEMO's 4.8%. These results support gains in feasible coverage across the full suite and in objective quality on the common feasible subsets.
Critical weaknesses
  1. Details of the agent interactions The paper explains the modeling and search loops, and Appendices A and K provide a useful execution example. A fuller explanation of the interactions between agents would help readers understand and evaluate the procedure. In particular, please specify the complete prompts and response schemas, and explain how diagnostic feedback, retained probes, and previous responses are selected for subsequent requests. These choices affect what the agent knows about a failed attempt and how it is guided toward a correction. A complete interaction trace would help make the process clearer and easier to follow.
  2. Construction and validation of CoPBench CoPBench covers several problem families and instance sizes. For the full 1,440-task suite, the construction and validation of the natural-language descriptions merit further explanation. Please describe how the instances and descriptions were produced, and how their agreement on the intended objectives and constraints was checked. This is important because the reported feasibility and objective comparisons depend on evaluating the tasks specified by those descriptions.
Questions for authors
  1. Appendices A and K describe an execution of the current system. Could the authors provide the complete prompts and response schemas for the evaluated modeling, source-review, and search-design stages, together with a full request-response trace? The trace should show how diagnostic feedback, retained probes, and earlier responses are incorporated into subsequent requests. This would clarify the information supplied to each agent and the revision procedure used in the experiments.
  2. How were the instances and natural-language descriptions in the full 1,440-task CoPBench suite generated or collected? Could the authors describe how their correspondence was verified, particularly the checks used to establish agreement on the intended objectives and constraints?
  3. Could the authors provide results by problem family and instance size for the main 30-run CoPBench evaluation, including feasibility rates, runtime, and the conditional objective gaps reported in Table 9? This breakdown would show whether the aggregate improvements are consistent across different families and instance sizes.
  4. Could the authors examine a representative sample of OPTSPEC models before and after source review, reporting which semantic errors were corrected and which remained? A returned solution may satisfy the source requirements and match the reference objective even when the accepted model omits or adds a constraint. So it would be helpful to examine whether the recovered models are correct.
  5. Appendix E gives a default initial temperature of 1. Since changing objective units alters the acceptance probability of uphill moves when temperature is fixed, could the authors clarify whether the temperature is calibrated to the objective scale in the reported experiments?
4 comments
AuthorsOpenReview ↗

Response to Reviewer BFSV (1/2)

Dear Reviewer BFSV,

We thank the reviewer for taking the time to carefully review our work and for the constructive feedback that has helped us improve the quality of our manuscript.

We appreciate your recognition of the connection between model recovery and search design, and the improvements in feasible coverage and solution quality. AutoHOpt makes recovered mathematical structure directly usable for both candidate evaluation and task-specific search. The revision makes the information exchanged between stages, the construction of CoPBench, and the scope of the validation explicit.

W1 / Q1. Complete prompts, response contracts, and interaction context

Appendix O presents the complete recorded IndustryOR94 execution: the original task, modeling prompt and response, compiled evaluator, source-review prompt and probes, and search-design prompt and generated initialization, destroy, and repair functions. It specifies the compact declaration's field grammar, its compilation into OPTSPEC, and the six-field SEARCHIR response contract.

Sections 4.2 and 4.4 now state the context supplied to model and search revisions. Appendix O.7 gives the exact field-level rules of the frozen implementation:

  • A modeling revision receives the original source, complete current candidate, latest diagnostics, remaining budget, the six most recent failure summaries, and all retained probes.
  • Source review receives the complete candidate and source, compiled constraint directory, objective dependencies, decision shapes, and retained tests. Passing and failing probes are deduplicated into a bank capped at 12; exceeding the cap raises an error rather than discarding earlier tests.
  • Review-format repair receives the previous complete review and the four most recent format-error summaries. Format errors are returned to the review role before changing the mathematical model.
  • Search-design revision receives the unchanged model context and hash, previous complete program, and the failure history within the bounded four-attempt design loop.

IndustryOR47 supplies a concrete feedback-driven revision. A source-valid assignment with 1.1 hectares and 55 whole labor-days failed the expression work == labor * area because floating-point multiplication produced 55.00000000000001. The next modeling request contained the failed diagnostic and all four retained probes. The generated edit rewrote the same positive-rate relationship as work / labor == area; all four retained tests then passed, followed by two new passing tests. The appendix displays the actual follow-up fields, complete edit, and replay outcomes. This example shows how diagnostic evidence changes the next request and remains available after repair.

W2 / Q2. CoPBench construction and objective–constraint correspondence

CoPBench is a constructed benchmark of 1,440 industrial optimization tasks across four domains, ten subtypes, four scales, and three configuration profiles. Each domain and each scale contains 360 tasks; each domain–scale–profile cell contains 30.

For each task, a frozen configuration and seed determine the numerical instance, active operational rules, and a private feasible construction witness. The public input contains an LLM-assisted scenario description, explicit calculation rules, and a complete numerical annex. The calculation rules and annex are rendered from the same numerical instance used by the reference evaluator. This connects the task statement to the evaluated coefficients, entities, and policy scopes.

The construction paragraph in Section 5.1 and Appendix Q.4 now explain the correspondence with concrete examples. In facility location, transport cost includes customer demand, customer weight, and distance, whereas emissions use demand, distance, and the facility's emission factor. In scheduling, a due date contributes to tardiness, while an active hard deadline constrains completion. In UAV tasks, hard windows apply to service start, and an explicit flag specifies whether the final depot-return leg is required.

The release records schema checks and independently verified feasible witnesses for all 1,440 instances. A primary evaluator and separately implemented verifier check feasibility and recompute the objective and metrics, with absolute and relative agreement tolerances of 10−810^{-8}. The revision reports the scope of these numerical checks and exposes the objective terms, coefficients, and constraint scopes for direct inspection of the description-to-instance correspondence.

AuthorsOpenReview ↗

Response to Reviewer BFSV (2/2)

Q3. Results by problem family and instance size

Main-text Table 4 reports the 30-run evaluation, with the metric defined in Section 5.5. It establishes broader full-suite feasibility, 87.4% versus 79.9%, and a smaller conditional mean gap, 2.4% versus 4.8%, for AutoHOpt relative to NEMO. The common feasible intersection across all twelve methods includes all four families. Its existing 30-run composition is:

FamilyTasks per runMean common-feasible tasksMean common coverage
Facility location360127.635.5%
Manufacturing scheduling36077.921.6%
Chip placement360127.635.5%
UAV planning36090.325.1%

All four families therefore contribute to the matched objective comparison, and all four scales contribute to the full-suite feasibility denominator. The reported method comparisons are at suite level; the family table quantifies the composition of the common feasible set. We interpret the 7.5-percentage-point feasibility gain and reduction in conditional gap from 4.8% to 2.4% at this suite-level scope.

Q4. Inspecting model semantics before and after review

Model semantics and returned-solution validity are separate evaluation targets. We inspect the declared domains, objective terms, and constraint relationships against the source, using distinguishing assignments to test their behavior. The added diagnostics identify the type and provenance of each change.

  • IndustryOR94: the first model passes four recorded source-derived probes covering marginal purchasing costs and grade-specific composition requirements. The appendix displays the complete model, expressions, and tests.
  • IndustryOR47: the retained feasibility probe drives the numerical rewrite described above. The variable domains and mathematical relationship are preserved; this is a numerical representation repair.
  • OptMATH153: a separate archived diagnostic run contains two objective coefficients with positive signs where the source specifies negative signs. Its original objective probe evaluates the zero decision, where either sign yields zero, so the discrepancy remains after that review. A subsequent direct comparison with the signed source expressions identifies the error; a separately recorded retry restores both signs. Appendix R displays the original coefficients and the first and revised models' objective values at distinguishing nonzero assignments. These observations test objective terms and are not asserted to satisfy the full constraint system.

The cases make successful tests, an observed repair, and a remaining semantic error independently inspectable. We report them as traceable model-level diagnostics. Population-level semantic-correction rates would require a separately sampled before/after model audit; solution-level success rates are not used as estimates of model equivalence.

Q5. Temperature and objective units

The reported 30-run experiments followed Appendix E, using T0=1T_0=1 and geometric cooling γ=0.995\gamma=0.995. Temperature was expressed in evaluator-cost units, with no automatic calibration to objective scale. Acceptance used the unnormalized difference in minimization-oriented penalized cost.

For an uphill move with unchanged feasibility and hard-residual rank components, the acceptance probability at iteration kk is

-\frac{\widetilde f_M(x')-\widetilde f_M(x)} {\max(T_0\gamma^{k-1},10^{-12})} \right].$$ Under a positive cost rescaling $\widetilde f'_M=a\widetilde f_M+b$, preserving these probabilities requires scaling both $T_0$ and the numerical temperature floor by $a$, with $\gamma$ unchanged. Appendix E now states the experimental settings, cost units, absence of automatic scale calibration, and rescaling relationship explicitly.

Thanks for the response. The added prompts and interaction trace make it easier to see how the system retries after a failed attempt. The revision also explains how the 1,440 CoPBench tasks were generated, how the numerical instances were matched to their descriptions, and how two implementations checked feasible solutions. The comparison is clearer now: feasibility is measured across the full suite, while objective gaps are computed on the tasks where all methods found a feasible solution in a given run, using the best observed feasible objective as the reference.

The OptMATH153 example clears up my question about the correction. The two nonzero checks were done after the fact, and the corrected model came from a separate retry. Appendix E specifies the initial temperature T0=1T_0=1, the cooling factor γ=0.995\gamma=0.995, and that temperature is measured in the evaluator’s cost units. It also clarifies that temperature was not automatically calibrated to the objective scale. These revisions address the main concerns from my original review.

AuthorsOpenReview ↗

Thank You for Your Constructive Feedback and Updated Assessment

We sincerely thank you for your careful follow-up and for reconsidering your assessment and raising your score. We appreciate your recognition that the revisions address the main concerns from your original review.

Your detailed and constructive suggestions have helped us improve the clarity, precision, and overall quality of the paper, particularly in presenting the interaction process, benchmark construction, and evaluation protocol. We are grateful for the time and care you have devoted to reviewing our work and helping us strengthen the manuscript.

Reviewer_C3ZXOpenReview ↗
Rating
4
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

AutoHOpt translates natural-language optimization tasks into executable OPTSPEC models and generates model-conditioned SEARCHIR operators for adaptive large neighborhood search. Its central contribution is an explicit model-to-search interface that preserves model semantics during algorithm refinement. Results across ten external benchmarks, supported by ablations and controlled comparisons, provide convincing evidence of its effectiveness. The useful model-to-search interface and substantial, well-supported empirical gains justify acceptance. The remaining concerns primarily limit the interpretation of efficiency and industrial solution quality.

Critical strengths
  • Well-motivated architecture. Separating model recovery from search design makes their respective failures diagnosable. Executable source probes, retained counterexamples, and explicit model bindings provide useful safeguards against semantic drift.
  • Strong empirical results. AutoHOpt achieves the highest mean zero-gap success on eight of ten external benchmarks, improving over NEMO by 20.2 percentage points on IndustryOR and 9.5 points on MIPLIB-NL across 30 end-to-end repetitions.
  • Informative controls. Fixed-model ablations demonstrate substantial contributions from generated operators. Comparisons under shared resource ceilings further support the structured interface and iterative search beyond initialization alone.
Critical weaknesses
  • Table 1 retains method-specific operating budgets. Although Appendix J provides useful controlled comparisons, the end-to-end quality–cost tradeoff against strong complete systems such as NEMO remains less clearly established.
  • COPBENCH’s objective-quality comparison uses the common feasible intersection, covering approximately 29.4% of the suite. The reported quality gains therefore characterize this subset; performance on harder instances would benefit from further analysis.
Questions for authors
  1. How does AutoHOpt compare with NEMO under matched end-to-end resource budgets, or along success-versus-cost curves?
  2. Can the authors provide COPBENCH results by problem scale and pairwise quality comparisons with NEMO on their larger shared feasible subset?
3 comments
AuthorsOpenReview ↗

Response to Reviewer C3ZX

Dear Reviewer C3ZX,

We thank the reviewer for taking the time to carefully review our work and for the constructive feedback that has helped us improve the quality of our manuscript.

We appreciate your recognition of the novelty of our model-to-search interface and the strength of the empirical results, including the architecture, ablations, and controlled comparisons. Below, we clarify the scope of the computational-cost and CoPBench quality evidence.

Q1. End-to-end performance and computational cost

Table 1 compares complete AutoHOpt and NEMO pipelines on the same task collections and evaluation criterion, with GPT-5.6 sol used for their generative roles. Each method retains its established operating settings. Under these configurations, AutoHOpt improves reference-optimal success by 20.2 percentage points on IndustryOR and 9.5 points on MIPLIB-NL over 30 end-to-end repetitions.

Main-text Table 3 now presents the controlled success and actual computational costs together; Appendix J provides evaluation counts and stage-specific timing. The six search-route controls share per-task ceilings of 12 LLM calls, 96k tokens, 10,000 full evaluations, and 240 seconds, including modeling, program generation, pilots, and failed revisions. AutoHOpt achieves 83.5% / 78.8% success on IndustryOR / MIPLIB-NL, compared with 77.0% / 71.0% for direct code generation. The reported mean costs are 99 versus 104 seconds per attempted task, respectively, including failures.

The constructor-only comparison isolates the benefit of continued search. It uses the same mean five LLM calls and 46k tokens as the full method, while stopping at initialization. Continuing numerical search increases success from 61.0% to 83.5% on IndustryOR and from 54.0% to 78.8% on MIPLIB-NL, with mean total time increasing from 82 to 99 seconds. Thus, the generated program delivers substantial additional optimization gains without further LLM calls. These controlled results support the computational value of the model-to-search interface alongside the complete-system effectiveness gains in Table 1.

Q2. CoPBench scales and the scope of matched objective quality

The complete CoPBench comparison is now in main-text Table 4, with its conditional-gap definition in Section 5.5 and intersection analysis in Appendix I. CoPBench contains 360 tasks at each of four scales, and all 1,440 tasks contribute to the full-suite feasibility evaluation. AutoHOpt achieves 87.4% mean feasibility versus 79.9% for NEMO. The reported objective comparison uses each run's common feasible intersection across all twelve methods, averaging 423.5 tasks (29.4%). On this identical task set, AutoHOpt reduces the mean gap from 4.8% to 2.4%. These two endpoints establish broader feasible coverage over the suite and stronger objective quality on the matched set.

The direct AutoHOpt–NEMO intersection can cover a substantially larger set. For feasible task sets ArA_r and NrN_r in repetition rr, inclusion–exclusion gives

\geq \frac{|A_r|}{1440}+\frac{|N_r|}{1440}-1.$$ The reported feasibility means imply that the pairwise intersection covers **at least two-thirds of the suite on average**. This is a mathematical coverage bound, not a measured pairwise quality result. The existing tables report suite-level method performance and the composition of the twelve-method common feasible set; they do not provide separate scale-specific scores or objective gaps on the larger two-method intersection. Accordingly, the **2.4% versus 4.8%** comparison retains its explicitly defined common-set interpretation.

Thank you to the authors for their detailed responses. I strongly support the approach of incorporating metaheuristics into end-to-end optimization for operations research. The method combines accurate problem formulation with high-quality solutions, outperforming traditional exact-solver-based approaches across multiple datasets. I consider this an interesting contribution that brings automated metaheuristic algorithm design to LLM4OR, and I therefore maintain my recommendation to accept.

We sincerely thank you for the time and care you have devoted to reviewing our paper, and for maintaining your recommendation to accept. We appreciate your recognition of the contribution of bringing automated metaheuristic algorithm design to LLM4OR and its strong empirical results. Your thoughtful feedback has helped us clarify the motivation and methodology, and better communicate the value of connecting executable problem modeling with problem-specific heuristic search.

Reviewer_1fMtOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

In this paper, the authors proposed AutoHOpt, a framework that formulates and solves optimization problems based on natural-language description. AutoHOpt first formulates the optimization problem described by the text, and then designs a customized search process to solve the optimization problem.

The authors tested AutoHOpt on a variety of benchmarks against various baselines, and demonstrated that AutoHOpt achieves state-of-the-art results on most of the benchmarks. The performance metric was how close the solution is to the optimal solution. Ablation study shows the contributions from different components in the two stages of model formulation and search.

Critical strengths

State-of-the-Art Performance. The proposed AutoHOpt achieved the state-of-the-art results in most benchmarks, including the more challenging CoPBench introduced by the authors.

Critical weaknesses

Clarity and Novelty. I don't think the authors gave enough details about the proposed method. While the two-stage process is clearly described at the high level, details of each component are missing.

  • For model construction, Section 4.1 is a quite generic definition of what is formulated, but not how. Section 4.2 touches upon the idea of "learning from failures and reviewing successful models", but goes not give details of the algorithms. In general, many existing works can be considered as "learning from failures and reviewing successful models", for example, all the works that use MCTS, history of past models and solver feedback in the formulation process, and so on. It is not clear if and how the proposed model construction is different.
  • For solution search, Sections 4.3-4.5 are also a little handwaving in describing the details. Therefore, without detailed descriptions of the proposed method and discussions on the differences from existing works at the methodology level, it is hard to evaluate the novelty of the proposed method.
Questions for authors

Please see my main concern in Critical Weaknesses. Below are additional questions.

  • Could you give us motivation of using the search process, instead of a mature solver (e.g., Gurobi, CVX), to solve for the solution?
  • Could you explain why were the results on CoPBench not presented in the main result table (Table 1), because it was specifically introduced in this paper? The results on CoPBench by NEMO and AutoHOpt are briefly mentioned in Section 5.5. But it seems that the performance is better than on existing benchmarks. So is CoPBench more challenging or not?
  • The performance metric of optimality gap may result from two aspects: the correctness of the optimization problem formulation and the efficiency of the search algorithm. Could you give insights on the contributions of each aspect to the overall performance improvement?
5 comments
AuthorsOpenReview ↗

Response to Reviewer 1fMt (1/2)

Dear Reviewer 1fMt,

We thank the reviewer for taking the time to carefully review our work and for the constructive feedback that has helped us improve the quality of our manuscript.

We appreciate your recognition of AutoHOpt's empirical performance. AutoHOpt connects executable problem modeling to automated search design. The same mathematical interface defines how solutions are evaluated and exposes the relationships used to construct and improve them. The revised manuscript makes this connection inspectable through a complete execution, explicit representation contracts and controlled comparisons.

Main concern. Method details and methodological contribution

Model construction. The modeler emits a compact declaration of indexed sets, source parameters and units, typed decisions, objectives and hard relationships. Deterministic compilation resolves indices, checks units and domains, expands supported modeling components, and produces executable expressions together with the exact projection to runtime coordinates.

Semantic feedback. Review continues after successful compilation. Each relationship probe specifies a decision assignment, an observation target and an expected outcome derived from the source. The host executes the probe and compares its result with that expectation. Failed probes become regression evidence for subsequent revisions; changing a decision representation requires explicit migration that preserves the source-derived expectation. Thus, retained history contains executable semantic obligations. MCTS concerns exploration of candidate formulations; this protocol specifies the evidence used to assess and revise them. These mechanisms can be combined. Source probes also examine interpretations that execute successfully, such as marginal versus all-units pricing.

Search design. The accepted model supplies resolved data, domains, expressions, coordinate bindings and a context hash. The designer generates initialization, destroy and repair functions. Destroy identifies released coordinates; repair returns a complete candidate while preserving unreleased coordinates. Syntax, interface and binding checks precede a short pilot. Execution errors and observed violations guide algorithm revision under the fixed model. The ALNS host supplies acceptance, operator adaptation and the feasible-incumbent archive; numerical iterations use the shared evaluator without additional LLM calls.

Appendix O supplies the full IndustryOR 94 input, all three recorded requests and responses, complete representation contracts, compiled evaluator and generated functions. Its replay reproduces the model, four passing probes and the recorded five-million-yuan solution. Section 4.3 now presents the worked mathematical example, and Table 3 reports the fixed-model controls and their mean computational costs. This makes the connection between source semantics and executable search behavior inspectable.

Q1. Motivation for using heuristic search instead of a conventional solver

Heuristic and exact approaches are both widely used in practical optimization. Our motivation comes from their different formulation requirements. Heuristic search can operate once the decision variables, objective and constraint checks are executable. OPTSPEC supplies this interface to our heuristic runtime and cannot be passed directly to Gurobi. A solver-based route requires an additional translation into a supported formulation, potentially involving linearization, logical encodings or other reformulations. These steps can require additional modeling expertise and implementation effort.

AutoHOpt automates the heuristic route by connecting executable problem modeling to customized search. On IndustryOR, our reproduction obtains an operational modeling-success rate of 93%, compared with 65% for NEMO. Here, modeling success means producing a feasible decision that passes all constraints in the original problem. Mean reference-optimal success reaches 83.5% versus 63.3%.

These results illustrate AutoHOpt's main advantage: reducing the formulation burden while achieving strong end-to-end solution quality. Conventional solvers offer powerful optimization capabilities once a correct, supported formulation is available. AutoHOpt provides a practical route that connects natural-language requirements directly to executable modeling and effective heuristic search.

AuthorsOpenReview ↗

Response to Reviewer 1fMt (2/2)

Q2. Why CoPBench is evaluated separately and what it contributes

Table 1 evaluates attainment of the external benchmarks' reference optima under the reference-status policy in Appendix F. CoPBench serves a complementary purpose. It covers larger industrial workloads for which certified global optima are not available across the entire collection. Its objectives and constraints still support direct evaluation of returned plans, allowing meaningful comparisons of feasibility and solution quality.

CoPBench contains 1,440 semantically varied task instances across four industrial domains, ten optimization subtypes and four scales. The tasks differ in operational requirements, constraint interactions, objectives and numerical data within these families. The scale progression extends this coverage to larger scheduling, facility-location, placement and UAV-planning problems. This creates a setting where heuristic methods can demonstrate both their ability to construct feasible plans and their ability to continue improving them.

Accordingly, main-text Table 4 reports feasibility over all 1,440 tasks and objective quality on each run's common feasible intersection across all twelve methods. The quality reference is the best feasible objective observed across the compared methods. AutoHOpt achieves 87.4% feasibility versus NEMO's 79.9%, with conditional mean gaps of 2.4% versus 4.8%.

CoPBench's community contribution is an industrial evaluation suite with semantic diversity, a systematic scale progression and scope for progressively better feasible solutions. These properties make it useful for continued advances in both automatic modeling and heuristic search. Section 5.5 explains the separate evaluation, and Appendix Q details the task structure and construction profiles.

Q3. Contributions of problem formulation and search

Thank you for this insightful question. Final solution quality depends on both recovering the problem requirements and effectively searching the resulting decision space.

The IndustryOR results illustrate this interaction. Under the operational definition in Q1, AutoHOpt achieves 93% modeling success and 83.5% reference-optimal success, compared with 65% and 63.3% for NEMO. These outcomes distinguish the ability to produce a source-feasible solution from the ability to improve that solution to the reference optimum.

Exact solvers are particularly effective when a correct formulation compatible with their modeling interface is available, and can certify optimality when the optimization completes. Heuristic search can work directly with executable objectives and constraints, while its attained solution quality depends on the generated operators and available search budget. Successfully producing a feasible solution therefore does not ensure that the reference optimum will be reached.

With accepted models fixed, replacing generated operators with fixed generic operators reduces success by 15.2/19.4 percentage points on IndustryOR/MIPLIB-NL (Table 2), and full search improves over the constructor-only checkpoint by 22.5/24.8 points, respectively (Table 3). These controls identify the contribution of search design.

AutoHOpt combines broader feasible coverage with effective subsequent optimization. This demonstrates the value of an end-to-end heuristic route for natural-language optimization and gives the community an additional approach with a useful balance between formulation requirements and solution quality.

Dear Reviewer 1fMt,

Thank you for the time and care you have devoted to reviewing our work and for your constructive feedback.

As the rebuttal discussion period is nearing its end, we would be grateful if you could review our responses and let us know whether our clarifications have addressed your concerns. If any questions remain, we would welcome the opportunity to discuss them and provide further explanations while the discussion is still open.

Thank you again for your time and thoughtful consideration.

Best regards,

The authors

Reviewer_Y5WcOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
3(+1)
Presentation
2(+1)
Soundness
2
Read review
Paper summary

In traditional autoformalization for optimization problems, An AI system is given an optimization problem in natural language and translates it to solver code such as gurobi, In AutoHOpt, they instead go from natural language to solved problem in two main stages. In the first stage, they formulate precisely an executable model of the problem (OptSpec). They verify it with AI-generated test cases. In the second stage, they make a problem-specific construction and destory/repair heuristic (SearchIR) that run in a ALNS engine (Adaptive Large neighborhood Search). The engine runs without a solver and it does not require any LLMs during the search. The authors find that for zero-gap success, they achieve better performance than existing methods on 8 out of 10 benchmarks, some with large gains. The authors also introduce a new benchmark CoPBench, which contains 1440 industrial benchmarking tasks.

Overall, the method does seem pretty exciting and novel, and there are a lot of cool reported results. But it is a bit difficult to understand how strong they are as it's currently written, since the numbers are only reported with exact match to optimal values and since the paper is a bit hard to read. If it gets cleaned up a bit, it would be an even stronger paper.

Critical strengths

From what I can tell, the general high-level idea is novel in taking natural language and translating it to a problem-specific ALNS. The numbers reported in the paper make this method look very strong, so if they hold up, this method is very promising. The authors evaluate the approach in many different areas on many benchmarks, and they also have many baselines and many runs. The model is fixed before the search, so the LLM is unable to make the problem easier by dropping constraints during the search. If the model finds all the constraints, the solutions is valid.

Critical weaknesses

It's pretty hard to follow the paper, both when it comes to visualization (such as Figure 1) and to having a lot of information that I wasn't able to parse, even after looking at it for a few minutes. To understand how the pipeline works, one needs to both look at the figure and read a couple of the sections, including some that are in the appendix. The paper could benefit from more details in some places, such as about the new benchmark. For instance, the coupled, tight and stress variants of CoPBench variant are not clearly defined. Comparing exactness rather than tolerance, like 1e-6, means it is a bit hard to tell if the reason their methods do better is because of the solver's slight inexactness and floating-point numbers, or if it is a real gain. It would be useful to also have some numbers with a small margin of error.

Questions for authors

If it is possible, it would be interesting to see what the impact of OptSpec-> Gurobi is, to understand how much of the gain comes from OptSpec and how much comes from the metaheuristics. It will be much easier to trust the paper if the numbers are reported with a tolerance, since I don't feel confident giving a higher score without this, but I would also love to understand if I'm misunderstanding something, and it makes more sense to do zero tolerance.

2 comments
AuthorsOpenReview ↗

Response to Reviewer Y5Wc (1/2)

Dear Reviewer Y5Wc,

We thank the reviewer for taking the time to carefully review our work and for the constructive feedback that has helped us improve the quality of our manuscript.

We appreciate your recognition of the potential of connecting natural-language modeling to problem-specific search. The revision makes the numerical criterion explicit, provides a complete implementation example, and defines CoPBench's construction profiles. We address the tolerance first because it directly determines how to interpret the reported gains.

1. Results under a numerical tolerance

We adopt source-feasible reference-optimal success with normalized objective gap

δ=∣fsrc(x)−z⋆∣max⁡(1,∣z⋆∣)<10−6.\delta=\frac{|f_{\mathrm{src}}(x)-z^\star|}{\max(1,|z^\star|)}<10^{-6}.

For tasks with a finite reference optimum, the criterion requires a source-feasible decision and objective agreement. All tasks remain in the denominator under the reference-status policy in Appendix F. Objective agreement is checked before display rounding, separately from feasibility and integrality. The normalization accommodates zero and negative reference objectives.

With this tolerance, each dataset's success rate changes by less than 0.1% in relative terms compared with exact matching. All table values remain unchanged at the reported precision. IndustryOR success remains 83.5% for AutoHOpt versus 63.3% for NEMO, and MIPLIB-NL remains 78.8% versus 69.3%. The gains of 20.2 and 9.5 percentage points therefore persist under numerical tolerance. Appendix F and the table captions now specify the criterion consistently across methods.

2. Complete representation definitions and implementation example

The modeling stage produces executable OPTSPEC decision domains, objectives and hard constraints. The search-design stage receives the accepted model's resolved data, expressions and runtime-coordinate projection, and generates initialization, destroy and repair functions. Numerical search uses the fixed evaluator throughout algorithm revision and execution.

Section 4.3 presents a worked mathematical example, and Sections 4.2 and 4.4 specify the feedback supplied to revisions. Appendix O provides the complete IndustryOR 94 task, all three recorded requests and responses, representation contracts, compiled objective and hard predicates, runtime-coordinate mapping, relation probes and generated code. Its offline replay reproduces the model, four passing probes and the recorded five-million-yuan solution. Together, the overview and executable example connect each stage's input, output and validation procedure.

3. Definitions of CoPBench's coupled, tight and stress profiles

These profiles control resource slack and interaction strength within each size tier. Coupled combines interacting rules with larger construction-relative margins; tight reduces margins and available alternatives; stress further increases these pressures. Profile and scale are crossed. Each profile contains 480 tasks, and each domain-by-scale-by-profile cell contains 30.

For example, facility capacity/policy-budget slack is 35%/18%/6%, placement occupancy density is 0.36/0.46/0.57, and the UAV hard-window slack base is 18/10/5 minutes for coupled/tight/stress. Appendix Q now specifies the domain-specific parameters and construction rules, including rounding and the applicability of optional constraints. Individual task configurations determine which policies are active. This distinguishes profile settings, problem size and task-specific constraint configurations.

4. Contributions of model-conditioned search

The fixed-model controls quantify the value of generated search after model acceptance. Replacing generated operators with fixed generic operators reduces reference-optimal success by 15.2/19.4 percentage points on IndustryOR/MIPLIB-NL (Table 2). Under common resource ceilings, full search improves over its constructor-only checkpoint by 22.5/24.8 points (Table 3). These comparisons identify gains from generated neighborhoods and continued search beyond producing the model and an initial decision.

Table 3 now places success and actual costs together in the main text. AutoHOpt uses a mean of five LLM calls, 46k tokens and 99 seconds per attempted task, including failed tasks, pilots and revisions. The numerical search reuses the generated program without further LLM calls.

AuthorsOpenReview ↗

Response to Reviewer Y5Wc (2/2)

5. Why OPTSPEC is designed for heuristic execution

The question about using Gurobi with OPTSPEC concerns the central motivation of AutoHOpt. OPTSPEC supplies executable objectives, hard predicates and decision bindings to a heuristic runtime. Gurobi requires an additional translation into an equivalent formulation through its supported modeling interface. AutoHOpt uses directly evaluable relationships to reduce this reformulation burden and to guide task-specific construction and repair. Appendix N illustrates the distinction using IndustryOR94's marginal-cost rule and an equivalent ordered-segment MILP.

Our detailed discussion of the formulation requirements and the respective contributions of modeling and optimization appears in our responses to Reviewer 1fMt, Q1 and Q3. Together with the fixed-model controls in Table 3, these results support executable modeling coupled with model-conditioned search.

Reviewer_FyxsOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces AutoHOpt, an LLM-based workflow for solving optimization problems from natural language problem descriptions. In the first stage, an llm translates the problem into an executable intermediate representation (OptSpec), which is checked against the problem requirements and refined when errors are detected. In the second stage, the llm generates a constructor for inital solutions, along with destroy and repair operators for an adaptive large neirborhood search (ALNS) framwork. The generated search program components are tested for syntax and interface error, followed by a short ALNS tiral, and are refined if needed. ALNS then use them to search for better solutions, utilizing OptSpec to evaluate feasibility and objective values. The paper also introduces CoPBench, a benchmark comprising 1440 optimization tasks across industrial domains.

Critical strengths
  1. AutoHOpt proposes an interesting framework that uses an executable intermediate representation to guide the generation of LNS operators for different optimization problems and evaluate candidate solutions. This connection between mathematical modeling and automated search design is a meaningful contribution that may inspire further research.

  2. The experiments are extensive, with strong empirical performance and substantial improvements over the evaluated baselines across multiple benchmarks. The evaluation is further strengthened by reporting the mean and standard deviation over 30 independent runs.

Critical weaknesses
  1. The search relies on an accepted and executable OptSpec. Since OptSpec already specifies the objective and constraints, it is unclear why generating additional LNS components (initialization, destroy, and repair functions) per problem is preferable to translating the mathematical model into solver code and using a conventional optimization solver, where applicable. The paper lacks a direct comparison using the same accepted OptSpec representation that would clarify the benefit of the additional search design stage.

  2. The presentation lacks sufficient detail and context. After carefully reading Section 4.1 and Appendices A-E, I could not find a complete definition of the OptSpec schema, prompt templates for the current pipeline, or a complete example showing the generated OptSpec and corresponding SearchIR implementation. This makes it difficult to understand how the central representations are constructed and connected. Other issues include quantities introduced without clear definitions, such as Eqs. (3)--(4) and (13).

  3. The evaluation requires an exact zero objective gap, including for tasks involving continuous variables and numerical solver outputs. The paper does not sufficiently justify this criterion.

Questions for authors
  1. Could the authors provide a complete worked example showing the inputs and outputs of both stages, including (1) the input prompt, the generated OptSpec, and its compiled evaluator, and (2) the input to the search designer, together with the resulting SearchIR code for the initialization, destroy, and repair functions?

  2. With the objective evaluation tolerance set to zero, how are numerical discrepancies from floating point computations or solver tolerances handled across the evaluated methods?

  3. Appendix F (Source-feasible zero-gap success) states that returned decisions are independently checked against the original requirements and their objectives recomputed from source data. How are the original objective functions and feasibility checkers obtained across the evaluated benchmarks, and how are returned solutions mapped to the decision variables used by these evaluators

Minor questions: 4. How are the composable modeling components introduced in Section 4.1 constructed? How frequently are they used across the evaluated benchmarks, and what is their contribution to performance?

  1. App. B and C provide qualitative analyses of several IndustryOR instances. Did the authors encounter any potentially problematic ground truths during this analysis, similar to the issues reported by NEMO (Section 4.3)?

  2. CoPBench is presented as a main contribution and comprises 1440 tasks. Do these tasks represent semantically distinct optimization problems or primarily parameterized instances of a smaller number of problem formulations?

8 comments
AuthorsOpenReview ↗

Response to Reviewer Fyxs (1/3)

Dear Reviewer Fyxs,

We thank the reviewer for taking the time to carefully review our work and for the constructive feedback that has helped us improve the quality of our manuscript.

We appreciate your recognition of the connection between executable modeling and automated search design. The revisions clarify the representations, their connection to search, and the evaluation protocol.

W1. Executable modeling and model-conditioned search

AutoHOpt connects executable problem semantics to automated heuristic design. OPTSPEC defines decision domains, objectives and feasibility predicates in directly evaluable form. Gurobi cannot optimize this representation directly: a translation must produce an equivalent model through its supported interfaces. Executable objectives and predicates do not themselves supply that formulation. Heuristic search can operate on the original decisions through these evaluations, reducing solver-specific reformulation for supported constructs while preserving the accepted objective and hard predicates.

IndustryOR 94 makes this distinction concrete. A refinery purchases q∈[0,1500]q\in[0,1500] tonnes of crude A with decreasing marginal prices. OPTSPEC directly represents

C(q)=10000 min⁡(q,500)+8000 min⁡(max⁡(q−500,0),500)+6000 max⁡(q−1000,0).C(q) = 10000\,\min(q,500) + 8000\,\min(\max(q-500,0),500) + 6000\,\max(q-1000,0). One equivalent explicit MILP retains the original purchase and blending decisions, adds t1,t2,t3∈[0,500]t_1,t_2,t_3\in[0,500] and y2,y3∈{0,1}y_2,y_3\in\{0,1\}, and imposes

q&=t_1+t_2+t_3,\\ t_2&\leq500y_2,&t_1&\geq500y_2,\\ t_3&\leq500y_3,&t_2&\geq500y_3, \end{aligned}$$ with cost $10{,}000t_1+8{,}000t_2+6{,}000t_3$. The ordering conditions preserve the marginal-price semantics: without them, a 100-tonne purchase could cost 600,000 instead of 1,000,000 yuan. Gurobi's piecewise-linear API can automate the reformulation. OPTSPEC expresses the source rule in one cost component and delegates its executable expansion to the compiler. Section 4.3 now presents the worked model-to-search example; Appendix N gives the full reformulation comparison in blue. **SEARCHIR supplies the search procedure that an evaluator leaves unspecified.** Evaluating feasibility and objective quality does not determine how to construct a feasible decision or improve it. SEARCHIR uses model structure to generate construction and coordinated destroy and repair moves, then searches under the fixed evaluator. Holding the accepted model fixed, generated operators improve success over fixed generic operators by 15.2/19.4 percentage points on IndustryOR/MIPLIB-NL (Table 2). Under the common budget, full search also improves over its constructor-only checkpoint by 22.5/24.8 points (Table 3). These controls establish the contribution of search design after model acceptance. **The complete system achieves both broader feasible coverage and high reference-optimal attainment.** In our IndustryOR reproduction, AutoHOpt returns source-feasible decisions on 93% of tasks versus 65% for NEMO. Table 1 reports mean reference-optimal success of 83.5% versus 63.3% over 30 independent runs, a 20.2-percentage-point gain. AutoHOpt achieves the highest mean on eight of ten benchmarks. Table 1 evaluates the complete natural-language-to-solution pipelines; the fixed-model controls identify the value of the generated search. Together, these results support **direct executable modeling coupled with structure-guided search**. **W2 and Q1. Representations, prompts and a complete execution** Appendix O makes the model-to-search interface fully inspectable through IndustryOR 94. It includes the complete original task, all three system prompts and user payloads, the generated compact model, compiled objective and hard predicates, exact runtime-coordinate projection, relation probes, and the full initialization, destroy and repair code. The interface map distinguishes the compact declaration, typed mathematical IR, executable OPTSPEC and generated SEARCHIR envelope, with their field contracts and complete case-specific generated objects. An offline replay reconstructs all three requests exactly, executes the saved responses and reproduces the recorded model and search program. All four probes pass; 10,000 ALNS iterations return the same solution with profit of five million yuan. Appendix P displays a standalone exact-rational checker that recomputes feasibility and profit directly from the public requirements, together with its input, individual check outcomes and altered-decision tests. The complete prompt-to-program example and compiled evaluator expressions are shown in Appendix O. Section 4.1 now defines the inventory quantities and relationship diagnostics in Eqs. (3)–(4). Appendix E defines the operator weight and mean segment reward in Eq. (13). These additions connect the mathematical interface to an executable, replayable instance of the method.
AuthorsOpenReview ↗

Response to Reviewer Fyxs (2/3)

W3 and Q2. Numerical objective agreement

Thank you for highlighting numerical precision. We adopt reference-optimal attainment at normalized gap below 10−610^{-6},

δ=∣fsrc(x)−z⋆∣max⁡(1,∣z⋆∣)<10−6.\delta=\frac{|f_{\mathrm{src}}(x)-z^\star|}{\max(1,|z^\star|)}<10^{-6}.

This accommodates floating-point discrepancies in continuous decisions and solver outputs. The source objective is recomputed in its original units before display rounding; source feasibility and integrality remain separate checks. NEMO likewise uses a 10−610^{-6} relative-objective threshold; our denominator retains the manuscript's handling of zero and negative reference objectives.

The precision check changes each dataset's success rate by less than 0.1% relative to exact matching. All table values remain unchanged at the reported precision. Appendix F now specifies this criterion consistently across the benchmark comparisons.

Q3. Source evaluators and decision mappings

Source validation and reference-optimum comparison are separate operations. The original task description and numerical inputs specify the decision domains, objective terms and operational rules. The assessment implementations use released mathematical data and reference declarations, including NL4Opt gold declarations for formulation assessment and MIPLIB-NL public CSVs for numerical reconstruction; separately implemented source-based checks for IndustryOR, MAMO and OptMATH; and separately held numerical instances and domain verifiers for CoPBench.

An explicit mapping records entity labels, index order, units and declared scaling while preserving submitted quantities. AutoHOpt's compiler records the mathematical-to-runtime projection. The source adapters identify the saved quantities required by each checker. IndustryOR supports field pointers, label mappings and unit conversions; MIPLIB-NL uses declared variable/index references bound to model and result hashes.

Appendix P gives the complete standalone implementation for the recorded IndustryOR 94 decision. buy_A denotes additional purchased tonnes. The mathematical quantity xA,IIx_{A,II} maps to the literal key ["A","II"] inside blend, denoting tonnes of crude A assigned to grade II. The source checker added in this revision implements the original purchase, material and composition rules independently of the generated evaluator. It recomputes 14 million yuan revenue minus 9 million yuan purchasing cost, yielding 5 million yuan profit. Its full input, individual outcomes, five rejected altered candidates and five purchasing-cost test points are displayed in the appendix. Reference comparison then uses the frozen decision's source-recomputed objective.

Q4. Construction, usage and contribution of composable components

The components are parameterized mathematical templates in the compiler. The LLM selects source-grounded semantics and supplies expressions, indices, units and event choices. Deterministic expansion produces derived expressions and hard constraints using the same typed evaluation path as explicit formulas. The templates cover batch purchasing, inventory balances, activation links, shift coverage and tiered pricing. IndustryOR 94's marginal_cost declaration illustrates this mechanism in Appendix O.

Appendix O.6 also reports a usage audit of three archived runs. The 16 October IndustryOR run has components in 14 of its 86 reviewed compact models, with 17 declarations. The archived NL4Opt current88 and MAMO-Complex NEMO153 runs have zero component declarations in their 88 and 149 reviewed models, respectively. Each declaration count matches the corresponding compiler-expansion audit. These are per-batch artifact counts; Table 1 reports repeated end-to-end benchmark aggregates.

The components reduce repeated arithmetic and indexing in generated models, centralize checks of units, event timing and threshold semantics, and preserve traceability to the originating requirements. Their role is reliable formulation through reusable mathematical expansion, followed by search under the resulting fixed evaluator.

AuthorsOpenReview ↗

Response to Reviewer Fyxs (3/3)

Q5. IndustryOR reference-quality reassessment

Yes. Our reassessment examines the 15 IndustryOR cases flagged in the NEMO-based audit. We recover source-feasible plans for eight cases, 16, 22, 58, 73, 77, 80, 90 and 98. Seven cases remain flagged for correction or clarification, 2, 29, 65, 69, 71, 76 and 97.

The added instance-level audit separates source ambiguity, variable-domain interpretation and reference-objective consistency. The recovered plans establish feasibility for these eight cases; reference objectives and other annotations are checked separately. The main comparison retains all 100 IndustryOR tasks and their original references. This preserves a common evaluation set while making the individual data issues and recovered feasible cases explicit.

Q6. CoPBench's semantic coverage and community role

CoPBench comprises 1,440 industrial task instances across four domains, ten optimization subtypes and four scales. Semantic variation concerns the operational requirements and interacting constraints within these families, alongside numerical and scale variation. Examples include machine eligibility and shared personnel in scheduling, resource and coverage rules in facility location, geometric restrictions in placement, and capability and timing rules in UAV planning. Appendix Q provides the subtype and scale breakdown.

Its purpose is to extend language-to-optimization evaluation to industrial settings where both feasible coverage and continued objective improvement matter. Separately held feasibility checkers and best-observed feasible references support this evaluation even when a certified optimum is unavailable. In main-text Table 4, AutoHOpt reaches 87.4% full-suite feasibility versus NEMO's 79.9%, and reduces the mean gap on the common feasible intersection from 4.8% to 2.4%. CoPBench provides a community benchmark for modeling, scale and solution quality, with scope for progressively better feasible plans.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal Summary

Dear Area Chair,

Thank you and the reviewers for the time and care devoted to evaluating our submission. We would like to briefly summarize the outcome of the rebuttal discussion.

Four reviewers now recommend acceptance. Y5Wc, 1fMt, and BFSV raised their ratings to 3 (weak accept), while C3ZX maintained 4 (clear accept).

The main revisions clarify:

  • The method and motivation. Complete prompts, schemas, interaction traces, and worked examples make the OPTSPEC-to-SEARCHIR workflow inspectable. Fixed-model controls establish the contribution of generated heuristic search.
  • Evaluation rigor. We adopted the 1e-6 objective-gap tolerance and clarified source-based validation, decision mappings, correction provenance, and temperature settings.
  • CoPBench construction and evaluation. We explain how the 1,440 tasks were generated and checked, and distinguish full-suite feasibility from objective quality on the common feasible intersection.

1fMt explicitly confirmed that the concerns about the method, new benchmark, and motivation for heuristic search were addressed. BFSV likewise confirmed that the main concerns were addressed.

Fyxs, who raised their score from 1 to 2, remains the only reviewer recommending weak rejection. Their remaining questions about the motivation for heuristic search and methodological novelty overlap with issues discussed with other reviewers. We provided a further response supported by the fixed-model controls and reformulation discussion. Y5Wc explicitly recognized the approach as original and promising, while C3ZX reaffirmed the contribution of bringing automated metaheuristic design to LLM4OR.

We hope this summary assists your assessment of the revised manuscript. We sincerely appreciate your coordination and all reviewers’ constructive contributions to improving the paper.

Best regards,

The authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.