acceptodds
Under review as a conference paper at ICLR 2027

Autoresearch in Mixed-Integer Programming via Idea Pooling and Algorithm Tree Search

Abstract

Despite recent progress in autoresearch, applying it to practical operations research problems, which are typically formulated as NP-hard mixed-integer linear or nonlinear programs (MILPs or MINLPs), requires more systematic and efficient research capabilities to manage competing ideas and experimental trajectories. We introduce AutoMIP, a reusable agent skill for organizing long-horizon autoresearch in MIPs through idea pooling and algorithm tree search. AutoMIP maintains a persistent pool of complementary candidate ideas, and organizes executable experiments in an algorithm tree, allowing the agent to preserve untried hypotheses, refine promising algorithms, and switch to new methodological directions from historical states. AutoMIP achieves the highest final success rate for MILP or MINLP benchmark cohorts. In MIPLib, AutoMIP reaches new best solutions for 31 out of 60 instances, more than current autoresearch frameworks; In MINLPLib, AutoMIP is more advantageous with a success rate of 52 out of 60. The complementary effects of idea pooling and algorithm tree search are evidenced by the ablation study.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Autoresearch, MIP, idea pool, algorithm tree

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_SWrLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes AutoMIP, an autoresearch framework for mixed-integer linear and nonlinear programming. It combines a persistent idea pool with an algorithm tree to manage research directions and reusable experimental states. The agent can refine the current approach or switch to an alternative direction by restoring a compatible historical state. The paper evaluates AutoMIP on MIPLIB Open and Hard instances and MINLPLib, comparing success rates, runtime, token consumption, and final solution quality. Ablation experiments examine the contributions of idea pooling and algorithm tree search.

Critical strengths
  1. The persistent idea pool is an interesting design. It preserves untested research hypotheses independently of the current experiment, allowing the agent to explore alternative directions without losing previous ideas.

  2. The algorithm tree explicitly connects research directions with compatible historical states and executable artifacts. This allows the agent to resume experiments from earlier states rather than always continuing from the latest implementation.

Critical weaknesses
  1. The reported efficiency statistics contain internal inconsistencies. In Table 1, EvoX's overall mean token consumption is 2,297,150, exceeding every reported interval mean, whose maximum is 1,586,613. If the intervals form disjoint groups covering all runs, the overall mean must be a weighted average and cannot exceed every group mean. Table 2 reports an AutoMIP mean runtime of 2,035 seconds. However, the 30 successful runtimes in Table 9 sum to 27,398 seconds. Counting the remaining 30 runs at the stated 3,600-second deadline gives an overall mean of 2,256.63 seconds. The reported value therefore requires a different aggregation or failure-handling rule that is not clearly explained. These inconsistencies directly affect the claimed runtime and token advantages. The authors should clarify the aggregation rules and reconcile the reported statistics with the per-instance results.

  2. The claims of new best-known solutions are not sufficiently supported by the reported experimental evidence. The paper claims new best solutions on 31/60 MIPLIB instances and 52/60 MINLPLib instances. However, the per-instance tables mainly report success labels, runtime, and tokens, without providing the reference objectives, final objectives, or improvement magnitudes. Improving a supplied starting incumbent does not necessarily establish a new best-known benchmark result. The paper also does not clearly specify the success criterion for MINLPLib. Although an independent verification procedure is described, the corresponding per-instance numerical verification results are not reported. Without these results, it is difficult to verify the claimed new records and final-objective wins.

Questions for authors
  1. Could the authors clarify the exact aggregation rules for runtime and token consumption, including how unsuccessful runs are handled? In particular, how are EvoX's overall token mean in Table 1 and AutoMIP's 2,035-second mean runtime in Table 2 obtained from the detailed results?

  2. Could the authors provide the reference objective, final verified objective, and improvement magnitude for each successful MIPLIB Open and MINLPLib instance? How are new best-known solutions distinguished from improvements over starting incumbents, and what exact success criterion is used for MINLPLib?

5 comments
AuthorsOpenReview ↗

Response to Reviewer SWrL

Dear Reviewer SWrL,

We sincerely thank the reviewer for recognizing the novel design of AutoMIP, particularly the persistent idea pool and algorithm tree for managing research directions and reusable experimental states. We acknowledge the concerns regarding the paper's presentation and soundness and address them point by point below.

[W1, Q1] Clarification of the inconsistent results.

Thank you for identifying the aggregation inconsistency. Indeed, the time intervals form disjoint groups covering all runs. As we stated in Metrics paragraph on page 7, w/s column counts wins/solved runs in each interval, whereas time and token columns average over the runs assigned to that interval, including unsuccessful runs. The columns under Total average over all 60 instances. For an unsuccessful run, we use its recorded elapsed wall-clock time and token consumption rather than substituting fixed budget values.

This aggregation rule illustrates the number of instances solved within each time interval, the number of wins when multiple methods solve the same instance, and thus the number of instances that remain unsolved after the full long-term run. It also successfully provides a comprehensive comparison of different autoresearch from perspectives of time and token in different time intervals and over the entire execution period.

EvoX token consumption in Table 1. The previous inconsistency resulted from a miscalculation. We have corrected EvoX's 6--12 h token mean in Table 1 from 1,244,128 to 2,774,485. The previous value did not include the token consumption of all unsuccessful runs assigned to that interval, which caused the inconsistency. For EvoX, we recorded the solved 6, 5, 3, and 1 runs in the four disjoint groups, respectively. The final group however contains the other 45 unsuccessful runs, which we should have included in the average token calculation. Consequently, the corrected interval mean give

$\frac{6\times (306{,}763)+5\times (720{,}454)+3\times (1{,}586{,}613)+46\times (2{,}774{,}485)}{60} \approx 2{,}297{,}150$

which now agrees with the reported Total token mean. We have updated the results in our paper.

AutoMIP mean runtime in hard set. We apologize for the misleading results. The maximum budget for the Hard instance is 3,600 seconds. An autoresearch agent iteratively tries different methods for attaining the best-known objective per Hard instance, as we stated in the experimental setting. However, in our original paper, we recorded the time point for the agent finding the latest improved objective values, for each of the 30 unsuccessful runs (i.e., those that did not reach the target objective within the allocated budget), since it fails to find a better objective value till 3,600s. These recorded time points are less than 3,600s. Consequently, these time points lead to a smaller average value (2,035) over the total 60 instances than otherwise 2,256.63 seconds if we assign exactly 3,600 seconds to each of the 30 unsuccessful runs.

We agree that assigning the full 3,600-second budget to each unsuccessful run provides a more consistent and transparent measure of the total computational budget. Accordingly, we have updated the reported AutoMIP mean runtime from 2,035 to 2,257 (rounded from 2,256.63) seconds in Table 2 of the revised paper, calculated using the 27,398 seconds accumulated across the 30 successful runs and the full 3,600-second budget for each of the remaining 30 unsuccessful runs. We thank the reviewer for pointing out this issue. Inspired by your comment, we will release all computation logs upon acceptance to facilitate reproducibility and verification of the reported results.

[W2, Q2] Reference and final verified objectives, success criterion for MINLPLib, and improvements over starting incumbents.

Reference and final verified objectives. We have added the public reference objective and final verified objective for each instance in the Open and MINLPLib per-instance tables (Appendices E and J). The public reference objectives were released as recently as January 26, 2026, and September 2026, respectively. For these minimization instances, we define the absolute improvement as △=zref−zfinal\triangle=z_{ref}-z_{final}. Our criterion is: a result is counted as a new best-known solution only when its independently verified objective strictly improves the public best-known objective. Therefore, improving only the agent's own starting incumbent does not meet this criterion. Under this criterion, we confirm that the 31 Open and 52 MINLPLib successes are improvements over their respective public references; for example, 'allcolor58' improves from 258 to 221 (△=37\triangle=37), and 'z26' from -1198 to -1201 (△=3\triangle=3).

Success criterion for MINLPLib. MINLPLib uses the same success criterion as Open in MIPLib: within different intervals in 43,200 seconds, a research agent finds and independently verifies a feasible solution strictly better than the public best-known objective. If no feasible reference solution is known, finding and verifying a feasible solution counts as success. We have clarified this point in the revised manuscript, with the corresponding changes highlighted in blue on page 7. For your reference, also, we have added the corresponding MINLPLib task template to Appendix B, highlighted in blue on page 21.

Improvements over starting incumbents. Following your suggestion, we also have added results in Tables 23-25 with the corresponding analysis in Appendix K to quantify the improvements achieved over the starting incumbents on the Open, Hard, and MINLPLib instances. The results demonstrate that AutoMIP can substantially improve existing feasible solutions, with maximum relative improvements of 26.78% (‘allcolor58’), 8.01% (‘neos-5196530-nuhaka’), and 7.13% (‘acopf_caseactivsg70k_qcqp’), respectively. Notably, improvements exceeding 5% are achieved on 8 Open, 3 Hard, and 5 MINLPLib instances. These findings provide further evidence that AutoMIP can go beyond simply retaining the initial solutions and discover effective solving strategies that yield better objective values.

Dear Reviewer SWrL,

As the rebuttal period is approaching its deadline tomorrow, we would greatly appreciate it if you could take a moment to review our responses and the revised manuscript. Highlights in our responses will make it easier for you to locate and review the corresponding revisions in the paper. We appreciate any further feedback, so that we can still reply them before the discussion closes. Thank you very much for your time and consideration!

Reviewer_aonMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes AutoMIP, an LLM-based autoresearch framework for MILP and MINLP optimization. The method maintains a persistent pool of alternative research directions and an algorithm tree that records executable experimental states and artifacts. During search, the agent either refines the current direction or switches to another direction from a compatible historical state. The framework is evaluated on MIPLIB and MINLPLib against direct Codex, a sequential research loop, AutoEoH, and EvoX.

Critical strengths
  1. The experimental study is extensive and the reported results consistently show that AutoMIP improves task completion, final-objective wins, and token efficiency over the evaluated autoresearch baselines.

  2. The paper reports how success and objective quality evolve over time, rather than only the final outcome. These trajectories make the relative research progress of different methods easier to interpret.

Critical weaknesses
  1. The novelty of the research-control mechanism needs stronger clarification. ARTS [1] already models automated discovery as search over research hypotheses and executable experiment states: its scientist selects a historical parent state and the next hypothesis to explore, then builds and evaluates a new code state from that parent. This is conceptually close to AutoMIP's central idea of deciding what research direction to pursue and where to resume. The persistent unused-idea pool and artifact-compatibility rules are additional design choices, but the paper should explain more clearly what substantive advance they provide beyond this prior work.

  2. The paper does not show representative examples of the final algorithms discovered for individual instances. Figure 4 mainly reports convergence behavior, but does not expose the actual sequence of research directions, code/solver modifications, failures, and resulting candidate performance. Providing several real search traces together with the final executable solving pipeline would make it much clearer what AutoMIP actually discovers and how the proposed tree-search mechanism contributes to the result.

  3. The current evaluation is instance-level: AutoMIP conducts a fresh autoresearch process for each instance. It is therefore unclear whether a discovered solving strategy transfers to unseen instances from the same problem family. The paper should report the clean-run execution time of the final discovered solver/algorithm after the research process is removed, and evaluate whether the frozen strategy generalizes across instances.

  4. The evaluation does not include a direct same-budget Gurobi baseline on the original model. Reporting the objective attained by Gurobi under the same wall-clock budget would provide an important reference for judging whether the additional per-instance autoresearch process produces a practically meaningful advantage over simply spending the budget on a strong MILP solver.

Reference:

[1] Gurusha Juneja, Arnav Kumar Jain, Deepak Nathani, William Yang Wang, and Xin Eric Wang. "Learning the ARTS of Search for Automated Discovery." arXiv preprint arXiv:2606.21891, 2026.

Questions for authors
  1. How does AutoMIP differ from ARTS [1] beyond maintaining a persistent pool of unexecuted ideas and using artifact-compatibility rules for parent selection? Could the authors provide a more direct comparison or ablation that isolates this distinction?

  2. Could the authors provide representative complete search traces for several instances, including the generated research directions, code/solver changes, failed attempts, and the final executable algorithm?

  3. What is the standalone runtime and solution quality of the final discovered algorithm when rerun without the autoresearch process? Does the discovered strategy transfer to unseen instances from the same problem family?

  4. How does AutoMIP compare with running Gurobi directly on the original instance for the same total wall-clock budget?

Reference:

[1] Gurusha Juneja, Arnav Kumar Jain, Deepak Nathani, William Yang Wang, and Xin Eric Wang. "Learning the ARTS of Search for Automated Discovery." arXiv preprint arXiv:2606.21891, 2026.

5 comments
AuthorsOpenReview ↗

Response to Reviewer aonM

Dear Reviewer aonM,

We sincerely thank the reviewer for recognizing the effectiveness of AutoMIP and the comprehensiveness of our experimental evaluation. We also appreciate the positive assessment of the paper's presentation and informativeness, particularly the value of analyzing research progress over time through success and objective-quality trajectories.

[W1, Q1] Advance beyond ARTS.

Thank you for pointing us to the very recent work ARTS [1]. We agree that ARTS also formulates automated discovery by searching hypotheses and executable experiment states, where the scientist jointly selects a hypothesis and a historical parent state. We acknowledge this conceptual connection and have revised the paper to further clarify AutoMIP's novelty. Particularly, we have revised the related work to include ARTS and make our advances more explicit, with the revision marked in blue on page 3.

The substantive difference is in how research directions and their executable dependencies are explicitly managed throughout long-horizon optimization research. Specifically, AutoMIP introduces two complementary mechanisms:

  1. Persistent management of untried research directions. Unlike generating and sampling candidate hypotheses during individual search steps, AutoMIP maintains an explicit, persistent idea pool independently of the active experimental trajectory. Each candidate records its hypothesis, supporting evidence, distinction from previously attempted directions, starting requirements, potential failure risks, and lifecycle status. An unused direction remains available while the current direction is being refined and is consumed only when activated as a new branch. This allows AutoMIP to preserve and systematically revisit complementary research directions rather than relying solely on the next hypothesis-generation decision.

  2. Artifact-aware historical state selection. More importantly, AutoMIP does not select a historical parent solely based on its research potential. It explicitly establishes a compatibility relation between a candidate research direction and a historical experimental state. A state is eligible only if its available artifacts satisfy the starting requirements of the selected direction. This is particularly important for MIP/MINLP algorithm research, where different methodological directions may require different representations, solver configurations, algorithmic components, generated models, and intermediate files. AutoMIP therefore selects the starting state from which a proposed direction can actually be executed, restores its associated artifacts, and records the resulting dependency in the algorithm tree.

These mechanisms are integrated into an explicit search protocol that distinguishes local refinement (L-edges) from activating an unused research direction (R-edges), while maintaining the corresponding experimental dependencies and recoverable artifacts. Thus, the contribution is not merely hypothesis exploration, which ARTS has indeed demonstrated, but an explicit coupling of persistent hypothesis lifecycle management with artifact-constrained experimental-state selection for iterative MIP/MINLP algorithm development.

We also highlight the complementary roles of the idea pool and algorithm tree through our ablation results on the 20 shared Open instances. As reported in Table 3, AutoMIP achieves 9/15 wins/solved, compared with 4/12 for NoPool and 2/10 for NoTree. These results demonstrate the complementary contributions of our two main advances: preserving untried research directions and reusing compatible historical experimental states. Meanwhile, AutoMIP has outperformed the competing research agents in solution quality, token consumption, runtime over MIPLib and MINLPLib instances, which again demonstrate the effectiveness of the proposed advances.

[W2, Q2] Representative examples of the algorithms.

We thank the reviewer for the suggestion. We have added four per-instance search traces alongside analysis in Appendix D for allcolor58, zeil, z26, and supportcase38 instances. The figures and accompanying descriptions identify the initial strategy, the research directions selected at subsequent branches, changes to the solver procedure, unsuccessful attempts, and the historical states used when switching directions. For example, the zeil trace shows an infeasible row-cluster LNS branch and a SCIP search without improvement in the beginning of the autoresearch, followed by a return to an earlier Gurobi state and improvements through NoRel and cutoff refinement. The z26 trace shows unsuccessful random LNS, subsequent improved LP-root seeding and repair, and a final ReduMIS candidate. Each trace reports the final objective verified on the original MPS instance. Given these examples, we make it much clearer what AutoMIP actually discovers and how the research progresses to the final result. Given the paper limit for more illustrations, we will release the link to all the search logs upon paper acceptance.

[W3, Q3] Clean-run execution time and generalization of the found algorithm.

Thank you for this constructive comment. Although our main experiments focus on the autoresearch performance, we have added the evaluation with the discovered algorithms. We evaluated the discovered algorithms on instances totally unseen during autoresearch. We fixed each algorithm before the target run and measured its standalone wall time from launch through solution export and independent verification on the original instance. We have added Table 6 on page 10 in the revised paper to report the public reference objectives with runtimes, as well as our final verified objectives with runtimes. For your convenience, we also provide the table below with an analysis followed.

Research instance → unseen instanceDiscovered algorithmOfficial objective/timeVerified finalTime
sorrell7 → sorrell8Conflict graph + KaMIS-350/3,489s-350183.840s
sct1 → sct32Radius-32 local branching-17.887559/65h-17.652357124.427s
uccase10 → uccase9Bootstrap + fixed-window LNS10,993.131409/Not reported11,005.04751865.803s

The discovered algorithms demonstrate strong transferability, achieving reference-quality solutions on unseen MIPLIB instances with remarkably short runtimes. Specifically, the conflict-graph and KaMIS strategy discovered from sorrell7 matches the official reference objectives on sorrell8, obtaining -350 in 183.840 seconds. Notably, the official reference for sorrell8 required 3,489 seconds with Gurobi 9.0. The radius-32 local-branching strategy discovered from sct1 obtains -17.652357 on sct32 in just 124.427 seconds, remarkably close to the official reference objective of -17.887559, which was obtained using Gurobi 7.0 in approximately 65 hours (32 threads). Similarly, the fixed-time-window LNS strategy discovered from uccase10 obtains 10,995.047518 on uccase9 in only 65.803 seconds, compared with the official reference objective of 10,993.131409.

These results highlight the effectiveness of our approach in transferring automatically discovered optimization strategies to unseen instances, achieving exact reference matches on two instances and near-reference-quality solutions on the other two, while requiring dramatically less computation time than the reported reference runs. All final solutions passed independent feasibility checks on their original MIP models.

Despite the effective transferability, we emphasize that AutoMIP itself can be viewed as a standalone LLM-based algorithm for solving a broad range of integer programming instances, in an autoresearch manner. As demonstrated by the additional experiments reported in our response to your [W4, Q4], AutoMIP can outperform the mature commercial solver Gurobi under identical computational budgets, particularly on challenging instances.

[W4, Q4] Additional same-budget comparisons against solver-only Gurobi and SCIP.

We thank the reviewer for this valuable suggestion. Based on our pilot experiments, we found that the off-the-shelf solvers performed considerably worse than expected on the MIPLib and MINLPLib instances. Moreover, we considered the autoresearch baselines to be more appropriate and fairer comparators for our Autoresearch agent, and therefore did not include solver-only results in the original submission. However, we agree that comparing AutoMIP against strong off-the-shelf MILP solvers under identical wall-clock budgets is important for establishing the practical value of the proposed approach. To address this comment, we have conducted additional experiments using Gurobi and SCIP directly on Open and Hard instances in MIPLib.

Table: Comparison with solver-only Gurobi and SCIP on MIPLIB instances. Results for Open and Hard instances are presented in the upper and lower panels, respectively. “–” indicates that no instances are solved within the interval; therefore, no successful solving time is recorded.

Open instances (12-hour budget)

Method0–1 h w/s0–1 h time (s)1–3 h w/s1–3 h time (s)3–6 h w/s3–6 h time (s)6–12 h w/s6–12 h time (s)Total w/sTotal time (s)
AutoMIP8/81,4405/55,5452/211,2560/043,20015/1513,888
Gurobi0/0–0/16,4820/114,5860/043,2000/239,933
SCIP0/0–0/0–0/0–0/043,2000/043,200

Hard instances (1-hour budget)

Method0–10 min solved0–10 min time (s)10–20 min solved10–20 min time (s)20–30 min solved20–30 min time (s)30–60 min solved30–60 min time (s)Total solvedTotal time (s)
AutoMIP4385681821,47222,607141,810
Gurobi32630–0–11,92243,016
SCIP15920–0–11,89123,364

The results demonstrate a substantial advantage for AutoMIP on both Open and Hard sets. On the Open instances, under the same 12-hour budget per instance, AutoMIP reaches the target (improving over the best known solution) on 15 instances, whereas Gurobi and SCIP reach only 2 and 0, respectively. The advantage is also evident in the anytime performance: AutoMIP reaches the target on 8 instances within the first hour, compared with none for either baseline, and solves 13 instances within three hours, compared with only 1 for Gurobi and none for SCIP. Gurobi reaches its two targets after 6,482 and 14,586 seconds, respectively. The mean runtimes across all instances are 13,888 seconds for AutoMIP and 39,933 seconds for Gurobi.

The results on the Hard instances show a similar trend under a 1-hour budget per instance. AutoMIP reaches the historical best known solution on 14 instances, substantially outperforming Gurobi (4 instances) and SCIP (2 instances). The corresponding mean runtimes across all instances are 1,810, 3,016, and 3,364 seconds, respectively.

Overall, these additional experiments provide direct evidence that AutoMIP offers practical advantages over simply allocating the same computational budget to mature MILP solvers, particularly on challenging MIPLIB instances. We have added these comparisons and the corresponding results to the revised manuscript in Table 13. The corresponding analysis has been added in Appendix G, marked in blue on page 32. The per-instance results follow in Table 14 for Open instances and Table 15 for Hard instances.

Reviewer_1X5KOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents AutoMIP, a reusable agent skill for long-horizon autoresearch on mixed-integer linear and nonlinear programs (MILPs/MINLPs). AutoMIP maintains a persistent idea pool of complementary, untried research directions and records experiments in a traceable algorithm tree. At each step, the agent either refines the current direction along an L edge or switches to an unused pool candidate along an R edge; for a switch, it selects a compatible historical parent, restores that state's artifacts, and registers a new branch. The authors evaluate AutoMIP on MIPLib Open, MIPLib Hard, and MINLPLib against Codex, Loop, AutoEoH, and EvoX, and report a higher task-success count, more final-objective wins, and lower token consumption than the baselines.

Critical strengths

1. The paper addresses a real and practically important inefficiency in current LLM usage. It frames the problem around a familiar but underexplored issue: much of the token budget is spent on understanding the input intent and connecting the context, while only a small fraction is used to actually solve the problem. Applying this lens to long-horizon autoresearch on MILP/MINLP is timely, and the idea of explicitly managing research directions and experimental states is a concrete step toward shifting more of the budget from context management to actual problem solving.

2. The research direction is novel and well aligned with current research trends. The combination of a persistent pool of untried directions with a traceable tree of executable states is conceptually novel, and it connects naturally to ongoing work on agent memory, tree search, and automated algorithm design. Framing the approach as a reusable agent skill also makes it easy to understand and potentially reusable across agents and domains.

Critical weaknesses

1. Missing solver-only baseline undermines the central attribution.

The paper claims that the idea pool and the compatibility-aware algorithm tree drive the observed gains, but it never compares against a solver-only baseline. AutoMIP's agent is explicitly allowed to use Gurobi, HiGHS, SCIP, CPLEX, CBC, and OR-Tools (Appendix A.1), and the Open and Hard tasks are, respectively, “improve the best-known incumbent” and “reach a historical target within one hour”—both of which a tuned solver run with the same budget could plausibly achieve. The experiments, however, compare only against other LLM-agent scaffolds (Codex, Loop, AutoEoH, EvoX). Without Gurobi/SCIP-only results under the same 12h/1h budget—default and tuned—the reported 31 vs. 29 and 30 vs. 29 cannot be attributed to AutoMIP's search mechanisms rather than to additional solver time, parameter search, or problem-specific code. This is not a minor missing baseline; it is the control that determines whether the paper's main claim holds.

2. The most distinctive mechanism is not isolated by the ablation.

The paper's key novelty is the joint selection of a research direction and a compatible historical starting state. However, "NoTree" removes the tree, parent links, frontier, historical-parent selection, backtracking, and the associated artifact restoration as a single bundle, while "NoPool" removes the pool; neither isolates compatibility-aware parent selection. A "Tree + always continue from the current leaf" ablation and a "Tree + random compatible parent" ablation are needed to show that the compatibility check itself matters. Moreover, Kt(v,q)K_t(v,q) is defined only as a binary predicate (Section 3.3); the paper does not specify whether it is computed by the LLM, by matching artifact metadata, or by a rule, nor does it report its accuracy. If KtK_t is unreliable, the restored state may be incompatible and the mechanism can fail silently. As written, the ablation supports “tree + restoration as a bundle” at best, not the claimed joint direction/state selection.

Questions for authors

Questions

  1. What exactly is the root state n0 for each instance? Is it a single run of Gurobi/SCIP with default parameters, or the first script written by the agent? Do all compared methods (AutoMIP, Codex, Loop, AutoEoH, and EvoX) start from the same n0?

  2. Does the quality of the initial algorithm affect the efficiency and final outcome of the subsequent L/R search? If n0 is already strong, how much of the 31/60 success count comes from n0 rather than from the pool/tree?

  3. AutoMIP's agent can call Gurobi, HiGHS, and SCIP. Can you add solver-only results under the same budget (12h/1h), both default and tuned?

  4. The core of the paper is the joint selection of a research direction and a compatible historical starting state, but NoTree removes the tree, historical-parent selection, backtracking, and artifact restoration as a single bundle. Can you add the following two ablations to isolate the effect of compatibility-aware parent selection?

    • "Tree + always continue from the current leaf"
    • "Tree + random compatible parent"
  5. How is Kt(v,q)K_t(v,q) computed—by the LLM, by matching artifact metadata, or by a rule? What is its accuracy? If KtK_t is wrong, does the mechanism fail silently?

6 comments
AuthorsOpenReview ↗

Response to Reviewer 1X5K

Dear Reviewer 1X5K,

We sincerely thank the reviewer for recognizing the practical significance and novelty of AutoMIP framework, particularly its combination of a persistent idea pool and a traceable algorithm tree for long-horizon autoresearch. We also appreciate the positive assessment of the paper's presentation, informativeness, and potential impact.

[W1, Q3] Additional results of solver-only results.

We thank the reviewer for this useful comment. Following your suggestion, now we have added solver-only Gurobi and SCIP results on 20 Open instances with a 12-hour budget and 20 Hard instances with a one-hour budget (Appendix G). We set MIPFocus=1 for Open instances because our primary objective is to quickly find high-quality feasible solutions that improve upon the reference objectives. Reasonably, we set MIPFocus=0 for Hard instances since the task for Hard instances focuses on balancing the search for high-quality feasible solutions with closing the optimality gap to reference objectives. For your convenience, we display the results in the following table.

Table: Comparison with solver-only Gurobi and SCIP on MIPLIB instances. Results for Open and Hard instances are presented in the upper and lower panels, respectively. “–” indicates that no instances are solved within the interval; therefore, no successful solving time is recorded.

Open instances (12-hour budget)

Method0–1 h w/s0–1 h time (s)1–3 h w/s1–3 h time (s)3–6 h w/s3–6 h time (s)6–12 h w/s6–12 h time (s)Total w/sTotal time (s)
AutoMIP8/81,4405/55,5452/211,2560/043,20015/1513,888
Gurobi0/0–0/16,4820/114,5860/043,2000/239,933
SCIP0/0–0/0–0/0–0/043,2000/043,200

Hard instances (1-hour budget)

Method0–10 min solved0–10 min time (s)10–20 min solved10–20 min time (s)20–30 min solved20–30 min time (s)30–60 min solved30–60 min time (s)Total solvedTotal time (s)
AutoMIP4385681821,47222,607141,810
Gurobi32630–0–11,92243,016
SCIP15920–0–11,89123,364

The results demonstrate a substantial advantage for AutoMIP on both Open and Hard sets. On the Open instances, under the same 12-hour budget per instance, AutoMIP reaches the target (improving over the best known solution) on 15 instances, whereas Gurobi and SCIP reach only 2 and 0, respectively. The advantage is also evident in the anytime performance: AutoMIP reaches the target on 8 instances within the first hour, compared with none for either baseline, and solves 13 instances within three hours, compared with only 1 for Gurobi and none for SCIP. Gurobi reaches its two targets after 6,482 and 14,586 seconds, respectively. The mean runtimes across all instances are 13,888 seconds for AutoMIP and 39,933 seconds for Gurobi.

The results on the Hard instances show a similar trend under a 1-hour budget per instance. AutoMIP reaches the historical best known solution on 14 instances, substantially outperforming Gurobi (4 instances) and SCIP (2 instances). The corresponding mean runtimes across all instances are 1,810, 3,016, and 3,364 seconds, respectively.

Overall, these additional experiments provide direct evidence that AutoMIP offers practical advantages over simply allocating the same computational budget to mature MILP solvers, particularly on challenging MIPLIB instances. We have added these comparisons and the corresponding results to the revised manuscript in Table 13. The corresponding analysis has been added in Appendix G, marked in blue on page 32. The per-instance results follow in Table 14 for Open instances and Table 15 for Hard instances.

[W2, Q4] Ablation study of "Tree + always continue from the current leaf" and "Tree + random compatible parent".

We thank the reviewer for the suggestion. We have added the two requested parent-selection ablations in the revised paper. Specifically, we have added the results in Table 3 alongside the analysis in Section 4.2, highlighted in blue on page 8. We also explained the two suggested parent-selection in Appendix A.6, highlighted in blue on page 20.

Both ablations retain the candidate pool, exploration tree, L/R branching, and historical-state restoration, while changing the parent choice for a direction switch. As advised by the reviewer “Tree + Current” always continues from the current leaf; “Tree + Random” randomly samples a compatible historical parent.

Method0-1h1-3h3-6h6-12hTotal
w/stime(s)tokenw/stime(s)tokenw/stime(s)tokenw/stime(s)tokenw/stime(s)token
AutoMIP5/81,440317,3153/55,545912,7891/211,2561,960,9320/043,2002,076,9739/1513,8881,070,460
NoPool1/52,019233,3532/45,643450,3140/211,4601,168,3430/142,3342,874,2663/1221,8291,558,655
NoTree2/31,304202,3881/65,841518,7990/113,560604,1180/043,2002,935,2033/1024,2261,683,805
Tree + Current0/0--0/13,924327,5820/0--0/043,2003,050,0000/141,2362,913,879
Tree + Random0/21,918165,8650/0--0/0--0/043,2003,010,0000/239,0722,725,586

The results show that both alternative parent-selection strategies can occasionally attain the target, but neither matches the solution quality achieved by full AutoMIP. Specifically, in every instance where either ablation succeeds, AutoMIP obtains a strictly better final objective: 22671.1972463 versus 22671.1974264 for Tree + Current on shs1014; 1081.3584612 versus 1081.3584991 for Tree + Random on zeil; and −164208.6435547 versus −164208.6433270 for Tree + Random on supportcase35. Although the differences are numerically small, their consistent direction provides evidence that the choice of historical parent matters even when the exploration tree, compatible-parent candidates, and historical-state restoration are retained. In particular, simply continuing from the current leaf or randomly selecting a compatible historical parent does not reliably recover the most beneficial starting state for exploring a new research direction. These results therefore support the importance of AutoMIP's compatibility-aware parent-selection mechanism in improving the quality of the resulting solutions.

[W2, Q5] Details of KtK_t computation.

We thank the reviewer for raising this question. Kt(v,qt)K_t(v,q_t) is determined by the LLM agent through semantic assessment, rather than by a fixed metadata-matching rule. Specifically, the parent_hint provides an initial suggestion for a suitable starting node for the selected research direction. The agent then assesses whether the candidate parent's saved state supports that direction by examining its node summary, available artifacts, artifact dependencies, and applicability conditions. Thus, metadata and artifact information serve as evidence for the agent's compatibility judgment, rather than being matched against a predefined set of rules. The backtrack_check records the candidate parents considered, the selected parent, and the rationale for the selection, making this decision traceable. The state-management script performs structural checks, such as verifying node existence and candidate eligibility, but leaves semantic compatibility assessment to the LLM agent.

Regarding accuracy, we have not conducted an independent evaluation against ground-truth labels of compatible and incompatible parent–direction pairs. Such an evaluation would require constructing and manually annotating a representative dataset, which involves substantial effort given the semantic and context-dependent nature of compatibility. Therefore, we cannot report a numerical accuracy for KtK_t, and acknowledge this as a limitation of the current implementation. We have clarified the mechanism for determining Kt(v,qt)K_t(v,q_t) in Section 3.3, marked in blue on page 5. We also admit the limitation of the compatibility assessment in conclusion, marked in blue on page 10.

Solution Validity. An imperfect compatibility assessment does not necessarily hinder the overall search process, as the agent can use the outcomes of subsequent experiments to inform its research decisions and select alternative starting points when needed. Moreover, KtK_t guides the selection of historical states rather than determining the validity of candidate solutions. The best verified solution is maintained independently of the active search branch, and all reported solutions are independently verified against the original instance. Therefore, the compatibility assessment primarily serves as a search-guidance mechanism, with its potential impact reflected in search efficiency rather than solution validity. Our experimental results further demonstrate the reliability of solution validity: all reported solutions pass independent verification against their original problem instances. Moreover, AutoMIP consistently achieves target results across MIPLib and MINLPLib instances more efficiently than competing research agents and established solvers, which indicates that the proposed search mechanism is effective in guiding the agent toward high-quality solutions despite the possibility of imperfect compatibility assessments.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.