acceptodds
Under review as a conference paper at ICLR 2027

DecisionBrain: Agentic Solver Orchestration for End-to-End Operations Research

Abstract

Solving an operations research problem end to end from its natural-language statement is challenging: the requirements are implicit in the statement, solver performance varies across problem classes, and an optimal solution to an incorrect model may violate the requirements. Although language models and agent frameworks have facilitated such workflows, most existing systems lack an explicit representation for a solution strategy designed by the agent, restrict execution to specific solvers, and require the solving stage to verify its own output. We present DecisionBrain, an end-to-end agentic workflow that orchestrates heterogeneous solvers through strategy-level design over a validated method library. It returns a decision only after an independent feasibility review, which routes repairable failures back to the responsible stage. DecisionBrain and every open-source baseline run on the same DeepSeek-V4-Flash backbone. On the stress test suite Hard32-Fea, DecisionBrain returns a feasible decision on 81.3% of instances against 59.4% for the strongest baseline. On FrontierOR65-Fea, where a solver based on one monolithic mathematical optimization model already suffices, DecisionBrain reaches 86.2% against 84.6% for the strongest baseline. Ablations confirm that each mechanism contributes, and removing the algorithm library has the largest effect, 46.9 points on Hard32-Fea.

Primary area:
foundation or frontier models, including LLMs
Keywords:
large language models, LLM agents, tool use, agentic workflows, combinatorial optimization, operations research, constraint satisfaction, automated verification

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_2j6iOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

DecisionBrain orchestrates heterogeneous optimization methods through explicit strategy design, a validated algorithm library, and an operationally separate feasibility-review stage. It evaluates natural-language optimization workflows on FrontierOR65-Fea and the targeted Hard32-Fea stress test, with nested ablations of the proposed mechanisms.

Critical strengths

The system addresses an important failure of optimization agents: an executable model need not satisfy the original requirements. The shared backbone and explicit execution budgets improve interpretability of the open-source comparisons. Hard32 feasibility increases to 26/32 instances, and the review ablation demonstrates a useful distinction between producing feasible decisions and preventing infeasible returns. The appendices document benchmark selection, solver interfaces, and failure records in considerable detail.

Critical weaknesses

The central attribution to adaptive orchestration remains incomplete. Hard32 deliberately selects instances where a particular monolithic configuration fails while a family-specific method succeeds. Table 3 attributes 46.9 percentage points to library access; the design-stage comparison is restricted to Gurobi and improves feasibility from 8/32 to 11/32. Appendix G.1 further identifies standard family solvers behind the full system's accepted decisions. These results demonstrate the package's utility in the selected regime, but do not sufficiently distinguish adaptive composition from straightforward family-based dispatch within the same library.

Questions for authors
  1. Clarification from existing run records: What evidence distinguishes adaptive composition from dispatching each problem family to a fixed method in the same full library? Please relate this to the Gurobi-restricted design ablation and Appendix G.1. Evidence of composition-specific gains would strengthen the main contribution.
  2. Clarification: What caused the three internal/external feasibility disagreements on FrontierOR65-Fea: contract errors, checker errors, or another failure? Their causes would affect the assessment of the verification contribution.
3 comments
AuthorsOpenReview ↗

Orchestration Contribution 1: Experimental Comparison

Thank you for suggesting a comparison against fixed dispatch by problem family. This was particularly useful and directly motivated the new comparison and DecisionBrain-Flash policy. The earlier design ablation restricted to Gurobi examines strategy design within a fixed backend. The experiment below instead compares specialist dispatch by problem family with unrestricted orchestration. Appendix G.1 identifies the packages behind accepted decisions, but package identity alone does not describe whether their use involved construction, warm starts, or subproblem coordination. We apologize that the original run records are difficult to retrieve, but the new experiments directly address this question and provide evidence on the contribution of orchestration beyond fixed dispatch by problem family.

This comparison was not included in the original submission because restricting the model to prescribed packages while excluding composition of algorithms required additional execution controls. We have now implemented constrained design schemas and runtime import guards and completed the comparison on all 32 Hard32 instances.

We compare three systems. DecisionBrain (full system) can use one algorithm or compose several algorithms. Family Dispatch selects one algorithm within the package assigned to each problem family. DecisionBrain-Flash first tries one package and can later switch to the unrestricted DecisionBrain workflow. Family Dispatch assigns OR-Tools to PDPTW and JSSP, and PyVRP to VRPTW. It does not prescribe a particular solver within OR-Tools. DecisionBrain and DecisionBrain-Flash use the same available library. Family Dispatch uses the documentation for its assigned package. All three use the same backbone model and checking mechanism. Each task has a total budget of 120 minutes, and each solver execution is limited to 30 seconds. Each task has a CPU quota of three cores. Gurobi is unavailable in all three settings.

The results distinguish a regime where the prescribed single method is less effective from regimes directly handled by a specialist. On PDPTW, DecisionBrain improves feasibility, mean quality on the common feasible subset, and mean runtime across all tasks over Family Dispatch. When one solver is already effective, composition may add little or even reduce performance. VRPTW exhibits a balance between quality and runtime, while JSSP shows worse quality and longer runtime. Because real OR requirements vary in their constraints and interface needs, the ability to compose remains useful, but composition should not be the default for every instance. This motivates DecisionBrain-Flash.

Table 1. Feasibility on all tasks

FamilyTasksDecisionBrainFamily DispatchDecisionBrain-Flash
PDPTW2020/20 (100%)16/20 (80%)20/20 (100%)
VRPTW66/6 (100%)6/6 (100%)6/6 (100%)
JSSP66/6 (100%)6/6 (100%)6/6 (100%)
Overall3232/32 (100%)28/32 (87.5%)32/32 (100%)

Table 2. Mean reference gap on instances solved by all three systems (%)

FamilyTasksDecisionBrainFamily DispatchDecisionBrain-Flash
PDPTW163.624.446.86
VRPTW60.851.192.51
JSSP68.977.978.05
Overall284.174.506.18

Table 3. Mean total task runtime across all tasks (minutes)

FamilyTasksDecisionBrainFamily DispatchDecisionBrain-Flash
PDPTW2029.2340.4624.67
VRPTW617.2312.1512.45
JSSP619.1518.3619.95
Overall3225.0931.0121.49

Reference gap is (objective − reference objective) / |reference objective|, not a certified optimality gap. Table 2 uses identical instances for all three settings. Table 3 includes failed tasks and timeouts and measures runtime rather than monetary or token cost. Updated statistics use the PDPTW-12 Family Dispatch and JSSP-1 DecisionBrain-Flash reruns. These point estimates do not measure variation between independent runs.

Part 2 discusses the results and the design of DecisionBrain-Flash.

AuthorsOpenReview ↗

Orchestration Contribution 2: Adaptive Design and Findings

Part 1 presents the experimental setup and the three result tables.

PDPTW results

DecisionBrain solves all 20 PDPTW instances, whereas Family Dispatch solves 16. On those same 16 feasible instances, mean reference gap improves from 4.44% to 3.62%. Across all 20 tasks, mean runtime falls from 40.46 to 29.23 minutes. Thus orchestration improves delivery success, solution quality, and overall task cost in this family. This is not dominance on every instance or faster solving conditional on success. On the 16 shared successes, DecisionBrain averages 26.99 minutes versus 20.55 for Family Dispatch. Its runtime advantage across all tasks includes avoiding unsuccessful attempts. The extra time on shared successes also motivates DecisionBrain-Flash, since orchestration can be redundant when a direct method already succeeds.

Problems directly supported by specialist solvers

VRPTW reaches 6/6 feasibility with DecisionBrain and Family Dispatch. DecisionBrain improves mean reference gap from 1.19% to 0.85%, but increases mean runtime from 12.15 to 17.23 minutes. This is a balance between quality and runtime. On JSSP, DecisionBrain and Family Dispatch each reach 6/6, but DecisionBrain has a worse gap, 8.97% versus 7.97%, and longer runtime, 19.15 versus 18.36 minutes. Family Dispatch returns a better objective on all six JSSP instances.

The final DecisionBrain designs select the hybrid schema on 11 of these 12 VRPTW and JSSP instances. This documents a tendency to introduce a design with multiple components even when direct specialist methods succeed. It does not identify the causal contribution of each extra component.

DecisionBrain-Flash design and results

DecisionBrain-Flash gives the model up to 45 minutes to select one installed package and solve the complete problem with one algorithm. Implementation repairs are allowed within this same budget. If a candidate passes verification, the task finishes. If feasibility review requests a new algorithm design, or the 45 minutes expire without an accepted result, the system switches to the unrestricted DecisionBrain workflow. Both phases share the same total task budget of 120 minutes. The second phase receives the actual remaining time and may choose either one algorithm or a composition.

Of the 32 tasks, 28 finish without escalation. The other four trigger escalation because feasibility review requests a new algorithm design, not because the initial time budget expires. All four subsequently succeed.

DecisionBrain-Flash retains 32/32 feasibility and reduces mean runtime across all tasks from 25.09 to 21.49 minutes, a 14.34% reduction relative to DecisionBrain. On their common set of all 32 instances, mean reference gap increases from 3.56% to 5.53%. The quality loss is concentrated in PDPTW, which accounts for approximately 93% of the net increase in the overall mean gap. This is the same regime in which the DecisionBrain versus Family Dispatch comparison shows a quality benefit from allowing orchestration. DecisionBrain-Flash therefore gives up quality primarily in this regime to reduce overall runtime while preserving feasibility, rather than delivering a lossless acceleration.

The balance differs by family. On PDPTW, DecisionBrain-Flash achieves 20/20 feasibility in 24.67 minutes on average, but its gap is 6.86% on the subset solved by all three systems, worse than both alternatives. On VRPTW it averages 12.45 minutes, close to Family Dispatch, but its 2.51% gap is worse. On JSSP its gap of 8.05% improves on DecisionBrain and its objective is better on all six instances, but it is slightly worse than Family Dispatch on average. Its mean runtime is 19.95 minutes. These results are consistent with our expectation. For instances well matched to solvers in the library, DecisionBrain-Flash can improve solving performance, further illustrating the boundary of orchestration and the value of an adaptive policy.

Summary

Orchestration is useful when a single packaged method struggles, but it is not always beneficial when a specialist already solves the problem effectively. On PDPTW, DecisionBrain improves feasibility, quality on shared successes, and mean runtime across all tasks relative to Family Dispatch. The VRPTW and JSSP results show that extra orchestration can add cost or worsen the objective. DecisionBrain remains meaningful in practical settings because requirements vary in their constraints and the solver interfaces they need, so a single package may not directly cover the complete problem. DecisionBrain-Flash responds by trying one package first and allowing unrestricted strategies when needed. It retains 32/32 feasibility and reduces overall mean runtime by 14.34%, while giving up quality mainly on PDPTW, the same regime in which DecisionBrain shows a quality advantage over Family Dispatch. The higher success rate reflects API model improvements. These results and discussion are now in the main text.

AuthorsOpenReview ↗

Internal and External Feasibility Disagreements

Thank you for raising this concern. Following the end of our previous appointment, the original traces are no longer accessible. We reran all three cases and inspected their descriptions and checkers. We infer that inconsistencies between the descriptions and the dataset checkers are the most likely source of the disagreements.[ZL1.1] earl2005: The description requires immediate deactivation upon interception or defense-zone entry but leaves the discrete update order unspecified. The internal checker updates a[k], while the dataset checker updates a[k+1]. All 11 violations reported by the dataset checker match this one-step difference.[ZL2.1]

bollapragada2001: The evaluation statement specifies supplied direction cosines, but the dataset checker recomputes them from coordinates. With supplied coefficients, all 144 equilibrium equations satisfy the external tolerance of 10⁻⁵ (maximum residual 1.17 × 10⁻¹¹). With recomputed coefficients, 86 fail (maximum residual 2.78 × 10⁻³).

kress2019: The ambiguity concerns whether stationary positions are required at the operation's end time.[ZL3.1] For an operation lasting p time units and starting at t, the checker generated by the model requires[ZL4.1][ZL5.1] stationary positions through t+p−1; the dataset checker also checks t+p. We suspect this endpoint convention caused the original disagreement. The rerun satisfies both conventions and passes both checkers.

All four evaluated baselines also failed external acceptance on earl2005 and bollapragada2001, providing supporting context. These findings identify benchmark contract issues; the original external scores remain unchanged.

Reviewer_YTBXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

DecisionBrain addresses end-to-end operations research by combining explicit strategy design, a library of heterogeneous solvers, and a separate feasibility-review stage with targeted repair. It is evaluated on a constructed stress test, Hard32-Fea, and a broader suite derived from FrontierOR. The reported results show a substantial feasibility improvement on Hard32-Fea and a much smaller improvement on FrontierOR65-Fea.

Critical strengths

The workflow is clearly specified and addresses practical failures in translating requirements into executable solutions. Explicit interfaces and execution records make solver use and repair decisions easier to inspect.

Hard32-Fea results are promising: the system reports feasible solutions on 26/32 instances, compared with 19/32 for the strongest baseline (Table 2). External hidden checkers, controlled ablations, and detailed runtime and failure analyses strengthen the evaluation.

Critical weaknesses

The largest improvement comes from expanding the solver pool, while the design and review ablations are conducted only in the Gurobi-restricted setting (Table 18). A simpler agent with the same full library and budget is needed to establish the value of strategy-level composition beyond access to suitable specialist solvers.

The main gains are demonstrated on 32 instances from three families, selected specifically to distinguish specialist methods from a particular monolithic formulation (Appendix E.2). On FrontierOR65, DecisionBrain solves only one more instance than OR-LLM-Agent and takes approximately three times as long on their shared successful instances (Appendix F.2.1). Variability across independent runs is not reported, leaving the stability of these differences unclear.

This is a useful and carefully documented system, but I lean toward weak rejection because the evidence does not yet sufficiently isolate the contribution of the complete orchestration framework. A comparison against a simpler agent using the same full library would most directly affect my assessment.

Questions for authors
  1. With the full solver library, documentation, checking mechanism, and budget held fixed, how does DecisionBrain compare with a simpler agent that selects a solver without strategy-level composition?

  2. Were independent repeated runs performed, and what LLM sampling settings were used? If repeated runs are available, how variable are the feasibility results, particularly the design-ablation difference and the small improvement on FrontierOR65?

4 comments
AuthorsOpenReview ↗

Orchestration Contribution 1: Experimental Comparison

Thank you for asking whether orchestration adds value beyond selecting a suitable specialist from the solver library. This motivated the comparison with Family Dispatch and adaptive DecisionBrain-Flash policy below.

This comparison was not included in the original submission because restricting the model to prescribed packages while excluding composition of algorithms required additional execution controls. We have now implemented constrained design schemas and runtime import guards and completed the comparison on all 32 Hard32 instances.

Experimental design

You requested a comparison in which the model selects a solver from the full library. For Family Dispatch, we instead provide a fixed mapping from problem families to packages. We chose these packages from their documented capabilities and the solver choices observed in existing successful runs, as discussed in Appendix G.1. OR-Tools supports paired pickups and deliveries, time windows, and vehicle capacity. PyVRP directly supports the VRPTW constraints. OR-Tools also provides scheduling models for operation precedence and machine capacity. The available PyVRP interface does not claim general support for requiring each pickup to precede its paired delivery. The mapping therefore uses both domain knowledge and empirical observations from existing runs. It removes the need for the model to identify a suitable package and provides a stronger domain prior than unaided selection. This compares against fixed family dispatch, rather than autonomous selection from the full library without composition. DecisionBrain-Flash can select a package from the available library and can later allow composition.

We compare three systems. DecisionBrain (full system) can use one algorithm or compose several algorithms. Family Dispatch selects one algorithm within the package assigned to each problem family. DecisionBrain-Flash first tries one package and can later switch to the unrestricted DecisionBrain workflow. Family Dispatch assigns OR-Tools to PDPTW and JSSP, and PyVRP to VRPTW. It does not prescribe a particular solver within OR-Tools. DecisionBrain and DecisionBrain-Flash use the same available library. Family Dispatch uses the documentation for its assigned package. All three use the same backbone model and checking mechanism. Each task has a total budget of 120 minutes, and each solver execution is limited to 30 seconds. Each task has a CPU quota of three cores. Gurobi is unavailable in all three settings.

The results distinguish a regime where the prescribed single method is less effective from regimes directly handled by a specialist. On PDPTW, DecisionBrain improves feasibility, mean quality on the common feasible subset, and mean runtime across all tasks over Family Dispatch. When one solver is already effective, composition may add little or even reduce performance. VRPTW exhibits a balance between quality and runtime, while JSSP shows worse quality and longer runtime. Because real OR requirements vary in their constraints and interface needs, the ability to compose remains useful, but composition should not be the default for every instance. This motivates DecisionBrain-Flash.

Table 1. Feasibility on all tasks

FamilyTasksDecisionBrainFamily DispatchDecisionBrain-Flash
PDPTW2020/20 (100%)16/20 (80%)20/20 (100%)
VRPTW66/6 (100%)6/6 (100%)6/6 (100%)
JSSP66/6 (100%)6/6 (100%)6/6 (100%)
Overall3232/32 (100%)28/32 (87.5%)32/32 (100%)

Table 2. Mean reference gap on instances solved by all three systems (%)

FamilyTasksDecisionBrainFamily DispatchDecisionBrain-Flash
PDPTW163.624.446.86
VRPTW60.851.192.51
JSSP68.977.978.05
Overall284.174.506.18

Table 3. Mean total task runtime across all tasks (minutes)

FamilyTasksDecisionBrainFamily DispatchDecisionBrain-Flash
PDPTW2029.2340.4624.67
VRPTW617.2312.1512.45
JSSP619.1518.3619.95
Overall3225.0931.0121.49

Reference gap is (objective − reference objective) / |reference objective|, not a certified optimality gap. Table 2 uses identical instances for all three settings. Table 3 includes failed tasks and timeouts and measures runtime rather than monetary or token cost. Updated statistics use the PDPTW-12 Family Dispatch and JSSP-1 DecisionBrain-Flash reruns. These point estimates do not measure variation between independent runs.

Part 2 discusses the results and the design of DecisionBrain-Flash.

AuthorsOpenReview ↗

Orchestration Contribution 2: Adaptive Design and Findings

Part 1 presents the experimental setup and the three result tables.

PDPTW results

DecisionBrain solves all 20 PDPTW instances, whereas Family Dispatch solves 16. On those same 16 feasible instances, mean reference gap improves from 4.44% to 3.62%. Across all 20 tasks, mean runtime falls from 40.46 to 29.23 minutes. Thus orchestration improves delivery success, solution quality, and overall task cost in this family. This is not dominance on every instance or faster solving conditional on success. On the 16 shared successes, DecisionBrain averages 26.99 minutes versus 20.55 for Family Dispatch. Its runtime advantage across all tasks includes avoiding unsuccessful attempts. The extra time on shared successes also motivates DecisionBrain-Flash, since orchestration can be redundant when a direct method already succeeds.

Problems directly supported by specialist solvers

VRPTW reaches 6/6 feasibility with DecisionBrain and Family Dispatch. DecisionBrain improves mean reference gap from 1.19% to 0.85%, but increases mean runtime from 12.15 to 17.23 minutes. This is a balance between quality and runtime. On JSSP, DecisionBrain and Family Dispatch each reach 6/6, but DecisionBrain has a worse gap, 8.97% versus 7.97%, and longer runtime, 19.15 versus 18.36 minutes. Family Dispatch returns a better objective on all six JSSP instances.

The final DecisionBrain designs select the hybrid schema on 11 of these 12 VRPTW and JSSP instances. This documents a tendency to introduce a design with multiple components even when direct specialist methods succeed. It does not identify the causal contribution of each extra component.

DecisionBrain-Flash design and results

DecisionBrain-Flash gives the model up to 45 minutes to select one installed package and solve the complete problem with one algorithm. Implementation repairs are allowed within this same budget. If a candidate passes verification, the task finishes. If feasibility review requests a new algorithm design, or the 45 minutes expire without an accepted result, the system switches to the unrestricted DecisionBrain workflow. Both phases share the same total task budget of 120 minutes. The second phase receives the actual remaining time and may choose either one algorithm or a composition.

Of the 32 tasks, 28 finish without escalation. The other four trigger escalation because feasibility review requests a new algorithm design, not because the initial time budget expires. All four subsequently succeed.

DecisionBrain-Flash retains 32/32 feasibility and reduces mean runtime across all tasks from 25.09 to 21.49 minutes, a 14.34% reduction relative to DecisionBrain. On their common set of all 32 instances, mean reference gap increases from 3.56% to 5.53%. The quality loss is concentrated in PDPTW, which accounts for approximately 93% of the net increase in the overall mean gap. This is the same regime in which the DecisionBrain versus Family Dispatch comparison shows a quality benefit from allowing orchestration. DecisionBrain-Flash therefore gives up quality primarily in this regime to reduce overall runtime while preserving feasibility, rather than delivering a lossless acceleration.

The balance differs by family. On PDPTW, DecisionBrain-Flash achieves 20/20 feasibility in 24.67 minutes on average, but its gap is 6.86% on the subset solved by all three systems, worse than both alternatives. On VRPTW it averages 12.45 minutes, close to Family Dispatch, but its 2.51% gap is worse. On JSSP its gap of 8.05% improves on DecisionBrain and its objective is better on all six instances, but it is slightly worse than Family Dispatch on average. Its mean runtime is 19.95 minutes. These results are consistent with our expectation. For instances well matched to solvers in the library, DecisionBrain-Flash can improve solving performance, further illustrating the boundary of orchestration and the value of an adaptive policy.

Summary

Orchestration is useful when a single packaged method struggles, but it is not always beneficial when a specialist already solves the problem effectively. On PDPTW, DecisionBrain improves feasibility, quality on shared successes, and mean runtime across all tasks relative to Family Dispatch. The VRPTW and JSSP results show that extra orchestration can add cost or worsen the objective. DecisionBrain remains meaningful in practical settings because requirements vary in their constraints and the solver interfaces they need, so a single package may not directly cover the complete problem. DecisionBrain-Flash responds by trying one package first and allowing unrestricted strategies when needed. It retains 32/32 feasibility and reduces overall mean runtime by 14.34%, while giving up quality mainly on PDPTW, the same regime in which DecisionBrain shows a quality advantage over Family Dispatch. The higher success rate reflects API model improvements. These results and discussion are now in the main text.

AuthorsOpenReview ↗

Independent Runs, Sampling, and Experimental Setup

Thank you for this question. Each configuration was evaluated in one independent run because running the complete agentic evaluation is costly. Therefore, all compared agent baselines were evaluated using the same protocol.[ZL1.1][ZL2.1] Another reason is that DeepSeek retired V4-Flash and redirected it to V4.1-Flash on September 10, 2026.[ZL3.1] Insufficient time remained to repeat the full evaluation with the replacement model before the submission deadline.[ZL4.1]

Both sets of experiments use high reasoning effort and the default thinking-mode sampling settings: top_p=1.0, with temperature having no effect. The original experiments used DeepSeek-V4-Flash; the rebuttal experiments use DeepSeek-V4.1-Flash.

The original environment is documented in Appendix F, “Experimental Setup”: an Intel i9-13900K workstation with 125 GiB RAM running Ubuntu 24.04.3. The rebuttal experiments run on an AMD Ryzen 7 7445H machine under Ubuntu 24.04/WSL2, without Gurobi and with up to three concurrent tasks. Both retain a three-core solver budget per task, a 7,200-second task limit, and per-call solver limits of 600 seconds for FrontierOR and 30 seconds for Hard32.

The AC encouraged us to view this paper from a higher-level perspective. From that perspective, I find the novelty and methodological contribution of the paper limited, so I will maintain my assessment.

Reviewer_dqCHOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

DecisionBrain is an agentic workflow for solving operations research problems from natural-language descriptions and structured data. It combines explicit algorithm design, a validated solver library, and independent feasibility review with responsibility-directed repair. The system achieves substantial feasibility gains on Hard32-Fea and comparable feasibility to the strongest baseline on FrontierOR65-Fea.

Critical strengths
  1. The paper addresses a practical limitation of optimization agents: different problem structures require different solution methods. Supporting heterogeneous solvers and composing restricted subproblems is a useful direction for end-to-end OR systems.

  2. Separating candidate construction from feasibility checking is a sensible design. The review ablation demonstrates its value in preventing constraint-violating candidates from being returned.

Critical weaknesses
  1. The largest gain comes from expanding the solver library. The current experiments do not clearly show how much decomposition and composition add beyond choosing a suitable solver. The authors should compare solver selection alone against multi-component orchestration using the same library, documentation, and budget.

  2. Feasibility does not fully measure optimization performance. Two feasible solutions can have very different objective values. Appendix F reports objective gaps, but the main conclusions rely heavily on feasibility. Solution quality should play a larger role in the evaluation.

  3. The additional runtime is not clearly justified on FrontierOR65. DecisionBrain solves only one more instance than OR-LLM-Agent, while its average runtime is about three times longer. The paper should explain whether the gains in solution quality justify this cost.

  4. The Explanation stage is not evaluated. Its gate only checks whether the output is valid JSON. The paper does not assess whether explanations accurately describe the returned solution, executed methods, objective value, and solver guarantees.

Questions for authors
  1. How does multi-component orchestration compare with single-solver selection using the same library, documentation, and budget?

  2. How does the workflow detect and fix errors in its generated feasibility checker?

  3. How do you evaluate whether final explanations accurately describe the solution, executed methods, and solver guarantees?

6 comments
AuthorsOpenReview ↗

Orchestration Contribution 1: Experimental Comparison

Thank you for proposing a direct comparison between solver selection and orchestration of multiple components. Your suggested comparison was particularly valuable to us and motivated both the additional experiments and the DecisionBrain-Flash policy below.

This comparison was not included in the original submission because restricting the model to prescribed packages while excluding composition of algorithms required additional execution controls. We have now implemented constrained design schemas and runtime import guards and completed the comparison on all 32 Hard32 instances.

Experimental design

You requested a comparison in which the model selects a solver from the full library. For Family Dispatch, we instead provide a fixed mapping from problem families to packages. We chose these packages from their documented capabilities and the solver choices observed in existing successful runs, as discussed in Appendix G.1. OR-Tools supports paired pickups and deliveries, time windows, and vehicle capacity. PyVRP directly supports the VRPTW constraints. OR-Tools also provides scheduling models for operation precedence and machine capacity. The available PyVRP interface does not claim general support for requiring each pickup to precede its paired delivery. The mapping therefore uses both domain knowledge and empirical observations from existing runs. It removes the need for the model to identify a suitable package and provides a stronger domain prior than unaided selection. This compares against fixed family dispatch, rather than autonomous selection from the full library without composition. DecisionBrain-Flash can select a package from the available library and can later allow composition.

We compare three systems. DecisionBrain (full system) can use one algorithm or compose several algorithms. Family Dispatch selects one algorithm within the package assigned to each problem family. DecisionBrain-Flash first tries one package and can later switch to the unrestricted DecisionBrain workflow. Family Dispatch assigns OR-Tools to PDPTW and JSSP, and PyVRP to VRPTW. It does not prescribe a particular solver within OR-Tools. DecisionBrain and DecisionBrain-Flash use the same available library. Family Dispatch uses the documentation for its assigned package. All three use the same backbone model and checking mechanism. Each task has a total budget of 120 minutes, and each solver execution is limited to 30 seconds. Each task has a CPU quota of three cores. Gurobi is unavailable in all three settings.

The results distinguish a regime where the prescribed single method is less effective from regimes directly handled by a specialist. On PDPTW, DecisionBrain improves feasibility, mean quality on the common feasible subset, and mean runtime across all tasks over Family Dispatch. When one solver is already effective, composition may add little or even reduce performance. VRPTW exhibits a balance between quality and runtime, while JSSP shows worse quality and longer runtime. Because real OR requirements vary in their constraints and interface needs, the ability to compose remains useful, but composition should not be the default for every instance. This motivates DecisionBrain-Flash.

Table 1. Feasibility on all tasks

FamilyTasksDecisionBrainFamily DispatchDecisionBrain-Flash
PDPTW2020/20 (100%)16/20 (80%)20/20 (100%)
VRPTW66/6 (100%)6/6 (100%)6/6 (100%)
JSSP66/6 (100%)6/6 (100%)6/6 (100%)
Overall3232/32 (100%)28/32 (87.5%)32/32 (100%)

Table 2. Mean reference gap on instances solved by all three systems (%)

FamilyTasksDecisionBrainFamily DispatchDecisionBrain-Flash
PDPTW163.624.446.86
VRPTW60.851.192.51
JSSP68.977.978.05
Overall284.174.506.18

Table 3. Mean total task runtime across all tasks (minutes)

FamilyTasksDecisionBrainFamily DispatchDecisionBrain-Flash
PDPTW2029.2340.4624.67
VRPTW617.2312.1512.45
JSSP619.1518.3619.95
Overall3225.0931.0121.49

Reference gap is (objective − reference objective) / |reference objective|, not a certified optimality gap. Table 2 uses identical instances for all three settings. Table 3 includes failed tasks and timeouts and measures runtime rather than monetary or token cost. Updated statistics use the PDPTW-12 Family Dispatch and JSSP-1 DecisionBrain-Flash reruns. These point estimates do not measure variation between independent runs.

Part 2 discusses the results and the design of DecisionBrain-Flash.

AuthorsOpenReview ↗

Orchestration Contribution 2: Adaptive Design and Findings

Part 1 presents the experimental setup and the three result tables.

PDPTW results

DecisionBrain solves all 20 PDPTW instances, whereas Family Dispatch solves 16. On those same 16 feasible instances, mean reference gap improves from 4.44% to 3.62%. Across all 20 tasks, mean runtime falls from 40.46 to 29.23 minutes. Thus orchestration improves delivery success, solution quality, and overall task cost in this family. This is not dominance on every instance or faster solving conditional on success. On the 16 shared successes, DecisionBrain averages 26.99 minutes versus 20.55 for Family Dispatch. Its runtime advantage across all tasks includes avoiding unsuccessful attempts. The extra time on shared successes also motivates DecisionBrain-Flash, since orchestration can be redundant when a direct method already succeeds.

Problems directly supported by specialist solvers

VRPTW reaches 6/6 feasibility with DecisionBrain and Family Dispatch. DecisionBrain improves mean reference gap from 1.19% to 0.85%, but increases mean runtime from 12.15 to 17.23 minutes. This is a balance between quality and runtime. On JSSP, DecisionBrain and Family Dispatch each reach 6/6, but DecisionBrain has a worse gap, 8.97% versus 7.97%, and longer runtime, 19.15 versus 18.36 minutes. Family Dispatch returns a better objective on all six JSSP instances.

The final DecisionBrain designs select the hybrid schema on 11 of these 12 VRPTW and JSSP instances. This documents a tendency to introduce a design with multiple components even when direct specialist methods succeed. It does not identify the causal contribution of each extra component.

DecisionBrain-Flash design and results

DecisionBrain-Flash gives the model up to 45 minutes to select one installed package and solve the complete problem with one algorithm. Implementation repairs are allowed within this same budget. If a candidate passes verification, the task finishes. If feasibility review requests a new algorithm design, or the 45 minutes expire without an accepted result, the system switches to the unrestricted DecisionBrain workflow. Both phases share the same total task budget of 120 minutes. The second phase receives the actual remaining time and may choose either one algorithm or a composition.

Of the 32 tasks, 28 finish without escalation. The other four trigger escalation because feasibility review requests a new algorithm design, not because the initial time budget expires. All four subsequently succeed.

DecisionBrain-Flash retains 32/32 feasibility and reduces mean runtime across all tasks from 25.09 to 21.49 minutes, a 14.34% reduction relative to DecisionBrain. On their common set of all 32 instances, mean reference gap increases from 3.56% to 5.53%. The quality loss is concentrated in PDPTW, which accounts for approximately 93% of the net increase in the overall mean gap. This is the same regime in which the DecisionBrain versus Family Dispatch comparison shows a quality benefit from allowing orchestration. DecisionBrain-Flash therefore gives up quality primarily in this regime to reduce overall runtime while preserving feasibility, rather than delivering a lossless acceleration.

The balance differs by family. On PDPTW, DecisionBrain-Flash achieves 20/20 feasibility in 24.67 minutes on average, but its gap is 6.86% on the subset solved by all three systems, worse than both alternatives. On VRPTW it averages 12.45 minutes, close to Family Dispatch, but its 2.51% gap is worse. On JSSP its gap of 8.05% improves on DecisionBrain and its objective is better on all six instances, but it is slightly worse than Family Dispatch on average. Its mean runtime is 19.95 minutes. These results are consistent with our expectation. For instances well matched to solvers in the library, DecisionBrain-Flash can improve solving performance, further illustrating the boundary of orchestration and the value of an adaptive policy.

Summary

Orchestration is useful when a single packaged method struggles, but it is not always beneficial when a specialist already solves the problem effectively. On PDPTW, DecisionBrain improves feasibility, quality on shared successes, and mean runtime across all tasks relative to Family Dispatch. The VRPTW and JSSP results show that extra orchestration can add cost or worsen the objective. DecisionBrain remains meaningful in practical settings because requirements vary in their constraints and the solver interfaces they need, so a single package may not directly cover the complete problem. DecisionBrain-Flash responds by trying one package first and allowing unrestricted strategies when needed. It retains 32/32 feasibility and reduces overall mean runtime by 14.34%, while giving up quality mainly on PDPTW, the same regime in which DecisionBrain shows a quality advantage over Family Dispatch. The higher success rate reflects API model improvements. These results and discussion are now in the main text.

AuthorsOpenReview ↗

Handling Errors in Generated Checkers

Thank you for raising this question. In the evaluated workflow, checker errors are primarily addressed through the model’s own reasoning and self-correction during checker construction, rather than a dedicated checker-repair module. The reason for this choice is that such discrepancies were rarely observed in our evaluation: the three identified internal/external feasibility disagreements are most plausibly attributable to inconsistencies between the natural-language descriptions and dataset checkers, as discussed in our response to Reviewer 2j6i. In the rerun of kress2019, the generated checker even enforces requirements omitted by the dataset checker. The description requires serving at least one landside container from the next-earliest deadline group after the makespan. Removing that required service from the rerun solution is rejected by the generated checker but accepted by the dataset checker. This illustrates how the generated checker can prevent false acceptance by enforcing the stated requirements more completely. For the current benchmarks, these observations have not motivated a separate checker-detection and repair module. If checker implementation errors become prevalent in broader applications, explicit checker auditing and repair would be a natural extension.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Additional Clarification on Our Rebuttal

Thank you again for your careful review and constructive feedback. In our rebuttal, we added an independent Family Dispatch comparison and conducted additional comparative experiments. We also introduced DecisionBrain-Flash, an adaptive strategy that avoids unnecessary orchestration when a specialist solver is sufficient.

We fully understand that reviewers are busy, but if any concern remains unaddressed, we would be very grateful for any brief guidance you may be able to provide during the discussion period. Thank you again for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.