acceptodds
Under review as a conference paper at ICLR 2027

MAPLE: Memory-Augmented Planning with Language and Evolution

Abstract

Operational plans often need to be revised as requirements change over time. Responding to these changes requires updating the underlying optimization problem, preserving decisions that users wish to keep, and deciding whether to reuse previous solutions as starting points or search from scratch. We introduce MAPLE (Memory-Augmented Planning with Language and Evolution), an agent for optimization under changing requirements expressed in natural language. MAPLE constructs the initial optimization problem through a typed program interface and solves it with numerical solvers. For each subsequent request, it revises and solves the current problem, uses records of prior requests and a saved plan to identify and preserve decisions that users ask to keep, and selectively reuses previous solutions to initialize the new search. To evaluate agents under changing requirements, we introduce NLDO, a benchmark of 15 planning tasks, each followed by 12 natural-language updates, spanning five task types: resource selection and allocation, production scheduling, staff scheduling, delivery route planning, and computing task allocation to servers. In the main evaluation, MAPLE completes all 15 task sequences, achieving mean online single-objective quality of 0.951 and a mean Pareto hypervolume ratio of 0.875 across initial and updated task states.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
LLM Agents, Agentic Planning, Natural Language Planning, Dynamic Optimization

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_aoQBOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

MAPLE is an agent for optimization under changing natural-language requirements. It maintains an executable optimization program, event history, representative plans, solution archives, and evolutionary populations across successive user revisions. Three core components handle typed search-space scaffolding (TSS), live problem decomposition (LPD), and live state memory (LSM). A restart selector decides between warm and full initialization. The paper also introduces NLDO, a benchmark of 15 planning trajectories (180 updates) across five task types. MAPLE completes all 15 trajectories with mean scalar quality 0.951 and Pareto HV ratio 0.875, substantially outperforming six adapted baselines.

Critical strengths
  1. The formulation of "live dynamic optimization" is new at this intersection. Prior work addresses parts but not their combination. The separation of event history for entity lookup from representative plans for assignment retrieval is a clean architectural insight.
Critical weaknesses
  1. MAPLE is a complete system. The baselines generate solver code from scratch at each step without any of these components designed in the paper. The enormous gap on dynamic Pareto tasks (MAPLE 0.875 vs. next-best 0.084) primarily measures whether the method maintains evolutionary state, not the marginal contribution of MAPLE's language-understanding components. A fairer comparison would give baselines access to the same NSGA-II solver with fresh initialization at each step, isolating MAPLE's NL understanding and state-management contributions.

  2. NLDO and MAPLE were developed together, and the benchmark's update patterns match MAPLE's TSS design well. The problem instances are small. The three episodes per domain share significant structural repetition. Independent validation on externally-designed dynamic optimization benchmarks would strengthen the claims.

Questions for authors

Future work

How would MAPLE handle updates that require changing the decision-space structure (e.g., adding a new decision type not in the TSS catalog)? The current TSS catalog is fixed, and is this a fundamental limitation?

3 comments

We thank the reviewer for recognizing the novelty of the setting. We have revised and uploaded the paper in response to the reviewer’s feedback.


1. Question 1

This is not a fundamental limitation. The fixed TSS catalog limits what the current TSS runtime can represent directly, but developers can extend it to support new decision types. The catalog covers common decision types in evolutionary computation, including binary, bounded integer, bounded real, categorical, permutation, assignment, and optional assignment. Segments of these types can be combined in any number and order. When an update changes the decision space, LPD can add, remove, or resize the corresponding segments. All NLDO tasks were expressible using the existing catalog.

Some additional decision structures can be represented through indirect encoding. For example, before evaluating a candidate, MAPLE can decode a real-valued vector into an ordering by sorting its entries or into binary choices by rounding values in [0,1].

When a decision structure requires a dedicated type, a developer can implement its initialization, crossover, mutation, and repair operators and register the type in the TSS catalog. If both indirect encoding and extending the catalog are impractical, the untyped Workbench provides an alternative configuration. We evaluated this configuration in the w/o TSS ablation.


2. Weakness 1

MAPLE constructs an executable optimization problem from an initial natural-language request and revises it as subsequent requests arrive. Existing optimization agents primarily target one-shot problem formulation, and we are unaware of a directly comparable agent designed for successive natural-language updates.

To assess whether MAPLE’s formulations remain effective when search starts from scratch, we use the Always Full ablation. MAPLE interprets each request and updates the executable problem as usual, while NSGA-II initializes a fresh population at every update.

SettingMean HV ratio
MAPLE0.875
Always Full ablation0.825
Best-performing baseline0.084

All three results cover the initial state and all 12 updates in each of the six Pareto trajectories. Most of MAPLE’s observed advantage over the best-performing baseline persists with fresh initialization, showing that the optimization problems generated and revised by MAPLE support effective search without reusing earlier candidates. A shared NSGA-II backend, as the reviewer suggests, would allow a direct comparison of the formulations produced by different agents under the same search procedure.


3. Weakness 2

We agree that evaluation on independently designed dynamic optimization tasks would provide further evidence of MAPLE’s generalizability. As noted in Appendix A.4, such validation remains future work.

The underlying optimization problems in NLDO come from established benchmarks designed independently of MAPLE. We draw on their problem formulations and instance formats for selection and allocation (OR-Library, Beasley, 1990), flexible job-shop scheduling (Brandimarte, 1993), nurse rostering (Ceschia et al., 2018), green vehicle routing (Erdoğan & Miller-Hooks, 2012, and Bektaş & Laporte, 2011), and energy-aware cloud placement (Beloglazov et al., 2012). The nurse-rostering instances come from the public INRC-II datasets. For NLDO, we added the natural-language descriptions and 12 cumulative requirement updates per trajectory.

We are unaware of an independently designed benchmark that provides successive natural-language updates to an optimization problem. One next step is to express the changes specified by an existing dynamic routing benchmark as natural-language updates and evaluate MAPLE on the resulting trajectories.


We thank the reviewer for the feedback and look forward to further discussion.

I thank the authors for the response. The Always Full ablation partially addresses my first concern by showing that fresh initialization retains most of MAPLE’s advantage (0.825 vs. 0.875). However, it still uses MAPLE’s formulations. The large gap to the best baseline therefore still conflates formulation quality, solver backend, and state reuse. The independent-validation concern also remains: NLDO is co-developed with MAPLE, and the authors agree that external dynamic benchmarks are future work. I maintain my rating of 2

We thank the reviewer for the follow-up and for acknowledging that fresh initialization retains most of MAPLE's advantage.


1. Separating formulation quality, solver backend, and state reuse

We agree that these factors should be separated. Our existing controls do so to a large extent:

ComparisonWhat differsMean HV ratio
MAPLE vs. Always FullState reuse0.875 vs. 0.825
MAPLE vs. MAPLE w/o TSSFormulation (same NSGA-II backend)0.875 vs. 0.665
MAPLE vs. baselines at t00Formulation and backend, no state reuse0.832 vs. 0.440
MAPLE vs. baselines over t00-t12All factors, across updates0.875 vs. 0.084

MAPLE w/o TSS is close to the comparison the reviewer suggested: the model writes its own program from scratch, without MAPLE's formulations, and runs it with the same NSGA-II backend. It still reaches 0.665, far above the baselines. The baselines, in turn, drop from 0.440 at t00 to 0.084 over the trajectories, so most of the remaining gap arises from maintaining the problem across updates, which is the capability MAPLE targets.


2. Independent validation

We agree that external validation is desirable, and Appendix A.4 states this as a limitation. However, no existing benchmark expresses dynamic optimization through successive natural-language updates, so such a benchmark can currently only be built by extending existing ones. NLDO follows this route: its problem formulations and instances come from established benchmarks designed independently of MAPLE (OR-Library, Brandimarte, INRC-II, green vehicle routing, and energy-aware cloud placement), rather than from our own design. We added only the natural-language descriptions and update streams on top of them, which preserves the generality of the underlying problems as far as possible in the current setting.


We respect the reviewer's assessment and thank the reviewer for the detailed engagement.

Reviewer_ZRh8OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
3(+1)
Presentation
3(+1)
Soundness
3(+1)
Read review
Paper summary

The paper introduces MAPLE (Memory-Augmented Planning with Language and Evolution), a framework for solving optimisation problems that evolve through successive natural-language updates. Unlike approaches that solve each problem independently, MAPLE maintains historical context, preserves explicitly committed decisions, and determines whether to reuse previous search states or restart optimisation. Its primary contributions are: (1) Typed Search-Space Scaffolding (TSS), which uses an LLM to specialise an executable optimisation template with typed decision variables and predefined evolutionary operators; (2) Live Problem Decomposition (LPD) and Live State Memory (LSM), which interpret natural-language updates, resolve references to previous requests and assignments, and enforce retained decisions; and (3) a restart-selection mechanism that combines semantic assessment with code-level checks to choose between repairing a previous search state and restarting from scratch. MAPLE integrates numerical optimisation methods, including genetic algorithms and NSGA-II, for single- and multiobjective problems. The authors evaluate MAPLE on the NLDO benchmark, comprising 15 update sequences across five domains, with 12 updates per sequence. They report that MAPLE completes all 15 sequences, achieving a mean online single-objective quality score of 0.951 and a mean Pareto hypervolume ratio of 0.875. Comparisons with several existing optimisation-agent baselines suggest that MAPLE is effective in dynamic planning settings. However, the evaluation is relatively small, and further evidence from ablation studies, broader benchmarks, and computational-cost measurements would help establish the generality and sources of its improvements.

Critical strengths
  1. Explicitly addressing the challenges of persistent optimisation under evolving requirements. The paper distinguishes between preserving historical decisions as constraints and reusing previous search states to accelerate subsequent optimisation. This is a meaningful design distinction because an updated problem must respect explicit commitments without unnecessarily restricting the search for improved solutions. The proposed integration of structured memory and restart selection provides a promising approach to this challenge.
  2. A unified framework combining language understanding with numerical optimisation. MAPLE separates the interpretation and compilation of natural-language updates from the numerical optimisation process, using typed search-space scaffolding and established evolutionary algorithms. This provides a structured alternative to relying solely on LLM-driven optimisation and is supported by evaluation across five problem domains. The reported results are promising, with a mean online single-objective quality score of 0.951 and a mean Pareto hypervolume ratio of 0.875.
Critical weaknesses
  1. Limited experimental validation and generalizability. The evaluation covers only 15 update trajectories across five domains, which provides limited evidence that MAPLE generalises to diverse, complex real-world optimisation problems. Although the reported results are promising, the paper does not establish how robust the approach is to longer update sequences, conflicting requirements, or more challenging problem instances. Broader evaluation is needed to substantiate the claimed benefits.

  2. Insufficient isolation of the contributions of individual components. MAPLE combines several mechanisms, including typed search-space scaffolding, structured memory, historical decision preservation, and restart selection. The reported aggregate results do not sufficiently establish how much each component contributes to the overall performance or whether the restart-selection mechanism reliably outperforms simpler alternatives. Ablation studies and more detailed comparisons are needed to validate the necessity and effectiveness of these design choices.

Questions for authors
  1. Correctness of generated optimisation models: How do the authors verify that the LLM-generated Workbench faithfully captures the semantics of the natural-language problem and its subsequent updates? Are there quantitative measurements of model correctness beyond successful compilation and execution?

  2. Contribution of individual components: Do ablation experiments quantify the individual contributions of TSS, LSM, and the restart-selection mechanism? In particular, how much of the reported performance improvement comes from structured memory and restart selection compared with simpler strategies, such as always restarting or always reusing the previous search state?

  3. Robustness of historical decision preservation: How does MAPLE handle conflicting or infeasible commitments when a new request contradicts previously retained decisions? Does the framework explicitly detect and communicate infeasibility, and how frequently does this occur in the benchmark?

  4. Evaluation robustness and computational cost: How variable are the reported results across the 15 trajectories, and how does MAPLE compare with the baselines in terms of runtime, LLM calls, and token consumption? Could the reported improvements come at a substantially higher computational cost?

3 comments

We thank the reviewer for the constructive feedback. We have revised the paper accordingly and uploaded the updated version.


1. Question 1

Beyond compilation and execution, we use two quantitative checks: (1) A held-out evaluator checks the returned solutions against the task constraints independently of the generated Workbench. All 780 Pareto state–seed results from MAPLE pass this check (Table 12). In the w/o TSS ablation, the same evaluator identifies 140 invalid results that the generated programs report as feasible (Table 23). (2) The entity, field, and value resolved by MAPLE match the annotations in all 30 updates that require resolving a historical reference (Table 20).

These quantitative checks assess two aspects of fidelity to the natural-language requirements: whether the solutions produced by the generated Workbenches satisfy the specified constraints and whether MAPLE correctly resolves references to earlier requests. We also manually inspected several generated Workbenches and found that all inspected Workbenches implemented their corresponding natural-language requirements.


2. Question 2

We thank the reviewer for asking about individual components. The original submission included ablations of TSS, LSM, and restart selection, summarized below.

ComparisonResult
TSS / w/o TSSFeasibility rate: 100% / 80.6%. Mean HV ratio: 0.879 / 0.665 (Table 4).
Typed operators / generic samplingMean HV ratio: 0.774 / 0.618 (Table 24).
LSM: full / events-only / plan-onlyCorrect identification of the requested assignment: 18/18, 6/18, and 6/18, respectively. Mean HV ratio in search replay: 0.825, 0.261, and 0.258, respectively (Table 22).
Restart policy: MAPLE selector / Always Warm / Always FullMean HV ratio: 0.879 / 0.868 / 0.824 (Table 29).
Restart policy with the same incoming search state: MAPLE selector / Matched Warm / Matched FullMean HV ratio: 0.779 / 0.766 / 0.731 (Table 33).
Restart policy: MAPLE selector / numerical ruleMean HV ratio: 0.779 / 0.765 (Table 34).
Warm vs. Full after rescaling, relabeling, or task additionWarm achieves a mean HV ratio gain of 0.181 over Full across the six cases (Table 30).

The ablations demonstrate that TSS improves feasibility and solution quality, LSM helps identify and preserve requested assignments, and adaptive restart selection achieves a higher mean HV ratio.


3. Question 3

When a new request conflicts with a requirement to preserve part of the existing plan, MAPLE enforces the preservation requirement as a hard constraint and checks candidate plans against both requirements. If the search finds no plan satisfying all constraints, MAPLE reports that no feasible plan was found and lists the constraint violations (Appendix A.4). We found no explicit conflicts of this kind among the 180 updates in the main NLDO benchmark.

NLDO includes 141 updates that override earlier problem information, such as objectives and preferences. For example, P002-t10 replaces monetary cost with service time as the optimization objective. MAPLE applies the new requirement to the executable problem and keeps the earlier request in the event history for later reference. During search-state reuse, MAPLE can discard individual candidates that are no longer suitable for the updated problem.


4. Question 4

Variability across trajectories: In the main experiments, MAPLE achieves a mean online quality of 0.951 across the nine single-objective trajectories and a mean HV ratio of 0.875 across the six Pareto trajectories (Table 3). The sample standard deviations across trajectory means are 0.060 and 0.118, respectively, computed from the per-trajectory results in Tables 16 and 13. Single-objective quality varies relatively little across trajectories, while Pareto HV ratios vary more.

Computational cost: In the main experiments, MAPLE generates 4.1k output tokens per executed state, compared with 12.4k–24.0k for the baselines. Its measured LLM call time is 68 seconds per state, compared with 179–347 seconds for the baselines (Table 39). Although MAPLE makes more LLM calls per state (2.1 vs. 1.08–1.93, Table 39), most of its calls return short, structured responses. Because MAPLE reuses its Workbench across requests, only 4.0 of its 27.8 calls per trajectory, on average, are used to build or repair the program. The remaining calls locate or translate updates (Table 28).

Overall, MAPLE uses substantially fewer output tokens and less measured LLM call time per state than the baselines, while keeping input token use within their range (Table 39). Although MAPLE makes more calls per state, its average output per call is shorter.


We would be very happy to continue the discussion.

I appreciate the authors for the detailed response and substantial revisions. The additional independent evaluation, historical-reference analysis, and update-localisation results strengthen the evidence for MAPLE’s correctness and reliability.

The new ablations also provide more convincing evidence for the contributions of typed operators, TSS, and the combination of event history and accepted plans. The conflicting-update clarification is satisfactory, and the expanded cost analysis better explains MAPLE’s efficiency advantage.

My main remaining concern is generalisation: NLDO was developed alongside MAPLE, and its natural-language update streams have not been independently designed. The limited experiments across alternative models and independent programs also leave some uncertainty about broader robustness. However, these limitations are now acknowledged.

Overall, the revisions address most of my original concerns.

We sincerely thank the reviewer for carefully assessing our revisions and recognizing that they address most of the original concerns. We respond to the remaining concerns below.


1. Generalization beyond NLDO

We agree that evaluation on independently designed dynamic optimization tasks would provide further evidence of MAPLE’s generalizability. Although NLDO’s natural-language descriptions and update streams were developed for this study, its underlying optimization problems draw on established benchmarks designed independently of MAPLE. We draw on their problem formulations and instance formats for selection and allocation (OR-Library, Beasley, 1990), flexible job-shop scheduling (Brandimarte, 1993), nurse rostering (Ceschia et al., 2018), green vehicle routing (Erdoğan & Miller-Hooks, 2012, and Bektaş & Laporte, 2011), and energy-aware cloud placement (Beloglazov et al., 2012). The nurse-rostering instances come from the public INRC-II datasets. We extended these benchmark problems with 12 cumulative natural-language updates per trajectory.

We also evaluate MAPLE on problems outside NLDO from two benchmarks developed independently of MAPLE: 59 NLP4LP-hard instances released with OptiMUS and 20 BWOR cases released with OR-LLM-Agent. MAPLE achieves reference-objective agreement rates of 59.3% and 80.0%, respectively (Table 2). These tests assess performance on independent static problems, while evaluation on independently designed dynamic optimization tasks remains future work (Appendix A.4).

We are unaware of an independently designed benchmark that provides successive natural-language updates to an optimization problem. One possible next step is to express changes from existing external dynamic benchmarks as natural-language updates and evaluate MAPLE on the resulting trajectories.

To support use on user-provided tasks, we developed MAPLE Harness, a web application where users can submit planning requests and data files, issue successive natural-language updates, and inspect solutions after each update (Appendix A.16). A recorded MAPLE Harness session on a CNC scheduling problem outside NLDO, including a Warm update after a priority change, is available at https://anonymous.4open.science/w/MAPLE-demo-review-F98B/, and the Harness code is released anonymously at https://anonymous.4open.science/r/MAPLE-harness-core. We welcome the reviewer to try MAPLE Harness with their own problems and updates.


2. Alternative models and independently generated Workbenches

We agree that the robustness experiments remain limited in scope. Evaluating additional language models across the full benchmark requires substantial LLM API calls and repeated numerical searches. We therefore tested one additional model, Kimi k2.7, as a limited check of model sensitivity. With all other settings unchanged, MAPLE achieves a scalar quality of 0.937 with a 100% solve rate and a Pareto HV ratio of 0.766 with a 92.3% solve rate. With DeepSeek, the corresponding quality scores are 0.951 and 0.875, with 100% solve rates for both task types (Table 26). These results show that MAPLE remains effective with a second model, while also indicating some model-dependent variation. Evaluating more models remains future work.

Table 25 reports our test of sensitivity to independent Workbench generation on two episodes: P010 (routing) and P015 (cloud-resource placement). Each episode has three independently generated Workbench runs. All six runs achieve a 100% solve rate, with mean online quality of 0.695 for P010 and 0.787 for P015. Thus, the variation across these runs is in solution quality, not feasibility. Within individual trajectories, standard deviations across numerical seeds range from 0.003 to 0.076. These results provide a limited check of variation across Workbench generations; evaluating more episodes and program generations remains future work.


We thank the reviewer again for the constructive discussion, which has helped improve the paper. If you feel that our revisions address your remaining concerns, we would be grateful if you would consider updating your rating.

Reviewer_qjYeOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how optimization agents can maintain problems and revise solutions as users change their requirements through successive natural-language requests. MAPLE uses a restricted program interface to construct and update optimization problems, while established numerical solvers perform the search. It retains request history, accepted plans, and search candidates to resolve historical references, preserve requested assignments, and choose between reusing previous candidates and restarting. The authors introduce NLDO, a benchmark with 15 task sequences and 180 updates. Baseline comparisons, ablations, and additional controls evaluate solution quality, continuity across updates, and the roles of the main components.

Critical strengths
  1. The paper addresses a practically meaningful problem with a coherent system design. Resolving historical references, preserving requested decisions, and reusing earlier search candidates require different information and operations. MAPLE makes these distinctions explicit and brings them together in an executable workflow that fits the task well.

  2. The experiments provide useful support for the complete system and its main components. MAPLE completes all trajectories in the main run and remains stronger than the comparison methods under mathematical-prefix evaluation, which ignores auxiliary output-field failures. The TSS ablation supports the value of typed scaffolding, while the LSM controls show how event history and accepted plans complement each other. The action-matched restart controls and numerical-selector comparison provide further evidence for the value of restart decisions.

Critical weaknesses
  1. A discrepancy between a generated program and the requested behavior needs further clarification. The saved implementation in Figure 6 handles a 25% speed reduction by multiplying distance by 1.25, whereas Appendix A.6 specifies unchanged physical distance and a travel-time multiplier of 4/3. The authors independently score the final plans using the public formulas and report a sensitivity analysis of congestion scope. These checks are helpful, but the effect of the implementation difference on search results remains unclear. Since correctly translating requirement changes into optimization programs is central to the paper, this example leaves some concern about how faithfully the system implements requested updates.

  2. Reliability across independently generated programs remains uncertain. The complete success reported in the main experiment is encouraging, but each trajectory primarily uses one language-model run, with ten search seeds sharing the generated program and update decisions. The additional independent-program experiments are informative: in Table 25, both additional cloud-placement sequences fail the held-out validity checks at the final two updates, each achieving an 83.3% solve rate. The paper provides limited analysis of these failures, so it is unclear whether they mainly reflect incorrect constraint implementations or failure to find feasible solutions within the search budget. This limits my confidence in the system’s reliability over successive updates.

Questions for authors
  1. What caused the failures at t11–t12 in the two additional cloud-placement program sequences in Table 25? Were they due to incorrect constraint implementations, failure to find feasible solutions within the search budget, or other issues? Distinguishing these causes would help clarify the main source of the reliability issue.

  2. Appendix A.15 reports that changing the congestion scope has little effect on aggregate results. However, the saved implementation in Figure 6 also changes physical distance, while Appendix A.6 specifies a travel-time adjustment. Has the effect of this difference on search results been examined? If results from searches using the corrected travel-time evaluator are available, do the main conclusions remain unchanged?

1 comment

We thank the reviewer for the positive assessment and the precise questions. We have revised the paper in response to your comments and uploaded the updated version.


1. Question 1 / Weakness 2

Thank you for carefully identifying this issue. When rerunning MAPLE on the P010 routing episode and the P015 cloud-resource placement episode, we found that we had introduced a data-import error unintentionally: the replacement tables supplied at t11–t12 were not loaded. This left MAPLE with stale inputs for the final updates. In P015, it searched the new 64-job workload using the old machine capacities, under which no feasible placement existed. In P010, the same error lowered routing quality at t11–t12. Thus, these failures resulted from the data-import error rather than from incorrect constraint implementation or an insufficient search budget.

After correcting the data-import error, with all other settings unchanged, all three independent MAPLE runs for each episode achieve a 100% solve rate, consistent with the main results. The w/o-TSS runs had loaded the replacement tables correctly, so their results are unchanged. We have corrected Table 25, added the failure analysis of the w/o-TSS runs below, and carefully checked the revised paper for consistency.

Corrected Table 25 (changed rows in bold)

MethodModel runExecution completedSolve rate (%)Online qualityQuality if validNumerical SD
P010 (routing)
MAPLE0yes100.00.7560.7560.120
1yes100.00.7520.7520.037
2yes100.00.5770.5770.040
All3/3100.0 ± 0.00.695 ± 0.1020.6950.066
MAPLE w/o TSS0yes0.00.000–0.000
1yes100.00.4350.4350.016
2yes100.00.5720.5720.029
All3/366.7 ± 57.70.336 ± 0.2990.5030.015
P015 (cloud)
MAPLE0yes100.00.7920.7920.022
1yes100.00.8020.8020.014
2yes100.00.7680.7680.010
All3/3100.0 ± 0.00.787 ± 0.0170.7870.015
MAPLE w/o TSS0yes83.30.7510.9010.011
1yes100.00.0590.0590.001
2yes83.30.7290.8740.019
All3/388.9 ± 9.60.513 ± 0.3930.5770.011

Failure analysis for w/o TSS. P010 run 0 omits the one-route-per-vehicle check, so all of its updates are invalid. P015 run 0 records GPU-incompatible placements without clearing the feasibility flag, which invalidates t11–t12. P015 run 2 finds no placement accepted by its own evaluator after the t11–t12 resource replacement. Beyond these two episodes, we also relate these failures to the failure-mode analysis of all main trajectories (Table 23): run 0 is the original main sequence, and its two modes, an omitted hard-constraint check and a violation recorded without clearing the feasibility flag, account for all 140 invalid w/o-TSS results there, each reported as feasible by the program itself. Run 2 adds a third mode, and no failure occurs in any typed run.


2. Question 2 / Weakness 1

The t02 request in Appendix A.18 states, “Traffic around order O004 slows vehicles by 25 percent”, but does not specify whether the slowdown should affect distance or speed. Figure 6 shows the patch MAPLE generated, which interprets the request as a 25% increase in distance and scales distance by 1.25. The rule in Appendix A.6 instead interprets the request as a 25% reduction in speed, scaling travel time on arrival at O004 by 4/3 while leaving distance and emissions unchanged. We revised Figure 3 and Appendix A.6 to clarify the distinction between the generated patch and the congestion modeling rule used in the experiments.

To measure the effect of this difference on search, we replaced the congestion handling in the saved routing Workbenches with the rule specified in Appendix A.6. We then reran MAPLE, Always Warm, and Always Full, keeping all other code, restart actions, seeds, and search budgets unchanged (Appendix A.15, Table 44). The HV ratio changes by at most 0.007, compared with a seed-level standard deviation of 0.029. All plans remain feasible, and the method ranking is unchanged in every routing trajectory.

Mean HV ratio across the three routing trajectories (t00–t12)

MethodSearch with the Figure 6 patchSearch with the Appendix A.6 rule
MAPLE0.7680.764
Always Warm0.7490.742
Always Full0.6650.661

We thank the reviewer again for the thoughtful feedback and look forward to further discussion.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Global Response: Paper Revisions

We thank all reviewers for their valuable comments. We have revised the paper to address the comments and clarify the issues raised, and uploaded the updated version.


The main revisions are summarized below.

  • In Appendix A.13 (Table 25), we corrected the P010 routing and P015 cloud-resource placement results after identifying a data-import error in the t11–t12 reruns. With the error corrected and all other settings unchanged, all three independent MAPLE runs for each episode achieve a 100% solve rate. We also added a failure analysis of the w/o-TSS runs, whose results were unaffected by the import error.
  • In Appendix A.6 and Figure 3, we clarified the difference between the generated t02 patch, which increases distance by 25%, and the congestion modeling rule in Appendix A.6, which reduces speed by 25%. Rerunning the routing searches with the Appendix A.6 rule changed HV ratios by at most 0.007 and left the method ranking unchanged across all three routing trajectories (Appendix A.15, Table 44).
  • In Appendix A.13 (Tables 28 and 39), we report MAPLE’s model usage and compare per-state costs with the baselines. MAPLE uses 4.1k output tokens and 68 seconds of LLM call time per state, compared with 12.4k–24.0k tokens and 179–347 seconds for the baselines. We also report across-trajectory standard deviations of 0.060 for single-objective quality and 0.118 for Pareto HV ratio (Tables 13 and 16).
  • In Appendix A.4, we clarify NLDO’s design, describe how MAPLE handles conflicts involving retained plan requirements, and note the need to evaluate on independently designed dynamic tasks.

We have provided point-by-point responses to all reviewers and welcome further discussion.

AuthorsOpenReview ↗

To AC: Rebuttal Summary

Dear AC, SAC, PC,

Thank you for your time and for supporting the review process. During the rebuttal, we responded to every question raised by the reviewers, added new experiments and analyses, and uploaded a revised paper (see our Global Response).


The reviewers recognized the novelty of live dynamic optimization (qjYe, aoQB), the coherent system design (qjYe, ZRh8, aoQB), and the combination of language understanding with numerical solvers (ZRh8). Below we summarize how we addressed each reviewer's concerns.

Reviewer qjYe (Initial Score: 3)

  • Concern: Reliability across independently generated programs. We updated Table 25 and analyzed the remaining failures. All three independently generated MAPLE runs reach a 100% solve rate in both the routing and the cloud-placement episode, consistent with the main results. The new failure analysis traces the w/o-TSS failures to errors in the self-written programs, such as omitted constraint checks, whereas no typed run fails.
  • Concern: Effect of the Figure 6 patch on search. We clarified the difference between the generated patch and the congestion rule in Figure 3 and Appendix A.6, and reran the routing searches with the Appendix A.6 rule. HV ratios change by at most 0.007, within the seed-level standard deviation of 0.029, and the method ranking is unchanged in every routing trajectory (Table 44).

Reviewer ZRh8 (Initial Score: 2)

  • Concern: Correctness of generated programs. We showed that an independent held-out evaluator finds all 780 Pareto results of MAPLE feasible, and that MAPLE resolves all 30 historical references correctly (Tables 12, 20).
  • Concern: Contributions of individual components. We summarized the ablations of TSS, typed operators, LSM, and restart selection, each of which degrades performance when removed or replaced (Tables 4, 22, 24, 29–34).
  • Concern: Conflicting commitments. We explained that MAPLE enforces retained decisions as hard constraints and reports infeasibility with the violated constraints, and that 141 of the 180 updates already override earlier information.
  • Concern: Variability and computational cost. We added a stage-by-stage cost analysis and a comparison with all baselines. Because MAPLE maintains its program across updates, it uses 4.1k output tokens and 68 s of LLM call time per state, against 12.4k–24.0k tokens and 179–347 s for the baselines (Tables 28, 39).
  • After these revisions, the reviewer confirmed that "the revisions address most of my original concerns."
  • NLDO developed alongside MAPLE (new in the second round): this point was not in the initial review and is the same as Reviewer aoQB's Weakness 2. We replied that NLDO's underlying problems come from independently designed benchmarks, and provided further evidence of generality: MAPLE achieves 59.3% and 80.0% on the independent NLP4LP-hard and BWOR benchmarks (Table 2), remains effective with a second model, Kimi k2.7 (Table 26), and reaches a 100% solve rate in all independent Workbench runs (Table 25). We also provided MAPLE Harness, a web application on which reviewers can test MAPLE with their own problems and updates.

Reviewer aoQB (Initial Score: 2)

  • Concern: Fairness of the baseline comparison. We showed that most of MAPLE's advantage does not rely on reusing evolutionary state. The Always Full ablation, which initializes a fresh NSGA-II population at every update, still reaches a mean HV ratio of 0.825, compared with 0.875 for the full MAPLE and 0.084 for the best baseline (Table 29).
  • Question: Decision types outside the TSS catalog. We explained that this is not a fundamental limitation: other structures can be represented through indirect encoding, new decision types can be registered in the catalog, and the untyped Workbench remains available.
  • Concern: NLDO developed together with MAPLE. We acknowledged this in Appendix A.4 and showed that NLDO's underlying problems come from established benchmarks designed independently of MAPLE.
  • Second round. The reviewer acknowledged that "fresh initialization retains most of MAPLE's advantage" and asked to further separate formulation, solver backend, and state reuse. Our existing controls do so: state reuse accounts for 0.875 vs. 0.825; MAPLE w/o TSS, a program written from scratch and run with the same NSGA-II backend as the reviewer suggested, still reaches 0.665, far above the baselines; and the baselines drop from at most 0.440 at t00 to at most 0.084, so most of the gap arises across updates. On independent validation, no benchmark with successive natural-language updates exists yet, so NLDO extends independently designed benchmarks to preserve generality.

We hope these responses have resolved the reviewers' concerns. We sincerely thank the AC for the hard work in coordinating the review and discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.