acceptodds
Under review as a conference paper at ICLR 2027

EKO: Evolving Knowledge for Interpretable Expensive Multiobjective Optimization

Abstract

Evolutionary algorithms and surrogate-based methods lack interpretability because they treat expensive multiobjective optimization problems as a black-box numerical search. However, real world problems have valuable but unused domain background that can guide search under scarce evaluations, including variable meanings and known mechanisms. Although large language model (LLM) optimizer can use domain background, existing methods neither inherit nor validate their reasoning, so fail to accumulate correct domain knowledge. We propose EKO, an evolutionary framework whose genetic representation is knowledge, which records reasoning and its validation outcomes in natural language. EKO implements knowledge evolution through three agents: the Selector chooses parent knowledge, the Generator produces offspring and new knowledge, and the Reflector validates this knowledge. EKO repeatedly updates reasoning to build global knowledge, delivering beyond a Pareto set an interpretable characterization of the problem landscape with supporting evidence. Since knowledge evolution is independent of variable representation, changing only the problem description in natural language avoids designing specialized operators for different variable types. Experiments on benchmarks from different domains demonstrate that EKO achieves competitive performance and adaptability while remaining interpretable. Ablation studies and knowledge audits confirm component effectiveness and knowledge traceability.

Primary area:
optimization
Keywords:
Expensive Multiobjective Optimization, Evolutionary Algorithms, Large Language Models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_dnfDOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes EKO, an LLM-based framework for expensive multi-objective optimization in which natural-language reasoning is treated as inheritable knowledge. Knowledge is defined as a record of the reasoning behind a candidate and its validation outcome. Three agents replace parts of the evolutionary loop: a Selector chooses parent knowledge and an exploit/explore directive, a Generator retrieves domain background, parent knowledge, and global knowledge to produce an offspring with an unvalidated reasoning record, and a Reflector compares the registered hypotheses with observed objective changes, updates global knowledge and the evolution state, and provides feedback for the next round. The protocol is intended to be independent of variable representation. Experiments use six continuous real-world multi-objective tasks with 50 real evaluations, comparing against NSGA-II, MOEA/D, EHVI, qLogEHVI, HOLLM, and LLAMBO; additional binary, categorical, and permutation tasks test cross-representation behavior.

Critical strengths
  1. Treating an LLM’s reasoning as an inheritable, validated natural-language record, rather than as a one-shot prompt or an ever-growing context, gives the evolutionary loop a persistent object to select, retrieve, and revise.

  2. Selector, Generator, and Reflector have distinct inputs, outputs, and responsibilities, and the loop closes with real evaluations rather than LLM self-critique. The Reflector’s deterministic validation of registered atomic hypotheses is a sensible way to prevent unvalidated LLM assertions from entering the global archive. The supplementary execution trace makes this concrete.

  3. The paper does not only test continuous benchmarks; it changes the problem interface to binary, categorical, and permutation spaces without representation-specific numerical operators, and EKO remains best on the two binary and two permutation tasks.

Critical weaknesses
  1. The comparison does not control the LLM-call budget, which is consequential because EKO invokes three agents per round while the LLM-based baselines may use different numbers of calls. All methods share the same real-evaluation budget, backbone, temperature, and token limit, but not the same number of LLM queries, tokens consumed, or wall-clock cost. Under a 50-evaluation budget, extra LLM calls can be a major advantage, especially for the Selector and Reflector steps. Without a matched LLM-call or token budget, the reported gains over HOLLM and LLAMBO are difficult to attribute to the knowledge-evolution mechanism rather than to more LLM computation. This is a central fairness issue.

  2. The main comparison uses six continuous tasks with 50 real evaluations, and the search spaces are small to medium (3–7 variables in the continuous tasks; 5–52 in the non-continuous tasks). The paper itself notes that large-scale problems are unresolved. Several reported gains are small—0.4% on CrossBarrel_MO and 0.7% on Fullerene_MO—and the CrossBarrel result is not marked significant in Table 1. The categorical tasks are a further limitation: qLogEHVI is better on both NB201 and LNP3. A claim of broad superiority across representations is therefore not supported; the honest claim is narrower.

  3. Each atomic hypothesis is validated by a binary sign comparison between the registered direction and the observed parent-relative change. This checks direction only, not magnitude, uncertainty, or whether the observed change is distinguishable from noise. A “consistent” record can be a coincidental sign match, and a “contradicted” record can reflect a noisy evaluation rather than a wrong mechanism. The audit in Figure 4 reports aggregate proportions of consistent and contradicted outcomes, but it does not relate knowledge quality to later performance, nor does it show that validated knowledge actually changes subsequent decisions in a beneficial way. Since the paper’s main selling point is evidence-linked, interpretable knowledge, this is a material gap.

  4. Three-agent LLM evolutionary loops, LLM-guided parent selection, and LLM-generated candidates are close to existing work such as LLAMBO, HOLLM, MOLLEO, and LLEMA; the paper’s distinguishing claim is knowledge inheritance plus validation. That distinction is plausible but not yet sharply demonstrated against those methods, especially because the global-knowledge ablation has limited effect on most tasks. In addition, the main text does not report LLM calls per run, token consumption, latency, knowledge-archive size, retrieval/compression rules, or how prompt length is controlled as the archive grows. The supplementary material helps, but the main paper does not give enough cost information to judge whether EKO is practical under an expensive-optimization budget.

Questions for authors
  1. Can you report the number of LLM calls, tokens consumed, and wall-clock time for EKO and each LLM-based baseline, and rerun the comparison under a matched LLM-call or token budget? This would directly test whether the gains come from the knowledge-evolution loop rather than from additional LLM computation.

  2. Can you strengthen knowledge validation beyond binary sign agreement. For example, by repeating evaluations, using confidence intervals, or testing whether the observed change exceeds a noise threshold, and show a causal link between validated knowledge quality and subsequent search performance? The current audit reports aggregate consistency but not whether the archive improves decisions.

  3. Can you evaluate EKO on more tasks, larger search spaces, and larger evaluation budgets, and explain why qLogEHVI outperforms EKO on the categorical tasks? The current evidence supports a narrow claim on small/medium tasks, not a general cross-representation advantage.

  4. How is the global knowledge archive retrieved and compressed as it grows? What are the prompt-length and token-cost implications for longer runs, and does the LLM-backbone sensitivity result hold on tasks other than CrossBarrel_MO?

Reviewer_zcnuOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

EKO is an LLM-based framework for expensive multiobjective optimization that maintains natural-language knowledge across search iterations. Experiments cover six continuous and six non-continuous tasks. The framework offers a useful approach to making optimization trajectories inspectable. However, inconsistencies in the central validation procedure and insufficient evidence isolating the contribution of validated knowledge make it below acceptance.

Critical strengths

Promising results across multiple variable representations under a shared evaluation budget. EKO achieves the highest mean final hypervolume on all six continuous tasks, with reported statistically significant improvements over the best baseline on four, and leads on four of the six non-continuous tasks.

Critical weaknesses
  1. The validation baseline is inconsistent between the method and execution trace. Equations (7)–(8) define objective changes relative to an individual anchor parent, whereas Appendix E, p. 29, explicitly uses the mean of both parents. This can reverse the validation outcome. For example, solution_000033 has F2 = 4.070 and anchor solution_000005 has F2 = 4.002, giving an anchor-relative change of +0.068. The trace instead reports approximately −0.06, using the parent mean. These alternatives produce opposite labels for a predicted decrease. It remains unclear which procedure generated the reported results.
  2. Directional agreement does not establish the reliability of the associated mechanism explanations. The validation rule checks the aggregate objective change after a potentially multivariable edit. It cannot independently establish variable-specific effects, physical mechanisms, or validity beyond the observed transition. Figure 4 audits registered direction predictions but does not directly assess the correctness or generalization of the resulting global archive. The evidence supports traceability more strongly than reliable problem understanding.
  3. The ablations do not isolate the contribution of validation. Removing parent knowledge reduces performance, but the information retained in this ablation is insufficiently specified. Without preserving identical parent configurations and objective values while selectively removing reasoning or validation labels, the results cannot establish that validated reasoning, rather than numerical history or ordinary reflective memory, drives the gains. This distinction is also important for assessing the contribution beyond existing reflection-based agents.
  4. The practical scope requires qualification. Appendix A.1 uses interpolated objectives for the materials tasks, omits the original CarSideImpact constraints, and does not enforce the PCE10 simplex constraint. These choices limit conclusions about realistic constrained optimization. Actual LLM token usage, latency, and cost are also unreported. Additionally, the stated PCE10 improvement over the strongest baseline should be approximately 3.47%, rather than 5.7%.
Questions for authors
  1. Which validation baseline was used in the experiments Can the authors provide corresponding raw records and clarify whether the discrepancy affects Figure 4, knowledge updates, or optimization performance?
  2. What exactly is removed in the parent-knowledge ablation Can the authors provide a controlled comparison that preserves numerical history while removing reasoning or validation labels?
  3. Can archive statements be evaluated on independent perturbations, distinguishing directional prediction accuracy from mechanism correctness and broader generalization?
  4. What batch size and knowledge-update schedule were actually used, and what are the measured LLM costs How sensitive are the conclusions to enforcing the omitted constraints?
Reviewer_4WP5OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

EKO is an LLM-driven evolutionary framework for expensive multiobjective optimization. A Selector chooses parents and a search directive; a Generator proposes offspring with preregistered directional predictions; and a Reflector checks predictions against evaluated outcomes and updates inherited and global natural-language knowledge. The intended contribution is an inspectable knowledge-evolution loop returning both a Pareto approximation and an evidence-linked archive. The paper reports experiments using 50 evaluations and ten seeds across six continuous and six non-continuous benchmarks, claiming the highest mean hypervolume on all continuous tasks and four non-continuous tasks, supplemented by knowledge-source ablations, prediction audits, and backbone comparisons.

Critical strengths
  1. Preregistering directional predictions and retaining contradictory outcomes creates an inspectable optimization history, making the proposed knowledge mechanism more concrete than free-form post-hoc explanations.

  2. The formulation spans continuous, binary, categorical, and permutation tasks, outlining a potentially useful representation-agnostic interface for knowledge-guided optimization.

Critical weaknesses
  1. The empirical comparison is not yet sufficiently supported by the submitted implementation. The inspected baseline and evaluation paths appear inconsistent with the stated experimental protocol. In particular, the NSGA-II objective wrapper reads keys that do not match those returned by the benchmark, replacing the actual objectives with zeros. Other discrepancies concern the evaluation-budget configuration, standalone qLogEHVI model fitting, and the construction of run-dependent hypervolume reference points. These issues could affect baseline strength and the comparability of the reported results. However, it remains unclear whether the inspected paths produced Tables 1–2 and the convergence curves. The authors should identify the actual experimental pipeline and reconcile it with the submitted implementation. If the reported experiments used the affected paths, the relevant comparisons should be rerun and HV recomputed under the stated shared-budget and shared-reference protocol.
  2. The reliability of the validated knowledge is not yet sufficiently established. The main method validates predictions against the sampled anchor, whereas the execution example in Appendix E uses the mean of the selected parents. This discrepancy can change whether a directional prediction is judged consistent and should be reconciled. More broadly, agreement between predicted and observed objective changes supports the net effect of an edit, but does not by itself establish the correctness of the variable-level mechanisms retained in the archive. The inconsistencies between the CarSideImpact domain background and the benchmark’s variable dependencies make this distinction particularly relevant. The paper should provide stronger evidence that the validation process identifies and corrects misleading explanations.
  3. The experiments do not sufficiently isolate the contribution of knowledge validation and inheritance. The knowledge-source ablations show that the supplied information affects performance, but do not clearly separate the benefit of inherited reasoning and its validation from access to numerical observations and other contextual information. A controlled comparison that preserves the available observations while selectively removing validation or inherited reasoning would more directly support the central claim. The methodological distinction from existing approaches that accumulate hypotheses or calibrate LLM guidance through feedback also needs clearer justification. A focused comparison under compatible information and evaluation budgets would be more informative than a broad set of additional baselines.
Questions for authors
  1. Which exact implementation and configuration produced Tables 1–2 and the convergence curves? Please reconcile the identified baseline discrepancies with the stated protocol and confirm whether the reported HV values were computed using the shared reference points in Table 3.
  2. Which reference was used for validation in the reported experiments: the sampled anchor or the parent mean? How does the validation and knowledge-update process identify and correct misleading mechanistic explanations, including those in the CarSideImpact example?
  3. Does EKO’s advantage remain when the qLogEHVI coverage supplement is disabled or its contribution is otherwise controlled for? Separately, what is the effect of removing validation or inherited reasoning while preserving access to numerical observations and comparable context/call budgets? Please clarify what information the −Parent ablation removes.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.