acceptodds
Under review as a conference paper at ICLR 2027

Self-Structuring Latent Action Spaces for Black-Box Instruction Optimization

Abstract

Instruction optimization for large language models (LLMs) is a high-dimensional discrete black-box optimization problem in which reliable feedback comes only from costly model evaluations. Existing methods search directly in raw text or use LLMs as high-frequency mutation operators, which makes the search hard to analyze and ties generation-side cost to population size and optimization depth. We propose Self-Structuring Latent Action-Space Evolution (SLAE), which represents each instruction as an integer-coded genotype that selects a syntactic scaffold and slot-wise semantic actions and is mapped deterministically to text. Routine search runs in this categorical genotype space without LLM calls, while the space itself evolves: Dynamic Structural Evolution (DSE) adds newly discovered scaffolds, and Adaptive Action-Space Refinement () adds slot-level actions when search stagnates. Because genotype coordinates are nominal categories, we show that differential-evolution arithmetic on them depends on how categories are indexed and give a label-equivariant variation rule. Under matched evaluation budgets, SLAE with integer-coded differential evolution achieves the best results on all nine language understanding and generation benchmarks we evaluate. In controlled studies under the main experimental protocol, disabling the expansion mechanisms costs 8.7 accuracy points on AG News and 5.3 on a held-out task with a different model family, while on AG News these mechanisms account for about 0.1% of the measured token budget.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Models, Black-box Optimization, Latent Space Co-evolution

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_M7ksOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SLAE, which divides instructions into semantic scaffold and slot-wise semantic actions, with one gene responsible for selecting the scaffold and other genes responsible for iteratively generating semantic actions, and then generates the final prompt text through a deterministic mapping. The paper also proposes two spatial expansion methods, DSE and A2R; the former adds new grammatical scaffolds, and the latter supplements new semantic actions when the search stagnates. Experiments are conducted on two types of tasks, NLU and NLG, and compared with other baselines, SLAE achieves the best results.

Critical strengths

The method in this paper is easy to understand, and judging from the experiments in the paper, it achieves good improvements.

Critical weaknesses
  • Experimentally, this paper only tests on an outdated small-scale model (Qwen2.5-7B) and outdated classification and summarization generation tasks, which completely fails to demonstrate the effectiveness of the proposed method and is entirely disconnected from the current development of LLMs.

  • The extensive gaps in the experiments make the effectiveness of the method unconvincing at present, as it lacks testing on advanced closed-source models (such as Claude-5.5 and GPT-6) and lacks evaluation on high-difficulty math, coding, and long-horizon Agent tasks.

Questions for authors

The SLAE method itself decomposes a prompt into two parts, a scaffolds and slots. Is such a structure suitable for the forms of tasks that people currently focus on, such as Agent tasks? I think the task of prompt evolution seems to have become outdated. At present, more attention is focused on harness evolution, and the task of harness evolution almost completely subsumes prompt evolution.

3 comments
AuthorsOpenReview ↗

The evaluation model is held fixed across methods; the contributions are properties of the search procedure

Thank you for the review, and for noting that the method "is easy to understand" and "achieves good improvements" in the paper's experiments. The remaining points concern the scope of the evaluation, and we answer them in turn.

Two points on the summary. To clarify how slot genes work: they select among existing actions, and no LLM call is made per candidate; new actions are generated only when search stagnates. This is what reduces generation-side calls from O(Np⋅T)O(N_p·T) to O(T)O(T). The paper also makes a contribution at the level of the search operator: differential-evolution arithmetic on nominal genotype coordinates depends on how categories are indexed, and the label-equivariant landing rule removes this dependence (Proposition 1). This result does not depend on the evaluation model.

What the experiments establish. Within the setting the paper studies, controlled comparisons demonstrate the effectiveness of the method. Across nine tasks, the methods in each table are re-run under the same evaluator, task splits, and budget (Section 4.1). Section 4.5 then tests alternative explanations of the gains: it switches expansion off, changes the landing rule, and replaces the prompt pool with random initialization. Switching expansion off lowers accuracy on AG News by 8.73 points (Table 6).

Models used in the paper. The controlled study of Section 4.5 and the comparison of generation-side tokens with EvoPrompt in Section 4.2 also use Llama-3.1-8B-Instruct, on CoLA. There the expansion effect is +5.25: it replicates on a held-out task with a different model family.

Model scale and task set. The nine tasks are the nine datasets evaluated by EvoPrompt (ICLR 2024), chosen for a like-for-like comparison with the closest baseline (Section 4.1), whose classification results are themselves reported on a 7B open model (Alpaca-7b). Work of this kind continues to be published at this model scale: GEPA (ICLR 2026) uses Qwen3-8B as its open-weight model, and PromptQuine (ICML 2025) uses Meta-Llama-3-8B-Instruct for all methods in its classification comparison, with SST-2, Subj, and AG News among its datasets. Models are replaced every few months; the paper's contributions are properties of the method and do not expire with a model: its cost structure, equivariance, and the attributability of improvements to explicit search events.

Closed-source frontier models. The paper studies the feasibility of a method: instruction search in a structured genotype space, with no LLM call per candidate, that still improves on text-space search under the same evaluator. The evaluation model is a controlled variable shared by every method, not the object of study. The more expensive the model an optimizer calls, the more the reduction from O(Np⋅T)O(N_p·T) to O(T)O(T) generation-side calls saves.

Math, coding, and long-horizon agent tasks. These lie outside the scope of this paper, as its conclusion states.

The scaffold-and-slot structure and agent tasks. At the level of representation, the decomposition acts on instruction text and requires only that the text contain components with fixed roles that can be exchanged. The prompts in an agent harness are such text: a system prompt states a role, a task, tool-use rules, and an output format.

Harness evolution and prompt evolution. We agree that harness evolution subsumes prompt evolution, and recent harness papers say so themselves: HarnessEvolve (arXiv:2609.00829) describes the harness that self-evolving agents optimize as "prompts, skills, tools, and execution logic", and The Last Harness You'll Ever Build (arXiv:2604.21003) describes harness engineering as "designing the prompts, tools, orchestration logic, and evaluation criteria that make a foundation model effective". Prompts are thus among the things a harness optimizer rewrites with an LLM. Prompt evolution also remains current at this venue: GEPA (ICLR 2026) describes itself as "a reflective prompt optimizer for compound AI systems" that "evolves only the set of prompts", and its title is "Reflective Prompt Evolution Can Outperform Reinforcement Learning". Because the larger problem contains this step, it inherits the step's generation cost and the question of whether its search operator is sound. In a new comparison under the same evaluator, SLAE uses about 0.8% of GEPA's generation-side tokens, with comparable accuracy on AG News, TREC, and SST-5 (revised Section 4.2).

I do not agree with the unqualified argument that models are updated every few months and that a paper’s contribution lies in the method itself. Although methodological contributions are important, LLM research should also be evaluated under settings representative of the field at the time the research is conducted. A method that works on Qwen2.5 may be entirely ineffective on Qwen4.

For example, a method might generate large quantities of synthetic mathematical training data and demonstrate substantial gains on an outdated model. However, if the generated queries are too simple to benefit more capable open-source models, and the method neither supports nor addresses the generation of more challenging problems, its practical relevance under current conditions remains unclear. Strong results on an older model alone would not establish the method’s continued value.

A paper’s contribution may indeed lie in its method, but this does not remove the need to establish its relevance to contemporary models and tasks. The method should demonstrate benefits under reasonably current and representative settings, rather than relying solely on outdated ones. I therefore maintain my original score.

AuthorsOpenReview ↗

Run unchanged on a later, larger model, SLAE keeps its generation-side saving on every task and its accuracy lead on most

Thank you for the follow-up. We accept the criterion as stated: the method "should demonstrate benefits under reasonably current and representative settings". The risk your comment raises, a method that works on Qwen2.5 but not on a later model, can be tested directly by replacing the model and changing nothing else. The comparison below does this with Qwen3.8-27B-FP8. With every setting unchanged, SLAE keeps its generation-side saving on every task and its accuracy lead on AG News and SST-5, and everything it adds to its search space is written by Qwen3.8-27B-FP8 itself.

Setting. Your example measures relevance by the benefit to "more capable open-source models". Relative to the paper's Qwen2.5-7B, Qwen3.8-27B-FP8 is such a model: it belongs to Qwen's latest publicly released open-weight generation and has nearly four times as many parameters. Every method uses it, in non-thinking mode, both to evaluate and to generate. The baselines are the DE variant of EvoPrompt, the paper's closest baseline, and OPRO; in both, the LLM writes every candidate instruction, whereas in SLAE it writes only the content that builds and expands the search space. The protocol is that of the paper's same-evaluator comparison (revised Section 4.2 and Appendix C.5), with the same three tasks and three seeds; the three methods share the initial prompt pool, the fitness subset, and the candidate-evaluation budget. SLAE keeps every setting of that comparison, and nothing is tuned on Qwen3.8-27B-FP8.

Result. SLAE uses about 3.7% of EvoPrompt's generation-side tokens and about 2.8% of OPRO's, and under 5% of either on every task. At this cost, it has the highest mean held-out accuracy of the three methods on AG News and SST-5, two of the three tasks; on both, it also varies least across seeds.

Replacing the model replaces the writer of everything SLAE adds to its search space. Your example describes a method whose generated queries stay "too simple" as the models they are meant to train become more capable, because the method cannot generate harder ones. In SLAE, as in the paper's setup (Section 4.1), the model the instructions are optimized for also builds and expands the search space. In these runs, Qwen3.8-27B-FP8 extracted the slot phrases from the pool prompts, filled the action sets, wrote new actions when the search stagnated (A²R), wrote new scaffolds (DSE), and scored every candidate; apart from the initial prompt pool, which all three methods share, no generated content in the search comes from an earlier model. The instructions SLAE returns draw on this content: none of the nine is a pool instruction, eight contain a phrase that Qwen3.8-27B-FP8 wrote, and four contain one it wrote during the search.

The case for SLAE therefore no longer rests on results from "an older model alone".

Reviewer_QpuaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

SLAE searches over discrete scaffold and semantic-slot combinations, avoiding per-candidate LLM rewriting while expanding the scaffold library and action sets during optimization.

Critical strengths
  1. The representation makes search easier to inspect and reduces generation-side calls. Table 2 reports a seven-task classification average of 85.51, 3.54 points above EvoPrompt(DE), with gains on two generation tasks as well. Random initialization, cold-start scaffolds, and the CoLA/Llama control show that the result is not wholly dependent on the manual prompt pool.

  2. The analysis of category-index arithmetic is clear, and the equivariance guarantee has an appropriately stated scope. Separating generation cost from evaluation cost also makes the efficiency discussion more useful than call counts alone.

Critical weaknesses
  1. The experiments do not isolate the value of adaptive expansion. Disabling DSE/A2R costs 8.73 and 5.25 points in Table 6, but also removes all newly generated scaffolds and actions. This establishes that expansion helps, without showing whether stagnation-based timing and elite-based slot selection outperform a fixed library of the same size. The cold-start control does not answer that question. The comparison with related optimizers is also incomplete: GEPA was published at ICLR 2026 and is discussed in the paper, but is not evaluated.

  2. Important implementation and evaluation details are missing. The paper does not explain how arbitrary prompts from the shared pool are encoded into the finite scaffold-slot representation, define the slot-fitness association, or give k_stag in Table 8. The later prompt in Figure 2 introduces two labeled demonstrations whose source and availability to baselines need clarification. Output handling matters too: the LE-minus-DE difference changes from +2.10 to +9.13 under post-processing in Table 6. A fixed answer-extraction protocol and an explanation of whether parents and offspring are rescored on the same development batch are needed to interpret the gains.

Questions for authors

On AG News and CoLA, how does full SLAE compare with a pre-generated library using the same generation and evaluation budgets, and with GEPA under the same evaluator? Please provide an end-to-end encoding and expansion example, including thresholds, parsing failures, and demonstration sources. Does the ranking hold when saved final prompts are rescored with a common answer-extraction rule?

1 comment
AuthorsOpenReview ↗

On TREC, a pre-generated library recovers about half of the expansion effect, and Tables 2 and 6 use one extraction rule

W1 and your question

SLAE versus a fixed library of the same size. You are right that the joint ablation of Table 6 removes the generated content together with the mechanism. We ran the control you describe on TREC with ten seeds. Table 6 shows the expansion effect on AG News and CoLA; the control adds TREC as a third task.

A pre-generated library keeps the generation budget but drops stagnation-based timing and elite-based slot selection: before search starts, the generator is called exactly as often as in the same seed's SLAE run, always on the initial best instruction with A²R rotating over the slots. The library and SLAE are matched in generation calls, evaluation budget, and the number of new items tried per generation, and the library is of comparable size (622 vs. 587 items over the ten runs).

Both new comparisons use Qwen2.5-7B-Instruct for evaluation and generation, and held-out accuracy is scored with the same extraction rule as Tables 2 and 6.

Paired differences in held-out accuracy:

Mean ± s.e.Positive seeds
SLAE − library+2.76 ± 1.377 of 10
Library − SLAE-static+3.28 ± 2.537 of 10
SLAE − SLAE-static+6.04 ± 1.798 of 10

So the library recovers about half of the expansion effect, and generating during search adds the remaining 2.76.

In two seeds the library falls 8 to 11 points below SLAE-static; SLAE never falls more than 4.4 below it. In six SLAE runs the final prompt uses content generated from a current best that already contained in-search content, which a library built from the initial best cannot contain.

The revised Section 4.5 replaces its caveat on the joint ablation with this control and its result; the revised Appendix D gives the protocol, settings, stability across seeds, and development trajectories.

GEPA under the same evaluator: SLAE uses about 0.8% of GEPA's generation-side tokens. We compared SLAE with the official GEPA implementation on AG News, TREC, and SST-5 (three seeds), with GEPA's limit on example evaluations set to the evaluation budget of the population-based methods. Accuracy is comparable: SLAE − GEPA is −2.73, +3.73, and −0.86, respectively (mean +0.05), each about the size of the standard deviations across seeds (1.6 to 4.2). The revised Section 4.2 reports this comparison.

W2 and your questions

Encoding pool prompts. In the control, each pool prompt becomes a scaffold of its own: the generator copies its task-name, input, and output phrases verbatim into slots, and label words stay as fixed text. The filled scaffold must reproduce the prompt character for character; all 100 pool prompts in the control did.

Slot–fitness association. Among the top-ranked individuals, individuals are grouped by the action they select at a slot, and fitness is averaged within each group; the association of the slot is the highest group mean minus the lowest. Slots at which only one action is observed are skipped, and A²R expands the slot with the largest association (revised Section 3.3).

k_stag. The revised Table 8 lists the default, 2, which the control and the example in Appendix D also use.

Parents and offspring. Both are scored on the same development batch, which is fixed per seed for the whole run; each instruction is scored once and cached, so parents are not rescored (revised Section 3.2).

Figure 2. The two labeled sentences were written by the generator within one SMO rewrite, as the revised caption states; they occur in no data split. Baselines that rewrite prompts with the same LLM are equally free to write such sentences.

Output handling and rescoring. In Table 6, "post-processing" means cleaning generated scaffolds during search; it is not applied to model outputs. Held-out accuracy in Tables 2 and 6 uses the answer-extraction code of EvoPrompt's public release, unchanged: the prediction is the first label word in the output after punctuation is removed. So +2.10 and +9.13 come from two separate sets of search runs, both scored in the same way, and not from two ways of scoring the same outputs. Every number in Tables 2 and 6 is scored with this one rule, so the rankings they report are already rankings under a common extraction rule. The revised caption of Table 6 defines post-processing, and the revised Section 4.1 states the rule.

End-to-end encoding and expansion example. Appendix D follows one run of the control from pool prompts to final prompt: the top pool prompt becomes a four-slot scaffold that decodes back verbatim, A²R adds task-name phrases in generation 3, and SMO rewrites the current best scaffold in generations 8 and 17. Thresholds: A²R fired after k_stag = 2 generations without improvement. Parsing failures: generated content that fails a format check is discarded; across the ten runs, 23 of 285 scaffold rewrites were discarded, most for losing slot markers. Demonstration sources: no labeled example enters any prompt in this run.

Reviewer_ZFjPOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes SLAE, which encodes each instruction as an integer-coded genotype. The structural gene selects a syntactic scaffold, and semantic genes select word- or phrase-level actions for each slot. A deterministic function maps the genotype to instruction text. The inner loop runs differential evolution in this genotype space with zero LLM calls. The authors further show that integer-coded DE arithmetic depends on how categories are indexed, and introduce a label-equivariant variation rule that removes this dependence.

Critical strengths

This paper addresses the problems of existing instruction optimization methods that iteratively rewrite instructions with LLM calls, whose generation-side cost grows linearly with population size and generations and whose improvements cannot be attributed, by combining free evolution in an integer-coded genotype space with sparse space expansion upon stagnation. Under a matched evaluation budget, it achieves the best results on all benchmarks while reducing generation-side LLM calls from O(N_p·T) to O(T).

Critical weaknesses

Please refer to the Questions.

Questions for authors

(1) The abstract claims "achieves the best results on all nine language understanding and generation benchmarks," yet Table 3 on SAMSum/ASSET appears to include fewer baselines than Table 2.

(2) SLAE's advantage is the generation-side cost of O(T) versus O(N_p·T). However, all experiments are conducted at a single small budget point (N_p = 10, T = 20), and no results under larger budgets are reported, so how accuracy scales with the evaluation budget remains unclear.

(3) The criterion for A2R's slot selection is stated in a single sentence, "the slot whose choices are most associated with fitness variation," without defining which statistic is used.

1 comment
AuthorsOpenReview ↗

The abstract now names its baselines, and the generation-side saving holds at every budget and in a comparison with GEPA

Thank you for the review and for your support.

A new comparison under the same evaluator adds GEPA (ICLR 2026). SLAE uses about 0.8% of GEPA's generation-side tokens (about 9.0K per run, against 1.08M) and about 4% of its generation-side calls, with comparable accuracy on AG News, TREC, and SST-5 (SLAE − GEPA: −2.73, +3.73, −0.86; mean +0.05; revised Section 4.2). In a new control on TREC (ten seeds), where generation and evaluation budgets are matched, SLAE exceeds a library of comparable size generated before search by 2.76 points (s.e. 1.37; 7 of 10 seeds), and the library recovers about half of the expansion effect (revised Section 4.5).

(1) Baselines in Table 3

We agree that the abstract should say what its claim is made against. The revised abstract and contribution list now state it for each task family: the best results "on all seven language understanding benchmarks against human-authored, APE, APO, EvoPrompt, and PromptQuine baselines, and on both generation benchmarks against human-authored, APE, and EvoPrompt baselines". Table 3 follows the comparison that EvoPrompt (ICLR 2024) reports for these two tasks: MI, APE, and EvoPrompt's GA and DE variants. NI, PromptSource, and APO appear only in EvoPrompt's classification table; EvoPrompt's Section 2 notes that methods which learn from incorrectly predicted cases, such as APO, are constrained to tasks with standard answers and cannot be directly applied to generation tasks. PromptQuine's generation experiments are style transfer and jailbreaking. The revised caption of Table 3 states its baseline set.

(2) A single budget point

Within T = 20, the new control on TREC shows how accuracy changes with the evaluation budget (revised Appendix D). We track the gain in best development accuracy over the initial best, averaged over the ten seeds:

After 100 candidate evaluationsAfter 200 (full budget)Runs still improving in the second half
SLAE-static (fixed space)+6.70+7.062 of 10
SLAE (expansion)+7.61+9.458 of 10

So additional evaluations keep paying off when the space expands during search and largely stop paying off when it is fixed.

We kept the budget of EvoPrompt, the primary baseline (Np=10,T=20)(N_p = 10, T = 20), so that the main comparison is like-for-like (Appendix C.2). The cost saving carries over to larger budgets: text-space evolution makes N_p·T generation-side calls, and SLAE at most (1+psmo)(1 + p_smo)·T in expectation (Table 1). The ratio between the two, Np/(1+psmo)N_p/(1 + p_smo), does not shrink as T grows and increases with the population size; at the paper's setting it is 200 calls against an expected bound of 26.

(3) The statistic behind A²R's slot selection

You are right that the submitted version does not define the statistic. Among the top-ranked individuals, individuals are grouped by the action they select at a slot, and fitness is averaged within each group; the association of the slot is the highest group mean minus the lowest. Slots at which only one action is observed are skipped, and A²R expands the slot with the largest association. The revised Section 3.3 states this definition.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.