acceptodds
Under review as a conference paper at ICLR 2027

One Model, Many Modes: Evolving System Prompt Ensembles for Creative Generation

Abstract

Preference-aligned LLMs can exhibit **mode collapse**, concentrating their output distributions around a narrow set of high-reward responses. This limits creative generation, which requires quality, novelty, and diversity. Existing test-time methods often emphasize diversity, giving less attention to whether individual responses are both high-quality and novel. This distinction matters in practical use, where users may seek a single compelling, original response. We formalize creative generation as a multi-objective distributional optimization problem and introduce **SEPO**, a training-free framework for system prompt ensemble optimization. The framework combines prompt evolution with greedy max–min selection to identify behaviorally distinct system prompts under a quality constraint, then uniformly samples one prompt per response at inference time. Across four creative generation tasks with increasingly stringent quality requirements and three open-weight models, SEPO achieves the highest joint coverage at every reported tolerance: all three scores fall within 30% of their respective best observed values in 91.7% of scenarios, versus 58.3% for the strongest baseline, showing strong joint performance across quality, diversity, and novelty. Further analysis shows that expanding the optimized ensemble increases separation among prompt-conditioned response distributions. These results support **system prompt ensemble optimization** as a promising approach to improving LLM creativity at inference time.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Model, LLM Creativity, Mode Collapsing, Prompt Optimization, Open-ended Discovery, Agent Design

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_KsJLOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors introduce SEPO, a system prompt ensemble method for improving a model's diversity, quality, and novelty on different tasks.

Critical strengths
  • The method is light-weight and does not require additional compute

  • The authors tested SEPO against four test-time baselines on four datasets, which is pretty comprehensive

  • The three dimensions (i.e. quality, diversity, and novelty) are well explained and defined in the problem definition section.

  • Evaluation for novelty is pretty comprehensive and includes methods from LLM judge to embedding and n-gram metrics.

Critical weaknesses
  • SEPO is conceptually similar to Verbalized Sampling in the sense that it elicits different response variations by prompting the model

  • There’s no explanation for how the base system prompts are constructed or selected? Since system prompt ensemble is the core of the SEPO algorithm, this missing part seems essential. The paper would also benefit from conducting some causal analysis showing how different prompt additions or combinations lead to greater or lower diversity, as well as providing some qualitative examples of what the prompts look like in the main text.

  • [minor]: “Evaluation Tasks” appeared twice in section 5.

Questions for authors
Reviewer_d1phOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

In this work, the authors propose a new evolutionary search method to search for new system prompts to use for ideation, creative writing, and mathematical reasoning. The method is called SEPO and evolves a pool of candidate system prompts that give different outputs but still meet quality thresholds. Experimental evaluation is done with three open-weight models and four datasets with expected metrics including LLM as a judge, embedding distances, and lexical metrics. Results are increased diversity from baselines prompts with quality trade-offs.

Critical strengths

Overall, I find the writing very clear and the task nicely framed and set up (although I'm not sure about including mathematics as an application). The related work covers creativity-oriented decoding, prompt ensembles, and prompt optimization, and is fine overall. The framing makes the objective of finding different outputs while meeting quality thresholds easy to understand.

Critical weaknesses
  1. Comparison against other prompt-search methods and evaluation rigor. Experiments are relatively small-scale. The cross-method evaluation groups very different styles of methods, including sampling changes and prompt optimization. As far as I can tell, there is no other iterative prompt-optimization baseline. The baselines therefore have unequal budgets for finding good prompts. While it is useful to compare against the reported baselines, they are limited and do not tell the whole story.

A common use case of manually optimizing prompts is to provide few-shot examples. There is minimal discussion of this, no comparison, and no discussion of the optimized system prompts to see whether examples were created within them. I do not think this can simply be caveated as outside the focus.

For CreativeMath, please confirm that correctness is actually evaluated as well. I would likely focus on creative writing and ideation and keep mathematical reasoning separate, as the latter should be evaluated in a context where correctness can be verified. Claims that quality does not suffer also seem to contradict the results. These issues make it difficult to assess the contribution and the diversity-quality trade-off.

  1. Limited scientific insight into what prompt optimization discovers. I would say this method is a form of evolutionary search applied to system-prompt optimization for fixed tasks, so novelty is relatively limited. This could be fine with deep and rigorous experimental design and reproducible insight. It would be useful to analyze what is actually changing in the prompts: is this reward hacking, or is the method discovering good and interesting system prompts? The review of the outputs should go deeper into what the optimization discovers and whether the optimized prompts generalize across models or are model-specific. This is central to understanding the scientific contribution beyond the reported diversity metrics.

As further directions, applying the method to multi-task system prompts or multimodal settings could be useful.

Questions for authors
  1. Clarification: prompt-search comparisons and budgets. Are there comparisons with other iterative prompt-optimization methods? How do the budgets for finding good prompts compare, and how should we interpret the results given the unequal search budgets and lack of a few-shot comparison?

  2. Clarification: correctness and quality. For CreativeMath, is correctness independently evaluated in addition to the reported quality metrics? How do the claims that quality does not suffer reconcile with the reported quality trade-offs?

  3. Clarification: learned prompts and generalization. Were there length constraints on the learned system prompts? Do they contain few-shot examples? What do the discovered prompts reveal about what is being optimized, including possible reward hacking, and is there evidence that any of them transfer from one model to another?

Reviewer_HFhZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

Creative generation papers at test time. Formalizing creativity as three distribution-level objectives of quality/diversity/novelty, training-free optimization system prompt ensemble: LLM proposer evolves candidates (10 rounds ×4 steps ×4 seeds ×4 proposals), measures behavioral differences by JS divergence estimated with 2 rollout and 32 token truncation, greedy max-min selection under double quality threshold, and uniform sampling at inference time. 4 dataset × 3 model (1.7b-8b), the top result is 30% tolerance joint coverage 91.7% vs 58.3%.

Critical strengths

1.Splitting creativity into three distribution-level goals of quality, diversity and novelty and demonstrating that each pair is not interchangeable is a conceptual clarification of real value in this field. Table 1 has literature reference significance for combing the three-dimensional coverage of existing test-time methods.

2.The ablation advantage (12.9% decrease in diversity /22.4% decrease in novelty) of Max-min selection in action space (JS divergence) over text space (cosine similarity) provides direct evidence for the core claim of our method.

3.The training-free and "optimize once, reuse everywhere" cost structure has practical value for applications that cannot fine-tune their models.

Critical weaknesses

1.The overall framework is disconnected from its own formalization: Section 3 sets up the three-objective distributional optimization formalism, but Section 4's pipeline doesn't really optimize it - novelty JNov is never optimized directly, only indirectly through reference centralized inclusion of default instructions proxies; The optimality of uniform mixed sampling is not demonstrated. The framework is composed of multi-level heuristics such as clustering, seed selection, evolution, Max-min, and double-mass gate. Each data set requires about 640 offline optimization of proposals, which is contradictory to the positioning of "lightweight test-time method".

2.Results are messy and out of order: the main result plot (Figure 3, quality-diversity/novelty tradeoff) appears in Experimental Setup (Section 5) before any results discussion; The main body of the text is occupied by the smallest Qwen3-1.7B, and the other two models are all appendized, so the reader cannot judge the cross-model robustness from the text. Three sets of metrics (LLM refs/embeddings/n-grams) are separated between the main text and the appendix; The headline indicator "joint coverage" is a non-standard custom indicator, but it appears along with the Table in Table 2(b) and lacks preparation; Table 2 Simultaneously confounding the two granularities of per-dataset score and coverage; Figure 5 squeets the optimization trajectory, cue number-diversity curve, and PCA projection into one figure. The overall results need to be rearranged in the order of "main conclusion → robustness → mechanism → ablation", and the main results should be upgraded to all three models.

3.Lack of empirical rigor: the JS divergence is estimated with 2 rolloutions per pair and a 32 token cutoff, while the most noise-sensitive Max-min selection is built on it, and the alternative explanation of "in selection noise" is not excluded. The SEP control without equal search budget cannot distinguish the gain from the optimization algorithm or the search budget. There is no seed level stability report for the prompt set selection results. All verification is limited to 1.7B-8B small model.

Questions for authors

1.Could you run SEP (your unoptimized ensemble) under the same total LLM-call budget as SEPO — i.e., regenerate/resample its prompt pool multiple times and keep the best — and compare? If budget-matched SEP matches SEPO's joint coverage, the contribution of the optimization algorithm itself is falsified — the gains would come from search budget, not search design.

2.Could you report how the selected ensemble changes as JS divergence is estimated with 2 / 4 / 8 / 16 rollouts per pair? If the selected prompts turn over substantially with more rollouts, the max–min selection is falsified as operating on noise rather than on behavioral differences.

3.Could you compare uniform mixture sampling against quality-weighted mixture sampling over the same optimized ensemble? If quality weighting recovers the quality drop (≈3.8→3.4) at no diversity cost, the uniform-sampling protocol — a core design choice — is falsified.

4.Could you evaluate on at least one ≥30B model whether prompt-conditioned behavior distributions remain separable at all? If separability collapses with scale, the framework's ceiling — being bounded by the base model's conditionability — is demonstrated, and the claims should be scoped accordingly.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.