One Model, Many Modes: Evolving System Prompt Ensembles for Creative Generation

Preference-aligned LLMs can exhibit **mode collapse**, concentrating their output distributions around a narrow set of high-reward responses. This limits creative generation, which requires quality, novelty, and diversity. Existing test-time methods often emphasize diversity, giving less attention to whether individual responses are both high-quality and novel. This distinction matters in practical use, where users may seek a single compelling, original response. We formalize creative generation as a multi-objective distributional optimization problem and introduce **SEPO**, a training-free framework for system prompt ensemble optimization. The framework combines prompt evolution with greedy max–min selection to identify behaviorally distinct system prompts under a quality constraint, then uniformly samples one prompt per response at inference time. Across four creative generation tasks with increasingly stringent quality requirements and three open-weight models, SEPO achieves the highest joint coverage at every reported tolerance: all three scores fall within 30% of their respective best observed values in 91.7% of scenarios, versus 58.3% for the strongest baseline, showing strong joint performance across quality, diversity, and novelty. Further analysis shows that expanding the optimized ensemble increases separation among prompt-conditioned response distributions. These results support **system prompt ensemble optimization** as a promising approach to improving LLM creativity at inference time.