acceptodds
Withdrawn from ICLR 2027

Breaking Homogeneity: Diversifying Persona Sets for Creative LLM Outputs

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Language models often produce homogeneous responses to open-ended tasks; such homogeneity can spawn groupthink—the convergence of ideas toward a singular and potentially suboptimal decision. We formulate persona diversification as a set-level conditioning problem and study two orthogonal design choices: selecting versus generating personas, and space-filling versus frontier-seeking diversity. We instantiate this design space with four methods spanning coverage and dispersion subset selections, uniform-coverage sampling, and evolutionary persona generation. Evaluations on the Alternative Uses Task (AUT), Infinity-Chat, and Divergent Association Task (DAT) show the benefits of the proposed methods across tasks and creativity objectives. On AUT, evolutionary persona generation increases response diversity by 78.8%, originality by 26.1%, flexibility by 49.5%, and holistic creativity by 13.9% over task-only prompting, while maintaining 98.5% validity; on Infinity-Chat, it nearly doubles persona-induced response separation relative to random personas. Moreover, evolutionary personas compose with creativity-optimized prompting, further increasing its response diversity by 18.6% and creativity by 6.3%. These results establish persona-set geometry as a task-agnostic mechanism for eliciting divergent LLM outputs, and support persona diversification as a reusable complement to prompt optimization.

Primary area:
foundation or frontier models, including LLMs
Keywords:
large language models, computational creativity, persona diversification, diverse subset selection, evolutionary optimization

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_g2n8OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper studies persona diversification through two axis, selecting personas from an existing pool and generating new ones. It introduces four algorithms based on different diversity objectives, with theoretical guarantees for some of them, and evaluates whether diversified persona sets improve the diversity and creativity of LLM-generated responses using automatic metrics and LLM-as-judge.

Critical strengths

The main strength of the paper is its systematic formulation of persona diversification, organizing the problem along two axes and proposing four corresponding methods. The theoretical guarantees for the selection and sampling algorithms provide a useful foundation for the proposed framework, although these guarantees are limited to persona-space objectives.

Critical weaknesses

1. Writing, organization, and clarity

The manuscript frequently reads like AI-generated text that has not been sufficiently reviewed or polished. There are recognizable patterns of AI-assisted writing, including excessive jargon, unnecessarily complicated phrasing, and abrupt transitions between concepts without adequate explanation. While the use of AI writing tools is not inherently problematic, submitting text without careful revision is concerning, particularly in an academic paper where clarity, precision, and coherent argumentation are essential.

Beyond these stylistic issues, the paper lacks a clear narrative and hierarchy of information.

For example, the abstract reports numerous individual percentage improvements, making it read more like a conclusion than a concise summary of the problem, approach, and main findings.

The introduction does not adequately explain the proposed framework. While the general idea of exploring the relationship between persona diversity and response diversity is understandable, the actual approach remains vague. In lines 073–079, four methods are introduced without sufficient intuition about what each method does or how they differ. Furthermore, one of the framework's main axes, "space-filling coverage versus frontier-seeking separation," is not clearly explained in the text, forcing readers to rely on Figure 1. However, Figure 1 itself is not sufficiently informative, as the formulas are presented without adequate explanation of their notation or meaning.

Similarly, the paragraph beginning at line 080 suddenly introduces the concept of the "structural geometry of persona sets" without adequately explaining what it means in this context. This issue recurs throughout the paper, as technical terms and concepts are frequently introduced without sufficient motivation or explanation, making the narrative unnecessarily difficult to follow.

The mathematical presentation could also be improved. Equations (2)–(4) describe standard distance measures, such as cosine distance, and could largely be moved to the appendix. Including these basic definitions in the main text adds clutter without providing much insight, while more important parts of the framework remain underexplained. Conversely, Equations (5)–(6) and the accompanying discussion in lines 162–170 are introduced rather abruptly, without clearly explaining their purpose or how they connect to the subsequent algorithms.

2. Motivation and justification of the methods

The modeling choices and algorithms in Section 3 would benefit from stronger motivation and more intuitive explanations. Although the paper provides formal definitions, theoretical guarantees, and references to established techniques, it is not always clear why these particular formulations were chosen or what intuition supports them.

Many of these optimization and sampling techniques have connections to established problems in the literature. Making those connections more explicit, particularly in the main text, and explaining the intuition behind each modeling choice would help readers understand the methodology and distinguish its contributions from existing approaches.

As currently presented, the section emphasizes mathematical formalism and theoretical guarantees without sufficiently developing the reasoning behind the proposed design choices.

3. Concerns about the evaluation methodology

The evaluation methodology raises significant concerns. Previous work has shown that LLM-as-a-judge and automated creativity metrics are not always reliable [1], and that LLM-based creativity judgments can exhibit weak agreement with human evaluations, particularly for subjective and holistic aspects of creativity [2,4]. Furthermore, average cosine similarity alone is insufficient to establish output homogeneity [3].

Although the paper employs multiple evaluation metrics, it does not directly evaluate whether human judges perceive the outputs of the proposed methods as more creative or diverse. Given the known limitations of automated evaluation, the absence of such human evaluation makes it difficult to determine whether the reported improvements correspond to meaningful gains in creativity. This substantially weakens the empirical evidence supporting the paper's main claims.

4. Limited evidence of benefits beyond simple prompt variation

As discussed in [3], relatively simple changes in prompt wording and specificity can substantially increase measured response diversity. Therefore, observing greater diversity after introducing different persona instructions is not particularly surprising on its own. This raises questions about how much of the reported improvement reflects the proposed diversification framework rather than the more general effect of prompt variation.

Although the paper compares against random personas and creativity-enhanced prompting, the results do not sufficiently establish the practical advantage of the proposed approach over simpler prompt-diversification strategies, especially given its additional complexity. Stronger comparisons against inexpensive prompt-variation baselines would help clarify the contribution.

References

[1] https://aclanthology.org/2026.eacl-long.297/

[2] https://arxiv.org/abs/2608.30754

[3] https://arxiv.org/abs/2609.33936

[4] https://arxiv.org/abs/2309.14556

Questions for authors

None

Reviewer_wss3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper is interested in breaking homogeneity of LLM responses through persona creation and selection criteria. The propose 3 theoretically justified algorithms and 1 heuristically justified algorithm for this purpose, each of which is targeted at different objectives defined within the paper (i.e., related to uniform coverage and maximal spread). The objectives and algorithms are all based on distances in a representation space. The theoretical algorithms generally rely on ILPs with exact solutions to attain theoretical guarantees. Empirical validation is done on a few models and datasets, mostly relying on the objectives defined within the paper or other automated evaluation strategies (LLM-as-judge).

Critical strengths
  • The paper's studied problem is interesting and useful, it is well motivated in the introduction, and the eventual methods and experiments largely tie back to the initial questions of interest producing a coherent research paper
  • The paper provides some theoretical motivation for a handful of algorithms it proposes - the results rely on generally known arguments and appear to be correct at a glance. the writing of these results is tailored to a computer science audience, making it somewhat understandable to some human communities.
  • the presentation of results addresses core questions one might have about the methods and is coherently presented.
Critical weaknesses

As a general view point, i feel the breadth of this paper is too large and the depth to shallow to allow contribution of generalizable scientific knowledge. My broad suggestion would be for the paper to focus on less content overall (e.g., one or two algorithms) and provide a more thorough investigation of these. I expand on a number of aspects below:

  • while the reliance on distance measures makes the methods analytically tractable, the methods and related evaluations are naturally subject to biases of LLM internal representations. however, the paper primarily uses automated methods (LLM-as-judge, embedding measures) to evaluate, which does not isolate the effect of such bias in results interpretation. Since the methods also optimize objectives that are defined using automated representations (embeddings), it is hard to see actual downstream utility on human preferences. Lexical metrics, designed experiments, specific rubrics + human validation of LLM judge, or actual human evaluation seems important to interpret the results accurately.
  • the interpretation of the automated metrics is difficult (e.g., what does a 1pt bump in diversity mean? or author defined metrics like dispersion?) it is rarely clear what the results actually mean for practical use besides interpreting seemingly small bumps. uncertainty quantification also does not help much because intervals are comparatively minuscule despite the apparent independent data point being the seed (n=5, are these values correct?).
  • the theoretical results mostly rely on re-formulation in terms of a distance problem and application of known results from optimization. while this is not inherently bad, I find it difficult to interpret actual algorithms, domain-specific insights, and key takeaways from the main text, despite often working in theoretical cs areas. this limits the utility of these theoretical results and connections to use in practice or insights for future algorithm design. for example, i did not understand the initial calibrated coverage algorithm until spending some significant time in the appendix. i think this paper is often trying to do to much, which severely inhibits clarity outside the possibility of repair with one additional page. while the appendix (a whopping 70 pages) could allow someone to implement these algorithms, i do not necessarily consider these theoretical results a generalizable scientific contribution, because an individual research looking to build on these ideas is unlikely to page through the appendix, and the key ideas and takeaways are not sufficiently well summarized in the main text.
  • by shifting to a distance view based on internal representations, and conducting all theory in that space, the authors theoretical results risk over-promising when they do not actually optimize the target objectives (coverage and diversity in the text space). as noted, the empirical results do not adequately address this. for instance, domain adaptation (tradition of Ben-David et al.) and extensions to representation learning (e.g., dissertation of Han Zhao) highlight the possibility of optimizing in representation space without actually optimizing the intended objective. this prompt selection problem is easily viewed in terms of representation learning, but there is limited discussion of this, and mitigations to potential problems, in the methods design.
  • there are typically inherent tradeoffs to prioritizing diversity and achieving task success, which are not discussed in this work. since diversity is optimized through user personas this also has potentially significant ethical ramifications, motivating a need for such analysis.
  • for a paper without generalizable theoretical insights, the empirical validation is quite limited (spanning one model, one independent judge, one embedding model, two datasets). Higher variability in automated methods could help to reduce concerns about potential biases introduced by the theoretical modeling choices.
Questions for authors

Please, address any misconceptions or direct questions in the weaknesses. thank you

Reviewer_sU8KOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates how to reduce the homogenization of creative outputs from Large Language Models by optimizing a set of personas. The authors propose a framework defined by two dimensions: selecting personas from an existing pool or generating new ones, and either maximizing coverage within the semantic space or exploring its boundaries. Based on this framework, the paper introduces four methods: COVERAGE, DISPERSION, UC-MCMC, and Evolutionary TextGrad, and evaluates their performance across three tasks: AUT, Infinity-Chat, and DAT. Experimental results demonstrate that structuring the diversity of a persona set can enhance certain creativity metrics and can be effectively integrated with existing prompt optimization techniques.

Critical strengths
  1. The problem is clearly defined, and the framework offers a degree of generalizability. By decomposing diversity into "space-filling" and "frontier-seeking" components, and further distinguishing between selection and generation, the authors establish a well-defined design space. This taxonomy facilitates a better understanding of the trade-offs between different diversity objectives.

  2. The work incorporates mathematical modeling and theoretical analysis. COVERAGE frames the selection problem as a threshold-based coverage optimization, while DISPERSION employs a max–min distance objective. Furthermore, UC-MCMC provides theoretical analysis regarding region allocation and asymptotic convergence.

  3. The experiments span multiple tasks and utilize a diverse set of evaluation metrics. The paper assesses response diversity, originality, flexibility, overall creativity, and effectiveness. In the context of the AUT, the "Evolution" approach improves diversity by 78.8% and overall creativity by 13.9% compared to standard task prompts; these gains are further enhanced when combined with optimized creative prompts.

Critical weaknesses
  1. Insufficient coverage of the most relevant existing work and an inadequate articulation of the paper’s original contribution. The paper fails to cite or discuss BILLY[1] (EACL 2026). Like the current work, BILLY enhances creative generation in LLMs by combining information from multiple personas. Although the technical approaches differ, there is a clear overlap in research motivation and objectives; the paper must clarify its substantive innovations relative to this existing work.

  2. Lack of direct comparison with established methods for diversity-based selection. The experiments primarily compare the proposed selection and generation methods against "Random" and "Typical" baselines. While the related work section touches upon existing diversity optimization techniques, such as Determinantal Point Processes (DPP), the experimental evaluation lacks direct comparisons with DPP, standard k-center, or farthest-first selection strategies. Consequently, the current results are insufficient to convincingly demonstrate the advantages of the proposed selection mechanism over other well-established approaches.

  3. The core conclusions regarding creativity rely primarily on automated evaluation, lacking direct evidence from human assessment. The overall creativity metrics for the AUT are derived from scores provided by an independent LLM "judge." Given that the primary objective of this research is to enhance subjective creativity, it is essential to validate the findings through human preference evaluations, specifically to distinguish between genuinely valuable, novel ideas and outputs that merely deviate semantically from common, conventional responses.

[1] BILLY: Steering Large Language Models via Merging Persona Vectors for Creative Generation

Questions for authors

Please see weakness.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.