acceptodds
Under review as a conference paper at ICLR 2027

Creative Priming: Decoupling Process from Product to Escape the Creative Paradox

Abstract

Many of the most consequential advances promised by AI, from curing diseases and inventing new technologies to formulating new scientific theories, are fundamentally creative acts. Yet creativity is uniquely difficult to learn because it faces a creative paradox: learning to repeat past examples of creative success makes those specific examples more likely to recur, inhibiting the very creativity desired. This work argues that creativity is better modeled as an explicit process rather than as a property of individual artifacts, and proposes to learn the underlying generative principles behind the creative process. In particular, to escape the paradox, we introduce Creative Priming (CP), wherein a trained priming model generates a prompt-specific primer that assists a response model in being creative, explicitly decoupling the creative process from response generation. The priming model learns through GRPO, rewarding primers that lead to responses that are novel, high-quality, and diverse, thereby reinforcing the intermediate steps that yield creative responses without reinforcing any specific response. Evaluating CP on NoveltyBench and comparing against response models directly trained for the same properties, we find that priming produces a 39–300% relative increase in the number of responses that are simultaneously novel, high-quality, and semantically distinct. Additionally, although the primer is only trained to improve creativity for a small open-source model (Qwen3-8B), the creative principles it learns transfer to frontier API models, increasing the number of semantically distinct responses they generate by 4.5–10× and diversity by 2.5–4.6×, while maintaining or improving quality. CP thus shows that decoupling the creative process from the responses themselves offers a concrete escape from the creative paradox.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
creativity, large language models, reinforcement learning

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_iKF7OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper aims to address the "creative paradox". If a model produces an interesting answer and training reinforces that answer, similar answers become more likely. That helps reproduce a past success, but creativity requires continuing to produce different ideas. The authors propose learning reusable ways of generating creative answers, through primers.

Critical strengths
  1. The research of creativity is an important subject, this paper addresses an interesting paradox. Generating novel samples once is not enough, they should continue be surprising over time.

  2. Separating the trainable priming model from the frozen response model is a useful design choice in my opinion. The experiments in table 3 show that the learned guidance generalizes beyond the response model used during training, allowing the same priming model to be reused without retraining each responder.

Critical weaknesses
  1. An answer can remain different from the fixed reference set even after the system has generated it many times. The diversity metrics capture variation within the evaluated response set, but do not track novelty against an accumulating output history. Although the authors acknowledge this limitation and propose addressing it in future work, the experiments leave open whether Creative Priming can sustain novelty over time. This limits the evidence supporting the claim that the method escapes the “creative paradox.”

  2. Two papers that might be useful to discuss are - [1] and [2] (below). Importantly, the authors should discuss and distinguish their approach from the one presented in [2].

[1] Creative Preference Optimization, Ismayilzada et al. (EMNLP 2025 Findings) [2] Guiding Large Language Models via Directional Stimulus Prompting. Li et al. (NeurIPS 2023)

Questions for authors
  1. Did the authors try to use an evolving set of references?

  2. Can the authors provide a more isolated experiment to distinguish between the contribution of the creative priming and the gain from the model separation?

  3. It seems to me that for the comparisons of the creative priming with other methods, the authors standardized advantages for all methods, whereas the original DARLING removes standard-deviation normalization. Can the authors please clarify this or provide a fair comparison using the DARLING original setup ?

  4. Can the authors clarify what shows that learning primers avoids reinforcing recurring answer patterns, given that rewarding a primer can also make the ideas it induces more likely? Have you checked whether the diversity reward reflects variation among high-quality, novel answers, or is partly driven by low-quality outliers?

2 comments

Thank you for the thoughtful questions and for the new references! We believe that your input can help to improve the paper, and we appreciate your efforts in this regard.

Strengths: We're glad that you found the creative paradox to be interesting, and fully agree that sustained creativity over time is an important direction to investigate. We hope that this work can inspire many more researchers to creatively explore the idea of creativity in AI.

Weaknesses:

  1. Yes, that is very true, and we wholly agree with you, which is why we included that in the discussion section (lines 477-485 + 517-519)! We believe this is a very good measure to track, but for the purposes of an academic study it makes for very difficult comparisons. As we mentioned in the discussion section (lines 517-519), using an evolving reference set makes comparisons across models very complicated since each model will create a unique reference set, and therefore the metrics are no longer an equal comparison.
  2. Thank you for pointing these out. The first paper, creative preference optimization, is currently mentioned in the background section (line 135), which has some obvious differences from creative priming, the main differences being that they directly optimize the response model and they train the model with a fixed dataset using a variation of DPO as opposed to online learning as in GRPO. On the other hand, there are a lot of similarities in the motivations of their rewards and ours (eg. the use of diversity, quality, and novelty). For the second paper, there are some very obvious similarities and differences with creative priming. First, both propose to train a separate model which generates inputs for another model. However, the settings and assumptions underlying directional stimulus prompting are quite different from creativity. In their setting, they assume that the desired output from the response model is known ahead of time (eg. the model needs to include some specific keyword in its response), and the guidance from the directional stimulus prompting is intended to help the response model fulfill these expectations. On the other hand, our setting is completely open-ended, we explicitly do not know ahead of time what we want the model to write, which is the whole point of creativity! Thank you for pointing both of these out, we will add a reference to directional stimulus prompting in the revised version.

Questions for authors:

  1. See the response to weakness # 1
  2. Can you clarify what that would mean? It's unclear how to separate these two things as they are intertwined. There are 2 things that we can think of which might be what you're looking for: (1) the untrained creative priming model (second row in Table 1) which shows the results of using the same model for creative priming and for response generation, and, (2) the bottom right cell in Figure 6 of the appendix where the trained creative priming model is used for both generating the primer and responding. Please let us know if you meant something different!
  3. Yes, we used exactly the original setup and hyperparameters from the DARLING setup. To be explicit, we did not normalize by standard deviation for DARLING.
  4. This is a very good question, and a good point. We believe that the compound metrics of creativity used for evaluations in this study are proof that the priming method avoids reinforcing recurring answers better than the direct training baselines. Table 1 shows that, across all compound metrics of creativity (which reflect only variation among high-quality, novel responses), creative priming leads to more responses that are simultaneously high-quality, diverse, and divergent from the reference set (from an untrained model). Separately, and possibly of interest, there is a slight variation of the statement you're making: since the creative priming model is being updated by GRPO, it is exposed to the creative paradox on it's own level, and is prone to repeating primers that scored highly in the past. We are aware of this, and are not trying to propose that creative priming is a perfect method. Rather, we believe that identifying the creative paradox allowed us to think about how to avoid it, and the creative priming is one such method. There are likely other methods which will perform even better than creative priming.
Reviewer_iKF7OpenReview ↗

Response to authors

Thank you for the detailed rebuttal (I appreciate the easy-to-follow, transparent and short responses.)

The revisions to the paper are informative, they have clarified my concerns.

Regarding question 2 - I meant (2) and missed it in the appendix. This answers my question.

I maintain my positive scoring.

Reviewer_qFRNOpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
3(+1)
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This paper introduces the Creative Priming prompting method to improve LLM creativity by separating a prompt-generator from an output-generator. The aim is to disentangle product and process components in LLM generations for creative output.

The method is interesting and easy to implement making it a very useful tool.

Critical strengths

This is an excellent paper on a relevant and popular topic.

Strengths:

  1. The intro and literature background are well written and capture the importance and societal relevance of the topic.
  2. The Creative Priming approach is promising.
  3. The design/methods are solid and well argued.
Critical weaknesses

Weaknesses:

  1. Is creativity notoriously challenging to teach to humans? Based on the literature, I disagree with some of your claims in this paragraph. The literature does point to it being difficult to improve with training with small to moderate effects, but that it can’t be taught is not really the claim as the underlying assumption is that humans are innately creative – but it would be great to be able to improve this. Here’s a meta-analysis on training for creativity to help you improve this part of your lit background and substantiate claims: Sio, U. N., & Lortie-Forgues, H. (2024). The impact of creativity training on creative performance: A meta-analytic review and critical evaluation of 5 decades of creativity training studies. Psychological Bulletin, 150(5), 554–585. https://doi.org/10.1037/bul0000432

  2. The effectiveness of your priming method stands and falls with the quality of your scoring method. Most of your scoring methods reduce to the effectiveness of diversity scoring. I wonder if your novelty/diversity scoring method meets rigorous psychometric standards. Scoring diversity with GPRO assigns the most value to unique responses in the dataset. This method of uniqueness scoring (in essence inverse frequency relative to a dataset) was abandoned in traditional creativity scoring in the field of psychology. Do the reasons for abandoning this scoring method still hold when scoring novelty in AI using the diversity metric? This paper lists the concerns of uniqueness (c.q. inverse frequency) from a psychological perspective: Silvia, P. J., Winterstein, B. P., Willse, J. T., Barona, C. M., Cram, J. T., Hess, K. I., Martinez, J. L., & Richard, C. A. (2008). Assessing creativity with divergent thinking tasks: Exploring the reliability and validity of new subjective scoring methods. Psychology of Aesthetics, Creativity, and the Arts, 2(2), 68–85. https://doi.org/10.1037/1931-3896.2.2.68

  3. Missing human comparison (including prompt optimization for humans?) What I fear based on the LLM-to-LLM cycle is that while increasing diversity for LLM outputs that this comes no where near the kind of creativity people -especially with optimized prompts- would display on such tasks. However, without a human comparison this is just a hypothesis. I realize including such a study is a big ask. But I do think that given the framing of the paper about enhancing human creativity this deserves to be a point of discussion and direction for future work.

Very minor:

  • on page 4 line 177 it is unclear what you mean by the “clipped GPRO objective”
Questions for authors

Questions for authors:

  1. In intro: Can you update your paragraph/framing of creativity being notoriously challenging to teach to humans? Or argue why this isn't necessary?

  2. In methods: Can you argue why the diversity metric has stronger psychometric properties that the historically used uniqueness metrics from psychology? I think that there are good reasons to continue with this metric, but am curious to hear your thoughts.

  3. In results/discussion: What would a fair human comparison look like?

8 comments

Thank you for the thoughtful comments on our work, we understand the effort it takes to read and understand a paper. We greatly appreciate your effort in helping us to improve the paper!

Strengths: We're glad that you thought this was an excellent paper overall, and we highly agree that it's a very important, and timely topic. In particular, we're glad that you found the method to be interesting, simple, and promising. We hope that many people will be inspired by the creative paradox and use creative priming as a jumping off point for much better methods in the future.

Weaknesses:

Re: "Is creativity notoriously challenging to teach to humans?"

  • You brought up an interesting distinction. Just to make sure that we understand clearly, you agree that it is difficult to teach creativity to humans, but your concern is that humans are already assumed to be creative in the first place? Is that correct? Either way, we're happy to update the introduction with a stronger statement, just want to make sure we understand your point first.

Re: "The effectiveness of your priming method stands and falls with the quality of your scoring method."

  • Unfortunately, we were only able to view the abstract and part of the introduction of the paper you sent as its behind a paywall, so we weren't able to see the exact arguments that they point out against using inverse frequency as a creativity score. However, we feel that it's worthwhile pointing out some differences in the settings under which inverse frequency is used in human studies compared with our study. The major difference is that in human studies, the responses from an individual are being compared with responses from other study participants or the general population (cross-sectional study). On the other hand, in our study, you might think of our metrics of creativity being a longitudinal study where the responses of an individual are only compared with their own past responses (in this case its an AI model). The paper that you linked claims that "over half of the variance in divergent thinking could be explained by dimensions of personality", and since our metrics of creativity are comparing individual models to themselves, it would be hard to argue that variances in creativity are due to personality changes as they are from the same model. We are assuming that the meta-analysis from the paper you linked is mostly focused on cross-sectional studies, so it's possible that our assumptions are wrong and this reasoning is not accurate.
  • Additionally, in the limitations paragraph we do admit that the metrics that we use are indeed flawed and in the future, as metrics improve, so too will the models trained using those metrics.

Re: "Missing human comparison"

  • This would be a very interesting study! Indeed, since our definitions of creativity are designed from the motivation that responses should change over time (the same way that humans change over time), we could collect a baseline set of responses from study participants, and then have them respond again periodically (eg., 1 month later, 4 months later, 1 year later, etc.). Obviously, that is not feasible for the current study, but would make for a very interesting comparison!

Re: "The clipped GRPO objective":

  • The standard GRPO objective is clipped between 1−ϵ1-\epsilon and 1+ϵ1+\epsilon. This is not anything that we introduced, but we understand how our phrasing could introduce confusion. We have simplified it by saying "the resulting GRPO objective"
Reviewer_qFRNOpenReview ↗

Thanks! 2 weaknesses remain

Re Re: "Is creativity notoriously challenging to teach to humans?"

You brought up an interesting distinction. Just to make sure that we understand clearly, you agree that it is difficult to teach creativity to humans, but your concern is that humans are already assumed to be creative in the first place? Is that correct?

No the distinction I want to emphasize is training vs teaching. Training interventions (in say 2 to 10 sessions) don't really lead to large or stable changes in people's creativity, so this is indeed difficult to induce (see ref above). Teaching creativity (across an educational curriculum) is something entirely different. I think that since your work refers to an "short-term" intervention this would be more analogous to training than teaching, and you could leave open whether or not education can help creativity emerge and improve in people (imho: a contentious claim).

Re Re: "The effectiveness of your priming method stands and falls with the quality of your scoring method."

Unfortunately, we were only able to view the abstract and part of the introduction of the paper you sent as its behind a paywall, so we weren't able to see the exact arguments that they point out against using inverse frequency as a creativity score.

sorry about the paywall, but should be available through uni library. I will outline the main points if needed below. These come up in other papers and should be easy to identify with a literature search of openly available work.

However, we feel that it's worthwhile pointing out some differences in the settings under which inverse frequency is used in human studies compared with our study...

I see that I confused within-model inverse frequency (i.e., diversity or inverse homogenization of model responses) with across-model -full dataset- inverse frequency (e.g., uniqueness/novelty/originality). So, my psychometric concerns with using full data inverse frequency are not relevant to the current paper. However, I am concerned about the claims of improved novelty using your method as you only measure within system (modelXconfiguration) diversity and not between system (across all models x configs) uniqueness/novelty/originality? I get that modelXconfiguration diversity is a pressing concern in AI creativity. However, it seems also important to know to what degree the different methods push the modelXconfiguration to generate responses that are dissimilar to the other configurations (i.e. the whole dataset).

The paper that you linked claims that "over half of the variance in divergent thinking could be explained by dimensions of personality", and since our metrics of creativity are comparing individual models to themselves, it would be hard to argue that variances in creativity are due to personality changes as they are from the same model.

yes, personality is completely irrelevant in this context. I was referring to a part of the paper that reviews the psychometric properties of uniqueness scoring.

Thank you so much for the helpful feedback, and for engaging with us in this process!

Re: concerns about our metrics

We were able to get a copy of the paper you linked: Silvia, P. J., Winterstein, B. P., Willse, J. T., Barona, C. M., Cram, J. T., Hess, K. I., Martinez, J. L., & Richard, C. A. (2008). Assessing creativity with divergent thinking tasks: Exploring the reliability and validity of new subjective scoring methods.

First of all, it's important that we acknowledge that our metrics are not perfect and can likely be further improved. Additionally, in the limitations section, we explicitly call out one way in which our metrics are limited, which is that the metrics themselves are built on top of models which do not perfectly capture quality or semantic distance. However, as methods for measuring quality and distance improve, so too will our training and evaluation methods.

Next, if we're understanding correctly, you're referring to a section titled: "Problems with Uniqueness Scoring"? If that is the case, then we address the 3 main concerns below.

  1. Uniqueness scoring confounds fluency and creativity

Based on this quote from the paper you cited, it seems that they are referring to tests of creativity that ask the subject to give a potentially unlimited number of responses: "Critics of divergent thinking research point out that uniqueness scores (the number of unique responses) are confounded with fluency scores (the total number of responses)". Indeed, there would be a confounding factor there between fluency and creativity in such studies. However, in our evaluation setup, each model gives the same number of responses, preventing any issues stemming from a mismatch in the number of responses each method.

  1. Statistical rarity is ambiguous

To summarize the point the author makes in their work: it's sometimes difficult to tell the line between very creative responses and random or inappropriate responses, and also sometimes very ordinary responses can mistakenly be considered creative just because they don't exist in some pre-existing reference set. Yes, the subjectivity of creativity is an issue that our work deals with as well. In particular, the first issue, where its difficult to tell the difference between a creative response and an inappropriate response is something that we want our LLM-as-a-judge to determine, but it's not guaranteed to work (the same as humans also struggle with this, since it's oftentimes subjective). However, there is some good news regarding the second concern. Since we are focused on measuring p-creativity (novelty is only considered with respect to the model itself) then ordinariness should only be considered with respect to the models past responses. This is exactly what our metrics are measuring, and so we do not believe this particular concern is an issue with our metrics.

  1. Uniqueness scoring penalizes large samples.

Yes, this was an issue that we ran into when developing our metrics! In particular, we ran into this issue when working on the compound metrics of creativity. After we filter out responses that do not pass the quality and deviation thresholds, we wanted to measure the spread of the remaining set. However, the size of the set may be different for different methods. To give an extreme example, a method that only had 2 items above the thresholds will have a diversity calculated only as the distance between those 2 points, meaning that it's very easy to have a high diversity for that method. On the other hand, a method where all 64 responses are high-quality and high deviation will struggle to achieve such a high diversity because it's much more likely that some responses are similar. However, this would not capture the fact that the method with 64 responses also likely genuinely does have a larger spread! Therefore, rather than using the cosine distance to calculate diversity (a biased version of spread), we turned to Vendi score and semantic clustering, which give measures of spread that are not biased by the size of each set.

Re: concerns about cross-method diversity

We agree that cross-model diversity is an important problem, but we think that maybe it's outside the scope of this current work? However, we can envision that a method motivated by this work could utilize our notion of deviation and rather than train a model to deviate from a reference set composed of its own responses, it could use the responses from another model and therefore push away, to create more diversity across models.

On a related note, we did do both a quantitative and qualitative analysis of the style for each method's responses (see lines 464-469, appendix H, and figures 8-13), finding that there is indeed a very different style for each method: DGRPO is very terse, DARLING increases in length using many more headers, bold, and lists, while CP stays stylistically most similar to the reference response model.

Reviewer_XAZtOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This paper introduces the notion of Creative Paradox, arguing that learning from prior successful creative outputs encourages a model to reproduce similar outputs, contradicting the core goal of novelty. The authors propose Creative Priming (CP) as a solution, where rather than training the response model directly, they train a separate priming model that generates prompt-specific guidance for the frozen response model. To train the priming model, they combine several reward components on the frozen model's induced responses with primer-level regularizers. The authors show the superiority of their approach against direct response model optimization-based methods on their proposed compound creativity metrics. They also show that their trained primer model transfers to four unseen API-based models without any response-model-specific training.

Critical strengths

The core idea of separating the creative guidance from response generation is simple and intuitive. While learning textual guidance has been explored in prior work, optimizing a prompt-specific policy for creativity is a useful direction. The cross-model transfer experiments are particularly interesting, showing that learned guidance can improve response diversity of unseen API models without sacrificing quality.

Critical weaknesses

The experiments don’t fully establish the connection between the proposed method and the creative paradox. Table 1 mainly shows that CP achieves a better quality-diversity trade-off than directly trained models. But it doesn’t show that these models struggle because they reinforce previously successful responses (since DGRPO and DARLING explicitly reward response diversity). Could an alternative explanation be that freezing the response model helps preserve quality while the primer encourages more diverse responses? Also, since the priming model itself is trained with GRPO, by the same paradox argument, could it not learn to reproduce previously generated successful primers? Although the diversity reward components in the primer model training are meant to prevent this, no ablation experiments were shown on this.

The priming model uses multiple reward components and a larger response-rollout budget during training (each training prompt in CP gets 8 primers x 8 responses compared to 8 responses in the baselines). This makes it difficult to isolate the contribution of decoupling itself. A comparison with baselines using more closely matched rewards and computational budgets (for example, using a similar no. of responses for the baselines or a CP variant with response reward terms only) would be helpful to clarify the decoupling contribution.

Questions for authors

NA

3 comments

First of all, thank you for taking the time to read and think critically about our work. We greatly appreciate your feedback and help in improving the paper!

New Experiment We will start off by highlighting a new experiment included in a revised version of the paper (see the revised version of the paper, with all new content highlighted in red, including additions to sections 1, 4, 5, 6 and appendix J). We believe the results of this experiments may be useful in reconsidering some of the weaknesses you claimed.

In this experiment, we scale the group size for DGRPO from 8 (original results) to 16, 32, and 64 (matching the total number of responses per prompt for CP). We believe that this is a very direct evaluation of the creative paradox: if directly reinforcing successful responses limits creativity, then we should find some ceiling where additional on-policy samples does not lead to increased creativity.

Scaling results:

MethodOn-policy samplesDiversityQualityDeviationFilt-VendiFilt-DistinctVUS-VendiVUS-Dist
DGRPO (G=8)834.077.325.62.857.59168.8658.0
DGRPO (G=16)1633.874.726.92.727.92167.2672.8
DGRPO (G=32)3231.681.528.03.9115.3174.2834.5
DGRPO (G=64)6428.669.133.52.959.70150.2586.9
CP8 primers33.485.725.04.7022.8186.51003.2

These results show the limits of scaling the number of on-policy samples for DGRPO. Looking at the simple metrics, increasing on-policy samples for DGRPO leads to improvements in deviation, but losses in both diversity and quality. The compound metrics show the consequences of this tradeoff, even at empirically determined best group size (G=32), DGRPO still has worse performance than CP on every compound metric. Ultimately, at G=64, every compound metric of creativity decreases from the peak score. Even though scaling up to G=64 gives DGRPO a compute advantage versus lower values of G, that additional compute neither closes the gap nor produces continual improvements in creativity, providing intriguing further evidence for the creative paradox: directly reinforcing successful responses can improve creativity only up to a point, after which reinforcing more responses begins to undermine system-level creativity.

Strengths: We're glad that you found our insight of separating creative guidance from response generation as an intuitive workaround for the creative paradox. And glad that you found our existing results to be interesting. We hope that many others will be inspired by the creative paradox and can use this work as inspiration to develop methods even better than creative priming.

Weaknesses:

Re: "The priming model uses multiple reward components and a larger response-rollout budget during training (each training prompt in CP gets 8 primers x 8 responses compared to 8 responses in the baselines)." and "The experiments don’t fully establish the connection between the proposed method and the creative paradox."

  • Thank you for the great suggestion! We took this to heart and ran a series of experiments to evaluate the impact of response-rollout budget during training. In fact, as we showed above and in the revised paper, these new results provide further support for the creative paradox, rather than the budget explanation. Scaling up the group size does not lead to continuously increasing creativity.

Re: "Could an alternative explanation be that freezing the response model helps preserve quality while the primer encourages more diverse responses?"

  • We fully support this explanation. In fact that is exactly what this paper suggests: decoupling the creative guidance from response generation is a crucial component of how we propose to get around the creative paradox. This explanation was already included in the discussion section (lines 460-462 of the revised paper).

Re: "Also, since the priming model itself is trained with GRPO, by the same paradox argument, could it not learn to reproduce previously generated successful primers? Although the diversity reward components in the primer model training are meant to prevent this, no ablation experiments were shown on this."

  • That's a very good point, and we believe this is true, but doesn't actually impact the metrics that we care about: downstream creativity of the response model. This suggestion does lead to an interesting thought for the future: can the creative priming model write a primer for a creative priming model (to allow for even more creativity in the primers themselves).
Reviewer_XAZtOpenReview ↗

Thank you

This is great. I updated by rating to 3

Thank you for engaging with us in the discussion period, and for reconsidering your original score. Best of luck on your own submissions if you have any!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.