acceptodds
Under review as a conference paper at ICLR 2027

Mind Wandering as a Decoding Rule for Creative Writing with Language Models

Abstract

Language models write fluent but predictable fiction, and the usual remedy, a higher sampling temperature, trades coherence for creativity. Human writers describe another source of surprise: mind wandering, the drift of attention away from the task at hand. We propose mind-wandering decoding, an inference-time rule under which a word the model has just written can open a bounded episode. For a few dozen tokens, the model predicts from a second context that splices a remembered passage between the story's opening and the text so far, then returns to the story, which is never edited. Episodes can be cued by the word, start spontaneously, or be scheduled by a controller trained with policy gradients under a wandering budget. In one science-fiction story world with a 957-entry dictionary of passages set in it, we generate 2,064 stories with Qwen2.5-1.5B and Qwen2.5-7B and score them with an LLM judge. For the 1.5B model, wandering raises both creativity and coherence over vanilla sampling (by and points on a five-point rubric), in every opening and beyond the creativity–coherence frontier of temperature and min- decoding. The gain tracks the breaking of repetition loops and depends neither on which passage an episode brings nor on what triggered it; the controller matches the best fixed schedules but learns mostly how much to wander, not when. At 7B, a model that rarely loops and whose coherence sits at the judge's ceiling, the gain disappears. As in incubation accounts of human creativity, what helps is less the remembered passage than the interruption itself.

Primary area:
foundation or frontier models, including LLMs
Keywords:
mind wandering, decoding, creative writing, story generation, large language models, inference-time methods, constrained reinforcement learning, LLM-as-a-judge

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_FsBYOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes an inference-time control method for creative writing through temporary context injection, inspired by the human writers' mind-wandering process. Specifically, a dictionary mapping certain words to passages is built before generation to provide content for temporary injection. During the generation process, an eligible dictionary word can trigger injection of its associated passage. A rule-based or trained router determines how much passage text to inject and how many new tokens to generate under the changed context. After this "wandering" process ends, the previous context will be restored, keeping the new story tokens and discarding the injected passage. Future tokens then continue to be generated based on the original context and the generated prefix so far. In experiments, the authors report that the proposed method improves Qwen2.5-1.5B's creativity and coherence over the temperature sampling and prompting baselines in one tailored story world, but shows little to no benefit at the 7B scale.

Critical strengths
  1. The paper draws on creative incubation to suggest an interesting way to alter decoding without retraining the generator, which might be insightful for the relevant community.

  2. The authors honestly report negative results and disagreements between evaluation methods.

Critical weaknesses
  1. Limited Generalization and Impacts: A Highly-Tailored Setting and Small Models: The eight opening prompts and dictionary passages share one story world, and the study covers one model family with little demonstrated benefit at 7B. It is not clear how the demonstrated performance improvement would hold in more open-ended creative writing across different models. Also, there is no sufficient discussion on transferability and building cost of the word-passage dictionary, leaving concerns about how such a dictionary can be built for general domains.

  2. Missing Human Calibration on Metrics: In the domain of LLM creative writing evaluation, there are many existing approaches from previous works [1, 2]. The proposed LLM-judge rubric needs validation against human judgments, leaving concerns about the reliability of the evaluation. The learned controller adds creativity and coherence in its reward (Eq. 3), and that weighting needs clearer justification when the objectives conflict (e.g., Figure 2 already shows metrics tradeoff analysis for the tested decoding methods).

  3. Significant Computational Overheads: As the proposed method change the context, each wandering episode requires a fresh prefill (Algorithm 1, Section 3), which is not predictable and impact the major parallelism mechanism for normal LLM decoding. The resulting latency and computation overhead would increase significantly and needs to be measured. Without a matched-cost comparison, it is unclear whether the gains over simpler decoding justify the extra work.

  4. The presentation overall is hard to follow: Several terms like "episodes", "second context", "step" are used without prior definition. The notation and the connection between the equations and the decoding procedure are also hard to follow. For example, the connection to [3] does not make the basic context injection operation easier to understand. A clearer walkthrough should connect the repeated episodes in Eq. 1, the number of passage sentences in Eq. 2, and the start and cooling rules. The "When to wander" part is particularly confusing. For the untriggered variant, a worked example would help readers follow how the reported waiting-time estimate sets the start probability and resulting amount of wandering.

References:

[1] Gómez-Rodríguez, Carlos, and Paul Williams. "A confederacy of models: A comprehensive evaluation of LLMs on creative writing." Findings of the Association for Computational Linguistics: EMNLP 2023.

[2] Wang, Yichen, et al. "Optimizing diversity and quality through base-aligned model collaboration." ICML 2026.

[3] Hinton, Geoffrey E. "Training products of experts by minimizing contrastive divergence." Neural computation 14.8 (2002): 1771-1800.

Questions for authors
  1. How would you generalize the method and dictionary to open-ended creative writing beyond this story world?

  2. How are the proposed metrics aligned with human judgments?

  3. What is the extra inference cost compared to vanilla decoding, and do the quality gains hold at matched compute?

  4. Please make the method clearer step by step, and explain what each experiment establishes.

Reviewer_K5PDOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Mind-Wandering Decoding, a reasoning-stage decoding method inspired by human mind-wandering, intended to alleviate the trade-off between creativity and coherence in creative writing with language models. Unlike conventional approaches that boost generation diversity by tuning the temperature, the authors present Two-Stream Decoding: during normal generation, the model briefly switches to a context augmented with an additional Memory Passage, generates a fixed number of tokens, and then reverts to the original context. The authors further design multiple triggering strategies including Cued, Uncued, Mis-cued and Untriggered Wandering, and train a Controller via reinforcement learning to adaptively decide when and at what depth to trigger wandering.

Critical strengths

This work translates the transient attentional shift observed in human mind-wandering into an explicit Two-Stream Decoding mechanism. Without parameter updates or edits to already-generated text, the method alters subsequent generation trajectories through temporary context injection. The design enjoys good interpretability and implementation simplicity, and offers a new angle for investigating how context switching shapes the generative behaviour of language models. Moreover, ablation experiments across Cued, Uncued, Mis-cued and Untriggered Wandering reveal that performance gains do not strongly depend on the semantic relatedness of the Memory Passage or the triggering paradigm. These findings provide valuable evidence for unpacking the underlying mechanism of the method.

Critical weaknesses

My primary concern is that the empirical results in the paper appear to support an explanation rooted in context interruption / repetition breaking, rather than a distinct mind-wandering mechanism.

As shown in Section 5.3, the semantic relevance of the Memory Passage, the mapping between trigger tokens and memory content, and the presence of dedicated triggers all show no clear significance. Meanwhile, Section 5.2 demonstrates that improvements in creativity correlate strongly with reduced repetition rates.

Nevertheless, the paper mainly compares against temperature scaling, min-p sampling and prompting techniques. It lacks systematic comparisons against established repetition-suppression baselines such as repetition penalty and no-repeat n-gram blocking, as well as simpler temporary context perturbation baselines.

It therefore remains unclear whether Mind-Wandering Decoding delivers benefits orthogonal to standard repetition mitigation, or whether its effects stem primarily from a computationally more expensive form of repetition interruption.

Questions for authors

Could the authors add comparisons against repetition penalty, no-repeat n-gram blocking, or other dedicated repetition-suppression strategies? In particular, I would appreciate results comparing creativity, coherence and inference cost under matched repetition rates.

Additionally, given the similar performance across Cued, Uncued and Untriggered settings, could the authors evaluate simpler context perturbation schemes — for example, brief interventions with randomly sampled irrelevant text that do not rely on narrative memory? Such experiments would help disentangle the independent contribution of the proposed method.

Reviewer_fx6NOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The main contribution in this manuscript is an interesting new decoding rule drawn from the drafting process in human writers that temporarily slips an in-world passage into a second context while a LM writes a story. They design it s.t. a certain cue in the story acts as a trigger that brings in a passage from their dictionary written for the same fictional world. Then, the model continues generating from the story and eventually goes back to the original context. The authors test random / mismatched passages and a controller that decides when to start an episode. On Qwen2.5-1.5B, they find higher LLM-judged creativity and coherence and lower 4-gram repetition than vanilla sampling but not at 7B. Overall, I believe the evidence is a good start but I don't think it is clear that this intervention helps.

Critical strengths
  1. The authors compare cued, uncued, and mis-cued passages, as well as a passage placed in the prompt, a learned controller, and different decoding settings. So their intervention is diverse.
Critical weaknesses
  1. Their main result uses one LLM judge, one science-fiction world, and also has no human ratings. From my understanding, they also use one judge for the tournament and don't match the creativity ranking of be original baseline at 1.5B.
  2. The cue or adding context could also have caused the gain (needing an ablation). Also, they analyze the repetition-creativity correlation across conditions so it's not a strong isolation of the difference.
  3. The 1.5B stories are 10% less words even though the same sentence budget and the method needs a passage dictionary and additional inference.
  4. There seem to be a few discrepancies, Section 5.2 says “the next tokens cannot copy from the story’s recent past,” but Eq.1 and Sec.3 put the prefix in both contexts. Is it referring to the inserted context? Also, Section 4 says GPT-5.6 Terra is the dictionary writer and Claude Sonnet 5 is the judge but Section 6 says the judge is the dictionary writer’s model family.
Questions for authors
  1. Does a different judge change the ranking of the method?
  2. Is there a simpler repetition-control or context-interruption method that matches the gains (at similar compute and word length)?
  3. Does the model attend to the full generated prefix during an episode, and what does the copying statement in Section 5.2 mean?
Reviewer_5oxYOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes mind-wandering decoding, an inference-time decoding rule that aims to mimic human mind wandering to introduce surprise and mitigate repetitive, predictable outputs in LLM creative writing. Specifically, when the model generates designated trigger words, it temporarily decodes for a short window from an auxiliary context that splices in a pre-written lore passage from a dictionary before returning to the main story stream. The authors evaluate over 2,000 stories across Qwen2.5-1.5B and 7B using an LLM judge, finding that the method improves creativity and coherence in the 1.5B model primarily by breaking repetition loops, though the gains disappear on the 7B model.

Critical strengths
  • Creative and unconventional problem framing that explores an understudied but important problem
  • Thorough internal exploration of the proposed decoding rule across diverse model and input variants (cued, uncued, and mis-cued wandering, as well as an RL-trained controller)
  • Fairly honest and transparent reporting of negative results at scale (i.e., clearly acknowledging that the benefits disappear on the 7B model where repetition is less of an issue)
Critical weaknesses
  1. Overall, “mind wandering” is heavily undertheorized. The motivation for the design is weak, relying on a single psychological study whose results are taken out of context (Gable et al. examine offline incubation during breaks away from a task, not mid-sentence cognitive interruptions during the physical act of drafting). In addition, the paper completely overlooks both cognitive models of writing (Hayes, 2012) and foundational literary theory and writing pedagogy on digression, defamiliarization (Shklovsky, 1925; Elbow, 1973; Brooks, 1984). Lacking any theoretical grounding in how diversion functions in storytelling, it is unclear how the paper makes sure that it operationalizes "mind wandering" in a way that mirrors human cognition of creativity.
  2. Insufficient baselines with only 1 under-specified prompt, and different sampling params (which really should just be an ablation in the appendix instead of a method to compare against). The paper does not compare against relevant steering and decoding methods (to think of a few, general domain methods like contrastive decoding (Li et al., 2023) and context-aware decoding (Shi et al., 2024), and creativity-specific ones like verbalized sampling (Zhang et al., 2026) and top-H decoding (Potraghloo et al., 2025)). Furthermore, the only prompting baselines evaluated are one vague instruction (“Be maximally original”) and a random word insertion, completely omitting standard persona, style-conditioned, or iterative brainstorming prompts.
  3. Many signs of LLM writing. Just to name a few:
  • “A drifting mind does not rewrite what is on the page; it lets something else into working memory for a while and then returns” (row 46)
  • “the cue does not matter; the intrusion does” (row 395)
  • “what mind wandering buys a language model” (row 440) Notably, the AI use statement does not disclose any use of LLMs in the drafting of the paper.
  1. The models evaluated are outdated and small.

Brooks, 1984. Reading for the Plot: Design and Intention in Narrative. Harvard University Press, 1984. Elbow, 1973. Writing Without Teachers. Oxford University Press, 1973. Li et al., 2023. Contrastive Decoding: Open-ended Text Generation Needs Contrastive Decoding. (ACL 2023) Potraghloo et al., 2025. Top-H Decoding: Adapting the Creativity and Coherence with Bounded Entropy in Text Generation. (arXiv 2025) Shi et al., 2024. Trusting Your Evidence: Hallucinate Less with Context-Aware Decoding. (ACL 2024) Shklovsky, 1925. Theory of Prose. Dalkey Archive Press, 1990. Zhang et al., 2026. Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversity. (ICML 2026)

Questions for authors

Table formatting: The tables (especially Table 1 and Table 2) have no vertical space between the caption and the table itself, which makes the layout cramped and congested. Some vertical spacing could help a lot.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.