Mind Wandering as a Decoding Rule for Creative Writing with Language Models

Language models write fluent but predictable fiction, and the usual remedy, a higher sampling temperature, trades coherence for creativity. Human writers describe another source of surprise: mind wandering, the drift of attention away from the task at hand. We propose mind-wandering decoding, an inference-time rule under which a word the model has just written can open a bounded episode. For a few dozen tokens, the model predicts from a second context that splices a remembered passage between the story's opening and the text so far, then returns to the story, which is never edited. Episodes can be cued by the word, start spontaneously, or be scheduled by a controller trained with policy gradients under a wandering budget. In one science-fiction story world with a 957-entry dictionary of passages set in it, we generate 2,064 stories with Qwen2.5-1.5B and Qwen2.5-7B and score them with an LLM judge. For the 1.5B model, wandering raises both creativity and coherence over vanilla sampling (by $0.26$ and $0.23$ points on a five-point rubric), in every opening and beyond the creativity–coherence frontier of temperature and min-$p$ decoding. The gain tracks the breaking of repetition loops and depends neither on which passage an episode brings nor on what triggered it; the controller matches the best fixed schedules but learns mostly how much to wander, not when. At 7B, a model that rarely loops and whose coherence sits at the judge's ceiling, the gain disappears. As in incubation accounts of human creativity, what helps is less the remembered passage than the interruption itself.