A Hint from the Past: Guiding Diffusion Language Models under Context Updates

Reasoning assistants must accommodate changes to a user's requirements while a solution is being generated. In multi-step problem solving, this requires adapting to new constraints or objectives after reasoning has begun. The partial answer provides potentially useful derivations and solution strategies that can guide the response to these changes. We investigate whether selectively using text from the previous partial answer during generation can improve correctness beyond retaining it in context. To this end, we introduce HiPE, a method that guides diffusion language models under updated conditions by injecting selected passages as revisable hints during generation. Implemented in DiffusionGemma, HiPE uses the current generation state to determine intervention timing, attention to extract candidate passages, and predictive distributions to select their destinations. It requires no additional training and constructs interventions from existing decoder forward passes. On AIME 2024 and 2025, HiPE improves accuracy over regeneration by 2.22 and 3.33 percentage points, respectively, with fewer decoder forward passes on average. It achieves higher accuracy than all evaluated initialization-based reuse methods across mathematics and coding tasks.