Solve One Problem with Another: When Linear Attention Meets Masked Diffusion

Long-context language modeling needs a global view of the answer and economical token-level computation. We present LaLanD (Linear Attention Language Diffusion), which unifies discrete diffusion with linear attention: diffusion organizes global answer refinement across rounds, while a bidirectional RWKV-7 denoiser integrates the full context to supply token predictions at every unresolved position. We connect the denoising objective to linear-cost sequence mixing and derive a cost–error criterion that states when an additional denoising round improves the generated distribution under fixed-partition stochastic sampling; Lean-verified proofs establish nested refinement and a training-coverage bound, and an enumerable task confirms the predicted regimes. Following this theory, we build a 3.3B model from RWKV-7 World v3 data with 30B tokens resampled from its own pretraining corpus. It raises the 20-task LongBench mean of its initialization from 11.8 to 19.5 with gains on 18 of 20 tasks, reaches a 68.3% ten-task average, and improves RACE and 4k bAbILong QA1, while retaining a fixed-size recurrent state and $O(K(H+N))$ mixing for context length $H$, answer length $N$, and $K$ rounds, and decoding all answer positions in parallel within each round.