Alaya-EVOKE: From Linear-Scaling Supervision to Endless World
Abstract
Interactive world models must simultaneously support persistent memory, responsive user interaction, and long-horizon generation, yet these requirements place conflicting demands on the underlying model. Maintaining history in the denoiser context or key-value cache incurs growing cost over time, forcing a trade-off between session length and retained memory, while low-latency interaction typically relies on few-step generation whose capabilities are ultimately bounded by its teacher. Alaya-EVOKE (Evoke) addresses both limitations by externalizing persistent world state and redesigning the teacher for long-horizon interactive generation. Scene geometry is maintained in an external, camera-indexed world state bank, from which only information relevant to the current view is retrieved, keeping the denoiser context bounded as the session grows. Rather than treating the teacher as a fixed high-quality generator, the teacher is explicitly designed for long-horizon supervision. Its sparse attention scheme combines chunk-wise grouping, retrieval of selected distant frames, and a linear-attention global state, yielding linear growth in activation memory and computation while enabling supervision over long temporal horizons. Such supervision exposes content drift that remains locally plausible within short windows, while per-chunk conditioning enables prompt changes and event control throughout the sequence. A 30-second long-horizon distribution-matching objective, applied under self-forced rollouts, transfers both capabilities to a three-step student that uses no classifier-free guidance (CFG), improving resistance to long-term content drift while preserving responsive conditioning. With bounded context and recurrent external memory, Evoke supports open-ended, continuously evolving generation; on a single H200 at , each s chunk is generated in s. With three sampling steps, Evoke ranks first on the WBench navigation split and achieves competitive results on VBench-2.0, VBench-Long, and VBench-I2V.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.