WorldLineBench: Toward Process-Oriented Evaluation of Video-Grounded Streaming Interaction

Streaming video generation endows video, for the first time, with a true worldline: a continuously unfolding visual trajectory that remains open to real-time user intervention. This transforms video generation into a fundamentally new interactive task that requires timely responses while preserving an already-established visual history, posing new challenges to existing evaluation paradigms centered on endpoint quality. To this end, we present WorldLineBench, to our knowledge the first large-scale video-grounded benchmark for streaming interaction, comprising 1,000 synthetic and 1,000 real videos spanning five content contexts and seven interaction types. Synthetic videos provide controlled instruction coverage while real videos probe whether interactive capability generalizes to natural scenarios. We adopt a 45-second long-horizon design supporting multiple rounds of history-dependent interaction, constructing branching worldlines from different interactions at designated nexus points. Under a process-oriented evaluation covering four metrics across two capability dimensions, WorldLineBench uniformly evaluates seven representative models from three paradigms, revealing consistent gaps across responsiveness, semantic fulfillment, and history-dependent preservation.