acceptodds
Under review as a conference paper at ICLR 2027

Sekai2: From World Exploration to Interactive World Modeling

Abstract

Large-scale exploration videos provide rich observations of the real world, but interactive world modeling demands more than diverse visual coverage: models must preserve scene state over long trajectories, respond consistently to camera motion, and remain coherent when previously observed regions are revisited. Existing video datasets rarely combine minute-scale continuity, camera trajectories, temporally grounded semantics, and explicit revisit structure. Building on the broad real-world exploration coverage of Sekai, we introduce Sekai2, extending the data paradigm from world exploration to interactive world modeling. Rather than simply expanding the original corpus, Sekai2 re-curates selected Sekai footage and combines it with newly collected perspective and panoramic videos under a unified construction and annotation pipeline. Sekai2 contains 128,892 clips totaling 2,826 hours across 113 countries or regions, including 1,453 hours of continuous two-minute sequences. Each released clip is paired with a camera trajectory and hierarchical clip- and segment-level annotations that disentangle subject motion, environmental dynamics, static scene content, and camera behavior, yielding 649,597 temporally grounded segments. To strengthen long-range spatial supervision, we further include 982 panoramic sequences totaling 119 hours that preserve non-linear routes, loops, and revisits. Fine-tuning experiments show consistent improvements across multiple world-model benchmarks, while extensive data-quality evaluations further validate the reliability of Sekai2 for long-horizon and camera-controllable generation. The complete data and annotations will be released at https://anonymous.4open.science/r/Sekai2.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.