acceptodds
Published as a conference paper at NeurIPS 2026

WorldForge: Forging Unified World Modeling into Video Generation

Boming Tan, Xiangdong Zhang, Ning Liao, Jingtao Zhang, 张雨晴, Xiaoqiu Zhong, Xiaosong Jia, Xue Yang, Shaofeng Zhang, Yanyong Zhang

Abstract

Despite impressive progress in video generation, existing models remain limited to surface-level plausibility, lacking a coherent and unified understanding of the world. Prior approaches typically incorporate only a single form of world-related knowledge or rely on rigid alignment strategies to introduce additional knowledge. However, aligning the single world knowledge is insufficient to constitute a world model that requires jointly modeling multiple heterogeneous dimensions (e.g., physical commonsense, 3D and temporal consistency). To address this limitation, we introduce WorldForge, a unified framework that integrates complementary world knowledge into video generators via a Joint World Modeling Paradigm, jointly predicting video pixels and features from foundation models to capture temporal dynamics, spatial geometry, and semantic consistency. However, naively optimizing these heterogeneous objectives can lead to visual instability and temporal flickering. To mitigate this issue, we propose Consistent Constraint Annealing (CCA) to progressively regulate world-level constraints during training, and Multi-Source Inner-Guidance to enforce learned world priors at inference. Extensive evaluations show that WorldForge improves world consistency, outperforming Wan2.1 by 2.26 points on VBench.

Related papers

See more related submissions when the market opens.