acceptodds
Under review as a conference paper at ICLR 2027

WoCA: World Simulation with Compositional Assets

Abstract

World simulation aims to model how environments evolve and predict alternative futures with different possible outcomes. Representing these possibilities calls for a model that retains object appearance, captures how geometric states evolve, and renders the resulting observations. In this paper, we propose World Simulation with Compositional Assets (WoCA), a new object-centric world simulation approach with three core components: a set of reusable appearance assets, a trajectory diffusion model, and an asset-conditioned video renderer. We extract reusable object appearance tokens from reference observations and bind them to time-varying poses. A separate trajectory diffusion model predicts future object states and interactions from history states and visual context. To render these assets under new states, our video renderer combines global token conditioning with localized spatial features, connecting each object's stored appearance to its location in the generated video. This disentangled factorization makes appearance and motion separately addressable: object appearances can be reassigned while pose trajectories are held fixed, and trajectories can be edited while the same appearance assets are maintained. It also provides an explicit object memory that remains available across recursive state predictions for long-horizon world simulation. WoCA therefore brings video generation closer to an editable world representation where persistent entities can participate in multiple possible futures. Experiments on MOVi-D and Waymo Open show improved video quality over Neural Assets under appearance and trajectory control and demonstrate long-horizon simulation with persistent appearance assets.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.