acceptodds
Under review as a conference paper at ICLR 2027

SyncWorld: Many Views, One World

Abstract

An interactive world observed by multiple players must support different views of the same evolving scene. We introduce a view-decoupled architecture that separates shared world evolution from observer-specific video generation. Joint actions advance an authoritative state, geometric projections express that state in each observer's image plane, and one shared video model renders the observations. Adding an observer reuses the learned weights through another renderer call. We instantiate the architecture with engine-authoritative dynamics and a video-diffusion backbone adapted through geometric and status conditioning, gated residual adapters, and LoRA. We also construct a CS:GO dataset pairing multi-observer video with cameras, actions, events, and state. We will release the five-map raw collection and 84,066 processed Dust2 clips, approximately 118 cumulative hours. To evaluate cross-view geometric compatibility, we introduce SymMVC, a symmetric reprojection residual using dense correspondences and calibrated cameras. Our renderer achieves 24.53 dB PSNR versus 15.53 dB for additive residual conditioning under matched training data and budget, and a 480-pixel SymMVC residual of 0.5259 versus 5.1 under shuffled state. A native trajectory drives ten concurrent views, and autoregressive evaluation reaches 12 windows, approximately 60 seconds.

Primary area:
generative models
Keywords:
world models, video generation, multi-view consistency, geometric conditioning, diffusion models, interactive environments

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_SRvgOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes SyncWorld, which is a view-decoupled architecture for multi-observer interactive video generation. It decouples the global state evolution and observer view rendering in the world model. The global state evolution comes from the game server itself, while an improved video-diffusion model renders the observation of each view based on the input condition from the game server. This paper also introduces a large-scale dataset collected in the CS:GO game, covering five maps. Experiments show that SyncWorld can achieve better visual quality than baselines.

Critical strengths
  1. This paper gives a clear system framing that separates "who advances the world" and "who observes it". Especially, it supports varying number of observers in the inference.

  2. This paper contributes a large-scale multi-observer FPS dataset with detailed annotations such as camera calibrations and actions. This open-source dataset can help the training and evaluation of other works in this community.

Critical weaknesses
  1. This method relies on a stand-alone game engine to model the world states. This violates the high-level principle and motivation of world models, i.e. modeling the world evolution and dynamics. The proposed method is more like a neural render instead of world model. Besides, this significantly limits the application range of the proposed method. For example, it can never be applied to the real world.

  2. Given the above weakness, one central claim, cross-view consistency, largely comes from the oracle conditions from the game engine rather than any specific designs of the model. This also makes the comparison with previous works unfair, as previous methods don't have access to the game engine and they have to model the world states by themselves.

  3. The model is quite heavy with more than 14B parameters, so it is almost impossible to render multiview observation in real time. This fact makes the claim of "interactive" world model unrealistic.

  4. The evaluation is conducted on a map from the training set, so the cross-map generalization ability is not demonstrated.

  5. The presentation can be further improved. Currently, it is not easy to understand the method, and Figure 1 can be more clear to show the detailed architecture.

Questions for authors

Please consider replying to the weaknesses.

1 comment
AuthorsOpenReview ↗

Clarifying the world-model scope and contribution

We thank the reviewer for raising the important question of how SyncWorld should be positioned relative to pixel-only world models. We clarify the scope of the model and the contribution below.

SyncWorld is built on LingBot-World, a pretrained video world-model backbone. Fine-tuning this backbone with structured geometric and status conditions preserves its temporally coherent video prediction and autoregressive continuation capabilities. The conditioning design determines how one shared evolving state is realized across multiple observers.

Our current setting separates two roles. The CS engine advances the authoritative state and interaction consequences, while the adapted world-model backbone generates observer-specific videos conditioned on projections of this shared state. These are different questions. The source of the state transition and the ability to generate coherent, controllable visual observations of that state need not be implemented by the same component. We therefore describe SyncWorld as a world-model-based multi-observer generation framework. We do not claim that the video model independently learns the underlying CS game dynamics from pixels.

The learned renderer is valuable because a conventional engine renderer follows predefined assets, rendering rules, and a fixed visual pipeline. A generative video renderer can produce diverse visual observations under the same structured state, including different appearances, camera styles, scene details, and prompt-controlled presentations. This is relevant to game development and interactive content creation, where reliable interactive state transitions and flexible visual generation are both important. A future extension could make the underlying 3D structure generative as well, so that both the 3D state and its video observations are generated.

The shared engine state is intentionally part of the current setting. We do not present cross-view consistency as evidence that the video model has independently learned the engine’s rules. Instead, our new metrics evaluate whether the learned renderer faithfully realizes one shared state across observers. SyncWorld obtains 0.6681 Cross-view Presence and 0.2973 Cross-view Action, while Cross-Group Shuffle obtains 0.4312 and 0.0675. The Time-Shift control obtains 0.4806 and 0.1444. These controls show that the measured consistency depends on synchronized state and correct observer grouping.

We agree that the claim of interactive operation should be supported by explicit runtime measurements. We will report latency, peak GPU memory, and total GPU cost as the observer count increases, distinguishing serial, batched, and parallel execution. If the measurements do not support real-time operation at the current model size, we will narrow the wording accordingly.

The current quantitative model evaluation is concentrated on Dust2, so we do not claim demonstrated unseen-map generalization. We will also revise Figure 1 to make the data flow explicit:

shared state evolution → observer-specific condition projection → shared video renderer

This clarification separates the capabilities supplied by the engine from those learned by the video world-model backbone and makes the intended contribution easier to assess.

Reviewer_cigVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper present SyncWorld, a video world model for multiple observers that keep the evolution of the shared world state apart from the video each observer see. A CS game engine supply the ground-truth world dynamics, and these is projected into geometric and player-status conditions for every observer. A shared LingBot-World backbone, adapted with gated residual adapter and LoRA, then generate each observer's video independently. The authors also builds large multi-observer CS dataset and introduce SymMVC to measure geometric consistency across view. In the experiments, visual quality and how faithfully the actors follow their conditions both improves, and the paper also show generation with ten observers and long-horizon autoregressive continuation.

Critical strengths

1.The paper give a practical framework that keep the shared evolution of the world state separate from the video generated for each observer. Because a common 3D state is projected into independent rendering condition, SyncWorld can handle several observer without any extra model parameters. I see this as useful system-level approach to simulating a world from multiple views. 2.The large CS dataset and the structured evaluation is both valuable contributions. In the matched comparison with Direct Residual, the method improve visual fidelity and how well the actors lines up with their conditions. The SymMVC metric the authors propose also offer useful way to look at how geometrically compatible the generated view are with each other.

Critical weaknesses

The method isn't very novel. Earlier work like MultiWorld and MultiGen have already looked at shared world representations and video generation with multiple agent. SyncWorld instead lean on an existing game engine to supply full world dynamics and player states, and the video model only render observations based on those states, which is fixed in advance. The main parts that gets trained are standard geometric conditioning, gated residual adapter, and LoRA. That's reasonable engineering design, but the paper don't show any substantial new world-modeling capability, and it doesn't separate its algorithmic contribution clearly enough from existing frameworks that generates from a shared state.

Questions for authors
  1. Since world dynamics and player states are supplied by the CS engine, what is the primary learned world modeling contribution beyond conditional video rendering? How does the approach differ fundamentally from existing engine-guided rendering frameworks?

  2. How does SyncWorld improve upon MultiWorld and MultiGen in terms of multi-agent interaction, shared-state consistency, and scalability? Can the authors provide a more controlled comparison that isolates these contributions?

  3. SymMVC measures geometric reprojection residuals but does not explicitly evaluate object identity or dynamic events across views. Can the authors provide quantitative cross-view identity and action-consistency evaluations?

  4. The strict SymMVC evaluation contains only eight view pairs, six of which have viewing-angle differences below 10 degrees. How does the method perform under larger viewpoint changes, wider baselines, and more challenging occlusions?

  5. The ten-observer demonstration covers approximately 10 seconds, while the 60-second autoregressive evaluation is reported separately. Can the authors evaluate long-horizon generation with multiple observers simultaneously and quantify cross-view consistency over time?

  6. All reported model experiments use Dust2 despite the dataset covering five CS maps. Can the authors provide cross-map evaluations, particularly on unseen maps, to demonstrate generalization beyond the training environment?

1 comment
AuthorsOpenReview ↗

Novelty, cross-view consistency, and long-horizon evaluation

We thank the reviewer for recognizing the value of separating shared state evolution from observer-specific video generation, as well as the contribution of the dataset and structured evaluation.

Our contribution is a deliberate separation of responsibilities. The CS engine advances the authoritative shared state and interaction consequences, while the adapted LingBot-World backbone generates observer-specific videos conditioned on projections of this shared state. SyncWorld therefore studies view-decoupled multi-observer generation based on a pretrained video world model, rather than claiming that the video model independently recovers CS game dynamics from pixels. The framework combines a shared evolving state interface, observer-specific geometric and status conditioning, gated residual adapters with LoRA, and cross-view evaluation under synchronized state queries. The same learned weights can be reused as the number of observers changes, while each observer receives its own projected condition and generated observation.

We agree that MultiWorld also supports shared-state and multi-view generation. Our claim is not that prior methods lack these capabilities. The compared works use different environment interfaces, state representations, temporal horizons, and learned components. SyncWorld is evaluated in a multi-player CS:GO environment with persistent 3D geometry, calibrated cameras, multiple synchronized observers, player actions, and engine-level state transitions, which creates a demanding combination of geometric, temporal, and multi-agent constraints. We therefore report native-interface system comparisons and a more controlled Direct Residual comparison. Direct Residual uses the same LingBot backbone, training split, structured conditions, and comparable training budget, and is intended to isolate the effect of our gated conditioning design. For MultiGen, we could not locate publicly released official code or checkpoints for a controlled same-environment reproduction, so we do not claim to have performed such a reproduction.

SymMVC evaluates geometric compatibility through calibrated-camera reprojection. To complement it, we introduce two GT-anchored metrics. Cross-view Actor Presence Consistency measures whether an actor that should be visible in both synchronized views is detected in both generated views. Cross-view Action Consistency evaluates whether the same actor, action head, and time query receive consistent action evidence in both views. All models use the same fixed GT queries, failed queries remain in the denominator, and action scores are aggregated within each head before the five-head macro average.

MethodCross-view PresenceCross-view Action
LingBot Base0.24890.0702
Direct Residual0.49520.1767
Vid2World0.17810.0896
MultiWorld0.39680.1717
Solaris0.32230.1038
SyncWorld0.66810.2973
Time-Shift0.48060.1444
Cross-Group Shuffle0.43120.0675

These metrics measure shared-state-conditioned presence and action realization. They do not independently verify appearance ReID or causal attacker-victim attribution. We also agree that the original strict SymMVC subset is concentrated on relatively small viewpoint differences. The new actor-level metrics provide complementary evidence on the synchronized evaluation groups, while very wide baselines and severe occlusions remain limitations of the current evaluation.

For long-horizon generation, we evaluate synchronized multi-observer trajectories in consecutive autoregressive windows. Each observer uses its own previous generated context without resetting with GT RGB frames. SyncWorld maintains higher cross-view scores across the reported windows. In the 0–20 second evaluation, its Presence / Action scores are 0.6808 / 0.3232 in the first window and 0.6505 / 0.3096 in the final window. We report scores per window because GT support can vary over time.

Finally, we agree that the current quantitative model experiments are concentrated on Dust2, although the data collection covers five maps. We therefore limit the corresponding claim to the evaluated environment and do not present the current results as evidence of unseen-map generalization.

Reviewer_TQTZOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a new framework, namely, SyncWorld, to generate multi-observer videos in an interactive world. The proposed framework first leverages a game engine to provide multiple conditions such as map geometry, player states, and calibrated cameras. By using these conditions as input, a video diffusion model is introduced to generate first-person videos for different observers, where a condition-gated adapter is integrated to align condition embeddings into the feature space of LingBot-World. The authors introduce condition-gated residual adapters to incorporate the structured conditions into a pretrained video model. In addition, the paper builds a new dataset to train the diffusion model. Experiments show that the proposed method outperforms the state-of-the-art approaches in terms of visual fidelity and actor condition fidelity.

Critical strengths
  1. The paper builds a new large-scale dataset consisting of 271,568 first-person videos and accompanying records (approximately 48.66 TB). The dataset is valuable and can benefit the community and future research on multi-player interactive world models.

  2. The proposed method outperforms several recent state-of-the-art approaches, as shown in quantitative experiment results,

Critical weaknesses
  1. I am confused about the motivation of the proposed method. The method relies on an existing game engine to provide geometric and status conditions for the video diffusion model. Since the game engine is already available, why not directly use it to render first-person videos for multiple players? It is unclear why a video diffusion model is needed in this setting. It would be better to provide a clearer explanation of the motivation and advantages of using video diffusion instead of conventional game-engine rendering.

  2. The input conditions of the proposed method are different from those of state-of-the-art approaches, raising concerns about comparison fairness. The proposed method uses the game engine to provide future information such as future world-state trajectory, depths, and masks. However, the approaches in the main experimental comparison, such as Vid2World, Solaris, and MultiWorld, use different input conditions. These differences make it difficult to determine whether the improvements come from the proposed architecture or the stronger conditioning information.

Questions for authors
  1. Does the proposed method lead to a large time cost ? It would be better to provide inference latency, GPU memory usage, or computational cost as the number of observers increases, since the proposed method is proposed for interactive applications.
1 comment
AuthorsOpenReview ↗

Motivation, comparison fairness, and scalability

We thank the reviewer for recognizing the value of our large-scale multi-observer dataset and the improvements in visual fidelity and actor-condition fidelity.

Regarding the motivation for using a video diffusion model, the game engine and the learned renderer serve different purposes. The engine provides an interactive and temporally coherent state space, including scene geometry, player states, actions, and their consequences. The video world-model backbone then maps this structured shared state to observer-specific visual observations.

A conventional engine renderer follows predefined assets, rendering rules, and a fixed visual pipeline. A generative video renderer can produce diverse visual realizations under the same underlying state, including different appearances, camera observations, scene details, and prompt-controlled visual presentations. This is relevant to game development and interactive content creation, where reliable state transitions and flexible visual generation are both valuable. Our current system uses the engine as an authoritative state interface and studies how one shared generative renderer can realize this state for multiple observers. A future extension is to generate the underlying 3D structure as well, so that both the 3D state and its video observations become generative.

We agree that the compared systems use different interfaces, so an absolutely identical input protocol is unavailable. We therefore distinguish two types of comparison. Direct Residual uses the same LingBot backbone, training split, structured geometry/status conditions, and comparable training budget as SyncWorld. This is the controlled comparison for our gated residual conditioning design. SyncWorld improves over Direct Residual from 21.3329 to 22.5274 PSNR, from 0.3437 to 0.2956 LPIPS, and from 923.2 to 838.1 FVD.

Comparisons with Vid2World, MultiWorld, and Solaris are native-interface system-level comparisons. Their input interfaces, temporal horizons, and learned components differ from ours. We preserve each method’s intended interface, match the evaluation clips and protocol as closely as possible, and avoid interpreting these results as strict same-input architectural ablations. We have clarified this distinction in the revision.

The reviewer is also correct that observer scalability should be reported explicitly. We will add end-to-end latency, peak GPU memory, and total GPU cost as the number of observers increases, with clear timing boundaries for state projection, condition construction, diffusion sampling, and VAE decoding. We will distinguish serial, batched, and multi-GPU execution.

We hope this clarification makes clear that SyncWorld is not intended to duplicate the engine’s state-transition mechanism. It combines an authoritative interactive state interface with the controllable and diverse visual generation capability of a video world model.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.