acceptodds
Under review as a conference paper at ICLR 2027

MULTICS: Generative Multi-Player World Model with Hierarchical Communication

Abstract

Multi-player interactive world models aim to simulate shared environments from multiple players’ perspectives, but achieving robust generation while scaling to more players remains challenging. Existing approaches that rely on fully-connected communication (e.g., Solaris) incur quadratic attention costs as the number of players grows, while centralized approaches (e.g., γ-World) suffer from interference between unrelated events in a shared global representation. To address these challenges, we present Multics, a multi-player video generation framework with hierarchical communication that structures information exchange across player, group, and global levels. This hierarchy introduces group tokens into self-attention as intermediate group-level representations that aggregate locally relevant player information. These tokens connect to global tokens for cross-group communication, while the hierarchical topology limits direct mixing between unrelated local events. Group-token parameters are shared across group slots, allowing the same communication module to accommodate varying numbers of players without introducing new group-specific parameters. We further propose budget-constrained dynamic grouping based on recent camera trajectories, which adaptively groups relevant players while limiting unnecessary communication. Experiments demonstrate that Multics achieves state-of-the-art performance in 4- and 6-player settings, with improved cross-player consistency, player-specific action and camera controllability, and interference robustness. Moreover, Multics generalizes from 4-player training to 6-, 8-, and 10-player inference without retraining, while reducing self-attention computation compared with fully-connected communication.

Primary area:
generative models
Keywords:
hierarchical communication, video generation, multi-player

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 2 reviews, above 85% of the others.
Reviewer_mRsNOpenReview ↗
Rating
3
Confidence
3(+1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

MULTICS is a multi-player video world model with hierarchical communication. It organizes communication at the player, group, and global levels. Group tokens collect information from related players and connect to global tokens. Player groups are updated from recent camera trajectories under a fixed communication budget. Shared group-token parameters allow the model to handle different player counts. Experiments on OpenCS2 and EgoTeam compare MULTICS with fully connected, centralized, and group-based communication. The evaluation covers consistency, controllability, visual quality, efficiency, cross-domain transfer, and inference with more players. The experiments are generally complete and the artchitecture has good scaling potential. I therefore give the paper a score of 3: Borderline Accept.

Critical strengths
  • The hierarchical design is reasonable. Local groups support communication between related players, while global tokens connect different groups. The model can therefore exchange both local and global information.

  • Dynamic grouping is useful. Groups are updated using recent camera trajectories. This is more flexible than fixed grouping and keeps the communication cost bounded.

  • The experiments are fairly complete. The paper compares several communication structures and evaluates the main claims from multiple aspects. The baselines also use the same base model and dataset.

Critical weaknesses
  • The current ablations do not fully isolate the main components of MULTICS. They compare complete communication variants and different grouping strategies, but several factors still change together. In particular, the paper does not separately test direct attention between players in the same group or overlapping group membership. It is also unclear whether fixed, random, and dynamic grouping use the same group size, player coverage, overlap pattern, and communication budget. The authors should start from the full model and remove one component at a time, this would show which parts of the hierarchical design actually improve consistency and controllability.

  • The conditioning design needs more explanation. Keyboard actions use cross-attention, mouse controls use a temporal module, and camera trajectories use Q/K adapters. Why are three different mechanisms needed? Mouse motion and camera trajectories both describe viewpoint changes. How does the model avoid redundant or conflicting conditions? Were more unified designs tested?

  • The evaluated scale is still modest. Training uses four or six players, and evaluation stops at ten. This shows some extrapolation, but not the limit of the method. How do quality, group coverage, memory use, communication cost, and latency change with many more players?

Questions for authors

see weaknesses

5 comments
AuthorsOpenReview ↗

Response to Reviewer mRsN [Part 1/3]

We sincerely thank the reviewer for your insightful comments and recognitions to our work, especially for acknowledging that:

  1. Our proposed hierarchical design is reasonable, with local, global communication levels.
  2. Our proposed dynamic grouping is useful, updated using recent camera trajectories
  3. The dynamic grouping offers more flexibility while keeping the communication cost bounded.
  4. The experiments are complete, comparing several communication structures and evaluates the main claims from multiple aspects.
  5. The comparisons are fair, with baselines also using the same base model and dataset.

We have polished the paper, added the experimental results and made clarifications in the revised version (marked in green). Note that the following polishments have been made according to your advice:

  • The ablation studies on w/o direct intra-group attention and w/o overlapping membership, and details about fixed, random, and dynamic grouping are included in the Appendix C.2 "Ablation on Different Attention Mechanisms", and "Ablation on Topology-Matched Random Grouping".
  • The quantitative results of getting camera from actions to show the robustness encountering different conditions, are included in the Appendix C.8 "Analysis of Robustness of Camera Trajectories".
  • The quantitative results of memory use, communication cost, and latency change with many more players are included in the Appendix C.10 "Analysis of Scalability".

Thanks again for your very constructive comments, which have helped us improve the paper quality significantly! Below we would like to provide point-to-point responses to all the raised questions:

W1 More ablations needed to isolate the main components of MULTICS: separately test direct attention between players in the same group or overlapping group membership; whether fixed, random, and dynamic grouping use the same group size, player coverage, overlap pattern, and communication budget.

A1: 1) We ablate direct intra-group attention and overlapping group membership separately. Removing direct intra-group attention substantially reduces GeoKPM and camera accuracy, while removing overlapping membership produces a smaller decline in GeoKPM. These results demonstrate that our method improves both consistency and controllability by enabling effective hierarchical communication that structures information exchange across player, group, and global levels.

MethodGeoKPM ↑PSNR ↑SSIM ↑LPIPS ↓Action ↑RotErr ↓TransErr ↓CamMC ↓FID ↓FVD ↓
w/o direct intra-group attention42.30%17.790.49590.431193.82%2.55813.6715.1326.97156.2
w/o overlapping membership52.84%18.120.50060.418094.10%2.53111.8113.3626.38159.1
Ours54.17%18.100.50050.418494.17%2.43311.7313.2726.39162.3

2) We provide details about fixed, random, and dynamic grouping.

  • Both the fixed and dynamic grouping (ours) in Table 2 have closely matched average group sizes (2.214 for fixed grouping vs. 2.206 in dynamic grouping, the same below), identical local player coverage (100%), similar overlap frequencies (42.83% vs. 41.17% of latent frames), and similar average communication costs (37.97M vs. 37.56M logical attention edges). Both use the same communication budget cap of 64.17M.
  • For random grouping, we further test the setting (denoted as "topology-matched random") under the same group size, player coverage, overlap pattern and communication budget as dynamic grouping, which isolate the effect of assigning players to groups based on relevance. We show the results as below. The results indicate that random grouping leads to decreases in GeoKPM and camera accuracy, demonstrating that dynamic grouping improves consistency and controllability by enabling relevant information exchange among multiple players.
GeoKPM ↑PSNR ↑SSIM ↑LPIPS ↓Action ↑RotErr ↓TransErr ↓CamMC ↓FID ↓FVD ↓
Topology-matched random52.06%18.090.50040.418894.13%2.50012.7814.1526.35159.9
Ours54.17%18.100.50050.418494.17%2.43311.7313.2726.39162.3
AuthorsOpenReview ↗

Response to Reviewer mRsN [Part 2/3]

W2: Why are three different mechanisms needed? Mouse motion and camera trajectories both describe viewpoint changes. How does the model avoid redundant or conflicting conditions? Were more unified designs tested?

A2: 1) We follow previous works to adopt different mechanisms: we follow Gamefactory [A] to use cross attention for discrete keyboard actions, and a temporal module for continuous mouse actions. The ablation in work [A] proves the effectiveness of this action conditioning method. For camera poses, we follow work [B] to adopt a similar mechanism, injecting camera into diffusion latent space by adapters.

2) There exist works [C, D] that incorporate keyboard, mouse actions and camera poses into model training similar to ours.

  • In the aspect of redundancy: in our work, the actions and camera poses serve different roles instead of providing redundant conditions: actions serve as controls for independent players, and camera poses are used for providing global reference coordinate system for multiple players.

  • In the aspect of conflicting conditions: both mouse inputs and camera poses are time-aligned signals from the same gameplay rollout and with little conflicts. Besides, camera trajectories can also be estimated by integrating player actions. Thus, the mouse and camera conditions describe the same viewpoint changes rather than independently or in conflict. To prove this, we follow Lingbot-world[E] to use actions to integrate for camera poses, starting from each player’s first-frame pose. We provide quantitative results of action-derived camera trajectories ("Action-camera") as follows, and the results show comparable performance with our default setting.

    GeoKPM ↑PSNR ↑SSIM ↑LPIPS ↓Action ↑RotErr ↓TransErr ↓CamMC ↓FID ↓FVD ↓
    Action-camera55.41%18.080.49920.418194.11%2.56011.8813.5126.18157.6
    Ours54.17%18.100.50050.418494.17%2.43311.7313.2726.39162.3

3) As for the unified design for keyboard, mouse, and camera poses conditions, a possible way is to integrates multiple conditions via unified full-attention mechanisms like in work [F], which we leave for future exploration.

AuthorsOpenReview ↗

Response to Reviewer mRsN [Part 3/3]

W3: Quality, group coverage, memory use, communication cost, and latency change with many more players

A3: 1) We provide the results of group coverage, memory use, communication cost, and latency change with many more players as follows. As OpenCS2 provides up to 10 players, our quantitative evaluation of quality is limited to that range, which is reported in Table 1 (our method shows advantages on quality from 4 to 10 players). To assess system scalability beyond 10 players, we stress-tested the same checkpoint on larger player sets up to 30 players and measured peak GPU memory, communication cost, and DiT latency for fully-connected, centralized and our methods. Local group coverage is applicable only to our method: fully-connected attention directly connects all players, whereas centralized attention enables all players to exchange information through shared hub tokens. Thus neither baseline constructs local groups, so we report this metric as NA for both.

2) The results show our scalability advantages beyond 10 players: at 30 players, our method uses 68.4x less communication than fully-connected method and reduces DiT latency from 10.59 s to 1.49 s (7.1x speedup). Besides, our method also shows 2.36x less communication and with 1.4x speedup in latency than centralized method. Peak GPU memory grows similarly across methods. Group coverage remains around 91% in the larger-player settings.

Values are ordered as fully-connected / centralized / ours:

PlayersGroup coverage (%)Peak GPU memory (GiB) ↓Communication cost (M) ↓DiT latency (s) ↓
497.39/NA/NA30.54 / 31.49 / 30.78414.69 / 145.46 / 37.110.343 / 0.317 / 0.261
696.90/NA/NA38.49 / 39.39 / 38.761,036.73 / 213.94 / 62.870.622 / 0.453 / 0.348
1090.99/NA/NA54.45 / 55.29 / 54.733,110.18 / 350.92 / 146.591.429 / 0.719 / 0.534
1490.48/NA/NA70.40 / 71.18 / 70.856,289.48 / 487.89 / 188.302.589 / 0.997 / 0.734
2090.99/NA/NA94.14 / 94.82 / 94.5413,131.89 / 693.35 / 293.155.001 / 1.404 / 1.009
3090.99/NA/NA133.94 / 134.47 / 134.5130,065.11 / 1,035.79 / 439.7110.590 / 2.080 / 1.489

References

[A] Yu J, Qin Y, Wang X, et al. GameFactory: Creating New Games with Generative Interactive Videos[C]//2025 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2025: 11590-11599.

[B] Bai J, Xia M, Fu X, et al. Recammaster: Camera-controlled generative rendering from a single video[C]//2025 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2025: 14834-14844.

[C] Wang Z, Liu Z, Li J, et al. Matrix-game 3.0: Real-time and streaming interactive world model with long-horizon memory[J]. arXiv preprint arXiv:2604.08995, 2026.

[D] Xiong Z, Song Y, Kang H, et al. ActWorld: From Explorable to Interactive World Model via Action-Aware Memory[J]. arXiv preprint arXiv:2606.17730, 2026.

[E] Team R, Gao Z, Wang Q, et al. Advancing open-source world models[J]. arXiv preprint arXiv:2601.20540, 2026.

[F] Ju X, Ye W, Liu Q, et al. Fulldit: Multi-task video generative foundation model with full attention[J]. arXiv preprint arXiv:2503.19907, 2025.


Please don’t hesitate to let us know if there are any additional clarifications or experiments that we can offer!

Reviewer_qCbVOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Multics is a diffusion video generator for first-person videos in a syncronyzed multi-player environment. Theuy use a pretrained model (Wan2.2-5B). Literature has leveraged fully-connected attention (Solaris) and to reduce cost, routing all exchange through centralized hub tokens (γ-World). They add an intermediate level of shared group tokens between the players and the global tokens, in a hierarchical way. They propose grouping dinamically from the overlap of their camera viws under a communication budget equal to the centralized cost. They validate on OpenCS2 with 4 and 6 players, on generalization from 4-player training to 6, 8 and 10 players, and on EgoTeam

Critical strengths
  • Scaling attention in multi-entity environments that are equivariant under permutations is a nice and useful topic, relevant to other tasks beyond video generation.
  • The paper is easy to read, and the contribution is easy to understand.
  • The comparison is fair: all baselines are re-implemented on the same backbone, changing the attention type. The method gives consistent gains in cross-player consistency and generalizes from 4 to 6,8,10 players wihtout retraining
Critical weaknesses
  1. Key claims are under-measured.
  • Interference robustness, one of the three headline properties, has no quantitative metric. It’s shown only through qualitative examples
  • In the grouping ablation (Table 2), group-based communication achieves a better FVD than the proposed method, and fixed grouping achieves better PSNR and SSIM. More generally, FVD and FID are computed on only 100 multi-player samples per model, with no seeds or confidence intervals, so differences of a few points may not be meaningful.
  • IDM accuracy is about 94% for all methods. What is the IDM’s accuracy on real videos (its upper bound)? Can you report accuracy per action type?
  • The efficiency advantage holds only against fully-connected attention. Compute is on par with the centralized baseline
  1. Limited evaluation scope and design assumptions.
  • Models are trained only with 4 and 6 players, although OpenCS2 has 10. With 4 players there are only two groups and little risk of cross-interference in the shared group and global tokens, so the setting where hierarchy should matter most is never used for training. The 4→10 test also favours bounded-communication methods over Solaris, which sees far more tokens than in training.
  • Grouping relies on ground-truth camera trajectories. These are conditioning inputs, but in a real interactive setting only the actions would be known; the actual camera positions depend on the world (walls, collisions) and would need to be generated.
  • Cameras are expressed in a frame anchored at the first player’s initial frame. Isn’t it breaking the permutation equivariance claimed in Appendix A, since reordering the players changes the anchor, and with it every player’s camera input.
  • The dynamic grouping is hand-designed, but only its budget is ablated (Fig. 7). The choice of M, the kernels and their hyperparameters are neither ablated nor reported, and “fixed grouping” (Table 2) not seems to be defined.

Minor: Related work misses hierarchical and group-wise relational modelling in multi-agent trajectory prediction, e.g., hypergraph-based methods such as GroupNet (Xu et al., CVPR 2022) and MART (Lee et al., ECCV 2024).

Questions for authors

Q1. Can you quantify interference robustness, for example by measuring content leakage between unrelated players? Q2. Why does the group-based strategy achieve a better FVD than yours? How sensitive is FVD with 100 samples per model? Can you report results over several seeds? Q3. If all methods were trained with 10 players, would Solaris still underperform? With 4 players there are only two groups (M = ⌈log₂4⌉ = 2), so there is minimal risk of cross-interference in the shared tokens (group and global). Q4. To compute groups, you rely on ground-truth camera locations. Could you try with noisy locations? How robust is your proposed attention to them? And what if ground-truth camera locations are not available in a real situation, e.g., when conditioning only on actions? In a real rollout, camera positions would depend on the generated video, which in turn depends on the grouping. Q5. IDM accuracy is about 94% for all methods. What is the IDM’s accuracy on real videos, and can you report it per action type? Q6. Do the outputs change when the player order is permuted, given the camera frame anchored to the first player? Q7. Is attention computed jointly over all players and frames in the window? Have you considered decoupled communication, i.e., temporal attention within each player and social attention across players in separate steps? This would reduce the cost from O((N·T·L)²) to O(N·T²·L² + T·N²·L²), with L spatial tokens per frame. Q8. The player-relevance score uses only camera poses and ignores occlusion by walls. How often are players grouped who cannot actually see the same content?

11 comments
AuthorsOpenReview ↗

Response to Reviewer qCbV [Part 1/6]

We sincerely thank the reviewer for your insightful comments and recognitions to our work, especially for acknowledging that:

  1. Our work addresses a useful problem in multiplayer video generation by designing scalable, permutation-equivariant attention for multi-entity settings.
  2. Our work is also useful for other relevant multi-entity tasks beyond video generation.
  3. Our method consistently improves cross-player consistency.
  4. The proposed method can generalize from 4 to 6, 8,10 players without retraining.
  5. The paper and contributions are easy to understand.

We have polished the paper, added the experimental results and made clarifications in the revised version (marked in green). Note that the following polishments have been made according to your advice:

  • The quantitative results for interference robustness are included in the Appendix C.5 "Quantitative Results on Robustness Interference"
  • The results of IDM’s accuracy on real videos and per action type are included in the Appendix C.6 "Action Accuracy per Action Type".
  • The comparison results and analysis with Solaris trained with 10 players are included in the Appendix C.7 "Additional Comparison of Models Trained and Infer on 10 Players".
  • The results and analysis of noisy camera or action-derived camera trajectories are included in the Appendix C.8 "Analysis of Robustness of Camera Trajectories".
  • The player reordering results and analysis are included in the Appendix C.9 "Sensitivity of Player Order".
  • The ablation studies on M, the kernels and their hyperparameters are included in the Appendix C.2 "Ablation on Number of Available Group Slots M , Kernels and Their Hyperparameters".
  • Relevant works of hierarchical and group-wise relational modelling are cited in the Section 2 "Communication Topologies".
  • The explanation of "fixed grouping" is added in the Section 4.4 "Impact of Dynamic Grouping".

Thanks again for your very constructive comments, which have helped us improve the paper quality significantly! Below we would like to provide point-to-point responses to all the raised questions:

W1.1 & Q1: Quantitative metrics for interference robustness, such as measuring content leakage between unrelated players

A1: 1) We provide quantitative results for interference robustness as below, compared with baselines (Solaris, γ-World) in both 100 and 200 samples. We use Leakage Rate (LR) as the metric to measure content leakage between unrelated players. The LR is the fraction of irrelevant event–view pairs for which an evaluator detects the specified effect, based on the false-positive rate [A, B]. We use GPT-6 Astra as a visual evaluator to determine whether the generated video shows the specified effect within the designated region.

LR ↓Solarisγ-WorldOurs
100 samples3.07%8.28%2.53%
200 samples5.67%7.64%4.50%

2) The results show that our method consistently outperforms baselines in interference robustness. We attribute the lower leakage rate to hierarchical aggregation, which limits direct mixing between unrelated local events.

AuthorsOpenReview ↗

Response to Reviewer qCbV [Part 2/6]

W1.2 & Q2: Why does the group-based strategy achieve a better FVD than yours? The sensitivity of FID/FVD with 100 samples over several seeds per model? Fixed grouping achieves better PSNR and SSIM?

A2: 1) FVD/FID:

  • We evaluate each model with three additional random seeds, using 100 four-player samples per seed (400 player-view videos per seed) as follows.

  • Across these 1200 player-view samples, our method achieves comparable mean FVD to Group-based (157.59 v.s. 153.51), while achieving lower FID (27.91 v.s. 26.24). Besides, our method improves cross-player consistency (GeoKPM) by a large increase of 12.25% (54.17% v.s. 48.26%).

    SeedGroup-based FVD ↓Ours FVD ↓Group-based FID ↓Ours FID ↓
    42156.34155.1927.9525.92
    2026148.51150.0627.3526.15
    6789155.68167.5128.4226.64
    Mean153.51157.5927.9126.24

2) PSNR and SSIM:

Our method achieves comparable PSNR and SSIM to fixed grouping in Table 2: PSNR is of 0.33% differences (18.10 v.s.18.16), and SSIM is of 0.93% differences (0.5005 v.s. 0.5052). In contrast, our method improves consistency in GeoKPM by 6.82% (54.17% v.s. 50.71%).

W1.3 & Q5: IDM accuracy is about 94% for all methods. What is the IDM’s accuracy on real videos (its upper bound)? Can you report accuracy per action type?

A3: We follow works [C, D] to calculate the IDM accuracy per action type and on real videos that represents the upper bound (96.28%), and provide results as follows. The second row (GT) shows the results on real videos. We reorder the 23 actions in descending order of percentage of active action (denoted as "positive rate"). Our method shows consistent improvements in action accuracy, with 10/13 in positive rate larger than 0.1%, showing the effectiveness of our method.

forwardrightleftsprintbacksneakattackpickItemswapHandsjumphotbar.8hotbar.7hotbar.1drophotbar.3usehotbar.6hotbar.2ESChotbar.4hotbar.5hotbar.9inventory23-action average
Positive rate37.67%29.62%28.91%28.83%10.85%8.44%4.73%3.27%0.77%0.41%0.29%0.26%0.13%0.05%0.03%0.03%0.03%0.02%0.00%0.00%0.00%0.00%0.00%—
GT89.80%83.09%86.92%85.54%91.56%90.53%98.62%95.69%98.42%96.25%99.29%99.46%99.62%99.94%99.90%99.97%99.93%99.93%100.00%100.00%100.00%100.00%100.00%96.28%
γ-World69.98%73.87%80.30%77.64%87.69%86.41%98.11%95.42%97.69%97.12%98.83%99.13%99.65%99.93%99.95%99.91%99.92%99.93%100.00%100.00%100.00%100.00%100.00%93.98%
Solaris70.13%74.67%79.39%75.79%86.64%87.08%97.94%95.81%97.48%96.96%98.73%99.06%99.60%99.93%99.97%99.92%99.90%99.93%100.00%100.00%100.00%100.00%100.00%93.87%
Ours71.84%75.22%79.45%77.95%87.40%87.37%98.18%96.56%97.84%96.75%98.92%99.15%99.67%99.92%99.96%99.95%99.95%99.92%100.00%100.00%100.00%100.00%100.00%94.17%
AuthorsOpenReview ↗

Response to Reviewer qCbV [Part 3/6]

W1.4: The efficiency advantage holds only against fully-connected attention. Compute is on par with the centralized baseline

A4: We agree with the comments, which are also indicated in our manuscript:

1) The efficiency advantage holds against fully-connected method, as stated in the submission Line 362-364 ("As shown in Figure 6, fully-connected communication scales quadratically with player count, while Multics substantially reduces computation (9.6 vs. 69.1 TFLOPs with 10 players) ")

2) Our method shows comparable computation efficiency with centralized baseline, which is also indicated in the submission Line 364-365 ("remains comparable to centralized and group-based methods.").

3) We would like to clarify the scope of our efficiency claim. Hierarchical communication improves cross-player consistency, player-specific controllability, and robustness to interference at a computational cost comparable to that of the centralized baseline and substantially lower than that of fully connected attention.

W2.1 & Q3: Models are trained only with 4 and 6 players. If all methods were trained with 10 players, would Solaris still underperform? With 4 players there are only two groups, so there is minimal risk of cross-interference in the shared tokens (group and global).

A5: 1) We provide quantitative results with methods trained on 10 players. We reimplement Solaris and our method using the same base model and dataset, with identical training configurations and iterations.

2) The results show that our method still outperforms Solaris on 9 out of 10 metrics, demonstrating the effectiveness of our method and its scalable communication mechanisms both for generalization (4-->10 players in Table 1) and for the same training/inference setting (10-->10 players).

3) We attribute the effective performances of our method to a) introducing intermediate group-level representations between player- and global-level communication, reducing the direct mixing of heterogeneous information within a shared space; b) hierarchical structure that improves effective multi-player representation learning. In contrast, Solaris concates the representations of all players in the shared self-attention, and learns across all spaces without distinguishing the importance of different information, which can lead to information mixing (also shown in Table 1).

4) We would also like to clarify that training with fewer players and testing with more (e.g., 4-->10 players in our generalization experiments) is more practical for real-world deployment than training and testing with 10 players. This is because (a) training data involving larger numbers of players are typically harder to collect, (b) the number of players at inference time is often unknown in advance and may exceed the largest player count seen during training, and (c) training on clips with more players typically takes longer for the same number of training steps.

GeoKPM ↑PSNR ↑SSIM ↑LPIPS ↓Action ↑RotErr ↓TransErr ↓CamMC ↓FID ↓FVD ↓
Solaris53.73%16.4540.46850.469293.94%3.65412.49514.9718.26108.508
Ours54.59%16.6190.46920.466394.10%3.44811.97314.33417.467126.905

Comments on the paper

5 comments
AuthorsOpenReview ↗

Response to Reviewer FVeG [Part 1/2]

We sincerely thank the reviewer for your insightful comments and recognitions to our work, especially for acknowledging that:

  1. The proposed communication design addresses a meaningful trade-off in multi-player generation.
  2. The intermediate group representations provide a reasonable way, helping to organize local exchange while retaining communication across groups.
  3. The experimental fair comparisons provide useful evidence of our method's effectiveness.
  4. The experiments of player-count generalization shows effectiveness.
  5. The improvements in geometric consistency, generation quality, measured DiT latency and inference with more players than used in training support the practical value of the approach.

We have polished the paper, added the experimental results and made clarifications in the revised version (marked in green). Note that the following polishments have been made according to your advice:

  • The comparisons of asymmetric interactions and our method are included in the Appendix C.11 "Analysis of Asymmetric Interactions".
  • The quantitative evaluation results on robustness interference are included in the Appendix C.5 "Quantitative Results on Robustness Interference".

Thanks again for your very constructive comments, which have helped us improve the paper quality significantly! Below we would like to provide point-to-point responses to all the raised questions:

W1: The grouping criterion does not sufficiently account for asymmetric interactions between players. Visual relevance need not be mutual: one player may observe another who is looking elsewhere. The paper should establish whether the grouping policy allocates sufficient communication to these important relationships under the budget constraint.

A1: 1) We additionally consider asymmetric interactions between players. We incorporate asymmetric visibility into grouping by aggregating the two directed pose-based visibility scores by taking the root mean square (RMS) under the same budget constraint. Unlike the geometric mean in Eq.(8), RMS preserves a nonzero contribution when only one player observes the other, allowing asymmetric visual relevance to influence local communication allocation.

2) We provide quantitative results of RMS and our default setting as below. Relative to the default, all metrics differ by less than 1.1%, showing comparable performance across all ten metrics.

GeoKPM ↑PSNR ↑SSIM ↑LPIPS ↓FID ↓FVD ↓Action ↑RotErr ↓TransErr ↓CamMC ↓
RMS54.1118.120.50070.417726.35161.794.192.42411.8513.39
Ours54.1718.100.50050.418426.39162.394.172.43311.7313.27

3) The comparable results show that under the budget constraint, the grouping policy allocates sufficient communication to these asymmetric interactions between players. This can be due to the reasons: a) A zero mutual-view score does not suppress scene relevance, since the final score is R=max⁡(Rscene,Rmutual)R=\max(R_{\mathrm{scene}},R_{\mathrm{mutual}}), where both terms are aggregated over time. One-sided visibility pairs can therefore still receive direct local communication when their scene relevance is high enough for them to be grouped together; b) Global tokens provide an indirect communication path between players in different groups. Thus, asymmetric interactions can be supported through both local and global pathways under the default budget.

AuthorsOpenReview ↗

Response to Reviewer FVeG [Part 2/2]

W2: The lack of quantitative evaluation on robustness interference. A useful multi-player model must both suppress irrelevant effects and preserve relevant cross-player responses. Evidence separating these two cases would more convincingly establish the claimed advantage of the communication design.

A2: 1) We provide quantitative results for interference robustness as below, compared with baselines (Solaris, γ-World) in both 100 and 200 samples.

  • We use Leakage Rate (LR) as the metric to measure content leakage between unrelated players. The LR is the fraction of irrelevant event–view pairs for which an evaluator detects the specified effect, based on the false-positive rate [A, B]. We use GPT-6 Astra as a visual evaluator to determine whether the generated video shows the specified effect within the designated region.

  • The results show that our method consistently outperforms baselines in interference robustness. We attribute the lower leakage rate to hierarchical aggregation, which limits direct mixing between unrelated local events.

LR ↓Solarisγ-WorldOurs
1003.07%8.28%2.53%
2005.67%7.64%4.50%

2) We totally agree that a useful multi-player model must both suppress irrelevant effects and preserve relevant cross-player responses.

  • From the perspective of suppressing irrelevant effects: the results in reponse 1) provide quantitative results for quantifying content leakage among unrelated players.
  • From the perspective of preserving relevant cross-player responses: we use metrics of cross-player consistency and player specific controllability in action/camera accuracy. The results in Table 1 in manuscript show that our method improves the relevant cross-player responses both in consistency and control accuracy.

References

[A] Li Y, Du Y, Zhou K, et al. Evaluating object hallucination in large vision-language models[C]//Proceedings of the 2023 conference on empirical methods in natural language processing. 2023: 292-305.

[B] Wang J, Wang Y, Xu G, et al. Amber: An llm-free multi-dimensional benchmark for mllms hallucination evaluation[J]. arXiv preprint arXiv:2311.07397, 2023.


Please don’t hesitate to let us know if there are any additional clarifications or experiments that we can offer!

Thank you for the detailed rebuttal and additional experiments. The analysis of asymmetric interactions and the new quantitative evaluation of interference robustness have addressed most of my concerns. While the asymmetric-interaction analysis could still benefit from more targeted evaluation, I find the additional evidence helpful in supporting the proposed communication design. I maintain my original score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.