- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents Gamma-World, a generative multi-agent world model for interactive video generation. The method introduces Simplex Rotary Agent Encoding to represent agent identities using vertices of a regular simplex, and Sparse Hub Attention to reduce the computational cost of cross-agent communication. A bidirectional diffusion teacher is distilled into a causal student for streaming generation with KV caching. Experiments in Minecraft environments show improvements in video quality and inference efficiency, along with qualitative examples of generalization from two to four agents.
Critical strengths
The paper addresses a practical difficulty in scaling interactive world models to multiple agents. Dense attention across agents becomes expensive as the number of viewpoints increases, and the proposed Sparse Hub Attention offers a relatively simple way to reduce this cost. The complexity analysis in Section 3.3 indicates linear scaling with the number of agents when the block size and hub capacity are fixed. Figure 3 provides supporting measurements: with eight agents, DiT latency decreases from 611 ms using dense attention to 246 ms using Sparse Hub Attention. This is a substantial reduction and makes the approach relevant to interactive generation.
The generation results are also promising. In Table 1, Gamma-World reduces FVD from 333.8 to 184.1 on Memory and from 443.1 to 280.0 on Consistency compared with Solaris. Table 2 provides some evidence for the proposed architectural components. Replacing learned view embeddings with Simplex Rotary Agent Encoding reduces FVD from 256.3 to 228.5, and adding Sparse Hub Attention further reduces it to 223.4. The improvements are not uniform across all metrics, but the overall results suggest that the proposed design can improve generation quality while reducing computational overhead.
Critical weaknesses
My main concern is the claimed permutation symmetry of Simplex Rotary Agent Encoding. The paper shows that agents represented by vertices of a regular simplex have equal pairwise distances in the rotary angle space. However, I am not sure this is sufficient to establish permutation equivariance of the complete attention computation. The attention scores also depend on the query and key representations and their relative rotations. Equal geometric distances between agent identities do not by themselves establish that permuting agent assignments produces correspondingly permuted attention outputs. Random assignment during training may improve robustness to agent ordering, but this is different from a formal equivariance guarantee. Since permutation symmetry is one of the main motivations for the encoding, the precise claim needs to be clarified. An explicit evaluation under different agent-identity assignments would also be useful.
The evidence for generalization beyond two agents is limited. Figure 5 shows qualitative four-agent generation results, but the main quantitative evaluations are conducted in the two-agent setting. Figure 3 demonstrates that the architecture scales favorably in terms of computation, which does not necessarily mean that generation quality and cross-agent consistency are maintained as more agents are introduced. In addition, the FID and FVD results in Table 1 do not directly measure whether an agent's actions are correctly reflected in other agents' observations. I would like to see quantitative results for four-agent generation, particularly on action controllability and cross-agent consistency. The real-world robotic examples in Figure 6 are interesting, but they are also qualitative.
I also have some concerns about the fairness of the baseline comparisons. The improvements over Solaris are substantial, but it is not sufficiently clear from the reported comparisons whether the training data, optimization budgets, and evaluation settings are fully matched. This matters because the performance differences could reflect factors beyond the proposed architecture. The encoding ablations are useful, but more explicit experimental configurations would help determine how much of the improvement comes from Simplex Rotary Agent Encoding and Sparse Hub Attention themselves.
Questions for authors
-
Does Simplex Rotary Agent Encoding guarantee permutation equivariance of the attention computation, or is the intended property robustness to agent permutations learned through random assignment? Could the authors provide either a formal clarification or an experiment comparing outputs under different agent-identity assignments?
-
Could the authors provide quantitative results for zero-shot four-agent generation? I am particularly interested in how cross-agent consistency, action controllability, and video quality change when moving from two to four agents.
-
Are the Solaris comparisons and agent-encoding ablations conducted with matched training data, optimization budgets, and evaluation protocols? More details would help assess the contribution of the proposed architectural components.
Thank you for preparing such a detailed rebuttal.
Q1/Q3. For the VLM evaluation results, could you provide the per-category VLM accuracies rather than reporting only the mean VLM accuracy? Also, I respectfully disagree with the claim that “These results show that Sparse Hub Attention achieves overall generation quality comparable to dense attention.” Gamma-World provides this analysis only for the two-player setting. Please consider narrowing the claim accordingly, rather than presenting it as a general conclusion.
Q2. There could be multiple reasons why the variant without hub attention does not work well, such as a mismatch with the training setting or sensitivity to sequence length. How does changing K affect the performance gap between models with and without hub attention? This analysis could help the audience better understand the effectiveness of hub tokens.