- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces Khora, a scalable multi-agent world model that supports both inference-time population expansion and generalization across geometrically distinct environments. Technically, the paper proposes a persistent Spatio-temporal Board (STBoard) to store scene and entity information, while geometry-conditioned rendering generates each agent’s view independently, which supports changing agent populations without modifying the renderer. Authors provide experimental results on visual quality and world consistency, runtime scalability with up to 80 agents, and dynamic agent insertion and removal, supporting the insight and effectiveness of the proposed method.
Critical strengths
Specifically, the strengths of the paper are:
- The paper is well motivated and architectural decomposition is clear and useful. Khora separates shared-state evolution from view generation using a persistent STBoard and geometry-conditioned rendering. This is a sensible design: consistency can be addressed at the state level, while rendering resources scale with observation demand. It also makes dynamics and visual-generation failures easier to diagnose separately.
- Clear improvements in both of video quality and action consistency. Compared with removing cross-agent state, FVD decreases from 51.6657 to 36.5529 for two views, 73.4917 to 38.6849 for four views, and 104.9082 to 53.6148 for eight views (approximately 29.3%, 47.4%, and 48.9% reductions, respectively.) These consistent improvements support the value of sharing dynamic information. As for action consistency, human ratings increase from 2.075 to 3.570. This is particularly relevant because visually plausible individual videos do not necessarily depict compatible interactions. These quantitative results demonstrate the effectiveness of the proposed work.
- Encouraging distributed runtime performance. In Table 20, across 1–80 agents, reported compute latency grows from approximately 107 to 117 ms, while per-view throughput declines only from 37.33 to 34.27 (2741.2984 / 80) FPS. Aggregate throughput reaches 2,741.3 view-FPS. These results suggest the architecture can exploit additional rendering hardware effectively within the measured range.
Critical weaknesses
Specifically, the weaknesses of the paper, or said, my main concerns about the paper are:
- Nearly constant latency depends on increasing hardware resources. The runtime setup uses one GPU per view (Line 2086:
One GPU is assigned per agent). Consequently, the result supports distributed scalability; it does not show how performance changes when more agents share a fixed GPU budget. A fixed-resource evaluation would make the practical efficiency advantage much clearer. - In table 3, world-consistency gains are comparatively modest. Ratings improve from 4.065 to 4.2133. Table 3 omits the confidence intervals promised in the text. Without uncertainty estimates, I cannot tell confidently whether this smaller improvement is reliable or perceptually meaningful.
- The highest-population quality evaluation has limited coverage. Eight-view evaluation uses 159 cases, versus 10,000 at two and four views. I would therefore avoid interpreting differences across population settings as a clean scaling trend. More balanced sampling and uncertainty estimates would strengthen the evidence.
- Strictly, a shared representation does not itself guarantee consistent generated observations. My concern is that the renderer may still contradict the conditioning state. Objective measurements of identity preservation, visibility, interaction outcomes, and long-horizon drift would be more convincing than visual quality and subjective ratings alone.
- The ablation study is limited to removing cross-agent state, which demonstrates the benefit of sharing information but does not isolate the contributions of the other proposed components. In particular, it remains unclear how much the learned motion residual, persistent scene memory, and geometry-based conditioning each contribute to the final performance. Targeted ablations, such as using only the deterministic motion model or replacing geometric conditioning with a simpler alternative, would help establish which design choices are essential.
Questions for authors
Here are questions for authors:
- Runtime clarification: With one GPU assigned per agent, do authors have measurements showing how latency, throughput, and memory usage change as the number of agents increases under a fixed GPU budget? If unavailable, please clarify whether the efficiency claim is limited to distributed scaling with proportional hardware resources.
- Statistical reliability: Could authors provide the confidence intervals mentioned in the user-study description, together with the numbers of participants and evaluated clips?
- Consistency evidence: Do existing evaluations measure agreement between generated observations and the underlying STBoard states, including identity, visibility, and interaction outcomes? Please clarify whether consistency is enforced by the rendering mechanism or encouraged through conditioning.
- Component contributions: Beyond removing cross-agent state, do authors have existing ablations isolating the learned motion residual, persistent scene memory, or geometry-based conditioning? These would help determine whether the benefits arise from the specific proposed design or primarily from access to shared structured state.
- Is it possible to provide more qualitative results including both good cases and failure cases to better demonstrate the effectiveness and limitations of the proposed method.
3 comments
Continuing our response, Khora w/o Cross-Agent State removes other agents’ dynamic state conditions while retaining static scene memory, ego state, and per-view latent history. Action consistency increases from 2.075 to 3.570 (+1.4950), and world consistency increases from 4.0650 to 4.2133 (+0.1483). The larger action gain matches the role of the dynamic conditions evaluated by this ablation.
We report the 95% confidence intervals for both criteria:
| Method | Action ↑ (95% CI) | World ↑ (95% CI) |
|---|---|---|
| Gamma-World | 2.027 [1.658, 2.433] | 2.786 [2.366, 3.221] |
| MIRA | 3.170 [2.771, 3.563] | 4.009 [3.596, 4.330] |
| Solaris | 3.000 [2.759, 3.250] | 4.105 [3.658, 4.465] |
| Khora w/o Cross-Agent State | 2.075 [1.802, 2.342] | 4.065 [3.755, 4.352] |
| Khora | 3.570 [3.217, 3.922] | 4.213 [3.895, 4.508] |
Paired gains compare Khora with Khora w/o Cross-Agent State. Both 95% CIs use bootstrap resampling: participants only, or participants and source cases.
| Metric | Paired gain | Participant 95% CI | Participant + case 95% CI |
|---|---|---|---|
| Action | +1.4950 | [1.0217, 1.9600] | [1.0005, 1.9913] |
| World | +0.1483 | [0.0867, 0.2200] | [0.0370, 0.2692] |
Both bootstrap analyses give positive paired-improvement intervals for both criteria. Means are displayed to three decimals; paired improvements use unrounded means.
Eight participants completed all 150 assignments in the paired study, yielding 1,200 ratings: 600 per variant and 200 per variant at each of 2, 4, and 8 views. Within each view count, a participant rates the same 25 source cases for both variants. Completed assignments cover 198 distinct two-view cases, 196 four-view cases, and 113 eight-view cases, identically paired across variants. Each rating includes Action and World scores. For the external baselines, 12 participants provided 338 ratings over ten source cases per model: Solaris 114, Gamma-World 112, and MIRA 112. We will include the uncertainty estimates in the revised manuscript.
STBoard supplies a common evolving state, and its projections condition rendering. Table 4 directly evaluates location, image motion, body-pose alignment, and foreground appearance against GT under recorded camera poses and supplied trajectories. CVC complements these actor-level measures with reconstructed geometry and color consistency.
We additionally evaluate target localization and joint visibility across observers:
| Method | Target recall ↑ (95% CI) | Joint recall ↑ (95% CI) |
|---|---|---|
| Khora | 40.42% [26.74, 61.10] | 26.72% [7.07, 55.84] |
| MIRA | 32.10% [18.17, 55.04] | 24.43% [4.07, 54.72] |
| Gamma | 26.00% [10.66, 50.29] | 21.84% [1.88, 51.14] |
The visibility evaluation uses 500 four-player cases with two-second clips. Localization recall counts generated detections at the corresponding locations of 673 detector-confirmed GT target observations. Joint recall uses 348 common-target observer pairs: when a target is visible to both GT observers, both generated views must detect it. Two simultaneous misses count as failure.
Khora has the highest recall on both measures. Bold and italics indicate the highest and second-highest point estimates; the intervals remain explicit. Part 3 provides action-conditioned pose prediction and matched visual ablations.
Continuing our response, we isolate the learned transition correction by comparing fixed kinematics with fixed kinematics plus the learned visual residual. All four pose-error measures improve at both horizons:
| Horizon / cases | State transition | ADE ↓ | FDE ↓ | Yaw MAE ↓ | Pitch MAE ↓ |
|---|---|---|---|---|---|
| 10 s / 1,000 | Fixed kinematics | 130.74 | 269.43 | 16.27° | 3.30° |
| 10 s / 1,000 | + learned residual | 102.97 | 229.67 | 14.16° | 2.56° |
| 30 s / 1,000 | Fixed kinematics | 305.12 | 714.79 | 19.38° | 3.46° |
| 30 s / 1,000 | + learned residual | 259.46 | 678.53 | 17.12° | 2.87° |
We evaluate 1,000 Dust2 cases at each horizon, each with two controlled players and two first-person views. RGB context is frames 0–12; predictions cover 13–332 (10 s) or 13–972 (30 s), at 32 FPS. Both variants use identical cases, initial states, actions, and per-view seeds, with no future GT-pose resets. ADE/FDE are 3D camera-position errors in original game-coordinate units; yaw uses the shortest circular angular difference. Angular errors are in degrees.
A matched visual comparison measures the downstream effect with the same renderer:
| Views | Pose source | PSNR ↑ (dB) | SSIM ↑ | LPIPS ↓ | FID ↓ | FVD ↓ |
|---|---|---|---|---|---|---|
| 2 | Fixed kinematics | 24.7913 | 0.7104 | 0.1862 | 10.2930 | 83.2843 |
| 2 | + learned residual | 26.2425 | 0.7130 | 0.1789 | 7.7098 | 36.5529 |
| 2 | GT-pose conditioning | 28.0482 | 0.7987 | 0.1374 | 7.1638 | 29.3661 |
| 4 | Fixed kinematics | 24.1049 | 0.6973 | 0.1889 | 14.9372 | 105.2849 |
| 4 | + learned residual | 25.2825 | 0.7020 | 0.1772 | 7.9821 | 38.6849 |
| 4 | GT-pose conditioning | 27.7358 | 0.7623 | 0.1637 | 11.6344 | 32.8530 |
| 8 | Fixed kinematics | 25.0482 | 0.7448 | 0.1594 | 36.8711 | 139.9372 |
| 8 | + learned residual | 26.5775 | 0.7573 | 0.1477 | 27.0311 | 53.6148 |
| 8 | GT-pose conditioning | 28.6497 | 0.8277 | 0.1329 | 24.3762 | 41.3781 |
The visual comparison uses frames 13–80: 10,000 cases at two views, 10,000 at four views, and 159 at eight views. All pose sources use matched cases and generation settings within each view count. GT-pose conditioning provides recorded future poses to the renderer, which still generates RGB. These visual cohorts are separate from the 1,000-case long-horizon experiments. The learned residual improves all five visual metrics over fixed kinematics at every view count.
We separately remove the geometry condition:
Evaluation cases: 10,000 for 2 and 4 views; 159 for 8 views. PSNR is in dB.
| Views | Geometry | PSNR ↑ | SSIM ↑ | LPIPS ↓ | FID ↓ | FVD ↓ |
|---|---|---|---|---|---|---|
| 2 | ON | 26.2425 | 0.7130 | 0.1789 | 7.7098 | 36.5529 |
| 2 | OFF | 10.6077 | 0.4235 | 0.6758 | 213.4300 | 1198.8300 |
| 4 | ON | 25.2825 | 0.7020 | 0.1772 | 7.9821 | 38.6849 |
| 4 | OFF | 10.9243 | 0.4193 | 0.6698 | 207.7100 | 1178.0400 |
| 8 | ON | 26.5775 | 0.7573 | 0.1477 | 27.0311 | 53.6148 |
| 8 | OFF | 12.8653 | 0.4519 | 0.5801 | 199.6600 | 934.6000 |
ON and OFF use the original evaluation cases and otherwise matched generation settings. Removing geometry degrades all five visual metrics, supporting its contribution independently of the motion-residual comparison. Within matched settings, bold marks the best value and italics the second-best among three settings.
The smaller eight-view cohort reflects the availability of trajectories with synchronized observations and corresponding action/state annotations for all eight agents. We collected all available valid cases: 159 cases (1,272 videos). The two- and four-view evaluations each use 10,000 cases. Each eight-view model comparison uses the same 159 cases, assessing higher-population visual quality on matched data within that setting. We will state this sampling protocol explicitly in the revised manuscript.
We thank the reviewer for recognizing the architectural decomposition, visual-quality and action-consistency improvements, and distributed scalability. We now report fixed-budget performance and profiled inference computation.
Multiple agents can share one device and reuse the same renderer. The original one-device-per-agent allocation isolates population-dependent latency from inter-agent device contention. All new hardware measurements use PPU-ZW810E devices with 96 GB of memory and unchanged generation settings.
Profiled computation grows linearly with requested views: 15.695680 TFLOPs/request for one player and 251.130885 for 16 players, with a constant 3.923920 TFLOPs per new view-RGB frame:
Each request produces four new RGB frames per player. Frame counts and total view-FPS sum over all players; Peak/GPU is the sampled peak memory per device. FLOPs count supported operators in five DiT forwards, one streaming VAE decoding call, and recognized projection/condition-construction operations. Counted operators include matrix multiplication, convolution, and Flash/Efficient Attention. A multiply-add is two FLOPs; TFLOPs denotes 10¹² operations.
On one device, the measured population ranges from 1 to 13 players:
With a fixed 16-player world, increasing the budget from 2 to 16 devices gives a 5.79× throughput improvement, or 72.42% of ideal linear strong scaling:
With eight players per device, increasing from 1 to 16 devices achieves 92.55% of ideal linear throughput scaling:
The eight-players-per-device rows define this scaling calculation; the one-device/13-player row separately reports a measured population setting. Part 2 provides user-study uncertainty and objective visibility evidence; Part 3 reports component ablations and the eight-view cohort.