- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper proposes OpenWorldAct, an interactive video world model that combines precise, continuous navigation control with temporally grounded open-vocabulary event execution. The paper points out a main challenge in collecting training data that are both accurate in low-level navigation control and diverse in semantic event generation. To address this challenge, the paper proposes ActMosaic, a complementary data engine that combines framewise annotated gameplay recordings with motion-suppressed synthetic videos with temporal annotated event descriptions. These two data streams provide separate supervision for navigation / event, and both get transformed into a unified language space for training. Then a Gameplay Harness is leveraged for generating consistent world rules and events. The paper also designs a dual-resolution architecture for real-time inference, which achieves 25FPS on 2 GPU.
Critical strengths
Clear presentation and a well-motivated problem
- The paper is easy to follow and the central problem is important and well-motivated. Since the controllability and customizability are key functionalities of world models. Navigation and Event Occurrence are both required for a game-like world model to be really usable.
Practical and scalable data strategy
- The ActMosaic data collection strategy that avoid navigation and event occurrence appear at same time is scalable. Even if videos with high-quality annotations of both types become available in the future, this data can still be put to use.
Stronger empirical event-control performance
- The method demonstrates much better event-response against different baselines.
An integrated real-time system
- The proposed system has a high degree of completion, including controllable long video generation, dual-resolution real-time streaming deployment, and stateful VLM world interaction harness.
Critical weaknesses
1. Reliability of synthetic event supervision
- The event-data construction pipeline firstly generates event descriptions with LLMs and then uses a video generation model to synthesize the videos. Although the paper mentions an event-adherence filter, it remains unclear how the method ensures the occurrence of the events and the accurate temporal alignment of these events.
2. Lack of real-time inference settings & results
- The paper reports several setting for the real-time inference (denoising steps / resolutions / AR chunk size / AR history size / GPU of 80G 989 TFLOPS), while some settings (Quantization / Kernel Usage / Compilation Usage / TinyVAE Usage) are not reported. And DiT throughputs are reported, while end to end latency (including latency of input group waiting , DiT, upscaler, VAE and gameplay harness) and end to end fps are not reported. It is better to complete these information for real-time inference.
3. Missing analysis of event fidelity in the dual-resolution architecture
- Table 2(c) compares 4-step generation at 360p, 480p, and 720p together with the 4+1 dual-resolution pipeline, showing a trade-off between efficiency and visual quality. However, the comparison does not include event-related metrics such as event completion, response rate, or temporal IoU. Could the authors report event-related metrics across different resolution, especially the direct 720p generation one?
4. Joint control composition remains imperfect
- The performance still decreases a lot when both navigation and event prompts are active. Could the authors report more details on these cases? (e.g. different camera & movement speed / different event types / multiple event at same time)
Questions for authors
See weaknesses.