- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents H3-WORLD, which aims to transform the coarse-grained understanding of motion already present in pretrained video generation models into temporally grounded action control. The method converts character and camera controls into compositional natural-language instructions, binds each instruction to its corresponding video latent interval, and uses single-egress attention routing to restrict the direct injection of action information into unmatched temporal intervals. Adaptation is performed using LoRA while keeping the MiniMax-H3 backbone frozen.
Critical strengths
-
I agree with the paper’s premise: powerful pretrained video models can already generate approximately correct motion from text, and developing fine-grained control interfaces should build on these existing capabilities as much as possible. Converting discrete controls into structured natural language and integrating them into the model’s native text pathway through lightweight adaptation is a reasonable technical approach. By training only 65.6M parameters, the method achieves effective character and camera control, supporting the feasibility of parameter-efficient adaptation for this task.
-
Expressing actions in natural language alone does not guarantee that the model will execute them at the correct time. The paper further introduces per-latent temporal binding and single-egress routing to constrain the injection of action information within H3’s shared bidirectional attention.
Critical weaknesses
-
In the rightward movement example in the second row of the demo videos, the dominant motion I observed appears to be forward movement or movement along the slope, with clear lateral movement to the right appearing only in the later part of the video. This leaves me uncertain whether the rightward movement command is executed promptly and continuously within the specified interval, rather than merely producing related motion near the end. In another pitch-down example, the character also appears to move forward. If the input does not request character movement, but instead asks the character to remain in place while only the camera tilts downward, this unintended character motion suggests interference between character and camera control.
-
This also leads to my main concern: separating character and camera actions in the textual representation does not automatically guarantee independent control of the generated behavior; single-egress routing primarily constrains direct information injection across temporal intervals and does not directly guarantee independence between different control dimensions.
Questions for authors
My questions mainly concern the demonstrated results and whether the method actually works. Please refer to the Weaknesses section for details.
3 comments
>>> Q Does the method work?
>>> A3 Of course, our method works. H3-World effectively controls character and camera motion, with clear improvements over frozen H3 in controllability, temporal grounding, and generalization.
(1) Effective action control. We compare frozen H3 and H3-World on the same subset of held-out clips.
| Method | Movement F1 ↑ | Yaw F1 ↑ | Pitch F1 ↑ | Macro F1 ↑ |
|---|---|---|---|---|
| Frozen H3 | 0.507 | 0.611 | 0.117 | 0.449 |
| H3-World | 0.510 | 0.832 | 0.417 | 0.588 |
H3-World improves yaw F1 by 0.221, pitch F1 by 0.300 (3.6×), and overall macro F1 from 0.449 to 0.588. These gains demonstrate that our adaptation substantially improves action controllability beyond H3's pretrained prior.
(2) Effective temporal control. As demonstrated in A2, H3-World follows changing action schedules that frozen H3 cannot. The temporal-binding and routing ablations further confirm that our method controls when actions occur, not just what motion is generated (Sections 4.3 and 4.5).
(3) Generalization. As shown in A2, movement and yaw remain controllable in unseen action combinations. On 73 unseen WBench cases, H3-World also improves the navigation score from 0.580 to 0.680 (Section 4.6).
Together, these results demonstrate that H3-World delivers effective, temporally grounded action control, and that the issues observed in individual demos do not represent its overall performance.
Dear Reviewer 38t6,
As the discussion period is approaching its end, we would like to kindly follow up on our rebuttal and ask whether you have any remaining questions or concerns.
We would be happy to provide further clarification if needed.
Thank you very much for your time and consideration!
Best regards,
The Authors
We sincerely thank Reviewer 38t6 for carefully examining our demonstrations and recognizing the value of building action control on pretrained video models. We address the concerns below with further analysis and quantitative evaluations.
>>> W1"Move right" and "tilt down" demos>>> A1Thank you for the careful observations. We acknowledge that unintended motion can occur in some cases, including the two highlighted demos. We identify the following possible causes.(1) "Move right" demo.
(2) "Tilt down" demo.
A2shows that H3-World exhibits a similar rate.These observations help explain the occasional control imperfections. We further examine action timing and control independence in
A2.>>> W2Independent and timely action control>>> A2Thank you for this insightful point. Indeed, our method may not guarantee perfect independence between camera control and character control. However, we've made tremendous efforts in the method design to fulfill the disentanglement between different control types.For example, we design factorized text clauses to represent character and camera actions separately, and temporal binding and single-egress routing to ensure that control signals are precisely aligned with their corresponding video latents.
We further evaluate these capabilities through four tests.
(1) Unintended character movement. We measure character movement under camera-only commands, using recorded gameplay as the reference, since camera rotation can also cause apparent character motion.
H3-World produces only 2.2 and 0.7 percentage points more detected character movement than recorded gameplay, respectively, indicating limited additional interference between character and camera controls.
(2) Single-action control. We evaluate 68 single-action cases from WBench, excluding 5 cases with combined actions.
H3-World improves the navigation score by +0.105 (95% CI [0.042, 0.167]) and reduces rotation error by 0.158, demonstrating more accurate control when only one action is requested.
(3) Action composition. We evaluate unseen combinations by adding tilt to seen movement + pan commands.
Movement and yaw remain comparable to their seen components, showing that the model can combine these controls in unseen combinations, although pitch remains weaker.
(4) Timely action execution. H3-World follows action schedules that change within a clip, including a mid-clip camera reversal that frozen H3 cannot follow (Section 4.3). Removing temporal binding or routing reduces yaw and pitch F1 by 47–61% (Section 4.5).
These results demonstrate that our model executes temporally specified actions, rather than merely producing related motion near the end of a video.
We add the unintended-movement analysis to Appendix B.7 and the composition results to Section 4.6, alongside the single-action WBench evaluation.