- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
CogOmniControl addresses multimodal control in professional video production, where storyboards, clay renders, and reference images convey intent rather than final pixels. A VLM is trained through SFT and RFT to form CogDirector, which interprets sparse conditions as a production plan and supplies representations to the video generator. The generator is then reinforcement-fine-tuned using rewards from the frozen CogDirector. At test time, CogDirector also selects evaluation tools to rank four candidate videos. The paper provides conditional analyses of misranking and selection regret, and introduces CogReasonBench and the 200-instance CogControlBench for reasoning and generation evaluation. The authors report improved intent following from domain-specific reasoning and further automatic-score gains from adaptive selection. Final evaluation uses an independent VLM, supplemented by human preference studies.
Critical strengths
-
The application is well motivated. Abstract requirements in storyboards and clay renders are poorly served by pixel matching alone, making intent interpretation a sensible design choice. Table 4 also supports the domain-specific reasoner's contribution, rather than relying only on comparisons between complete systems with different backbones.
-
Final evaluation includes an independent model and human studies, rather than relying solely on the training reward model. This provides positive evidence for the overall system, particularly its intent following under professional control conditions.
Critical weaknesses
-
One Avg in Table 3 is not reproducible. Normalizing as described and averaging the fifteen metrics equally gives 0.700 for VACE-Wan2.1, not 0.665. Additional weights or eligible-sample handling need to be specified. This does not change the overall open-source ranking, but materially changes the gain over that baseline.
-
The evidence for adaptive selection is incomplete. Changing tools and weights may also change the evaluation objective; the increase from 0.731 to 0.748 does not by itself establish the noise reduction assumed by the theory. I would particularly like to know whether both selectors use identical candidates and whether independent human evaluation favors the adaptive selections.
-
The authors exclude test clips, but training and testing still come from the same production pipeline. Separation of projects, characters, and adjacent shots is unclear (Appendix C.1). The results therefore do not yet distinguish adaptation within a familiar distribution from transferable intent understanding. This contributes to my current weak-reject recommendation.
Questions for authors
-
Could you provide the exact aggregation formula and per-instance results for Table 3?
-
Do fixed and adaptive selection use the same four candidates? How do they compare under human evaluation on that shared pool?
-
Are data split by project or shot sequence, and are results available on an unseen project?