- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
A multi-shot video system: an MLLM compiles a shot program, a causal Wan2.1 with cache resets executes it, a layered verifier triggers local re-generation of failed shots, and accepted repairs are distilled with DPO. It is evaluated on 240 programs against multi-shot and camera-control baselines; the appendix adds VBench, a 14B transfer and a human study.
Critical strengths
The verify-then-commit design with loop-off evaluation is sensible, and the verifier calibration (App. D) looks careful. Thresholds are set on a separate human-labelled pool with per-layer precision and recall, and F.3 openly reports a reward-hacking case where distillation suppressed motion.
Critical weaknesses
- The three claims that organize the paper are each supposed to be settled by a control that App. E.8 says has not been run: the comparisons "require actual generation runs and are not assigned numerical outcomes in this appendix" (Supp. p. 11, l. 586–589). What is reported instead is accounting rather than measurement. G.8 computes the final violation rate from the initial incidence, the physics recall and two conditional repair rates that the paper itself calls a check of "arithmetic consistency", "not independent validation" (Supp. p. 18, l. 932–938), and the abstract's "62%" reduction (p. 1, l. 023) is exactly that computation. The same applies to the flagged-shot breakdown in F.2 (Supp. p. 13, l. 670–671), whose layer rates are hard to reconcile with the loop-off violation rate the same appendix feeds into the identity. Nothing in the appendix distinguishes numbers produced by generation runs from numbers produced by the ledger, so the headline reduction cannot be read as an empirical result, and the claims that repair preserves the original contract and that distillation adds something beyond more supervision rest on controls that exist only as a specification.
- Evaluation appears partly circular. REC counts a re-entry as successful when "the ArcFace similarity ... is at least 0.50" (App. E.5, Supp. p. 10, l. 514–515), the same model used in the verifier, and camera error relies on the same VGGT "re-estimation" (App. D.3, p. 8, l. 404). The independent check uses "a different judge model and a re-trained detector seed" (App. F.3, p. 13, l. 701). Baselines receive programs through "one fixed template per system" (App. E.1, p. 8, l. 420), so the CDE gap of 1.46 vs 0.04 (main Table 1, p. 7, l. 333–335) favours scheduled resets by design. As a result, the identity, camera and physics gains may partly reflect the loop optimizing its own evaluators. The cut metrics mainly confirm that the scheduled switch works, rather than showing better generation quality.
- The paper's central efficiency argument is that targeted local revision is cheaper than resampling whole videos, but no matched-budget comparison supports it. Table 3 (p. 8, l. 378–386) contrasts local revision with "Best-of-3", "Refine ×3" and a semantic-only variant, none of which is defined in Appendices E or F, and none of which is per-shot resampling under the same verifier at a comparable budget. Table 4(c) (p. 9, l. 442–444) shows that replacing the single judge with the layered verifier already produces most of the reported improvement, so it is unclear whether the gain belongs to critique-guided editing or simply to rejecting more candidates, and the ablation of the loop therefore does not separate the two. Cost accounting is partial in the same way: verification is excluded from the reported evaluation counts although it runs on separate devices including a large judge, the relationship between the reported evaluation multiple and wall clock is stated to depend on a separately matched timing protocol (Supp. p. 7, l. 346–347), and the full ledger of model calls and hardware time is itself listed among the controls not yet run (Supp. p. 12, l. 634–639). The paper is also an assembly of existing parts (a compiler, a causal backbone, a verifier, DPO), and without per-component budgets and a matched-budget baseline the size of the contribution is hard to judge.
Questions for authors
- Table 3 (p. 8, l. 378–386): how are "Best-of-3", "Refine ×3" and "Semantic" defined, and what does per-shot resampling under the same layered verifier give at a comparable budget? The main-text claim that targeted revision is cheaper than "whole-video resampling" (p. 7, l. 375–377) needs that arm to be interpretable.
- Fig. 1 and Table 3 (p. 1, l. 041–049; p. 8, l. 410–412): which of the stage-comparison numbers come from the same sampling budget as the full loop, and what is the per-component cost of the compiler, verifier and revision passes? Without that split, it is difficult to distinguish the contribution of each component.