- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
DisCo improves camera-controlled video generation by converting continuous camera trajectories into discrete translation and rotation commands. These actions condition a Wan2.2-based model, which is subsequently distilled into a four-step autoregressive generator. The paper also introduces DisCoBench to evaluate action following, dynamic content generation, and long-horizon consistency.
Critical strengths
-
The controlled ablation in Table 5 provides useful evidence for the proposed representation. With the backbone, data, and training settings held fixed, discrete actions improve Action Score over both continuous pose and Plücker representations.
-
The evaluation covers complementary aspects of controllable generation. The appendix strengthens the evidence through human evaluation, multiple evaluation seeds, an external dataset, and continuous pose-error metrics, reducing reliance on the proposed discrete Action Score alone.
Critical weaknesses
-
The methodological novelty is incremental, and the explanation of the gains is stronger than the evidence supports. Discretization, action embeddings, and the distillation components build on existing approaches. Higher cosine similarity alone does not establish that continuous representations are insufficiently distinguishable or that feature entanglement causes control failures. Table 5 supports the effectiveness of the tested discrete encoding, but not a general disadvantage of continuous representations. The loss of motion-magnitude information and omission of pitch and roll further limit the scope of this conclusion.
-
No supplementary videos were provided in the materials available for this review. This limits direct assessment of action transitions, motion smoothness, dynamic content, and long-horizon consistency, which are central to the paper’s claims. Static figures and aggregate metrics provide useful evidence but cannot adequately reveal temporal artifacts or visually demonstrate the reported improvements.
Questions for authors
Can the authors strengthen the continuous-representation baseline, for example through improved normalization or an explicit feature-separation objective, to test whether the gains arise from discretization itself or from the particular encoding design?
4 comments
Q3. Strengthening the continuous-representation baselines.
A3. Thanks for this suggestion. Our continuous-versus-discrete comparison was specifically designed as a controlled ablation to ensure a fair evaluation. As described in Sec. 4.3 and Appendix B.4, the continuous pose, continuous Plücker, and discrete variants use the same Wan2.2 backbone, training data, training settings, and computational resources, varying only the camera-motion representation. Under these matched conditions, Table 5 reports Action Scores of 0.5245, 0.5563, and 0.6332, respectively. The discrete encoding therefore improves action following by 10.87 and 7.69 percentage points over the two continuous variants under the same training protocol.
We agree that improved normalization or an explicit feature-separation objective could further strengthen continuous encodings. Such experiments would complement the existing fair comparison by testing additional encoding or optimization choices. The current controlled results already support the effectiveness of our discrete design relative to the evaluated continuous representations.
Q4. No supplementary videos were provided in the materials available for this review.
A4. We provide video demos in the newly submitted supplementary material (demo_videos.mp4)
Official comment by reviewer
Thank you for the clarification. The contribution and the limitations of the action space are now clearer. I will keep my weak-accept recommendation
Thank you for your positive feedback! We sincerely appreciate your time and effort throughout the review process.
Q1. Methodological novelty beyond existing components.
A1. Thanks for raising this concern. Our contributions are a structured camera-action conditioning design for reliable action following, a controlled study of its representation-level benefits, and DisCoBench for evaluating complex visual exploration. We do not claim quantization, embedding lookup, or distillation themselves as new contributions.
This is non-trivial because assigning discrete labels alone does not specify how translation and rotation semantics should condition video generation or ensure reliable execution when commands change and combine over time.
To address this, we introduce separate learnable translation and rotation embeddings and fuse their projected combination directly with video features (Sec. 3.3), providing explicit conditioning for compositional camera commands. Our representation analysis examines the resulting feature structures (Sec. 4.2), and the controlled ablation in Table 5 establishes the benefit of this design: with the same backbone, data, and training settings, Action Score improves from 0.5245/0.5563 for pose/Plücker encodings to 0.6332.
We further introduce DisCoBench to evaluate command following and transitions, prompted dynamic content, and long-horizon consistency (Sec. 4). This benchmark complements the conditioning design by assessing whether camera commands are executed reliably while maintaining video quality and temporal coherence across these settings.
Q2. Evidence for representation gains and limitations of the action space.
A2. Thank you for highlighting these distinctions. Feature similarity is an empirical characterization of the learned action-conditioning space. In Sec. 4.2, Fig. 4 shows structured similarities among DisCo commands sharing translation or rotation components. Table 4 reports lower feature similarity for DisCo (0.7029 versus 0.8826–0.9913 for the evaluated continuous-action baselines), alongside better action following. These observations provide empirical support for our motivation to make action conditions more distinguishable.
The controlled ablation further establishes the benefit of changing the encoding. With the same backbone, data, architecture, and training settings, Table 5 reports an Action Score of 0.6332 for DisCo, versus 0.5245/0.5563 for pose/Plücker encodings, corresponding to gains of 10.87 and 7.69 percentage points. Our conclusion concerns the effectiveness of this design relative to the tested encodings under matched conditions. We agree that developing stronger continuous representations is a valuable direction; our focus here is a compact action interface for reliable execution of compositional camera commands.
Regarding the action space, we acknowledge that the current interface covers horizontal translation and yaw; it does not specify exact displacement or rotation angles and does not include pitch or roll, as discussed in Appendices B.2–B.3 and E(F in the revision). Appendix B.3 explains the choice of a compact vocabulary to maintain sufficient training samples per category. Table 9 further evaluates finer translation-direction vocabularies: the 27-, 39-, and 48-action variants score 0.6332, 0.5638, and 0.5073, respectively, supporting the compact design for action following under the evaluated conditions.
The discrete framework also offers a path to richer controls. Appendix B.3 discusses adding primitives for different movement speeds or rotation angles, while Appendix B.2 discusses extending the motion categories to pitch and roll. Combining direction commands with discrete magnitude tokens could approximate desired trajectories at a chosen bin resolution. These extensions would require appropriate training data and validation, and remain future work.