- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents Code Video Model. A coding agent turns a prompt into an executable scene program and renders a proxy video, and a frozen video model then generates the final clip. To keep the output from copying the proxy’s flat appearance, the authors propose a training-free strategy that applies proxy conditioning only during the early sampling steps. Experiments on their own benchmark show that it outperforms existing methods.
Critical strengths
This paper uses executable programs as a previs engine for video generation. Scene layout, object motion, camera trajectories, and timing are written explicitly into code, so the structure can be inspected and edited line by line, and the dynamics are computed from equations at the proxy level. The early-step conditioning strategy needs no training and no extra modules, yet it lets a frozen model follow both the proxy’s structure and the reference image’s appearance. The authors analyze the two native conditioning pathways with step-wise sensitivity and feature probes, which explains the mechanism well, and they show that the approach transfers to a closed-source model.
Critical weaknesses
The novelty of early-step conditioning is somewhat limited. Prior work such as SDEdit has already established that structure is mostly set in the early denoising steps, and recent training-free temporal conditioning methods also rely on this pattern. This paper uses a hard switch at a fixed cutoff step, so we suggest exploring content-adaptive schedules and giving a detailed justification.
The case for physical consistency is not yet sufficient. Dynamics are computed only in the proxy video, so after the cutoff the frozen model may drift from the trajectories the equations define, and physical validation relies mainly on qualitative examples. The recent PhysAgent work already uses an execute-and-verify loop, so we suggest quantitatively measuring the physical error in the final videos.
Questions for authors
Please refer to weaknesses.
3 comments
Official Comment by Authors
W2. Physical Consistency Evaluation
We thank the reviewer for suggesting a physical consistency evaluation. Following PhysAgent’s open-source evaluation procedure, we provide each judge with the final video, the task prompt, the reference image, and the physical configuration produced by our coding agent during proxy video generation. The judge uses this configuration to assess whether the observed contacts, motion, and object responses are physically plausible and consistent with the intended event.
Qwen-27B, GPT-5.6 Sol, and GPT-6 Astra each assign a score in to every evaluated video. For each method and judge, we average the scores across CodeVideoBench cases and multiply the mean by 100. We report Physical Consistency as a fourth dimension alongside Structure, Style, and Quality.
Table R12: Physical consistency scores of final videos on CodeVideoBench.
| Method | Qwen-27B | GPT-5.6 Sol | GPT-6 Astra |
|---|---|---|---|
| VACE | 81.73 | 75.64 | 76.32 |
| Go-with-the-Flow | 57.29 | 39.43 | 41.38 |
| Diffusion as Shader | 63.91 | 49.77 | 49.95 |
| Text-to-video | 83.23 | 77.05 | 74.45 |
| R2V | 82.45 | 77.59 | 81.77 |
| SDEdit | 82.50 | 78.32 | 78.73 |
| Code Video Model (Ours) | 84.05 | 79.14 | 82.73 |
Our Code Video Model achieves the highest Physical Consistency scores under all three evaluators, consistently outperforming all compared baselines. This demonstrates that our method also achieves strong physical consistency in addition to its performance on the original evaluation dimensions. The evaluation protocol and physical consistency results are now included in Appendix D.2 of the revised manuscript.
Thank you again for your thoughtful review. We would be happy to discuss any remaining questions and would be grateful if you could raise your score.
Dear Reviewer,
The discussion period is nearing its end, and we haven’t yet seen a reply from you. We encourage you to respond to the authors’ comments, especially if you have further questions about points that may affect your assessment.
Thank you for taking the time to engage in the discussion.
Best,
Your AC
Official Comment by Authors
Thank you for the careful review and for recognizing the key contributions of our work, including the use of executable programs as a previs engine for video generation, the early-step conditioning strategy, and our analysis on the pathway conditioning.
W1. Part 1. Novelty and Contribution
Our central contribution is using code as an executable previs for video generation, together with early-step conditioning as a technical method for realizing this approach using a frozen R2V model.
Our method is conceptually related to SDEdit only at a high level; technically, it differs substantially in its initialization and conditioning mechanisms.
Table R9: Key methodological differences between SDEdit and Code Video Model.
The following table compares both approaches using the same frozen MiniMax-H3 backbone (the full table appears as Table 1 in our main paper). Our method achieves a higher overall score under all three evaluators.
Table R10: Quantitative comparison between SDEdit and Code Video Model.
W1. Part 2. Content-adaptive Schedule for Proxy-Conditioning
We thank the reviewer for suggesting that we explore content-adaptive schedules for proxy conditioning. Figure 4(a) shows that proxy conditioning has a greater influence on generation in early denoising steps than in later ones. Based on this observation, we propose and evaluate an online schedule that selects a cutoff for each sample. At each denoising step i, we compute the sensitivity of each proxy pathway using Eq. (4) of the main paper and take the maximum of the two.
St,g=∥vtfull∥2vtfull−vt∖g2,di=maxg∈proxy-VAE,,proxy-Qwen-VLSti,g.
Both velocity predictions use the same latent and timestep, with full conditioning and with pathway g removed, respectively. At the first update i≥2 satisfying
di−1<τanddi<τ,
we disable both proxy pathways from update i onward, giving K=i−1. If the criterion is never met, K=N. Text and reference image conditioning are active throughout. We evaluated three thresholds on CodeVideoBench.
Table R11: Quantitative comparison of fixed and adaptive cutoffs for proxy conditioning.
The adaptive schedule modestly improved scores under all three evaluators, with τ=0.5 performing best. Fixed K=5 remains a practical default, offering similar generation quality with fewer model evaluations. The adaptive schedule and its quantitative evaluation are now included in Appendix F of the revised manuscript.