- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents a method to achieve auto-regressive video generation by distilling a teacher video generation model. The key insight of the paper is the mismatch between teacher and student models, since the teacher sees the whole video, whereas the student only sees the past. The solution to this is to constrain the teacher to only see the past as well. Camera-conditioned causal distillation employs a similar approach, using only the camera embeddings available up to the current frame. The results are very impressive, with extremely smooth videos, and no visible temporal seams due to chunking, which may be the case in some methods.
Critical strengths
1/ I found the paper in general to be well-written and clear. This is appreciated, especially in this domain, which tends to contain a lot of technical parts and details. 2/ The central premise and solution of the method are clear, and make a lot of sense. This makes a strong argument in favour of the paper. 3/ The results in the supplementary material are visually very impressive, and I congratulate the authors on them. This is obviously a huge plus for the paper.
Critical weaknesses
1/ While the paper was in general well-written, I thought that the jargon was a bit difficult to follow when diving into the details. I understand that much of it is unavoidable, but I felt that some could be a bit clearer, and in general it bogged the reading/understanding of the paper significantly. Examples:
- Line 180: "is rolled out on-policy". If I understand correctly, this is jargon borrowed from reinforcement learning, which the readers of the current paper are not necessarily familiar with, and which you have not, in any case, introduced. What is on-policy rolling out? I saw that it has indeed been used in other very recent preprints on autoregressive video generation, but I think it should be explained, otherwise the paper becomes difficult to read.
- Throughout the paper, you use "scoring", for evaluating the score, ie the gradient of the log probability of the video. I do not think this is common jargon, I would advise changing it. For me, it makes things slightly cloudier than they have to be. Especially in this domain, I feel like too much new jargon is introduced, making the papers difficult to read. If I am wrong, and this is common in auto-regressive diffusion-based video generation, please point me to some (published) papers, and I will accept the jargon.
- The use of the word "prefix". While I understand this is common terminology in LLMs, it is not so common in video generation models. While I understand you do not have space to define it in the abstract, you must define what it means in the beginning of section 3, otherwise some readers will have trouble following.
2/ I think you really need some notation for the different models, ie:
- teacher
- causal student variant
- block causal student variant
- prefix
In summary, I think that because the authors are so familiar with many different domains relevant to the present task, they have ommitted to define certain notions carefully. This, combined with the lack of notation in some cases, leads to long, technical sentences which are not easy to follow.
3/ Also, I think that the whole premise of the method, and the solution you propose, could have been explained a bit better. If I summarise it, it is: "during training of the student, the teacher sees the whole video, whereas the student is auto-regressive, and so only sees the past. This creates a mismatch between teacher and student. Our solution is to constrain the teacher to only see the past as well". (if this is incorrect, please correct me). I think a diagram could be of great help here. In particular, the paragraph "conditioning mismatch in causal training" is central to your premise, and slightly difficult to follow. Could you not put the diagram in the supplementary material illustrating which information is available for the teacher and the student ? And also maybe an illustration of your solution, CMD. It might help make your case clearer, and certainly more impactful.
4/ The beginning of Section 3 was difficult to follow for me. Some examples:
-
Line 182: "Building on base CMD". What is base CMD ?
-
Line 183: "the cached student-generated prefix used to generate it". By this, you mean the video generated by the student until time i-1, which as been put into the model to calculate K and V. Again, this could be clearer.
5/ I find it extremely odd that you put the related work in the supplementary material! This seems unfair: other authors will have made an effort to make space for the related work in the main paper, whereas you have bypassed it. My opinion would be to put it in the main paper.
Questions for authors
1/ In section 3.1, it seems like you present the conditioning mismatch as if this is the first paper to remark on it. Is this the case ? If not, can you cite the others (apart from Zhu 2026b, which you put). Rather than in section 3.1, this should be earlier, in related work, since it is your core premise.
2/ When looking at the factorisation in Equation 2, I felt that something was slightly odd. While I agree that the conditioning on B_{<i} is just a consequence of the definition of conditional probability, the conditioning on turns into . This is not necessarily obvious. Indeed, you are saying that future controls are independent of generated blocks in , which means that the user is not making future decisions based on what is happening in these previous blocks. This may be a hypothesis that helps simplify the model, but in practice is probably not true. I am not saying this should be changed, but the hypothesis should be made clearer. At first glance, it seems like what is being said is "the controls in the future are unknown, so therefore they are independent", but unknown does not mean independent. If I have misunderstood something, please explain further.
3/ Why do you not compare with other methods in your supplementary materials videos !? You have the videos, since you have the quantitative results, why not add a few (not all obivously) examples of other works ? This seems odd, since your results are so good.
Small comments:
3/ Line 139: you re-use to define it as a video (in equation 1 it was a generic datum). I think you could just remove here, since you never use it again. The confusion for me comes from the fact that everything which is a vector is in bold, and is normal font. I think it is a very good idea to put vectors in bold, but maybe you could just say without referring to at all. However, this is only a suggestion, and a very minor point, you do not have to take it into account.
4/ Line 147: Why do you use both "chunks" and "blocks", aren't they the same thing ? If so, I would only use one jargon (preferrably chunks), unless they are not the same.
5 Line 153: "We treat the input image as part of the ordinary causal history rather than something that is permanently retained beyond local attention windows." The meaning of this sentence is not clear to me.
6/ Equation 3. Is this equation useful ? Why is it useful to know that setting a block size of 1 just gives generation frame by frame ? This is obvious no ? You could equally say that setting gives a standard bidirection model, and it would not be that useful either. Maybe remove.
7/ Line 231: "These noised targets are arranged temporally", what does arranged temporally mean?
8/ Line 241: "Frozen causal teacher", is this the same one that you discuss in the beginning of the section 3.2 ? If so, it should be defined earlier.
9/ Equation 5: The notation is not great, can be easily confused for the conditioning in Equations 2-3. I would change this.
10/ Line 315: "We unproject rays". This is not very clear. The whole subsection is not super clear for me. What is ? It is not precisely defined it seems. I feel like the authors have attempted to make the section formal, but not really gone far enough, leaving the explanation a bit muddy. I would suggest either re-writing it, or adding a fuller explanation in the supplementary.
2 comments
Ok, thank you for the changes made, I feel it is clearer now. In light of this increased clarity, I will raise my score to accept.
Response to reviewer PJn7
Thank you for the careful reading. We have uploaded a revised paper and supplementary material, with all changes marked in red. The revision adopts the suggestions.
1. Terminology and notation
"On-policy", "prefix", "scores a target" and "Base CMD" are now defined at first use at the start of Section 4. "Block" and "chunk" are declared interchangeable. The corruption operator is renamed so that it no longer collides with the controls C. The frozen teacher is introduced where it is trained.
On "scoring": real and fake score networks are standard terminology in the DMD papers we build on (Yin et al., CVPR 2024; Yin et al., NeurIPS 2024). We agree the verb was undefined and now define it once.
Instead of a separate notation table, we define each term inline where it first appears, so that the definitions sit next to their use.
2. Premise and diagram
Your summary is correct. Figure 2 illustrates exactly this information-set difference, and the mismatch paragraph in Section 4.1 now points to it.
3. Related work in the supplementary
Agreed. The main paper now has a Related Work section (Section 2) that covers prior work on teacher-student mismatch (Causal Forcing, Causal Forcing++, Context Forcing); the extended survey remains in Supplementary S1.
4. Q2: the factorization in Eq. 2
No independence is assumed. The joint over frames and controls factorizes as ∏ip(Ci∣B<i,C<i)⋅p(Bi∣B<i,C≤i), and the model specifies only the second factor, so the user's controls are free to react to earlier frames. The only assumption is that Bi does not depend on future controls given the past. This strengthens the premise: when controls react to frames, future controls carry information about the present frame, which is exactly what a teacher given the full trajectory can exploit. We state this after Eq. 2.
5. Q3: videos of other methods
Added, as side-by-side comparisons with four baselines in the supplementary gallery, and as camera-controlled comparisons with SANA-WM.
6. Smaller comments
[Edit: in the latest revision the ablation tables are renumbered in the order they appear. Former Table 5 (short-video) is now Table 4, former Table 6 (long-video) is now Table 5, and former Table 4 (camera) is now Table 6. Point 5 now also mentions the camera-controlled comparisons.]