- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes SEGUE, a framework that tackles how streaming video models should handle a new instruction when the current frame is not aligned. The core idea is to adopt a training-free planner that writes the missing transition (following TD-schema). Authors also propose SpanDMD which assigns DMD residuals to the corresponding temporal chunks while still keeping the full rollout. Authors build/evaluate their methods on OpenTrans-360.
Critical strengths
-
The problem setting is simple, but important. In practical point of view, planning from what the model is currently generating is important but many previous/concurrent work just blindly feed the prompts. Adding a planner that interleaves different transitions seems like a practical solution.
-
Method is reasonably motivated and designed. I don’t find it specifically novel, but overall planner approach + benchmark sounds solid.
Critical weaknesses
-
My main concern is on SpanDMD. In w/o SpanDMD, authors concatenate all instructions into a single prompt and remove temporal masks together. SpanDMD also uses separate teacher evaluation (which probably increases the training cost), and assuming like, what if we have a teacher that is already capable of having accurate transitions (maybe per-latent/chunk text conditions)? Would we still need SpanDMD? I like Figure 9, it’s very helpful for understanding what it feels like but this also brings a question like if we give enough time between transition wouldn't the original rollout also do the right generation? Colors are turning from Red -> Blue but slowly, so I’d expect if it's just generating a smoother transition and we are changing conditions so fast. In general, SpanDMD for me, feels like a small trick on how/where we apply DMD and partial application of DMD for clips are already discussed in longer rollout DMD papers (though not aligned semantically like this) so it feels more as an incremental contribution.
-
It’s getting common in this field, but I feel like the technical novelty and amount of new components are very limited. The basic idea of creating intermediate transition plans is great, but I think the method does not provide insights beyond that. Adding a planner with some schema is relatively straightforward (and obviously, some improvements are expected), and I feel like SpanDMD is essentially a small variation of how we apply DMD loss. Since the amount of novelty can depend very subjectively, I would also consider other reviewers’ comments for the final score.
Questions for authors
Please refer to my weakness section for the questions.
2 comments
Dear Reviewer zw2S,
Thank you again for your thoughtful feedback. We have provided additional experiments and clarifications to better explain the motivation and contributions of our approach. We hope our responses help address your concerns. If you find them convincing, we would sincerely appreciate your consideration of updating your assessment.
Best regards,
The Authors
We appreciate the reviewer's careful reading and insightful questions, particularly about SpanDMD. Your suggestions have helped us better examine its design choices and clarify the contributions of our work.
We agree that the original ablation changed conditioning and masking simultaneously. To better isolate their effects, we added three controlled alternatives in Sec. 5.3 / Table 2 (p. 10).
SpanDMD outperforms all three alternatives. Importantly, unmasked per-condition DMD preserves the same conditioning and full-rollout inputs, isolating the effect of span-specific masking, while cropped-span DMD isolates the benefit of full-rollout temporal context.
In our 60-second training sequence, we perform 12 DMD evaluations per sequence, but only one involves a prompt switch requiring L=2~5 teacher evaluations instead of one. Thus, the overall teacher-evaluation cost increases by only (11+L)/12 = 1.08–1.33 times, not L times. This very limited overhead is training-only, with no additional inference cost.
This is an insightful question. We would love to see future teachers support accurate per-chunk or per-latent text conditioning! SpanDMD could still provide span-specific supervision with full temporal context, although its additional benefit would need further evaluation. But currently, the bidirectional teachers operate within a 5–15s temporal context under a single text condition, making direct supervision of multiple short-lived prompts difficult. SpanDMD provides a practical solution under these existing teacher constraints.
We are glad the color-transition visualization was helpful. To examine whether longer durations resolve the issue, we added a 20-second color-transition experiment in Appendix A.9 / Fig. 12 (pp. 27–28), allowing approximately 3–4.5 seconds per post-switch color span.
The original rollout eventually reaches some target colors but still exhibits delayed responses and transient dimming or flicker. In contrast, the SpanDMD-trained generator transitions through intermediate hues more gradually and maintains a more stable appearance. We hope this additional visualization helps address your concern about whether longer transitions resolve these artifacts.
We appreciate the reviewer's perspective. Prior works such as Rolling Forcing[1] and RMD[2] have adapted DMD in different ways to address their respective challenges, improving the stability and quality of long-video generation. In interactive streaming generation, however, corner cutting is a common and persistent problem, which we further examine through quantitative and qualitative analyses in Appendix A.1 (pp. 16–17). SpanDMD is specifically designed to address this issue through span-specific supervision, helping the generator faithfully execute intermediate actions during prompt switches. Our results show that it visibly improves transition quality and effectively mitigates corner-cutting failures.
We also agree that temporal planning is becoming increasingly common, as seen in DirecT2V[3] and TempAct[4]. But interactive streaming brings a different challenge: the new prompt tells the model what to do next, but what it has already generated determines how to get there. This motivates state-grounded, dependency-aware planning rather than simply decomposing the target prompt. Together with SpanDMD, SEGUE connects planning what must happen with ensuring that the generator actually executes it, addressing a practical challenge in interactive streaming generation.
We hope this helps clarify the motivation and contributions of our work.
References
[1] Liu et al. Rolling Forcing: Autoregressive Long Video Diffusion in Real Time. 2026.
[2] Gao et al. Rollout-Marginal Distillation for Long-Horizon Autoregressive Video Generation. 2026.
[3] Hong et al. DirecT2V: Large Language Models are Frame-Level Directors for Zero-Shot Text-to-Video Generation. 2023.
[4] Wang et al. TempAct: Advancing Temporal Plausibility in Autoregressive Video Generation via Planner-Executor RL. 2026.