acceptodds
Withdrawn from ICLR 2027

No Corners Cut: State-Grounded Transitions for Mid-Stream Prompt Switches in Video Generation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Streaming video generators allow users to dynamically modulate video synthesis via mid-stream prompt switching. Existing streaming methods can respond to the updated instruction while still cutting corners, prematurely realizing goals or taking heuristic shortcuts that bypass necessary intermediate state changes needed for a plausible transition. In this study, we present SEGUE, a novel framework that makes this process explicit and trains the generator to execute these transitions faithfully. At each switch, a training-free planner parses the latest frame and prompts, writes a few segue prompts with roles and durations, and then hands control back to the user's prompt. Furthermore, to address the inherent difficulty of training causal models on short-lived temporal schedules without corrupting preparatory supervision, we introduce SPANDMD, which evaluates each active prompt using the full rollout as temporal context while retaining its DMD residual only within the prompt's assigned span. On OpenTrans-360, a benchmark of 1,800 switches that scores how the old state exits and the new one begins, SEGUE ranks first on all eight transition metrics and raises the overall score over the strongest baseline from 0.866 to 0.887. It also ranks first on four of six instruction-response metrics of StreamAV-Bench, while the planner transfers to frozen autoregressive generators without retraining.

Primary area:
generative models
Keywords:
Interactive Video Generation

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_zw2SOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes SEGUE, a framework that tackles how streaming video models should handle a new instruction when the current frame is not aligned. The core idea is to adopt a training-free planner that writes the missing transition (following TD-schema). Authors also propose SpanDMD which assigns DMD residuals to the corresponding temporal chunks while still keeping the full rollout. Authors build/evaluate their methods on OpenTrans-360.

Critical strengths
  • The problem setting is simple, but important. In practical point of view, planning from what the model is currently generating is important but many previous/concurrent work just blindly feed the prompts. Adding a planner that interleaves different transitions seems like a practical solution.

  • Method is reasonably motivated and designed. I don’t find it specifically novel, but overall planner approach + benchmark sounds solid.

Critical weaknesses
  • My main concern is on SpanDMD. In w/o SpanDMD, authors concatenate all instructions into a single prompt and remove temporal masks together. SpanDMD also uses separate teacher evaluation (which probably increases the training cost), and assuming like, what if we have a teacher that is already capable of having accurate transitions (maybe per-latent/chunk text conditions)? Would we still need SpanDMD? I like Figure 9, it’s very helpful for understanding what it feels like but this also brings a question like if we give enough time between transition wouldn't the original rollout also do the right generation? Colors are turning from Red -> Blue but slowly, so I’d expect if it's just generating a smoother transition and we are changing conditions so fast. In general, SpanDMD for me, feels like a small trick on how/where we apply DMD and partial application of DMD for clips are already discussed in longer rollout DMD papers (though not aligned semantically like this) so it feels more as an incremental contribution.

  • It’s getting common in this field, but I feel like the technical novelty and amount of new components are very limited. The basic idea of creating intermediate transition plans is great, but I think the method does not provide insights beyond that. Adding a planner with some schema is relatively straightforward (and obviously, some improvements are expected), and I feel like SpanDMD is essentially a small variation of how we apply DMD loss. Since the amount of novelty can depend very subjectively, I would also consider other reviewers’ comments for the final score.

Questions for authors

Please refer to my weakness section for the questions.

2 comments

We appreciate the reviewer's careful reading and insightful questions, particularly about SpanDMD. Your suggestions have helped us better examine its design choices and clarify the contributions of our work.

W1(a). Isolating the contribution of SpanDMD

We agree that the original ablation changed conditioning and masking simultaneously. To better isolate their effects, we added three controlled alternatives in Sec. 5.3 / Table 2 (p. 10).

MethodVBSMBSOSENSESUFTP
SpanDMD.995.897.802.890.796.961
Final-prompt-only DMD.986.866.749.864.785.945
Cropped-span DMD.983.871.755.859.783.951
Unmasked per-condition DMD.982.862.742.853.768.933

SpanDMD outperforms all three alternatives. Importantly, unmasked per-condition DMD preserves the same conditioning and full-rollout inputs, isolating the effect of span-specific masking, while cropped-span DMD isolates the benefit of full-rollout temporal context.

W1(b). Training overhead of SpanDMD

In our 60-second training sequence, we perform 12 DMD evaluations per sequence, but only one involves a prompt switch requiring L=2~5 teacher evaluations instead of one. Thus, the overall teacher-evaluation cost increases by only (11+L)/12 = 1.08–1.33 times, not L times. This very limited overhead is training-only, with no additional inference cost.

W1(c). Assuming a teacher with finer-grained text conditioning

This is an insightful question. We would love to see future teachers support accurate per-chunk or per-latent text conditioning! SpanDMD could still provide span-specific supervision with full temporal context, although its additional benefit would need further evaluation. But currently, the bidirectional teachers operate within a 5–15s temporal context under a single text condition, making direct supervision of multiple short-lived prompts difficult. SpanDMD provides a practical solution under these existing teacher constraints.

W1(d). Color transitions with longer durations

We are glad the color-transition visualization was helpful. To examine whether longer durations resolve the issue, we added a 20-second color-transition experiment in Appendix A.9 / Fig. 12 (pp. 27–28), allowing approximately 3–4.5 seconds per post-switch color span.

The original rollout eventually reaches some target colors but still exhibits delayed responses and transient dimming or flicker. In contrast, the SpanDMD-trained generator transitions through intermediate hues more gradually and maintains a more stable appearance. We hope this additional visualization helps address your concern about whether longer transitions resolve these artifacts.

W1(e) / W2. Technical novelty of SpanDMD and SEGUE

We appreciate the reviewer's perspective. Prior works such as Rolling Forcing[1] and RMD[2] have adapted DMD in different ways to address their respective challenges, improving the stability and quality of long-video generation. In interactive streaming generation, however, corner cutting is a common and persistent problem, which we further examine through quantitative and qualitative analyses in Appendix A.1 (pp. 16–17). SpanDMD is specifically designed to address this issue through span-specific supervision, helping the generator faithfully execute intermediate actions during prompt switches. Our results show that it visibly improves transition quality and effectively mitigates corner-cutting failures.

We also agree that temporal planning is becoming increasingly common, as seen in DirecT2V[3] and TempAct[4]. But interactive streaming brings a different challenge: the new prompt tells the model what to do next, but what it has already generated determines how to get there. This motivates state-grounded, dependency-aware planning rather than simply decomposing the target prompt. Together with SpanDMD, SEGUE connects planning what must happen with ensuring that the generator actually executes it, addressing a practical challenge in interactive streaming generation.

We hope this helps clarify the motivation and contributions of our work.

References

[1] Liu et al. Rolling Forcing: Autoregressive Long Video Diffusion in Real Time. 2026.

[2] Gao et al. Rollout-Marginal Distillation for Long-Horizon Autoregressive Video Generation. 2026.

[3] Hong et al. DirecT2V: Large Language Models are Frame-Level Directors for Zero-Shot Text-to-Video Generation. 2023.

[4] Wang et al. TempAct: Advancing Temporal Plausibility in Autoregressive Video Generation via Planner-Executor RL. 2026.

Dear Reviewer zw2S,

Thank you again for your thoughtful feedback. We have provided additional experiments and clarifications to better explain the motivation and contributions of our approach. We hope our responses help address your concerns. If you find them convincing, we would sincerely appreciate your consideration of updating your assessment.

Best regards,

The Authors

Reviewer_sSVDOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses mid-stream prompt switches in streaming video generation, where existing methods tend to jump straight to the new target and skip the intermediate steps a plausible transition needs. The authors propose SEGUE, which combines a training-free VLM planner that inserts short intermediate prompts based on the current frame with SpanDMD, a distillation objective that trains the generator to follow these short-lived prompts. They also introduce OpenTrans-360, a benchmark for evaluating transition quality at prompt switches. SEGUE achieves the best transition scores on this benchmark and competitive results on StreamAV-Bench, and the planner also improves existing generators without retraining.

Critical strengths
  1. The paper identifies a real and underexplored failure mode in interactive video generation. Smooth boundaries do not guarantee a plausible transition, and the benchmark results support this: visual smoothness is uniformly high across methods while old-state exit varies widely.
  2. The method is simple and practical. The planner needs no training, adds little latency, and transfers to frozen generators with consistent gains. SpanDMD is a clean, well-motivated fix for supervising short-lived prompts, and it comes with a sound conditional-score interpretation.
  3. The evaluation is thorough. It covers a new benchmark with reported human agreement, an external benchmark, twelve baselines including commercial systems, a user study, and ablations on each component, visual grounding, and planner context length. The authors are also candid about limitations, including what their ablations do and do not establish.
Critical weaknesses
  1. The novelty of the planner is limited. It is an VLM prompted with a structured template to insert intermediate prompts before the target prompt. This is close to existing temporal prompt planning such as DirecT2V and TempAct, and the main difference is that planning happens at switch time from the current frame. TD-Schema is essentially a prompt design rather than a technical contribution.
  2. The benefit of training with SpanDMD is small. Adding the planner to a frozen LongLive, with no training at all, already gets close to the full method on most transition metrics, for example in table 2 OSE 0.781 vs 0.802 and NSE 0.879 vs 0.890. The ablations make SpanDMD look stronger than it is, because the trained baselines it is compared against are worse than the original untrained LongLive. Most of the gain seems to come from the training-free planner.
  3. Standard benchmarks such as VBench are missing.
Questions for authors

See weekness.

2 comments

We sincerely thank the reviewer for the thoughtful assessment of our work and constructive feedback. We appreciate the concerns regarding novelty, SpanDMD's contribution, and evaluation. We address these points below with additional experiments and clarifications.

W1. Novelty of the planner

We appreciate this perspective. Temporal planning has indeed proven effective in works such as DirecT2V and TempAct, and our work builds on this direction. We focus on interactive streaming generation, where new instructions arrive mid-stream and transitions must account for the realized visual state. Compared with offline generation, this setting is more challenging because past states cannot be revised and new instructions may conflict with ongoing actions. By combining lightweight state-grounded planning with SpanDMD, SEGUE effectively mitigates corner-cutting failures by explicitly planning and faithfully executing plausible intermediate actions.

And the planner is also a simple, plug-and-play module that can be directly applied to generator backbones of different architectures and model sizes to improve transition quality. We hope this clarifies the motivation and contribution of our approach.

W2. Contribution of SpanDMD beyond the planner

We understand why the strong performance of LongLive + planner raises questions about SpanDMD's additional contribution. However, the released LongLive checkpoint was trained on 64 H100 GPUs, whereas SEGUE was trained on 32 A800 GPUs. Since training resources can substantially affect streaming video generation quality, we retrained LongLive using the same training resources and configuration as SEGUE, denoted as LongLive*, and evaluated it with and without the planner.

MethodVBSMBSOSENSESUFTP
LongLive*.973.821.713.851.745.930
LongLive* + planner.986.853.752.864.775.945
SEGUE (full).995.897.802.890.796.961

The planner consistently improves LongLive*, but a clear gap to full SEGUE remains (+0.044 MBS, +0.050 OSE, +0.026 NSE), suggesting that planning alone does not account for the full improvement. We are additionally running a LongLive* + SpanDMD experiment under the same configuration to further isolate the contribution of the distillation objective, and will report this controlled result during the discussion when completed.

To further isolate the contribution of SpanDMD, we also added final-prompt-only DMD, cropped-span DMD, and unmasked per-condition DMD in Sec. 5.3 / Table 2 (p. 10).

MethodVBSMBSOSENSESUFTP
SpanDMD.995.897.802.890.796.961
Final-prompt-only DMD.986.866.749.864.785.945
Cropped-span DMD.983.871.755.859.783.951
Unmasked per-condition DMD.982.862.742.853.768.933

SpanDMD outperforms all three alternatives. In particular, unmasked per-condition DMD keeps the conditioning and full-rollout inputs unchanged, directly isolating the benefit of span-specific masking; cropped-span DMD further isolates the benefit of full-rollout temporal context. These results support the contribution of SpanDMD beyond simply adding a planner.

W3. Evaluation on standard benchmarks

Thank you for this suggestion. We have added VBench-Long + CLIP evaluations in Sec. 5.2 (p. 8) and Appendix A.4 / Table 5 (p. 20). SEGUE ranks first on three of four visual-quality metrics, second on imaging quality, and achieves the highest CLIP score across all five post-switch segments. These results further support SEGUE's effectiveness beyond our proposed benchmark.

We hope these additional results help clarify the distinct contributions of the planner and SpanDMD, and the practical value of SEGUE.

Dear Reviewer sSVD,

We have completed the additional controlled experiment mentioned in our earlier response, where we train LongLive* under the same configuration with SEGUE.

MethodVBSMBSOSENSESUFTP
LongLive*.973.821.713.851.745.930
LongLive* + planner.986.853.752.864.775.945
LongLive* + planner + SpanDMD.993.891.803.886.791.955
SEGUE (full).995.897.802.890.796.961

With the planner retained, introducing SpanDMD alone raises OSE from .752 to .803 and MBS from .853 to .891, with consistent gains across all six metrics. The resulting performance is comparable to full SEGUE, suggesting that SpanDMD contributes meaningfully to transition quality.

We hope this controlled comparison helps address your concerns about SpanDMD's contribution. We welcome any remaining questions, and if you find these additional results convincing, we would be grateful if you could reconsider your assessment.

Best regards,

The Authors

Reviewer_d9tpOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper targets in-stream prompt switching in video streaming. The authors document that current video models "cut corners" by e.g. reaching the new prompt's goal through object duplication or abrupt state jumps. They propose (i) SEGUE a training-free state-grounded planner basically by leveraging inference scale/compute. At each switch, a VLM inspects the last generated frame plus the old and new prompts. It then writes up to 4 prompts to transition between states. They also propose (ii) SpanDMD, a training schedule for streaming distillation. Furthermore, they propose a benchmark, OpenTrans-360. It consists of 360 streams, 1,800 switches and 11 checklist metrics scored by InternVL3.5-38B. They show promising results on their own and previous benchmarks for controlling video streaming.

Critical strengths
  • The problem is well motivated and clearly named, contorlling video streaming is very relveant with Genie4 around the corner. Listed failure modes in Figs 1 and 3 are good motivation. Furthermore, current methods perform very poorly on e.g. boundary smoothness, which is also relevant to data filtering.
  • A training-free planner is a cheap great idea for inference scaling, agnostic to video generation models. This module alone improves on the metrics quite substantially.
Critical weaknesses
  • SpanDMD's central claim is not isolated. Sec. 3.3 and Fig. 2 motivate SpanDMD against two alternatives: final-prompt-only DMD and cropped clip-wise supervision. Neither is evaluated. The only ablation ("w/o SpanDMD") concatenates all prompts into one teacher condition. That changes conditioning and masking at once, not isolatetdly.
  • there is some confound between designing the method and the benchmark. The results are promising but desiging a benchmark and method hand in hand leaves it unclear how competitors clearly score on this new benchmark.
  • while overall headlines read great and bold, the underlying numbers are often brittle: e.g. 0.887 vs 0.866 without CI
Questions for authors
  • is there a way to build a fairer comparison with sota methods on this novel benchmark?
4 comments

We thank the reviewer for the positive assessment and constructive suggestions. We added a controlled SpanDMD ablation and clarified the fairness and statistical robustness of our evaluation.

W1. Isolating the contribution of SpanDMD

The original ablation indeed changed conditioning and masking simultaneously. We have now added the two requested alternatives, final-prompt-only DMD and cropped-span DMD, together with an additional unmasked per-condition DMD baseline in Sec. 5.3 / Table 2 (p. 10).

MethodVBSMBSOSENSESUFTP
SpanDMD.995.897.802.890.796.961
Final-prompt-only DMD.986.866.749.864.785.945
Cropped-span DMD.983.871.755.859.783.951
Unmasked per-condition DMD.982.862.742.853.768.933

The unmasked per-condition DMD baseline keeps the same per-stage conditioning and full-rollout input as SpanDMD but removes span-specific residual assignment. Restricting each residual to its assigned span improves OSE from 0.742 to 0.802 and MBS from 0.862 to 0.897, directly isolating the benefit of span masking. Compared with cropped-span DMD, retaining the full rollout as score-network context improves OSE from 0.755 to 0.802 and NSE from 0.859 to 0.890. We also expanded the SpanDMD explanation and added a direct vanilla-DMD-vs-SpanDMD comparison in Sec. 3.3 / Fig. 3 (pp. 5–6).

W2. Potential confound between the method and benchmark

We understand the concern about developing the method and benchmark together. We clarified in Sec. 4 and Sec. 5.1 that OpenTrans-360 evaluates the generated state transition itself, rather than whether a method follows a specific planning mechanism.

SEGUE and most trainable baselines use transition prompts derived from VidProM [1], exposing them to broadly similar training scenarios. OpenTrans-360 is constructed independently from structured combinations of subjects, settings, and events, expanded into six-stage prompt streams and filtered through automatic checks and human inspection. Its test prompts are disjoint from the VidProM training pairs. Moreover, the 11 evaluation metrics show strong agreement with human judgments (mean method-level Spearman 0.87; Appendix A.12, Table 17, p. 33).

To further address concerns about benchmark-specific bias, we also evaluated SEGUE on two external benchmarks, StreamAV-Bench and VBench-Long, together with CLIP scores (Sec. 5.2, p. 8; Appendix A.4–A.5, p. 20), and observed similarly strong performance.

W3. Statistical robustness of the reported gains

We appreciate this concern and have added paired 95% case-bootstrap CIs in Sec. 5.2 (p. 8), resampling complete prompt streams to preserve dependence among switches.

MetricMBSOSENSEScPSUFTPOverall
Improvement+.022+.019+.027+.017+.018+.016+.021
95% CI[.019, .026][.015, .027][.022, .035][.014, .022][.013, .025][.012, .022][.017, .031]

The confidence intervals are strictly positive, including the Overall improvement of +.021, suggesting that the improvements are statistically reliable across these transition metrics.

We also report relative margins in Table 1 (p. 8). SEGUE's 2.42% Overall improvement over the second-best method is substantially larger than the 0.23% gap between the second- and third-best methods.

Q1. Fairer comparison with SOTA methods

We agree that this is especially important for a newly introduced benchmark. Beyond the controlled OpenTrans-360 comparison discussed above, we also evaluated on 2 external benchmarks.

On StreamAV-Bench, SEGUE ranks first on four of six instruction/response metrics, as reported in Sec. 5.2 (p. 8) and Appendix A.5 / Table 6 (p. 20), including a 7.1% improvement in PVC. We additionally added VBench-Long + CLIP results in Appendix A.4 / Table 5 (p. 20), where SEGUE ranks first on three of four visual-quality metrics and achieves the highest CLIP score across all five post-switch segments.

We hope these additional experiments and evaluations help address your concerns and strengthen the evidence for our approach.

References

[1] Wenhao Wang and Yi Yang. VidProM: A Million-scale Real Prompt-Gallery Dataset for Text-to-Video Diffusion Models. NeurIPS, 2024.

Dear Reviewer d9tp,

Thank you again for your constructive feedback and positive assessment of our work. As the discussion phase approaches its end, please let us know if any questions or concerns remain. We would be happy to provide further clarification and sincerely appreciate your continued support.

Best regards,

The Authors

thanks for addressing my concerns, I will maintain my positive score and suggest acceptance.

Thanks so much for your positive feedback and support! We're really glad our rebuttal helped address your concerns. We truly appreciate the time you took to review our work and your recommendation for acceptance, even though we've now withdrawn the submission. Thanks again, and all the best!

Reviewer_imi2OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposed a SEGUE framework that

  1. explicitly plans transition trajectories uses an existing VLM as a planner to generate a few segue prompts to address the transition problems for mid-stream interactions for video generation.
  2. the planner's prompts are governed by a Transiiton Dependency Schema (TD Schema) designed by the authors
  3. a span-based technique called SpanDMD modifies standard Distribution Matching Distillation (DMD) introduces the complete noised rollout for global context for each local span.

The framework is demonstrated to be effective both quantatitvely and qualitatively on the purposely built benchmark OpenTrans-360.

Critical strengths
  1. The introduction of a pre-trained VLM as transition planner to generate segue prompts.
  2. The trained SpanDMD to take into account the global context (predicted over noisy roll-out).
Critical weaknesses
  1. No justification of the use of JoyAI-VL-Interaction over other VLMs.
  2. For lines 192-194, where the failures of corner cutting is unsupported, how do you know the undesired artifacts are introduced by instruction switching and not an intrinsic problem of the video generator.
  3. Math notations in 3.1 need to double check about clearity. For example, on Line 201 \rok\ro_{k} is introduced without the definition of roles. Also the cached state, realised state, conditioning prompt are all a bit confusing without alignemtn to the video generation context, how it is related to the frames, blocks, and what are the inputs to the DMD. A detailed standard DMD diagram would help, with the math notations introduced, and then transition to SpanDMD. Currently the diagram in Figure 2 is good for overall pipeline description, but the detailed digram would be good at introducing these formal math notations.
Questions for authors
  • Why JoyAI-VL-Interaction? How would your performance change if we vary the VLM?

  • The biggest gain in performance is OSE in Table 2, other metrics are all minimal - potentially satistically insignificant. Moreover, LongLive + Planner performing across the six metrics similar to SEGUE, any explanation on the key benefits of SEGUE that are not reflected by the metrics would strengthen the claims of better performance.

3 comments

Thank you for the thoughtful review and for recognizing the value of the state-grounded planner and SpanDMD. We have revised the paper substantially and added several targeted experiments.

W1 / Q1. Choice of planner VLM

We added a controlled VLM sensitivity study in Appendix A.6, Table 7 (p. 21), keeping the generator, TD-Schema, compiler, transition budget, and evaluation cases fixed. Performance is largely insensitive to the planner choice, while JoyAI-VL-Interaction provides the best quality–latency trade-off.

Planner VLMLatency (s)SPFSMCSPPVBSMBSOSENSESPScPSUFTP
JoyAI-VL-Interaction-INT40.44–0.850.9090.8690.9920.9950.8970.8020.8900.7830.8630.7960.961
Qwen3-VL-2B-Instruct4.81–9.820.8840.8610.9900.9930.8900.7940.8840.7810.8580.7910.961
Qwen3-VL-8B-Instruct5.40–10.630.9040.8660.9890.9960.8940.8050.8870.7830.8600.7930.958
InternVL3.5-8B2.65–4.580.8990.8670.9920.9950.8960.8000.8860.7780.8610.7890.956

Across the four VLMs, the mean per-metric variance is only 1e-5, while JoyAI-VL-Interaction ranks first or ties for first on 9/11 metrics and is substantially faster. We therefore use JoyAI as the default planner for efficiency.

W2. Corner cutting vs. intrinsic generator limitations

We recognize the importance of distinguishing intrinsic generator limitations from failures aggravated by mid-stream prompt switching. To disentangle these effects, we added a 100-case study in Appendix A.1, Table 3 and Fig. 7 (pp. 16–17), comparing the same event transitions when both events are known from the outset versus when the second event is introduced online.

MethodMBSOSENSESUFTP
Wan2.1-1.3B0.8850.7910.8840.8070.962
LongLive-1.3B (no switch)0.8790.7860.8790.7950.955
MemFlow-1.3B (no switch)0.8710.7700.8710.7860.939
LongLive-1.3B (switch)0.8310.7550.8590.7680.943
MemFlow-1.3B (switch)0.8510.6660.8150.6900.861
SEGUE-1.3B (Ours)0.8950.8080.8950.8140.968

For the same AR generators, corner cutting is substantially reduced when both events are specified from the outset. Fig. 7 (pp. 16–17) further shows that online switching can cause mug duplication or abrupt subject appearance, indicating that these failures are not solely intrinsic to the generator; SEGUE mitigates them under online updates. We revised Sec. 3.1 (p. 4) accordingly.

W3. Clarification of notation and DMD formulation

We apologize that these definitions were not sufficiently clear in the original submission. We rewrote Sec. 3.1 (p. 4) to explicitly define generation blocks, generated history, KV cache, realized state, the decoded boundary frame, and the role set.

We also expanded Sec. 3.3 (pp. 5–6) to introduce standard DMD before SpanDMD and added a new Fig. 3 (p. 6) directly comparing vanilla DMD and SpanDMD. The figure shows that vanilla DMD supervises the full rollout under one condition, whereas SpanDMD evaluates the same full rollout under each scheduled condition and applies each residual only to its assigned span.

Q2. Statistical significance and the benefit beyond LongLive + planner

We appreciate this concern. Since the transition metrics are already relatively high and tightly clustered among strong baselines, small absolute differences can still represent meaningful improvements. To make the effect size more interpretable, we now compare our gain over the second-best method with the corresponding second-to-third gap, and additionally report paired 95% case-bootstrap CIs in Table 1 / Sec. 5.2 (p. 8).

Metric2nd vs. 3rdSEGUE vs. 2nd
MBS+0.34%+2.51%
OSE+0.26%+2.43%
NSE+0.12%+3.13%
Overall+0.23%+2.42%

Because the metrics are reported on a 0–1 scale, the absolute differences may look small at first glance. Relative to the much narrower gaps among the strongest baselines, however, the improvements are substantial.

Regarding the concern that LongLive + planner appears close to SEGUE, we note that the released LongLive checkpoint and SEGUE were obtained under different training-compute settings: the former was trained on 64 H100 GPUs, whereas SEGUE was trained on 32 A800 GPUs. This difference is non-trivial for streaming video training, where model quality is sensitive to both the available compute budget and hardware efficiency. To control for this factor, we retrained LongLive using the same training resources and configuration as SEGUE, denoted as LongLive*, and reevaluated it with and without the planner.

MethodVBSMBSOSENSESUFTP
LongLive*0.9730.8210.7130.8510.7450.930
LongLive* + planner0.9860.8530.7520.8640.7750.945
SEGUE (full)0.9950.8970.8020.8900.7960.961

The planner still consistently improves LongLive*, but a clear gap to full SEGUE remains, e.g. +0.044 MBS, +0.050 OSE, and +0.026 NSE, supporting the complementary role of SpanDMD in executing short-lived transition conditions. We are additionally running a matched LongLive* + SpanDMD experiment under the same configuration to further isolate the contribution of the distillation objective, and will report this controlled result during the discussion when completed.

We also recognize that aggregate metrics do not tell the whole story. In our blinded user study (Appendix A.2, Table 4), SEGUE is preferred over all 12 baselines overall and shows stronger transition naturalness. We also evaluated SEGUE on external benchmarks. It ranks first on four of six instruction/response metrics on StreamAV-Bench, including a 7.1% gain in PVC, and ranks first on three of four visual-quality metrics on VBench-Long + CLIP (Appendix A.4, Table 5), while achieving the highest CLIP score across all five post-switch segments.

We hope these additional results help clarify our contributions and address your concerns.

Dear Reviewer imi2,

As promised, we have completed the promised LongLive* + planner + SpanDMD experiment under the same training configuration.

MethodVBSMBSOSENSESUFTP
LongLive*.973.821.713.851.745.930
LongLive* + planner.986.853.752.864.775.945
LongLive* + planner + SpanDMD.993.891.803.886.791.955
SEGUE (full).995.897.802.890.796.961

Adding SpanDMD to LongLive* + planner further improves all six transition metrics, particularly **MBS (+.038), OSE (+.051), and NSE (+.022).

We hope this additional controlled comparison helps address your concerns about the benefits of SpanDMD beyond the planner. We would be happy to clarify any remaining questions, and if these results have addressed your concerns, we would sincerely appreciate your consideration of updating your assessment.

Best regards,

The Authors

Comments on the paper

2 comments

Dear Program Chairs, Area Chairs, and Reviewers,

We sincerely thank you for your time and effort in reviewing our submission and for the constructive feedback provided during the review process.

After careful consideration, we have decided to withdraw our submission from ICLR 2027.

We greatly appreciate your valuable comments and suggestions, which will help us improve our work.

Thank you again for your consideration and support.

Best regards,

The Authors

Thanks for your kind reply. We're glad the additional experiments helped address most of your concerns about SpanDMD. We totally see your point about novelty.

Planning and DMD are definitely not new ideas, but what we found interesting was the gap between planning a reasonable transition and actually getting the generator to follow it. Even with a good transition plan, the generator can still skip intermediate steps, which is what motivated us to look more closely at the training objective. And actually, running these additional experiments also helped us better understand what the planner and SpanDMD each bring to the table.

We'd still love to hear more about the novelty concerns you were planning to share, and it'd be great to discuss them further! Thanks again for taking the time to share your thoughts. We really appreciate it!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.