acceptodds
Under review as a conference paper at ICLR 2027

Resilient Streaming Video Editing

14 views

Abstract

Existing streaming video editors can exhibit ruminative behavior, over-relying on historical frames even as errors accumulate in the generation history. To address this limitation, we present JoyAI-Video-Edit, a resilient 16B-parameter autoregressive diffusion model for source-faithful, low-latency streaming editing. Specifically, we introduce Resilient-DMD, which selectively strengthens teacher-side source guidance during distillation when relative source attention falls below matched baselines, yielding a two-step generator. Long-Horizon Autoregressive Distillation (LHAD) extends this adaptive supervision to deeper rollout states while bounding activation memory. Our streaming engine combines bounded transformer state, single-frame VAE state handoff, and asynchronous execution to reduce end-to-end overhead. To support training and sustained-stream evaluation, we construct an editing data pipeline and introduce LongV2VBench, built on one-minute videos. Automatic and human evaluations show that JoyAI-Video-Edit outperforms the evaluated streaming baselines and remains competitive with strong offline editors. The system delivers end-to-end editing at 34.87 FPS on a single B200 GPU, with support for RTX 5090 and RTX PRO 6000.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Real-Time Video Editing, Autoregressive Diffusion

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 4 reviews, above 85% of the others.
Reviewer_UyuWOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper presents JoyAI-Video-Edit, a 16B-parameter autoregressive model for streaming video editing. Its main algorithmic component, Resilient-DMD, monitors source-versus-history attention and adaptively increases teacher-side source guidance when source attention falls below matched moving baselines. Long-Horizon Autoregressive Distillation extends training to deeper generated histories through forward-reverse source traversal and segmented optimization. The paper also develops a bounded-state asynchronous inference engine and introduces LongV2VBench, containing 229 one-minute editing instances. Evaluations report improvements over streaming baselines, competitive editing quality relative to offline models, and 34.87 FPS steady-state end-to-end throughput at 720p on a single B200.

Critical strengths
  1. The paper addresses a concrete failure mode specific to streaming editing: generated history can override changes in the current source. The proposed teacher-side guidance mechanism is practical because it introduces no monitoring or additional guidance branches at inference. The component ablations and comparison against the best tested static-guidance baseline provide meaningful evidence that adaptive source guidance improves editing quality.

  2. The work combines algorithmic improvements with a substantial deployment implementation. It reports full-pipeline throughput, chunk-latency distributions, component-level optimizations, and quality changes under deployment precision settings. The evaluation spans an existing benchmark, a new minute-long benchmark, qualitative comparisons, and human preferences, supporting the practical value of the resulting editor.

Critical weaknesses
  1. It is unclear whether low source attention reliably identifies editing errors. The appropriate reliance on the source can differ between, for example, recoloring an object and replacing an entire background, but the EMA baseline does not account for the editing task. Figure 9 shows that dynamic guidance works better than the tested fixed weight; it does not show whether the attention signal assigns stronger guidance to the right chunks. The authors could address this by measuring how well the signal predicts source-following errors across editing categories, or by shuffling guidance weights among samples within matched noise/depth bins. The latter would test whether matching the weight to the sample matters beyond the weight distribution itself.

  2. The minute-long benchmark shows better average editing quality, but does not directly measure recovery from history errors. The human study also uses clips shorter than ten seconds. A focused comparison against static guidance on events such as subject disappearance, measuring how long phantom content persists, would better support the resilience claim.

  3. The main streaming baselines with reported model sizes use 1.3B–2B parameters, whereas JoyAI-Video-Edit uses 16B parameters and a substantial foundation-training pipeline. This makes it difficult to attribute the cross-model quality gains to the proposed methods rather than model capacity and training data. The same-backbone ablations are helpful, but a smaller-scale implementation or a capacity-matched streaming baseline would make the comparison more convincing.

Questions for authors

How reliably does the attention score identify chunks that benefit from stronger source guidance? Does the same controller configuration work across editing categories and scene types, or does it require substantial manual tuning of the monitored layers, EMA settings, and guidance parameters?

3 comments
AuthorsOpenReview ↗

Response to Reviewer UyuW (1/1)

We thank you for the valuable feedback. Our responses below address each concern through controlled experiments and configuration details.

q1/w1: How reliably does the attention score identify chunks that benefit from stronger source guidance?

  1. Attention reliability and guidance allocation: As shown in Tables 13 and 14 in our response to Reviewer NFd5, we evaluate both source-following error prediction across editing categories and guidance allocation through random reassignment. The category-balanced AUROC of 0.764 and the 0.41-point drop in Overall after reassignment support the predictive value of attention feedback and the benefit of applying stronger source guidance to the appropriate chunks. Furthermore, we share the EMA reference across tasks, while both teacher branches retain the editing instruction to define task-conditioned source correction, as shown in Eq. 4.

w2: Recovery from history errors and phantom persistence

  1. History error recovery: We define response delay as the interval from the subject's disappearance in the source video to the disappearance of the corresponding phantom in the edited output. As shown in Table 16 in our response to Reviewer NFd5, the maximum delay is 0.3 s with Resilient-DMD, while phantom content persists for at least 4 s without it. These measurements support rapid source following and suppression of persistent history artifacts.

  2. Dynamic versus static recovery: Both static and dynamic guidance provide source-conditioned correction. On the same source-change cases, dynamic guidance reduces the maximum response delay from 1 s (three chunks) to 0.3 s (within one chunk) (Table 19), supporting the benefit of adapting correction strength to each chunk.

    CasesDynamicStatic
    Substantial source changes0.3 s maximum (within 1 chunk)1 s maximum (3 chunks)

    Table 19: Response delays under dynamic and static guidance after substantial source changes.

  3. Full-video human evaluation: As shown in Table 15 in our response to Reviewer NFd5, human evaluation of complete one-minute and five-minute videos yields win rates of 88–94% and 98–99%, respectively, for our method against the three streaming baselines. These results support sustained editing quality over the evaluated durations.

w3: Attributing cross-model quality gains to methods rather than model capacity and training data

  1. Capacity and data controls: As clarified in our responses to Reviewers 5ibZ and NFd5, Table 4 and Fig. 9 show improvements at a fixed 16B capacity, while Table 17 controls training compute and inherited pretraining through shared-checkpoint comparisons. These results support the contributions of Resilient-DMD and LHAD. Smaller models for edge devices, including smartphones, are also under development as future work.

q2: Does the same controller configuration work across editing categories and scene types without substantial manual tuning?

  1. Shared controller configuration: We use one controller configuration during distillation and the same distilled checkpoint across all editing categories and tested scenes. Since monitoring and guidance adaptation operate only during training, inference requires no task- or scene-specific controller tuning. Table 2 reports per-task performance, while Figure 20 and Tables 15 and 16 in our response to Reviewer NFd5 support sustained editing quality and source responsiveness.

  2. Monitoring layer selection: No specific choice of monitoring layers is required. Our layer-selection ablations showed 0.8% variation in final scores. We therefore use all layers for monitoring in the final configuration. Monitoring reuses the generator's existing queries and keys and adds no forward or backward model passes.

  3. EMA configuration: We use fixed EMA settings across tasks, without task-specific tuning: β=0.99, two exit-step noise bins, and three rollout-depth bins. Each EMA entry is initialized from its first available attention statistic and updated after each rollout. An entry is used for adaptive normalization once it has accumulated 24 attention measurements.

  4. Guidance configuration: We selected the guidance parameters through a standard hyperparameter search and reuse the same settings across tasks and in both short-rollout and LHAD training, without task-specific retuning. Table 5 and Fig. 10 report sensitivity results and guidance trajectories under alternative settings.

I thank the authors for the detailed response and for conducting the suggested controls.

The AUROC analysis and the guidance weight shuffling experiment satisfactorily demonstrate the predictive utility of the attention signal. Additionally, the response delay measurements provide concrete quantitative evidence regarding error recovery and phantom content suppression. While the capacity difference relative to external streaming baselines remains a contextual caveat, the same-backbone ablations and the new validation results adequately address my primary concerns.

Overall, the response clarifies the open questions, and I maintain my rating of 3 (Weak Acceptance). Please ensure that all newly presented experimental results—particularly the attention reliability tests and the response delay metrics—are fully integrated into the final manuscript.

AuthorsOpenReview ↗

Response to Reviewer UyuW

Dear UyuW,

Thank you very much for your positive feedback. We are glad that our responses have addressed your concerns, and we will incorporate all these revisions into the final manuscript.

Best regards,

Paper 6937 Authors

Reviewer_6XTCOpenReview ↗
Rating
3
Confidence
2
Informativeness
1
Presentation
1
Soundness
3
Read review
Paper summary

The authors present JoyAI-Video-Edit, a 16B-parameter autoregressive diffusion editor.

Then lines 57-67 from the paper's introduction are a dense set of details describing the details of their diffusion editor. There are many small contributions: Teacher-side source guidance during distillation. Comparing the generator’s attention to source and history tokens with moving baselines matched by layer, noise level, and rollout depth. When relative source attention falls, the teacher increases its source-conditioned correction for that chunk. This control is used only during training They further introduce Long-Horizon Autoregressive Distillation (LHAD) to bring adaptive supervision to deeper generated states. LHAD divides extended rollouts into seg- ments, reuses detached history, and accumulates gradients before an optimizer update. They designed a bounded-state asynchronous engine. A fixed transformer history window and static KV pool bound temporal memory; FP8 linear layers, fused operators, and CUDA graphs reduce execution overhead.

The abstract and introduction (from which the above text was taken) read like computer code translated into English. But computer code may be easier to read than the English of this paper, since the expectations would match the text that the reader encountered.

Critical strengths

This paper does perform well on the benchmarks of tables 1 and 2.

Critical weaknesses

The paper reads like computer code. And at the level of detail of a low-level computer language. It's not clear what the reader learns from the manuscript.

No variance is reported for quality scores. Many conclusions rest on differences of 0.02–0.06, e.g., the “limited sensitivity to K” claim (L. 462–463) and the FP8 quality claim (L. 961–965). Quality scores have no error bars or seeds, even though timing results do.

Questions for authors

none

1 comment
AuthorsOpenReview ↗

Response to Reviewer 6XTC (1/1)

We appreciate your careful review and constructive feedback. Below, we clarify the motivation and contributions of our work and provide additional analyses to assess the reliability of the reported results.

w1: Presentation clarity and what the reader learns from the manuscript

  1. Motivation and contributions: We apologize for the confusion caused by our presentation. Beyond mitigating exposure bias, streaming video editing must remain responsive to the evolving source. Our work is motivated by ruminative behavior: overreliance on generated history can keep a subject visible after it disappears from the source, as shown in Fig. 1. This raises the question of how to preserve continuity and consistency while following source changes without propagating history errors. We address this with Resilient-DMD for selective source correction, LHAD for training on deeper generated histories, and a bounded-state asynchronous engine for sustained 16B streaming in real-world user scenarios (Fig. 14). Under matched training conditions, dynamic correction improves Overall from 3.23 to 3.61 in Fig. 9. LHAD achieves 3.61, versus 3.38 and 3.35 for short-rollout training at matched GPU-hours and reference DiT compute, respectively, as shown in Table 17. Our engine delivers 34.87 FPS at 720p on a single B200. The revised abstract and introduction clarify our motivation, contributions, and supporting evidence, while implementation details remain in the method section and appendix.

w2: Quality scores lack variance, error bars, and seeds

Thank you for your suggestion. In our paper, we follow a VLM-based evaluation approach, as used in prior video-editing studies [1–3]. To quantify variability in quality scores, we re-evaluated the K ablations and deployment optimizations on LongV2VBench using 4 random seeds (42, 928797, 143836, 4753). All settings used the same seeds and benchmark cases, with consistent judge settings. Table 18 reports mean ± SD across repeated evaluations; error bars denote one sample standard deviation. The small observed score changes across the tested K values and within each deployment comparison support the stability of the corresponding ablation conclusions. We have incorporated the re-evaluated means and standard deviations into the ablation results in the revised manuscript.

ComparisonSettingLongV2VBench Overall (mean ± SD)
K sensitivityK=13.36 ± 0.0445
K sensitivityK=23.38 ± 0.0397
K sensitivityK=43.33 ± 0.0692
B200, 720×1248BF16, accumulation off3.62 ± 0.0433
B200, 720×1248FP8, accumulation off3.61 ± 0.0376
RTX 5090, 480×840FP8, fast accumulation off3.55 ± 0.0701
RTX 5090, 480×840FP8, fast accumulation on3.50 ± 0.0364

Table 18: Quality-score variability on LongV2VBench.

[1] OpenVE-3M: A Large-Scale High-Quality Dataset for Instruction-Guided Video Editing (2025).

[2] Kiwi-Edit: Versatile Video Editing via Instruction and Reference Guidance (2026).

[3] Bernini: Latent Semantic Planning for Video Diffusion (2026).

Reviewer_5ibZOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces JoyAI-Video-Edit, a 16B-parameter autoregressive diffusion model that converts a bidirectional video editor into a chunk-wise streaming editor with bounded temporal state. The authors proposed Resilient-DMD, a distillation scheme that monitors relative source attention against EMA baselines and adaptively scales the teacher's source-conditioned correction; Long-Horizon Autoregressive Distillation (LHAD), which segments extended rollouts and accumulates gradients to expose the model to self-generated error histories; and a bounded-state asynchronous inference engine using a fixed KV window, single-frame VAE handoff, FP8 and CUDA graphs. The authors also build a paired-data pipeline and LongV2VBench. Reported results show 3.72 overall on OpenVE-Bench and 3.61 on LongV2VBench with 34.87 FPS end-to-end at 720p on a B200.

Critical strengths
  1. The adaptive source-anchoring mechanism is empirically distinguished from alternatives. The diagnostic in Fig.9 separates dynamic anchoring from both "none" and "static" anchoring, reaching 3.61 while reducing the attention-ratio IQR from 0.15 to 0.11. This indicates that the effect is not merely a stronger fixed guidance weight.
  2. Table 4 isolates Resilient-DMD and LHAD and shows combined gains concentrated in the Local metric, which is consistent with the claimed long-horizon benefit. Table 11 provides per-configuration DiT latency/speedup, and Tables 7–8 and 12 detail pipeline, VAE, and multi-GPU scaling.
Critical weaknesses
  1. Attribution of gains to the adaptive controller is confounded by the training budget. Resilient-DMD adds a teacher-guidance branch and monitoring; the natural control is a matched-compute run with a fixed λ\lambda at equal wall-clock or steps. The "static" baseline in Fig.9 is described only as "the best fixed weight tested," with no indication of whether it received the same training duration or tuning effort. Does the dynamic controller's advantage survive a budget-matched comparison?
  2. The closest streaming baselines are small (1.3B–2B), while the proposed model is 16B. Table 1 shows StreamDiffusionV2 and LiveEdit at 1.3B and SANA-Streaming at 2B. There is no parameter-matched streaming comparison or compute-controlled ablation. Therefore, it is hard to distinguish whether performance improvements stem from the algorithm or model scaling.
Questions for authors
  1. For the static-anchoring comparison in Fig.9, was the fixed λ\lambda selected with the same hyperparameter search budget (and training steps) as the dynamic controller? It would be better to report a budget-matched control.
  2. Table 2 compares against 1.3B–2B streaming models at 480p/704p, while the introduced model runs at 720p. What is the score for the strongest streaming baseline when run at 720p, or for your model at 480p/704p?
  3. Does the bounded-state engine's memory and per-chunk latency remain flat beyond the one-minute horizon (e.g., at 5–10 minutes)?
1 comment
AuthorsOpenReview ↗

Response to Reviewer 5ibZ (1/1)

We appreciate your focus on fair comparisons and sustained streaming performance. Below, we address each concern regarding training budgets, model capacity, resolution, and long-duration operation.

q1/w1: Hyperparameter search budgets and training steps for static and dynamic anchoring

  1. Hyperparameter search budget: The static and dynamic variants use comparable total search budgets and GPU-hours per training run. We selected the static baseline by screening source-guidance strengths using Stage-1 inference on randomly sampled user cases, then training fixed-λ candidates under the same training recipe and choosing the best-performing one (Fig. 9).

  2. Matched training conditions: The static and dynamic variants in Fig. 9 are matched in training steps, duration, data recipe, samples, and cumulative data exposure.

  3. Teacher and monitoring overhead: Both variants use the same teacher source-correction branch (Eq. 4). Monitoring reuses existing generator queries and keys without additional model passes or backpropagation, and its overhead is included in the matched training budget.

  4. Benefits under matched budgets: With comparable search budgets and matched training conditions, dynamic guidance outperforms static guidance by 0.38 points in Overall score (3.61 vs. 3.23 in Fig. 9).

w2: Distinguishing algorithmic improvements from model scaling

  1. Model capacity clarification: Model capacity contributes to task-specific quality, while our proposed algorithm plays a key role in sustaining source grounding over long videos. At a fixed 16B capacity, our method improves Overall from 2.81 to 3.61, as shown in Table 4. Achieving strong editing quality together with high streaming throughput at this scale is challenging. Our engine delivers 34.87 FPS on a single B200 at 720×1248, exceeding the 14.51 FPS of the evaluated 2B SANA-Streaming baseline in Table 2. The experiments in Figures 9 and 10 and Tables 4 and 5 use the same backbone under controlled conditions; Table 3 keeps the model fixed while varying deployment hardware. We are also developing smaller models for edge devices, including smartphones, as part of our future work.

q2: Scores for the strongest streaming baseline at 720p or our model at 480p/704p

  1. Resolution explanation: As shown in Table 10, our model at 480×840 with FP8 achieves Overall scores of 3.60 on OpenVE-Bench and 3.54 on LongV2VBench, exceeding the strongest evaluated streaming baselines by 0.98 and 1.83 points, respectively. Our VAE's 24× spatial downsampling requires input dimensions divisible by 24, so we cannot directly evaluate at 704p. As shown in Tables 1 and 2, SANA-Streaming at 704×1280 and our model at 720×1248 have essentially the same total pixel count, despite the different aspect ratios used during training.

q3: Memory and per-chunk latency beyond the one-minute horizon

  1. Memory and latency stability: Our fixed history window, static KV pool, and bounded VAE state and queues keep retained temporal memory and per-chunk workload independent of stream duration under fixed settings, as described in Section 3.4 and Appendix C. Measurements in 1-, 5-, and 10-minute live sessions on RTX PRO 6000 showed that both GPU memory usage and per-chunk latency remained stable after warm-up. In real-world live-streaming tests, including 1-minute and open-ended sessions such as the example in Fig. 14, total display latency, including network communication and other overhead, remained stable at approximately 600 ms. Logged backend processing latencies were consistent with the P50/P95 values of 573.92/578.75 ms at 720×1248 and Q=1 reported in Table 12.
Reviewer_NFd5OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses overreliance on generated history in streaming video editing, which can prevent models from following changes in the current source video. It proposes JoyAI-Video-Edit, an autoregressive diffusion editor trained with Resilient-DMD, which selectively strengthens teacher-side source guidance when relative source attention falls below matched baselines. Long-Horizon Autoregressive Distillation (LHAD) extends training rollouts while bounding activation memory. The system also combines efficient cache management with asynchronous execution. Experiments on OpenVE-Bench and the proposed LongV2VBench show strong editing quality and efficient streaming inference.

Critical strengths
  1. The paper addresses a meaningful failure mode with a well-motivated training approach. The persistence of previously generated content after it disappears from the source is an important problem in streaming editing. Selectively strengthening source guidance provides a reasonable way to balance temporal consistency and responsiveness to the source. Table 4 supports the complementary benefits of Resilient-DMD and LHAD, while Figure 9 shows an advantage over the tested fixed-guidance baseline.
  2. The work combines strong editing performance with practical system design. The model performs favorably against the evaluated streaming baselines and remains competitive with strong offline editors. The paper also addresses execution overhead across the pipeline, including VAE processing, cache management, and asynchronous scheduling. The appendix provides useful analyses of deployment efficiency and the quality impact of inference optimizations.
Critical weaknesses
  1. The reliability of source attention as an indicator of editing errors remains insufficiently validated. Figure 9 demonstrates improvements in overall editing quality, but does not directly establish whether reduced source attention identifies problematic chunks. Lower source attention may reflect either legitimate editing transformations or excessive reliance on history, and spatial averaging may obscure localized errors. The effectiveness of dynamic guidance is supported more clearly than the proposed explanation of how the monitor detects failures.
  2. The evaluation provides limited evidence for stability over extended streaming durations. LongV2VBench evaluates one-minute videos, while the human evaluation in Appendix E.3 uses single-shot videos shorter than ten seconds. These human preferences therefore do not directly validate long-term stability. Aggregate scores also provide limited insight into when errors accumulate or how quickly the model recovers. Since LHAD extends training videos through forward–reverse traversal, its effectiveness on longer, non-repeating content warrants further investigation.
Questions for authors
  1. Could the authors analyze whether normalized source attention correlates with actual editing errors at the chunk level, particularly during subject exits, reappearances, or rapid motion? A control that randomly reassigns guidance coefficients across chunks while preserving their average strength would also help establish the importance of applying stronger guidance to the appropriate chunks.
  2. Could the authors evaluate longer, non-repeating videos and report how editing quality and source fidelity change over time? Measuring recovery after substantial source changes, together with human evaluation of complete long videos, would strengthen the long-term stability claim. A comparison with and without LHAD under matched training compute would also help isolate its contribution.
  3. Could the authors provide video comparisons with the evaluated baselines, along with ablation videos showing results with and without Resilient-DMD and LHAD? These would help assess temporal consistency, source fidelity, and error recovery beyond the selected frames shown in the paper.
4 comments
AuthorsOpenReview ↗

Response to Reviewer NFd5 (1/2)

Thank you for the constructive comments and valuable suggestions. We address your questions and concerns below with supporting analyses and complete video comparisons.

q1/w1: Normalized source attention, chunk-level editing errors, and guidance allocation

  1. Attention reliability: We analyze 8,950 chunks from 50 baseline-editor videos (3,000 s), covering subject exits and reappearances, shot changes, occlusions, and subject and camera motion across 5 editing scenarios. These changes can expose history-induced artifacts such as phantom content and structural distortion (Fig. 1). We compare rˉi\bar r_i, normalized against baselines matched by layer, noise level, and rollout depth, with human annotations of artifacts in the same generated sequences. AUROC is computed within each video and averaged within each editing category; equal weighting of the five category means yields 0.764 overall (Table 13). This supports an overall association between lower relative source reliance and editing artifacts. Guidance ablations separately evaluate the benefit of adaptive correction (Fig. 9; Table 14).

    Editing categoryMean within-video AUROC
    Overall0.764\mathbf{0.764}
    Global style0.734[0.523, 0.882]0.734_{[0.523,\,0.882]}
    Background change0.744[0.489, 0.974]0.744_{[0.489,\,0.974]}
    Local modification0.778[0.676, 0.880]0.778_{[0.676,\,0.880]}
    Local addition0.791[0.594, 0.974]0.791_{[0.594,\,0.974]}
    Local removal0.772[0.584, 0.933]0.772_{[0.584,\,0.933]}

    Table 13: Mean within-video AUROC by editing category; Overall equally weights the five category means. Subscripts show available 95% CIs from 5,000 video-level bootstrap resamples.

  2. Spatial averaging clarification: Deep query representations incorporate spatial context through self-attention, so a visually localized artifact can accompany changes in source reliance across multiple query positions. Spatial averaging retains coherent changes in these dependencies. When the remaining queries are stable and the affected proportion is fixed, averaging rescales the signal while preserving its ordering. Moreover, a visually localized artifact is not necessarily confined to a few contextualized query representations, so its associated changes in relative source reliance can remain observable after spatial averaging.

  3. Random guidance reassignment: We randomly reassign source-guidance coefficients across training chunks while preserving their mean strength, disrupting the controller's chunk-specific allocation. This mismatched allocation yields an Overall score of 3.20, which is 0.41 points lower than dynamic guidance (3.61) and 0.03 points lower than static guidance (3.23) (Table 14). This supports matching source-guidance strength to the appropriate chunks.

    MetricDynamicStaticRandom reassignmentNone
    Overall score3.613.23 (-10.53%)3.20 (-11.36%)3.19 (-11.63%)

    Table 14: Overall scores for different source guidance strategies. None denotes no source anchoring; percentages in parentheses indicate changes relative to Dynamic (3.61).

AuthorsOpenReview ↗

Response to Reviewer NFd5 (2/2)

q2/w2: Longer, non-repeating videos, recovery after source changes, and LHAD under matched training compute

  1. Long-video human evaluation: We have also conducted human evaluations on complete one-minute natural videos and additional five-minute real-world videos. As shown in Table 15, our method achieves win rates of 88–94% and 98–99%, respectively, against LiveEdit, SANA-Streaming, and StreamDiffusionV2. These baselines frequently exhibit structural distortion, flicker, and loss of subject consistency within the first minute, with further degradation over five minutes. The supplementary material provides five-minute examples from all four methods across all five editing categories for direct inspection of long-term editing quality.

    Counterpart1-min Win (%) ↑1-min Tie (%)1-min Loss (%) ↓5-min Win (%) ↑5-min Tie (%)5-min Loss (%) ↓
    LiveEdit94609910
    SANA-Streaming89929910
    StreamDiffusionV2881029820

    Table 15: Human preference on 1- and 5-minute videos; Win, Tie, and Loss are relative to our method.

  2. Temporal quality and fidelity: We assess 50 one-minute videos across five editing tasks (10 per task), dividing each output into 12 five-second segments. Editing quality is evaluated using the LongV2VBench protocol, while source fidelity is assessed through human evaluation. Figure 20 shows broadly stable overall editing quality and high source fidelity for our method, while LiveEdit and SANA-Streaming deteriorate over time. These results support sustained editing quality and source following throughout the evaluated minute.

    [The figure has been added in revised manuscript]

    Figure 20: Editing quality and source fidelity over time across five tasks, evaluated in 5-second segments.

  3. Recovery after source changes: We measure response delay and phantom persistence following subject disappearance, scene cuts, and other substantial source changes (Table 16). In the subject-exit example in Figure 8, our source-anchored output responds with an additional delay of <0.042 s, whereas the baseline without Resilient-DMD retains phantom content for >4.875 s, through the end of the clip. For the model without Resilient-DMD, apparent recovery results from the real subject reoccupying the phantom's location, which removes the visible mismatch without demonstrating autonomous correction of erroneous generated history.

    CasesWith Resilient-DMDWithout Resilient-DMD
    Substantial source changes0.3 s maximum (within 1 chunk)4 s minimum (12 chunks)

    Table 16: Response-time analysis following substantial changes in the source video.

  4. Ablation under matched training compute: On the same backbone, LHAD improves Overall from 3.38 to 3.61 (+0.23) and Local from 3.35 to 3.75 (+0.40), while Global remains 3.42 (in Table 4). We also conduct ablation experiments under matched training compute, as shown in Table 17. At 413 GPU-hours, LHAD scores 3.61 vs. 3.38; at matched reference DiT compute, short-rollout training uses 448 GPU-hours and scores 3.35. LHAD therefore retains its advantage under both budget controls.

    GPU-hoursTFLOPsOverall Score
    with LHAD4132.80 × 10⁸3.61
    without LHAD4132.58 × 10⁸3.38
    without LHAD4482.80 × 10⁸3.35

    Table 17: Training-budget and quality comparison with and without LHAD.

q3: Video comparisons with evaluated baselines and ablations with and without Resilient-DMD and LHAD

We have combined baseline comparisons and videos comparing the full model with variants trained without Resilient-DMD or without LHAD into a single compressed video in the updated supplementary material. Please re-download the supplementary material to view these comparisons.

Thank you for sharing the high-quality video results and for responding to my questions in such detail. The videos and your explanations fully address all of my concerns. I appreciate the time and effort you put into the rebuttal.

Dear Reviewer NFd5,

Thank you for your positive feedback and for taking the time to review our additional results. We are glad that our videos and explanations have fully addressed your concerns. We hope our work will contribute to further research in this direction. In light of the additional evidence, we would be grateful if you could consider raising your rating accordingly.

Best regards,

Paper 6937 Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Revision Clarification

Dear Reviewers and ACs,

We sincerely appreciate your constructive feedback. We have revised our manuscript and updated the supplementary material. The main manuscript changes are marked in purple.

In the main paper:

  1. Clarified the motivation and the roles of Resilient-DMD and LHAD.
  2. Added attention reliability, guidance allocation, and recovery analyses.
  3. Added one- and five-minute human evaluations and temporal quality and fidelity results (Figure 20).
  4. Clarified training-budget comparisons and added LHAD results under matched compute.
  5. Added score variability and observations of memory and latency stability in longer live sessions.

In the appendix:

  1. Added the final monitoring configuration and shared EMA settings (Appendix B.1).
  2. Added detailed attention, guidance, human evaluation, recovery, and training budget results (Tables 13, 14, 15, 16, 17 and 19).
  3. Added means and standard deviations for repeated evaluations (Table 18).

In the supplementary material:

  1. Added five-minute comparisons with LiveEdit, SANA-Streaming, and StreamDiffusionV2, and one-minute Resilient-DMD/LHAD ablation videos. Combined these comparisons into one compressed video with synchronized views and labeled playback speeds. Please re-download the updated supplementary material.

Thank you again for your time and feedback.

Best regards,

Paper 6937 Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.