acceptodds
Withdrawn from ICLR 2027

StreamErase: Towards Ultra-Fast Streaming Video Erasing

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Current video object removal methods suffer from prohibitive latency (minutes per video) and non-causal processing that prevents real-time interaction. We present StreamErase, enabling sub-second latency with up to 26 FPS streaming generation on a single GPU. Our approach first augments a text-to-video model with mask input to generate high-quality videos that erase target objects and synthesize coherent backgrounds, but does not perform inference on the fly. As such, we progressively transfer the erasing capability from a powerful bidirectional foundation model to an efficient unidirectional streaming eraser, enabling long-horizon erasing without sacrificing visual fidelity. Several key challenges arise when generating videos over long, potentially infinite horizons: (1) bridging the domain gap from training on finite length and extrapolating to long horizons, (2) sustaining high quality by preventing error accumulation, and (3) maintaining fast inference, without increasing computational cost as the context grows. To address these challenges, we carefully designed sliding-window causal attention with attention sinks. By incorporating self-rollout with attention sinks and KV cache rolling during training, we simulate inference-time extrapolations with a fixed context window, enabling constant-speed generation of arbitrarily long videos. Finally, we establish a dedicated benchmark, LongErase, for streaming long-video object removal. Our model achieves strong performance compared to existing video object erasing models and generalizes well to real-world video scenarios, uniquely enabling infinite-length streaming. With StreamErase, users can input a mask and see the erasing results unfold in real-time, delivering a truly interactive experience.

Primary area:
generative models
Keywords:
diffusion models, video erasing

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_eRVmOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

StreamErase combines effect-aware removal with causal distillation and a rolling KV cache for streaming video erasing. It reports 26.06 FPS on an H200 and evaluates long-video performance on 30-second clips.

Critical strengths

The speedup is useful in practice. Table 4 supports the guidance and effect-supervision modules, and Figure 5/Table 5 show that the sink and window choices affect stability.

Critical weaknesses
  1. I learned little beyond how to put the existing pieces together. Causal distillation, attention sinks, and rolling caches are established ideas. The guidance and loss improve removal quality, but the paper leaves me unclear about what new insight comes from combining them with streaming inference. The comparisons do not resolve this either. VideoPainter is cited but is missing from the long-video experiments; Object-WIPER is directly relevant to removing associated effects. RT-Remover and SVOR also deserve discussion, as do the streaming designs in LiveEdit and EditCtrl. I would not require experiments against unpublished or contemporaneous work, but a clearer account of the differences would help establish what this paper adds.

  2. The evidence for long-video erasing is too limited for the claims. The teaser says “Infinite-length generated video,” yet the quantitative long-video tests cover only 30 seconds. Figure 5 does show quality over time, and Table 5 includes a sliding-window comparison. Still, a fixed-size cache says little about whether removal stays correct minutes into a video, especially with the scene-change limitations acknowledged in Appendix F. Table 2 has no other method in its 30-second section, so the advantage over existing long-video methods is also unclear. LongErase's ground-truth construction and alignment are not explained, which further limits how much I can conclude from these results.

Questions for authors
  1. What is the main insight beyond combining the existing streaming machinery with effect-aware removal? A focused comparison with the closest applicable method would be more useful than additional component ablations.
  2. Is there evidence that removal remains reliable well beyond 30 seconds? Please clarify how LongErase ground truth was obtained and aligned. Without further evidence, the claims about infinite streaming with preserved quality should be narrowed.
Reviewer_veS4OpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

StreamErase converts a bidirectional video-erasing model into a one-step causal model through three-stage distillation. Region guidance and effect-erasing supervision target object-related effects, while attention sinks and a rolling cache support streaming. The method reports 26.06 FPS and 0.87-second latency on one H200. Experiments demonstrate substantial speed gains and strong synthetic-benchmark results,

Critical strengths
  1. The reported 26.06 FPS and 0.87-second latency support efficient, interactive video erasing in a streaming setting.
  2. Positive quality and ablation results. StreamErase results is the best on all four ROSE metrics. Component and attention-sink ablations support the design, with longer-video results extending to 30 seconds.
Critical weaknesses
  1. Mixed real-world quality. On VOR-Eval, EffectErase achieves better PSNR, LPIPS, and FVD, StreamErase only slightly improves SSIM. Despite strong ROSE results and faster inference, the evidence supports a speed-quality trade-off rather than consistent quality superiority.

  2. Possible residual effects. Figure 4, third row, retains visible ripples near the removed swan. Their origin requires reference-video verification, but the example does not convincingly demonstrate complete effect removal. Full-frame metrics also fail to isolate residual object-induced effects.

Questions for authors
  1. Can you provide uncertainty estimates and effect-region results to clarify the real-world quality trade-off?

  2. Can reference videos and aligned crops establish whether the remaining ripples in Figure 4, third row, are background motion or unremoved swan-induced effects?

If the authors could provide further validation of the method’s effectiveness, I would consider increasing my score.

Reviewer_MKYhOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes StreamErase, a streaming video object erasing framework. It first trains a bidirectional teacher with region guidance and an effect erasing loss, then obtains a one-step autoregressive student through causal initialization and DMD distillation, and combines attention sinks with rolling KV caches to achieve real-time long-video erasing.

Critical strengths

This paper extends streaming real-time generation to video object erasing with side effects, achieving about 26 FPS throughput and sub-second latency on a single GPU, which alleviates the long runtime and lack of interactivity of existing erasing methods. The method also models side effects such as shadows and reflections, and both the effect erasing loss and the multimodal region guidance bring consistent gains in the ablations. The authors build LongErase, a 30-second long-video erasing benchmark, and systematically analyze the effects of chunk size, sink size and window size, giving the work strong practical value.

Critical weaknesses

The paper claims to uniquely enable infinite-length streaming erasing, but prior work has adapted VACE for real-time autoregressive masked inpainting, and InstantViR achieves one-step causal streaming inpainting. We suggest adding these baselines and moderating the novelty claims. The effect erasing loss supervises attention maps with difference masks, which is similar in spirit to the object-effect attention in ObjectClear and the side-effect mask predictor in ROSE. We suggest explaining the essential difference and validating this module separately on the streaming model. The fixed single-sink, single-window context strategy offers limited novelty, and the authors acknowledge weak adaptability under drastic scene changes. Recent work such as DySink proposes dynamic sinks, so we suggest evaluating on videos with clear scene transitions and discussing adaptive caching.

Questions for authors

Please refer to weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.