acceptodds
Under review as a conference paper at ICLR 2027

FastEraser: Achieving Efficient and Effect-Aware Video Object Removal via Sparsity-Driven Distillation

Abstract

Effect-aware video object removal (VOR) aims to eliminate not only a target object but also its associated environmental effects, such as shadows and reflections. Although recent generation-based VOR methods achieve compelling reconstruction quality, their billion-scale architectures and iterative denoising processes incur substantial computational cost. Existing acceleration strategies either apply mask-guided spatial sparsity, which overlooks effects outside the input mask, or directly reduce the number of denoising steps, which introduces noticeable blur and noise. Few-step distillation offers a promising alternative, yet conventional Distribution Matching Distillation (DMD) relies heavily on the asymmetry induced by classifier-free guidance (CFG), making it poorly suited to VOR models that are inherently CFG-free. We propose FastEraser, an efficient framework for effect-aware VOR built upon Sparsity-Driven Distillation (SDD) and Pixel Reconstruction Loss (PRL). SDD replaces CFG-induced asymmetry with a dense-to-sparse attention discrepancy between the real and fake score models, restoring the driving signal required for stable few-step adaptation without restricting removal to the input mask. PRL further anchors the student prediction to the ground-truth video by decoding its denoised latent through a frozen VAE and applying direct supervision in RGB space, thereby preserving removal capability and high-frequency details. Extensive experiments on four benchmarks demonstrate that FastEraser achieves performance comparable to existing 50-step models using only 4 denoising steps, substantially reducing inference cost while maintaining effect removal quality and reconstruction fidelity.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Video editing, Video generation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_K8Y6OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose an efficient inference framework for Effect-aware Video Object Removal. Sparsity-Driven Distillation (SDD) combines dense attention in the real score model with sparse attention in the fake score model, exploiting their score discrepancy to guide few-step student distillation. Meanwhile, Pixel Reconstruction Loss (PRL) decodes the predicted denoised latents into RGB videos using a frozen VAE decoder and enforces pixel-level supervision against ground-truth videos. Beyond removing specified objects, the framework also eliminates associated visual effects, such as shadows, reflections, and illumination changes, even when they extend beyond the input mask.

Critical strengths
  1. Removing an object and its associated effects, such as shadows, aligns with human intuition. Through architectural and training innovations, FastEraser leverages Sparsity-Driven Distillation (SDD) and Pixel Reconstruction Loss (PRL) to retain effect-aware removal capabilities without requiring additional effect-region masks, while substantially reducing inference costs, demonstrating practical value.
  2. FastEraser achieves a substantial improvement in inference efficiency over EffectErase with 50 inference steps, reducing the DiT inference time from 86.0 seconds to just 4.6 seconds and significantly lowering the computational cost.
Critical weaknesses
  1. As shown in Table 1, FastEraser exhibits degraded temporal consistency across all benchmarks. Temporal consistency is particularly important in video tasks, as frequent flickering can lead to a very poor user experience. What causes this degradation, and how can temporal consistency be improved?
  2. FastEraser reduces the teacher model's denoising steps from 50 to 4. Would increasing the number of steps to 8 or 16 help maintain temporal consistency?
  3. Tables 4 in the main paper and Table 6 in the appendix both compare auxiliary losses. Why do they report different metric combinations and results?
  4. In Table 3, why does SDD outperform the Teacher (50 steps) baseline? What accounts for this improvement?
  5. In Figure 3, “Oberservation” is misspelled.
Questions for authors

Please refer to the Weaknesses.

Reviewer_qfSxOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

FastEraser accelerates effect-aware video object removal through Sparsity-Driven Distillation and Pixel Reconstruction Loss. A dense real-score model and sparse fake-score model guide four-step adaptation, while decoded RGB supervision preserves reconstruction quality. The method achieves broadly comparable spatial quality across four benchmarks and reduces reported DiT denoising time from 86.0 to 4.6 seconds.

Critical strengths

1.Presenting a well-motivated approach to efficient effect-aware video removal. Its distillation design addresses the high inference cost of existing methods while retaining competitive removal quality.

2.The experimental study is reasonably comprehensive. Comparisons across four benchmarks and several ablations provide useful evidence for the overall approach and its main design choices.

Critical weaknesses
  1. Temporal quality is not consistently maintained. RC-T is worse than EffectErase on all four benchmarks, please analyze and explain the reason.

  2. I am not fully clear on why the Gemini evaluation uses only the middle frame of each video (Appendix C, lines 750–753). Could the authors explain why this frame is representative of overall removal quality?

  3. Practical speedup remains unquantified. The reported 18.7× acceleration measures DiT denoising only. Appendix B acknowledges unchanged VAE costs, but provides no end-to-end runtime. Consequently, the actual speedup experienced by users remains unclear despite the substantial denoising improvement.

Questions for authors

see the weakness.

If the authors can adequately address my concerns, I would consider increasing my score.

Reviewer_8bZcOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

FastEraser speeds up video object removal while also removing associated effects, such as shadows and reflections. It introduces Sparse–Dense Distillation (SDD), which combines few-step distillation with sparse attention using a dense real-score model and a sparse fake-score model. The method operates without classifier-free guidance and adds a pixel reconstruction loss (PRL), computed after decoding through a frozen VAE, to provide supervision specific to the removal task. Evaluations on four benchmarks show that FastEraser reduces the number of denoising steps from 50 to 4, accelerating the DiT denoising component by approximately 18.7×. Although quality results vary across metrics, the method offers a competitive trade-off between removal quality and inference speed.

Critical strengths
  1. FastEraser addresses a useful practical problem: reducing the cost of video object removal while also handling effects such as shadows and reflections and preserving the surrounding content. The reported 18.7× acceleration of the DiT denoising component is substantial. Results on four benchmarks, covering paired and unpaired data, suggest a competitive balance between speed and removal quality.
  2. The ablations cover several relevant choices, including the fake-score configuration, auxiliary losses, optimization order, initialization, and sparsity. I found the PRL results particularly useful, as they show that task-specific reconstruction supervision improves reconstruction quality. The discussion of BLADE in Appendix A.1 also helps put the method in context.
Critical weaknesses
  1. I am still unclear about what is new in SDD and how strongly the proposed explanation is supported. Appendix A.1 discusses BLADE, and Figure 5 compares fake-score choices, but a dense-real/sparse-fake configuration already appears in BLADE’s public implementation. The paper should state more directly whether its contribution is a new training operation, an empirical study of this configuration, or its adaptation to video removal without CFG.The mechanism claim also seems stronger in the abstract than in Section 3.2, where it is presented as a hypothesis. Equation (6) and the reported curves show that the configuration can work, but they do not establish that the sparse fake score supplies a driving force analogous to CFG. Figures 3 and 5 would be easier to interpret with clearer definitions of the measured quantities, evaluation settings, and controls. Some of this may only require clarification.
  2. A few differences between the main tables and the ablations need explanation. The VOR-Eval teacher has a Gemini score of 6.12 in Table 1 but 6.02 in Table 3, despite identical RC-S/RC-T values. The final method also has a PSNR of 22.69 in Table 1, compared with 22.52 for SDD+PRL in Table 4, while both report an SSIM of 0.7410.These differences may come from different checkpoints or evaluation runs. As written, though, I cannot tell which results are directly comparable, which makes it harder to use the ablations to explain the final performance. Please specify whether these entries use the same datasets, model configurations, checkpoints, and evaluation runs.
Questions for authors
  1. What does SDD add beyond the configuration already used in BLADE? Is there an additional training operation, or is the main contribution an empirical finding about how this configuration works? It would help to separate the structural design, the proposed mechanism, and the adaptation to video removal without CFG.
  2. Could you give more details about the setup for Figures 3 and 5, including the dataset, metric definitions, and controls? For the fake-score comparison, were the student architecture, auxiliary supervision, noise sampling, update schedule, and training budget kept the same? Also, were the results checked across multiple runs, and if so, how much did they vary? This would make it easier to judge how much evidence there is for the proposed mechanism.
  3. Could you clarify which datasets and configurations were used in Tables 3–8? I also noticed that the teacher’s Gemini score is 6.12 in Table 1 but 6.02 in Table 3, while the final model’s PSNR is 22.69 in Table 1 but 22.52 in Table 4. Were these results obtained from the same checkpoints and generated outputs, or from different runs or settings?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.