acceptodds
Under review as a conference paper at ICLR 2027

DeOverlay: A Benchmark and Source-Anchored Flow for Video Overlay Removal

Abstract

Video overlay removal aims to recover scenes obscured by subtitles, titles, and logos while preserving surrounding content. Existing methods use temporal propagation or source-conditioned generation, but can leave residual overlays and distort scene structure when useful source information is obscured. We propose DeOverlay, a source-anchored flow model that reconstructs the clean video from the source latent with fixed source guidance. Specifically, we train a shared velocity network along paired source-to-clean paths using region-weighted flow matching, enabling successive corrections from intermediate states. To select local source cues, we develop visibility-normalized temporal aggregation that excludes masked positions and averages the remaining observations. Global source context and edit-region embeddings complement these cues through additive conditioning, retaining a single video-token sequence. To evaluate scene recovery across diverse overlay conditions, we introduce DVTE-Bench, comprising 1,631 paired videos across eight text-overlay categories, with OCR-screened clean targets, frame-aligned pixel masks, and a common evaluation protocol. Experiments on the complete release show that DeOverlay outperforms four baselines on all five reconstruction and perceptual metrics, reducing Mask LPIPS and Mask VFID by 58.3% and 70.3%, respectively, relative to the strongest evaluated baseline. Additional qualitative examples demonstrate the removal of graphic logos and stickers. We will make the complete benchmark, evaluation code, and test outputs available.

Primary area:
generative models
Keywords:
Video overlay removal, Source-initialized flow refinement framework, Spatial guidance

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_iJwQOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes DeOverlay, a source-anchored flow model for removing subtitles, titles, and logos from videos. Starting from the source latent, it reconstructs the clean video using region-weighted flow matching. Local cues come from visibility-normalized temporal aggregation, and global context is added through additive conditioning. The authors also introduce DVTE-Bench, which contains 1,631 paired videos across eight overlay categories. DeOverlay outperforms four baselines on all metrics. In my view, the technical novelty is somewhat limited, but the benchmark and released code are a useful contribution to the open-source community.

Critical strengths
  1. The method runs inference at competitive speed and needs relatively little GPU memory.

  2. Both the quantitative results and the qualitative examples show strong overlay removal and faithful scene reconstruction.

  3. The architecture adds source initialization and fixed local and global guidance, yet still uses a single video-token sequence.

Critical weaknesses
  1. Since this is both a benchmark and a method paper, I suggest reporting speed, peak GPU memory, and output quality under FP8 and NVFP4 where supported. Please also state the hardware and quantization settings. Controlled comparisons of this kind would show how well the architecture takes advantage of newer GPUs.

  2. It is still unclear how the method differs from existing fast video editing approaches. Please explain how source initialization, fixed guidance, and multi-step flow updates differ from distilled few-step object removal methods. Comparisons at similar computational budgets would make the contribution of the proposed design clearer.

  3. I could not find runnable code in the submission, so the implementation is hard to verify. Please provide anonymized code, or at least the full training and evaluation configurations. Please also describe the data construction procedure, the checkpoints used, and the preprocessing and postprocessing settings for the baselines.

  4. All experiments use a single 1.3B pretrained model. Results on a larger backbone, covering quality, speed, and memory, would show whether the method scales. They would also show whether its benefits go beyond what a bigger model alone provides.

  5. The source and diversity of the 350K paired clips need more detail, including the number of distinct source videos, the total duration, and the amount of duplication. Sample count alone does not show whether the data is enough to adapt a pretrained model. Still, data-scaling ablations and tests on real subtitles and unseen overlay styles would give stronger evidence of generalization.

Questions for authors
  1. How is your method different from distilled few-step object removal methods? How does it compare with them at the same compute cost?

  2. What are the speed, memory use, and output quality under FP8 and NVFP4?

  3. Does the method still help with a model larger than 1.3B?

  4. How many distinct videos do the 350K clips come from? Does the model work on real subtitles and overlay styles it has not seen?

3 comments
AuthorsOpenReview ↗

Rebuttal by Authors

Dear Reviewer iJwQ,

We sincerely thank Reviewer iJwQ for the detailed review and for recognizing the reconstruction quality, competitive inference speed, and single video-token sequence.

W2/Q1: How does DeOverlay differ from distilled few-step removal, and how does it compare at similar computational cost?

MiniMax-Remover obtains a few-step remover through staged training and minimax distillation. DeOverlay learns a source-to-clean velocity field directly from aligned video pairs. Its design has three connected parts:

  1. Paired flow training. A region-weighted objective supervises the velocity along source-to-clean latent paths, without requiring teacher-generated editing trajectories.
  2. Source initialization and persistent guidance. Inference starts from the source latent. Each update recomputes the velocity from the current state and time, while the source observations, mask, and network weights remain fixed.
  3. Visibility-aware additive conditioning. The local branch excludes masked observations and normalizes by their visible count; global context complements those observations. Both enter through addition, preserving one video-token sequence.

The new Figure 6 in Appendix B shows these updates on a difficult artistic title. The title progressively disappears, revealing the car and wet-road reflections. All columns show the same frame and crop from a single run with four equally spaced Euler updates.

The computational evidence separates reconstruction quality from the cost of source conditioning. On the full benchmark, six-step DeOverlay obtains 30.80 dB Mask PSNR, compared with 26.17 dB for six-step MiniMax-Remover. These settings require 12 and 6 network evaluations, respectively, because DeOverlay uses classifier-free guidance at scale 2. They therefore have different computational budgets.

To isolate conditioning cost, Table 9 uses the same weights and four transformer forwards:

Input organizationTotal tokensTime (s) ↓Incremental peak memory (GiB) ↓
Source-initialized stream13,3465.2661.744
Noise and source streams26,59413.0883.479

Adding the second stream increases transformer time by 2.49× and incremental peak memory by 2.00× under this protocol. This measures the conditioning overhead; quality at matched end-to-end latency has not been measured. Section 2 now clarifies the relation to minimax distillation, and Appendix B adds the recovery visualization.

W3: Training configurations, checkpoints, and baseline processing

Appendices A, D, and G describe the model, evaluated pipelines, and timing protocol. The settings are summarized below:

ItemSetting
Backbone and adaptationDense LingBot-Video 1.3B; rank-256 LoRA; patch embedding and source-guidance modules trained directly; VAE and text encoder frozen
GuidanceLocal/global hidden widths 512/512; local window of five latent positions; mask dropout 0.2
Training objectiveRegion-weighted flow matching; source-endpoint perturbation 0.05; overlay-region weight 6
OptimizationAdamW; learning rate 5 × 10⁻⁵; betas (0.9, 0.95); weight decay 0; 300 warm-up updates; gradient clipping 1.0; BF16
Training buckets41 frames at short side 768 for original long side ≥1,920; 65 at 768 for ≥1,280; 121 at 608 otherwise
Main inferenceSix Euler updates; guidance scale 2; resolution-shifted linear–quadratic schedule; short-side 768 bucket
Long-video processing81-frame windows; nine-frame overlap; eight-frame mirrored tail padding
Input and scoring masksGround-truth bounding boxes for mask-conditioned methods; aligned pixel masks for scoring

DVTE-Bench renders scene-aware text plans or speech-derived captions onto clean clips and retains the original frames as targets. The renderer produces frame-aligned masks. Target-frame OCR screening rejects 569 of 2,200 candidates, retaining 1,631 clips.

ProPainter, MiniMax-Remover, and DiffuEraser use shared bounding-box inputs and their recorded 8-pixel mask expansion for output processing. DeOverlay uses 12-pixel seamless output processing. The added VACE and EffectErase comparisons use official weights and seed 2025. VACE uses UniPC with guidance scale 5 and shift 16; EffectErase uses its flow-matching scheduler with guidance scale 1 and shift 5. Appendix D reports their native resolution settings.

W5/Q4: Training-data diversity and generalization to real subtitles and unseen styles

The 350K training samples are rendered pairs, comprising 200K conventional subtitle pairs and 150K pairs with other overlay types, including moving subtitles, mirrored subtitles, and watermarks. Each contains overlaid input frames, aligned clean frames, and an overlay mask. The training corpus is separate from all 1,631 DVTE-Bench clips; Appendix A now states the composition and separation explicitly.

Training subsetRendered pairsShare of training pairs
Conventional subtitles200,00057.14%
Other overlays (including moving subtitles, mirrored subtitles, and watermarks)150,00042.86%
Total350,000100%

Most application examples in Figure 5 and Appendix E are tests on real videos, including real subtitles, logos, and stickers. They demonstrate application beyond the controlled benchmark renderings. Paired DVTE-Bench results additionally show recovery across eight conditions, with DeOverlay leading Mask PSNR, SSIM, LPIPS, and DISTS in every category, including the new VACE and EffectErase comparisons. These results and real-video examples provide the current generalization evidence. Controlled data-scaling and explicitly held-out-font or held-out-style experiments have not been conducted.

W1/Q2: FP8 and NVFP4 quality, speed, and memory

Our reported results use BF16. The A100 used for our timing tests does not provide native FP8 or NVFP4 Tensor Core execution. Enabling FP8/NVFP4 inference is a substantive engineering and validation task, rather than a simple precision switch. It requires compatible hardware, integration of quantized operators into our source-conditioned video pipeline, and validation of reconstruction quality and temporal consistency across successive updates. In particular, the effect of quantization must be assessed over the complete reconstruction trajectory, since each update uses the preceding state. We have not yet carried out this evaluation. In future work, we plan to complete the pipeline adaptation and quality validation on compatible hardware and compare output quality, latency, and peak GPU memory under the same evaluation protocol

For the measured BF16 setting, Table 5 evaluates eight shared clips totaling 650 frames, processed serially on the same A100:

MethodE2E s/frame ↓E2E FPS ↑Sampled peak memory (GiB) ↓
MiniMax-Remover (six steps)0.37872.640510.776
DeOverlay (one update)0.32423.084712.236†
DeOverlay (four updates)0.48282.071212.236†

† The two DeOverlay settings run in the same process and share its externally sampled peak of 12.236 GiB; these are not separate per-setting peak measurements. GPU memory is sampled every 200 ms.

End-to-end timing includes decoding, inference, output processing, and encoding, excluding model loading. Appendix G specifies these measurement boundaries.

W4/Q3: Does the method help with a larger backbone?

The current 1.3B model achieves 30.80 dB Mask PSNR, 3.94 dB above the tested VACE-1.3B setting, and leads all five aggregate quality metrics on DVTE-Bench. Its four-update setting also achieves 2.07 FPS in the separate A100 timing test. These results show that the source representation and guidance are effective at this model size. Exploring larger checkpoints, with corresponding quality, speed, and memory measurements, is our next direction. The conclusion now identifies this direction for future work.

Thank you again for your time and thoughtful review. Please let us know if you have any further questions.

AuthorsOpenReview ↗

Kind reminder

Dear Reviewer iJwQ,

Thank you again for your time and effort in reviewing our paper. We know this is a busy period. We would like to kindly follow up on our rebuttal and hope that our responses have adequately addressed your concerns.

As the rebuttal discussion period is approaching its end, we would greatly appreciate it if you could take a moment to review our responses when convenient. If you feel your concerns have been addressed, we would sincerely appreciate your considering an updated evaluation. If anything remains unconvincing, we would be glad to discuss it together

Thank you again for your valuable feedback and consideration!

Best regards,

The Authors

Reviewer_sw2QOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

DeOverlay addresses video overlay removal using a source-initialized flow model with persistent local and global guidance. Visibility-normalized temporal aggregation selects unmasked source features for reconstruction. The paper also introduces DVTE-Bench, containing 1,631 paired clips across eight text-overlay categories. Evaluation reports substantial regional reconstruction improvements over four baselines, supported by component ablations and separate efficiency measurements. Broader comparisons and full-output preservation tests would strengthen the evidence.

Critical strengths
  1. DVTE-Bench covers eight overlay categories with clean targets and aligned masks, enabling consistent regional evaluation across 1,631 clips.

  2. The method is novel. The experimental results are strong overall, and the method performs well.

Critical weaknesses
  1. The task scope is relatively narrow. The quantitative evaluation focuses on synthetic text-overlay removal. Although qualitative examples include logos and stickers, the broader applicability of the proposed method remains unclear. The paper would benefit from explaining whether its contribution extends beyond this specialized setting.

  2. The comparisons don't include relevant methods such as VACE (Jiang et al., 2025) and EffectErase (Fu et al., 2026), and no evaluation on a general video object-removal benchmark is provided. Comparisons with these methods under appropriate task settings, together with broader benchmark evaluation, would help determine whether the improvements generalize beyond text-overlay removal.

Questions for authors

see the weekness.

If the authors can adequately address my concerns, I would consider increasing my score.

6 comments
AuthorsOpenReview ↗

Rebuttal by Authors

Dear Reviewer sw2Q,

We sincerely thank Reviewer sw2Q for the thoughtful review and for recognizing the benchmark, the method's novelty, and its reconstruction quality. We have completed the suggested baseline comparisons and clarified the benchmark contribution and real-video application evidence.

W2: How does DeOverlay compare with VACE and EffectErase, and what is its scope relative to general video object removal?

We evaluated the official VACE-1.3B and EffectErase checkpoints on all 1,631 DVTE-Bench clips, covering 136,090 frames per method. The methods receive the same input clips and ground-truth bounding boxes, and their outputs are evaluated against the same clean targets using the released pixel masks. The results are:

MethodMask PSNR ↑Mask SSIM ↑Mask LPIPS ↓Mask DISTS ↓Mask VFID ↓
VACE-1.3B26.860.81160.08450.07650.2244
EffectErase21.750.69500.17820.13210.6730
DeOverlay30.800.88220.04080.05070.0835

PSNR is averaged over finite video-level values, following the benchmark protocol.

Compared with VACE, the strongest evaluated baseline on these five metrics, DeOverlay improves Mask PSNR by 3.94 dB and reduces Mask LPIPS and VFID by 51.76% and 62.80%, respectively. The improvement also holds across overlay conditions: DeOverlay outperforms both additional baselines on PSNR, SSIM, LPIPS, and DISTS in each of the eight categories.

The submitted DeOverlay model was trained for video overlays, principally subtitles, artistic text, and watermarks. It was not trained for general video object removal. Extending the method to physical objects and evaluating it on dedicated object-removal benchmarks will be a focus of our future work.

W1: What does DVTE-Bench contribute to broader video overlay removal?

We address this concern through the benchmark's coverage of overlay-removal challenges and the real-video application results.

  1. Benchmark coverage of diverse reconstruction conditions. DVTE-Bench varies overlay layout, temporal visibility, appearance, and caption timing across eight categories. Standard subtitles provide a conventional layout; vertical, non-bottom, and multi-region text vary the shape and placement of covered regions. Moving text may expose a location in nearby frames, whereas persistent watermarks can keep it obscured. Artistic titles vary typography, outlines, and glow, while speech-aligned captions vary content and timing based on audio. Together with subtitle background panels, these overlays cover both fine strokes and contiguous scene regions. Their boundary patterns and filled areas also occur in graphic logos, making the evaluated reconstruction abilities relevant to broader graphic-overlay removal. Paired clean targets and aligned masks support a common assessment of how faithfully the covered scene is recovered across these conditions.

    The category counts, proportions, and corresponding reconstruction conditions are:

    Overlay categoryClipsClip shareMean mask coverageReconstruction condition
    Standard subtitle23214.22%3.85%Conventional lower-third layout
    Artistic title21813.37%16.92%Display typography, outlines, and glow
    Moving text23214.22%2.40%Changing occlusion locations across frames
    Vertical text21513.18%1.96%Vertical writing and elongated covered regions
    Non-bottom layout22313.67%4.36%Text outside the conventional bottom region
    Multi-region text22513.80%8.94%Simultaneous occlusion in distributed regions
    Persistent watermark21813.37%0.58%Persistent semi-transparent text marks
    Speech-aligned caption684.17%3.58%Audio-derived content and caption timing
    Total1,631100.00%5.47%

    Clip share is the category count divided by 1,631. Mean mask coverage first averages the pixel mask over all frames within each clip, then averages clips equally within each category; the total averages all 1,631 clips equally.

  2. Results on real videos. Most application visualizations in the main paper and appendix come from real-video tests. Figure 5 and Appendix E show removal of real subtitles, artistic text, watermarks, logos, and stickers. These examples demonstrate the model's practical performance on real overlays, complementing the controlled, paired evaluation in DVTE-Bench.

DVTE-Bench contributes a common paired evaluation of these reconstruction challenges, while the real-video examples demonstrate their practical relevance.

Thank you again for your time and thoughtful review. Please let us know if you have any further questions.

AuthorsOpenReview ↗

Comment by authors

Dear reviewer sw2Q,

Thank you again for your detailed review. As the discussion period ends on November 18, we would appreciate your feedback on whether our responses have addressed your concerns. We have added comparisons with VACE and EffectErase on all 1,631 DVTE-Bench clips, clarified the reconstruction challenges covered by the eight overlay categories, and explained the real-video subtitle, logo, and sticker examples. We have also clarified the scope of video overlay removal and identified general video object removal as a direction for future work.

We would be happy to address any remaining questions. If our responses and the revised manuscript have addressed your concerns, we would be grateful if you would consider updating your evaluation.

Sincerely, The Authors

Reviewer_sw2QOpenReview ↗

Thank you for your detailed rebuttal

Thank you for adding the comparisons. EffectErase’s reported performance seems quite low. Could you share a few side-by-side video demos to help clarify the performance gap?

Reviewer_3A56OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary
  • This paper proposes a video overly removal method that aims to recover scenes obscured by many kinds of texts, logs.
  • The proposed source-anchored flow model named DeOverly, reconstructs the clean video from the source latent with fixed source guidance.
  • Besides the new model, a new benchmark named DVTE-Bench, is constructed, comprising 1631 paired videos across eight text-overly categories.
  • The experiments result shows the proposed DeOverlay outperforms four baselines on all metrics.
Critical strengths
  1. The proposed benchmark is valuable for the the academic community
  2. The prosed model has some novel modules, and the performance results are good.
Critical weaknesses
  1. Although this dataset is highly valuable, there are some problems with it. The overall eight text-overly categories are quite similar and not very representative.
Questions for authors
  1. Why the proposed eight text-overly categories are chosen? Is it possible that more representative categories are included.
2 comments
AuthorsOpenReview ↗

Rebuttal by Authors

Dear Reviewer 3A56,

We sincerely thank Reviewer 3A56 for the careful review and for recognizing the value of DVTE-Bench and the reconstruction performance of DeOverlay.

W1/Q1: Why were the eight overlay categories chosen, and could the benchmark include more representative categories?

We chose the eight categories to cover different conditions for recovering the underlying scene. Their shared use of text makes paired evaluation consistent, while their layout, persistence, and appearance change the information available for reconstruction.

  1. Different recovery conditions. The eight categories cover controlled reconstruction conditions, and some visual attributes can occur in more than one category. Their sample counts, shares of the full release, mean mask coverage, and main recovery conditions are listed below.

    Overlay categoryClipsShare of 1,631 clipsMean mask coverageMain recovery condition
    Standard subtitle23214.22%3.85%Conventional lower-third occlusion
    Artistic title21813.37%16.92%Display typography, outlines, glow, and larger covered areas
    Moving text23214.22%2.40%Changing occlusion locations and visibility across frames
    Vertical text21513.18%1.96%Vertically elongated occlusions at varied positions
    Non-bottom layout22313.67%4.36%Occlusions outside the conventional subtitle band
    Multi-region text22513.80%8.94%Several spatially separated regions covered simultaneously
    Persistent watermark21813.37%0.58%The same location may remain hidden throughout a local window
    Speech-aligned caption684.17%3.58%Caption content and visibility intervals derived from speech
    Total1,631100.00%5.47%

    Sample shares use the full release of 1,631 clips as the denominator. Mask coverage is averaged over all pixels and frames within each clip, then equally across clips in that category. The total coverage averages all 1,631 clips equally.

  2. Measurable variation within the release. The category-level results also expose different difficulties: artistic titles give DeOverlay its lowest PSNR/SSIM, at 26.24 dB/0.8409, while vertical text gives its highest LPIPS/DISTS, at 0.0975/0.1015. Thus, the hardest condition depends on whether pixel fidelity or perceptual similarity is measured. These observations reflect the combined recovery conditions in each category.

  3. Transfer from artistic titles to graphic overlays. Our training data include artistic titles and stylized text with diverse shapes, outlines, and visual effects. These overlays share visual characteristics with graphic logos, providing training experience for reconstructing scenes beneath similar occlusions. Figure 5 and Appendix E illustrate the model's ability to remove logos and stickers from real videos. Adding paired graphic-overlay categories would extend DVTE-Bench to quantify these applications.

Section 3.1 now reports the mask-coverage statistics alongside the category rationale. Appendix D details the released composition, and Appendix H includes the category-level comparisons with the newly evaluated VACE and EffectErase baselines. Across all eight categories, DeOverlay leads the seven evaluated methods on Mask PSNR, SSIM, LPIPS, and DISTS.

Thank you again for your time and thoughtful review. Please let us know if you have any further questions.

AuthorsOpenReview ↗

Kind reminder

Dear Reviewer 3A56,

Thank you again for your time and effort in reviewing our paper. We know this is a busy period. We would like to kindly follow up on our rebuttal and hope that our responses have adequately addressed your concerns.

As the rebuttal discussion period is approaching its end, we would greatly appreciate it if you could take a moment to review our responses when convenient. If you feel your concerns have been addressed, we would sincerely appreciate your considering an updated evaluation. If anything remains unconvincing, we would be glad to discuss it together

Thank you again for your valuable feedback and consideration!

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.