- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes DeOverlay, a source-anchored flow model for removing subtitles, titles, and logos from videos. Starting from the source latent, it reconstructs the clean video using region-weighted flow matching. Local cues come from visibility-normalized temporal aggregation, and global context is added through additive conditioning. The authors also introduce DVTE-Bench, which contains 1,631 paired videos across eight overlay categories. DeOverlay outperforms four baselines on all metrics. In my view, the technical novelty is somewhat limited, but the benchmark and released code are a useful contribution to the open-source community.
Critical strengths
-
The method runs inference at competitive speed and needs relatively little GPU memory.
-
Both the quantitative results and the qualitative examples show strong overlay removal and faithful scene reconstruction.
-
The architecture adds source initialization and fixed local and global guidance, yet still uses a single video-token sequence.
Critical weaknesses
-
Since this is both a benchmark and a method paper, I suggest reporting speed, peak GPU memory, and output quality under FP8 and NVFP4 where supported. Please also state the hardware and quantization settings. Controlled comparisons of this kind would show how well the architecture takes advantage of newer GPUs.
-
It is still unclear how the method differs from existing fast video editing approaches. Please explain how source initialization, fixed guidance, and multi-step flow updates differ from distilled few-step object removal methods. Comparisons at similar computational budgets would make the contribution of the proposed design clearer.
-
I could not find runnable code in the submission, so the implementation is hard to verify. Please provide anonymized code, or at least the full training and evaluation configurations. Please also describe the data construction procedure, the checkpoints used, and the preprocessing and postprocessing settings for the baselines.
-
All experiments use a single 1.3B pretrained model. Results on a larger backbone, covering quality, speed, and memory, would show whether the method scales. They would also show whether its benefits go beyond what a bigger model alone provides.
-
The source and diversity of the 350K paired clips need more detail, including the number of distinct source videos, the total duration, and the amount of duplication. Sample count alone does not show whether the data is enough to adapt a pretrained model. Still, data-scaling ablations and tests on real subtitles and unseen overlay styles would give stronger evidence of generalization.
Questions for authors
-
How is your method different from distilled few-step object removal methods? How does it compare with them at the same compute cost?
-
What are the speed, memory use, and output quality under FP8 and NVFP4?
-
Does the method still help with a model larger than 1.3B?
-
How many distinct videos do the 350K clips come from? Does the model work on real subtitles and overlay styles it has not seen?
3 comments
Part 2
W5/Q4: Training-data diversity and generalization to real subtitles and unseen styles
The 350K training samples are rendered pairs, comprising 200K conventional subtitle pairs and 150K pairs with other overlay types, including moving subtitles, mirrored subtitles, and watermarks. Each contains overlaid input frames, aligned clean frames, and an overlay mask. The training corpus is separate from all 1,631 DVTE-Bench clips; Appendix A now states the composition and separation explicitly.
| Training subset | Rendered pairs | Share of training pairs |
|---|---|---|
| Conventional subtitles | 200,000 | 57.14% |
| Other overlays (including moving subtitles, mirrored subtitles, and watermarks) | 150,000 | 42.86% |
| Total | 350,000 | 100% |
Most application examples in Figure 5 and Appendix E are tests on real videos, including real subtitles, logos, and stickers. They demonstrate application beyond the controlled benchmark renderings. Paired DVTE-Bench results additionally show recovery across eight conditions, with DeOverlay leading Mask PSNR, SSIM, LPIPS, and DISTS in every category, including the new VACE and EffectErase comparisons. These results and real-video examples provide the current generalization evidence. Controlled data-scaling and explicitly held-out-font or held-out-style experiments have not been conducted.
W1/Q2: FP8 and NVFP4 quality, speed, and memory
Our reported results use BF16. The A100 used for our timing tests does not provide native FP8 or NVFP4 Tensor Core execution. Enabling FP8/NVFP4 inference is a substantive engineering and validation task, rather than a simple precision switch. It requires compatible hardware, integration of quantized operators into our source-conditioned video pipeline, and validation of reconstruction quality and temporal consistency across successive updates. In particular, the effect of quantization must be assessed over the complete reconstruction trajectory, since each update uses the preceding state. We have not yet carried out this evaluation. In future work, we plan to complete the pipeline adaptation and quality validation on compatible hardware and compare output quality, latency, and peak GPU memory under the same evaluation protocol
For the measured BF16 setting, Table 5 evaluates eight shared clips totaling 650 frames, processed serially on the same A100:
| Method | E2E s/frame ↓ | E2E FPS ↑ | Sampled peak memory (GiB) ↓ |
|---|---|---|---|
| MiniMax-Remover (six steps) | 0.3787 | 2.6405 | 10.776 |
| DeOverlay (one update) | 0.3242 | 3.0847 | 12.236† |
| DeOverlay (four updates) | 0.4828 | 2.0712 | 12.236† |
† The two DeOverlay settings run in the same process and share its externally sampled peak of 12.236 GiB; these are not separate per-setting peak measurements. GPU memory is sampled every 200 ms.
End-to-end timing includes decoding, inference, output processing, and encoding, excluding model loading. Appendix G specifies these measurement boundaries.
W4/Q3: Does the method help with a larger backbone?
The current 1.3B model achieves 30.80 dB Mask PSNR, 3.94 dB above the tested VACE-1.3B setting, and leads all five aggregate quality metrics on DVTE-Bench. Its four-update setting also achieves 2.07 FPS in the separate A100 timing test. These results show that the source representation and guidance are effective at this model size. Exploring larger checkpoints, with corresponding quality, speed, and memory measurements, is our next direction. The conclusion now identifies this direction for future work.
Thank you again for your time and thoughtful review. Please let us know if you have any further questions.
Kind reminder
Dear Reviewer iJwQ,
Thank you again for your time and effort in reviewing our paper. We know this is a busy period. We would like to kindly follow up on our rebuttal and hope that our responses have adequately addressed your concerns.
As the rebuttal discussion period is approaching its end, we would greatly appreciate it if you could take a moment to review our responses when convenient. If you feel your concerns have been addressed, we would sincerely appreciate your considering an updated evaluation. If anything remains unconvincing, we would be glad to discuss it together
Thank you again for your valuable feedback and consideration!
Best regards,
The Authors
Rebuttal by Authors
Dear Reviewer iJwQ,
We sincerely thank Reviewer iJwQ for the detailed review and for recognizing the reconstruction quality, competitive inference speed, and single video-token sequence.
W2/Q1: How does DeOverlay differ from distilled few-step removal, and how does it compare at similar computational cost?
MiniMax-Remover obtains a few-step remover through staged training and minimax distillation. DeOverlay learns a source-to-clean velocity field directly from aligned video pairs. Its design has three connected parts:
The new Figure 6 in Appendix B shows these updates on a difficult artistic title. The title progressively disappears, revealing the car and wet-road reflections. All columns show the same frame and crop from a single run with four equally spaced Euler updates.
The computational evidence separates reconstruction quality from the cost of source conditioning. On the full benchmark, six-step DeOverlay obtains 30.80 dB Mask PSNR, compared with 26.17 dB for six-step MiniMax-Remover. These settings require 12 and 6 network evaluations, respectively, because DeOverlay uses classifier-free guidance at scale 2. They therefore have different computational budgets.
To isolate conditioning cost, Table 9 uses the same weights and four transformer forwards:
Adding the second stream increases transformer time by 2.49× and incremental peak memory by 2.00× under this protocol. This measures the conditioning overhead; quality at matched end-to-end latency has not been measured. Section 2 now clarifies the relation to minimax distillation, and Appendix B adds the recovery visualization.
W3: Training configurations, checkpoints, and baseline processing
Appendices A, D, and G describe the model, evaluated pipelines, and timing protocol. The settings are summarized below:
DVTE-Bench renders scene-aware text plans or speech-derived captions onto clean clips and retains the original frames as targets. The renderer produces frame-aligned masks. Target-frame OCR screening rejects 569 of 2,200 candidates, retaining 1,631 clips.
ProPainter, MiniMax-Remover, and DiffuEraser use shared bounding-box inputs and their recorded 8-pixel mask expansion for output processing. DeOverlay uses 12-pixel seamless output processing. The added VACE and EffectErase comparisons use official weights and seed 2025. VACE uses UniPC with guidance scale 5 and shift 16; EffectErase uses its flow-matching scheduler with guidance scale 1 and shift 5. Appendix D reports their native resolution settings.