acceptodds
Under review as a conference paper at ICLR 2027

SEDiT: Mask-Free Video Subtitle Erasure via One-step Diffusion Transformer

Abstract

Existing video subtitle removal methods inpaint masked frames, so they must localize the subtitles first, and the accuracy of that segmentation limits the final quality. We present SEDiT, a one-stage video Subtitle Erasure approach built on a one-step Diffusion Transformer. SEDiT is mask-free at inference: it requires no mask, OCR, or segmentation at test time, and a coarse subtitle box is used only for subtitle-region loss weighting during training. Our key observation is that subtitle removal is often conditionally near-deterministic: once the subtitled video is given, the subtitle-free video is strongly constrained. Under an ideal conditional velocity field, a single Euler step corresponds to a conditional-mean estimator whose expected squared error equals the conditional variance of the target, which explains why one-step inference can work well for subtitle erasure while a multi-modal task such as stylization may still require multiple steps. Together with chunk-wise streaming inference, this enables one-step erasure on native 1080p videos of minute-scale duration. On our paired VSR-Bench-400 benchmark, SEDiT attains dB PSNR and FVD, achieving stronger raw-output fidelity than the evaluated mask-based baselines, which are given ground-truth subtitle boxes rather than detected ones, while taking s per 65-frame 1080p clip—about faster than Minimax-Remover and faster than EffectErase, and without step distillation. On real clips of roughly two minutes, primarily from television dramas and short-form dramas, subtitles are completely removed in under our strict full-clip criterion; supplementary videos provide further qualitative evidence on real video.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
video subtitle removal, one-step DiT, mask-free method

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 2 reviews, above 27% of the others.
Reviewer_uQEpOpenReview ↗
Rating
1(−1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents SEDiT, a mask-free video subtitle removal framework based on LTX-Video. It combines reference-token conditioning, LoRA fine-tuning, subtitle-region loss weighting, and one-step inference. The authors introduce VSR-Bench-400, a synthetic paired benchmark, and an optional distillation stage using real failure cases. Experiments cover reconstruction quality, inference speed, and minute-scale video processing.

Critical strengths
  1. The paper addresses a practical problem. Removing test-time subtitle localization can simplify processing for stylized or difficult-to-segment subtitles.
  2. The reported quality–speed trade-off is promising. Region-resolved metrics, a VAE reconstruction reference, and chunk-boundary experiments provide useful supporting evidence.
Critical weaknesses
  1. The comparisons confound reconstruction quality with input information. In Table 1, mask-based baselines lose all content inside the subtitle box, while SEDiT retains the background between characters and information beneath translucent text. Appendix G.6 acknowledges that stroke-level scoring does not remove this asymmetry. Quantitative comparisons with precise masks and composited baseline outputs are needed to establish how much of the advantage comes from the proposed model rather than the input protocol.

  2. The methodological novelty and the explanatory value of the theory are limited. Mask-free removal and sequence-concatenated conditioning have precedents acknowledged in the paper. The main method combines these with standard fine-tuning and loss weighting. The conditional-mean result is valid, but neither conditional uncertainty nor endpoint field error is measured, so it does not explain the trained model's one-step performance. A matched direct conditional-reconstruction baseline would clarify the contribution of flow matching.

  3. Real-video usability and the distillation gains are not sufficiently established. The 30-video usability criterion checks only subtitle residuals, allowing background damage or temporal artifacts to go unpenalized. The improvement from 25/30 to 29/30 therefore establishes more complete erasure, not overall usability. The distillation data size and additional training budget are also unspecified, and there is no matched continued-training or random-mining control to isolate the benefit of failure-driven distillation.

Questions for authors
  1. How do the results change when baselines receive precise subtitle masks and preserve unedited pixels through compositing?
  2. Under matched initialization, data, and training budgets, how does one-step flow matching compare with direct conditional latent reconstruction?
  3. Can you jointly evaluate residuals, background damage, and temporal consistency on real videos, and isolate failure-driven distillation from additional data and training?
Reviewer_Z6sPOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary
  • This paper introduces SEDiT, a mask-free, one-step flow-matching framework designed for video subtitle erasure.
  • Unlike conventional pipelines that rely on sequential OCR/segmentation and inpainting—which inherently bottlenecks performance - based on detection accuracy—the authors propose a one-step inference approach.
  • By theoretical analysis of the conditional velocity field, they show that a single Euler step can recover the conditional mean, assuming low conditional uncertainty due to strong video spatio-temporal context.
  • On their paired VSR-Bench-400 benchmark, SEDiT achieves very good results.
Critical strengths
  • The method is clearly described, and several limitations are explicitly acknowledged.
  • The method is well motivated and the experiment results validate its effectiveness.
  • The method runs much faster than competing approaches, this makes the approach highly viable beyond standard benchmark settings.
Critical weaknesses

None

Questions for authors

None

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.