- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces MFVSR, a benchmark for video subtitle removal comprising 3,000 paired videos, 16 languages, and 24 subtitle configurations. Experiments on six methods show that good subtitle restoration does not necessarily imply faithful background preservation or complete video outputs.
Critical strengths
The dataset construction is systematic. This paper examines subtitle regions, their bounding boxes, and full frames separately. The results show residual text, incorrect background reconstruction, and unwanted changes outside subtitle regions. Some methods also return incomplete videos.
Critical weaknesses
- Limited additional insight. Relative to existing diagnostic evaluation approaches, the contribution primarily consists of expanding the dataset, organizing subtitle variations, and integrating evaluation protocols. The main paper focuses on aggregate metrics and stratified comparisons, without a detailed analysis of individual failure cases. Qualitative failure examples are confined to Appendix E, where they illustrate errors but do not systematically explain why existing methods fail or establish new insights that could guide method development.
- Limited temporal and real-world validation. The human study provides limited support for the temporal component. The paper also reports no separate results on the 953 naturally subtitled videos. It therefore remains unclear how well the benchmark’s findings transfer to real-world subtitle removal.
- Incomplete evaluation of complex subtitle styles. Audit300 excludes subtitles with background boxes. Their separate evaluation omits preservation and BBox temporal metrics because the annotations may not cover the full subtitle area. This limits the benchmark’s coverage of complex styles.
Questions for authors
See weaknesses.