- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3(+1)
Read review
Paper summary
The paper studies video text editing, where target text must be correct in every frame and move coherently with the scene. The authors propose a trajectory-aligned glyph rendering reference that provides explicit per-frame glyph guidance following text position and perspective, and a depth-normalized recognizer feature supervision that uses multi-depth features from a frozen text recognizer with per-depth normalized errors. They also introduce VTEdit, a benchmark of 288 real-scene clips with 440 annotated text trajectories covering text replacement and text addition. Experiments on VTEdit report 0.9408 sentence accuracy and the highest user preference, with claimed advantages over image text editing methods, video editing methods, and commercial models.
Critical strengths
-
The problem is practical and clearly motivated: video text editing requires both per-frame glyph correctness and temporal coherence, which are not fully addressed by generic video diffusion models.
-
The trajectory-aligned glyph rendering reference is a sensible design for providing explicit stroke-level guidance while following text position and perspective across frames.
-
The depth-normalized recognizer feature supervision targets stroke errors that pixel-level diffusion losses may overlook, and the per-depth normalization is a reasonable way to balance features with different activation scales.
-
VTEdit covers both text replacement and text addition on real-scene clips, and the reported sentence accuracy and user preference suggest meaningful empirical gains.
Critical weaknesses
-
Methodological novelty is somewhat limited, and the closest prior work is not adequately compared. Trajectory-aligned glyph conditioning and multi-depth recognizer feature supervision are close to existing glyph-conditioned text editing and recognizer-supervised generation methods, and the technical distinction from the most relevant video text editing work is not established.
-
The paper emphasizes that edited text must “move coherently with the scene,” yet the reported results focus on text accuracy, background preservation, and user preference. Temporal consistency metrics, flicker analysis, and frame-to-frame stability should be explicitly reported.
-
VTEdit contains only 288 clips and 440 trajectories, and the balance between replacement and addition, language/font/complexity distribution, train/test separation, and annotation quality control are not sufficiently characterized.
Questions for authors
Please refer to the weaknesses.
4 comments
Response to Reviewer J5Ha (Part 2/2)
W2. The paper emphasizes that edited text must "move coherently with the scene," yet the reported results focus on text accuracy, background preservation, and user preference. Temporal consistency metrics, flicker analysis, and frame-to-frame stability should be explicitly reported.
We thank the reviewer for this comment. The paper already reports temporal consistency through the warping error over full frames in Table 1, where our method achieves competitive performance. We further add two metrics within each text trajectory to evaluate the edited text specifically: (1) Warp$_{crop}$ for temporal consistency and flicker, which computes the warping error with the flow of the source video and thus measures how much the edited text changes beyond the scene motion; and (2) Traj-ACC for frame-to-frame stability, the fraction of trajectories whose text is correctly recognized in all of their frames. As shown in Table J, our method achieves the lowest Warp$_{crop}$ and by far the highest Traj-ACC, showing that the edited text stays temporally stable and remains correct across frames. The comparison videos in the supplementary material further confirm this, where the compared methods often suffer from flickering or incorrect characters.
Table J. Temporal consistency, flicker, and frame-to-frame text stability on VTEdit.
| Method | ↓ | Warp$_{crop}$↓ | Traj-ACC↑ |
|---|---|---|---|
| FLUX-Text | 4.1516 | 6.6234 | 0.4477 |
| VACE | 1.5593 | 2.4837 | 0.1409 |
| VideoPainter | 2.0111 | 3.5565 | 0.0341 |
| UniVideo | 1.7399 | 5.3197 | 0.0000 |
| Kiwi-Edit | 1.5849 | 3.7647 | 0.0000 |
| Kling O3 | 2.4885 | 5.1401 | 0.0045 |
| Seedance | 1.4862 | 4.4496 | 0.4341 |
| SteerVTE | 2.3357 | 3.6991 | 0.3864 |
| Ours | 1.5433 | 2.2902 | 0.9159 |
W3. VTEdit contains only 288 clips and 440 trajectories, and the balance between replacement and addition, language/font/complexity distribution, train/test separation, and annotation quality control are not sufficiently characterized.
VTEdit is not small: it is comparable in scale to recent video text editing benchmarks. As suggested, we characterize each requested aspect below (Table K) and will add these statistics to the paper. We also report results on each subset, where our method consistently achieves the highest Sen.ACC on both tasks and both languages (Table L).
Scale: VTEdit contains 288 real videos, more than ViTeX-Bench [B] (157 evaluation videos) and comparable to VTE-Bench of SteerVTE (300 real videos). Every trajectory is annotated and evaluated in every frame (48.3 frames on average), giving more than 21,000 frame-level evaluations.
Replacement and addition: VTEdit covers both tasks, with 398 replacement and 42 addition trajectories (Table K), and our method achieves the highest Sen.ACC on both, including addition (0.8454 vs. 0.7542 for the second-best method; Table L).
Language, font, and complexity: most trajectories are Chinese (358 of 440), which is more challenging due to its complex strokes, and our method is the best on both languages (Table L). The text comes from real scenes (e.g., signs, screens, packaging, posters), covering diverse real-world fonts, and varies in length with frequent rotation, perspective distortion, large motion, or scale change (Table K).
Train/test separation: our training set (1,050 clips) and VTEdit are split by source video, so no source video appears in both.
Annotation quality control: after automatic annotation, we review every clip by hand: we correct wrongly recognized original text, fix boxes that do not fit or follow the text, and trim or remove text that is fully occluded or leaves the frame. The detailed procedure is given in Q7 of our response to Reviewer 7qME.
Table K. Statistics of VTEdit.
| Statistic | Value |
|---|---|
| Clips / trajectories / frame-level annotations | 288 / 440 / 21,243 |
| Replacement / addition trajectories | 398 / 42 |
| Chinese / English trajectories | 358 / 82 |
| Distinct Chinese characters | 740 |
| Text length, Chinese / English (characters) | 1–19 / 2–23 |
| Rotation >10° | 28.4% |
| Perspective distortion >10° | 9.5% |
| Motion >10% of frame diagonal | 37.3% |
| Scale change >1.5× | 17.5% |
Table L. Sen.ACC on the subsets of VTEdit.
| Method | Replacement | Addition | Chinese | English |
|---|---|---|---|---|
| FLUX-Text | 0.8480 | 0.7542 | 0.8392 | 0.8380 |
| VACE | 0.1675 | 0.0228 | 0.0937 | 0.4158 |
| VideoPainter | 0.0423 | 0.0175 | 0.0000 | 0.2143 |
| UniVideo | 0.0003 | 0.0000 | 0.0003 | 0.0000 |
| Kiwi-Edit | 0.0000 | 0.0000 | 0.0000 | 0.0000 |
| Seedance | 0.6224 | 0.3730 | 0.5887 | 0.6396 |
| SteerVTE | 0.4314 | 0.4563 | 0.4103 | 0.5361 |
| Ours | 0.9504 | 0.8454 | 0.9487 | 0.9040 |
[B] ViTeX-Bench: Benchmarking High-Fidelity Video Scene Text Editing. NeurIPS 2026.
Kind Reminder to Reviewer J5Ha for the Feedback on Our Rebuttal
Dear Reviewer J5Ha,
Thanks for your time and review. As the discussion period is approaching its end, we would really appreciate some feedback from you on our rebuttal.
There seems to be some misunderstanding regarding the novelty of our method. As clarified in our rebuttal, our techniques are fundamentally different from existing glyph-conditioned and recognizer-supervised methods, including the most relevant video text editing work, SteerVTE: we are the first video text editing method to embed perspective-aligned glyphs directly into every frame of the source video, and we reveal two overlooked properties of recognizer features that motivate our depth-normalized supervision. In a direct comparison, our method substantially outperforms SteerVTE on all metrics. With the newly added temporal metrics and VTEdit statistics, our method achieves the best text stability across frames and the highest accuracy on both tasks and both languages. We would sincerely appreciate it if you could consider updating your rating. If you have any further questions, please do not hesitate to let us know.
Thank you for your time and consideration.
Best regards,
Authors of #3938
Follow-up on Our Rebuttal
Dear Reviewer J5Ha,
Thank you again for your time. We notice that Reviewer 7qME has raised the rating, acknowledging that our ablations clarify the contributions of trajectory alignment and depth normalization, and that the direct comparison with SteerVTE fills an important gap in the evaluation. We hope that our clarifications on the novelty of our method and its distinction from SteerVTE have also addressed your concerns. As the discussion period is approaching its end, we would really appreciate your feedback. If you have any further questions, please do not hesitate to let us know. We look forward to your reply.
Best regards,
Authors of #3938
Response to Reviewer J5Ha (Part 1/2)
Thanks for the comments. Reviewer may have some misunderstandings, and we would like to clarify the key points before addressing each concern.
(1) Reviewer may view our techniques as close to existing glyph-conditioned and recognizer-supervised methods. We highlight that they are fundamentally different. First, to the best of our knowledge, we are the first video text editing method to embed perspective-aligned glyphs directly into every frame of the source video, rather than providing them as a separate condition. Second, we reveal two overlooked properties of recognizer features, namely that different depths are complementary yet lie on very different scales, and accordingly propose depth-normalized supervision. Our method also substantially outperforms the most relevant work, SteerVTE (Sen.ACC of 0.9408 vs. 0.4337).
(2) Reviewer may be concerned that VTEdit is too small. In fact, it contains more evaluation videos than ViTeX-Bench and is comparable to VTE-Bench of SteerVTE. We further characterize it in detail, and our method consistently achieves the highest Sen.ACC on both tasks and both languages.
W1. Methodological novelty is somewhat limited, and the closest prior work is not adequately compared. Trajectory-aligned glyph conditioning and multi-depth recognizer feature supervision are close to existing glyph-conditioned text editing and recognizer-supervised generation methods, and the technical distinction from the most relevant video text editing work is not established.
Our method is novel and fundamentally different from existing glyph-conditioned text editing methods, recognizer-supervised generation methods, and the most relevant video text editing work, SteerVTE, as detailed below. We also add a direct comparison, in which our method substantially outperforms SteerVTE (Table C in our response to Reviewer erwr).
1) To the best of our knowledge, we are the first video text editing method to embed the glyphs directly into the source video, rather than providing them as a separate condition. Existing methods (AnyText, FLUX-Text, and SteerVTE) draw the glyphs on a separate canvas or encode them into tokens, while the text region in the source is masked out or left unchanged. In contrast, we embed the target glyphs into the text box of the source video, replacing the original content. The model thus receives the glyphs together with their surrounding scene in a single input, and only needs to restyle the glyphs in place to match the scene, rather than generating the target text from a separate glyph condition.
2) Prior glyph conditions are not aligned with the perspective of the text, whereas our glyphs are perspective-warped into the text box in every frame of the source video. AnyText and FLUX-Text align the glyphs only with the position and orientation of the text box, and cannot capture perspective deformation; SteerVTE provides only a single horizontal, static glyph image. We instead warp the glyphs into the text box of every frame of the source video by a perspective transform, so that they cover the original content and align with the position and perspective of the text throughout the video. Removing this alignment drops Sen.ACC from 0.9408 to 0.8870 (Table F in our response to Reviewer 7qME).
3) Existing methods either supervise a single depth or directly sum the errors of multiple depths, whereas we reveal two overlooked properties of recognizer features and accordingly propose depth-normalized recognizer feature supervision. First, the shallow depth retains strokes while the deep depth is more related to character identity (Figure 2), so a single depth, as in AnyText, cannot capture both. Second, unlike VGG, the depths lie on very different scales (Figure 3), so directly summing their errors, as in JoyType, CharGen, and SteerVTE, lets the shallow depth dominate. Based on these findings, we normalize the error at each depth by the magnitude of its ground-truth features, which further improves text accuracy (Table 4).