acceptodds
Under review as a conference paper at ICLR 2027

Enhanced Video Text Editing with Trajectory-Aligned Glyph Rendering

Abstract

Video text editing aims to replace or add text in a video while keeping the rest of the video unchanged, which requires the edited text to be correct in every frame and to move coherently with the scene. Despite the remarkable progress of video diffusion models, they struggle to reproduce exact stroke structures and often produce garbled or wrong characters, especially for characters with complex strokes. To address this, we propose a trajectory-aligned glyph rendering reference that provides explicit per-frame glyph guidance following the position and perspective of the text, and a depth-normalized recognizer feature supervision that supervises the generated text on multi-depth features of a frozen text recognizer with per-depth normalized errors, targeting stroke errors overlooked by the diffusion loss. We further build VTEdit, a benchmark of 288 real-scene clips with 440 annotated text trajectories covering text replacement and text addition, which will be publicly released to facilitate future research. Experiments on VTEdit show that our method outperforms image text editing methods, video editing methods, and commercial models in text accuracy and background preservation, achieving a sentence accuracy of 0.9408, and receives the highest preference in a user study.

Primary area:
generative models
Keywords:
video editing, text editing, diffusion models, inpainting, video generation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_J5HaOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3(+1)
Read review
Paper summary

The paper studies video text editing, where target text must be correct in every frame and move coherently with the scene. The authors propose a trajectory-aligned glyph rendering reference that provides explicit per-frame glyph guidance following text position and perspective, and a depth-normalized recognizer feature supervision that uses multi-depth features from a frozen text recognizer with per-depth normalized errors. They also introduce VTEdit, a benchmark of 288 real-scene clips with 440 annotated text trajectories covering text replacement and text addition. Experiments on VTEdit report 0.9408 sentence accuracy and the highest user preference, with claimed advantages over image text editing methods, video editing methods, and commercial models.

Critical strengths
  1. The problem is practical and clearly motivated: video text editing requires both per-frame glyph correctness and temporal coherence, which are not fully addressed by generic video diffusion models.

  2. The trajectory-aligned glyph rendering reference is a sensible design for providing explicit stroke-level guidance while following text position and perspective across frames.

  3. The depth-normalized recognizer feature supervision targets stroke errors that pixel-level diffusion losses may overlook, and the per-depth normalization is a reasonable way to balance features with different activation scales.

  4. VTEdit covers both text replacement and text addition on real-scene clips, and the reported sentence accuracy and user preference suggest meaningful empirical gains.

Critical weaknesses
  1. Methodological novelty is somewhat limited, and the closest prior work is not adequately compared. Trajectory-aligned glyph conditioning and multi-depth recognizer feature supervision are close to existing glyph-conditioned text editing and recognizer-supervised generation methods, and the technical distinction from the most relevant video text editing work is not established.

  2. The paper emphasizes that edited text must “move coherently with the scene,” yet the reported results focus on text accuracy, background preservation, and user preference. Temporal consistency metrics, flicker analysis, and frame-to-frame stability should be explicitly reported.

  3. VTEdit contains only 288 clips and 440 trajectories, and the balance between replacement and addition, language/font/complexity distribution, train/test separation, and annotation quality control are not sufficiently characterized.

Questions for authors

Please refer to the weaknesses.

4 comments
AuthorsOpenReview ↗

Response to Reviewer J5Ha (Part 1/2)

Thanks for the comments. Reviewer may have some misunderstandings, and we would like to clarify the key points before addressing each concern.

(1) Reviewer may view our techniques as close to existing glyph-conditioned and recognizer-supervised methods. We highlight that they are fundamentally different. First, to the best of our knowledge, we are the first video text editing method to embed perspective-aligned glyphs directly into every frame of the source video, rather than providing them as a separate condition. Second, we reveal two overlooked properties of recognizer features, namely that different depths are complementary yet lie on very different scales, and accordingly propose depth-normalized supervision. Our method also substantially outperforms the most relevant work, SteerVTE (Sen.ACC of 0.9408 vs. 0.4337).

(2) Reviewer may be concerned that VTEdit is too small. In fact, it contains more evaluation videos than ViTeX-Bench and is comparable to VTE-Bench of SteerVTE. We further characterize it in detail, and our method consistently achieves the highest Sen.ACC on both tasks and both languages.

W1. Methodological novelty is somewhat limited, and the closest prior work is not adequately compared. Trajectory-aligned glyph conditioning and multi-depth recognizer feature supervision are close to existing glyph-conditioned text editing and recognizer-supervised generation methods, and the technical distinction from the most relevant video text editing work is not established.

Our method is novel and fundamentally different from existing glyph-conditioned text editing methods, recognizer-supervised generation methods, and the most relevant video text editing work, SteerVTE, as detailed below. We also add a direct comparison, in which our method substantially outperforms SteerVTE (Table C in our response to Reviewer erwr).

1) To the best of our knowledge, we are the first video text editing method to embed the glyphs directly into the source video, rather than providing them as a separate condition. Existing methods (AnyText, FLUX-Text, and SteerVTE) draw the glyphs on a separate canvas or encode them into tokens, while the text region in the source is masked out or left unchanged. In contrast, we embed the target glyphs into the text box of the source video, replacing the original content. The model thus receives the glyphs together with their surrounding scene in a single input, and only needs to restyle the glyphs in place to match the scene, rather than generating the target text from a separate glyph condition.

2) Prior glyph conditions are not aligned with the perspective of the text, whereas our glyphs are perspective-warped into the text box in every frame of the source video. AnyText and FLUX-Text align the glyphs only with the position and orientation of the text box, and cannot capture perspective deformation; SteerVTE provides only a single horizontal, static glyph image. We instead warp the glyphs into the text box of every frame of the source video by a perspective transform, so that they cover the original content and align with the position and perspective of the text throughout the video. Removing this alignment drops Sen.ACC from 0.9408 to 0.8870 (Table F in our response to Reviewer 7qME).

3) Existing methods either supervise a single depth or directly sum the errors of multiple depths, whereas we reveal two overlooked properties of recognizer features and accordingly propose depth-normalized recognizer feature supervision. First, the shallow depth retains strokes while the deep depth is more related to character identity (Figure 2), so a single depth, as in AnyText, cannot capture both. Second, unlike VGG, the depths lie on very different scales (Figure 3), so directly summing their errors, as in JoyType, CharGen, and SteerVTE, lets the shallow depth dominate. Based on these findings, we normalize the error at each depth by the magnitude of its ground-truth features, which further improves text accuracy (Table 4).

AuthorsOpenReview ↗

Response to Reviewer J5Ha (Part 2/2)

W2. The paper emphasizes that edited text must "move coherently with the scene," yet the reported results focus on text accuracy, background preservation, and user preference. Temporal consistency metrics, flicker analysis, and frame-to-frame stability should be explicitly reported.

We thank the reviewer for this comment. The paper already reports temporal consistency through the warping error EwarpE_{warp} over full frames in Table 1, where our method achieves competitive performance. We further add two metrics within each text trajectory to evaluate the edited text specifically: (1) Warp$_{crop}$ for temporal consistency and flicker, which computes the warping error with the flow of the source video and thus measures how much the edited text changes beyond the scene motion; and (2) Traj-ACC for frame-to-frame stability, the fraction of trajectories whose text is correctly recognized in all of their frames. As shown in Table J, our method achieves the lowest Warp$_{crop}$ and by far the highest Traj-ACC, showing that the edited text stays temporally stable and remains correct across frames. The comparison videos in the supplementary material further confirm this, where the compared methods often suffer from flickering or incorrect characters.

Table J. Temporal consistency, flicker, and frame-to-frame text stability on VTEdit.

MethodEwarpE_{warp}↓Warp$_{crop}$↓Traj-ACC↑
FLUX-Text4.15166.62340.4477
VACE1.55932.48370.1409
VideoPainter2.01113.55650.0341
UniVideo1.73995.31970.0000
Kiwi-Edit1.58493.76470.0000
Kling O32.48855.14010.0045
Seedance1.48624.44960.4341
SteerVTE2.33573.69910.3864
Ours1.54332.29020.9159

W3. VTEdit contains only 288 clips and 440 trajectories, and the balance between replacement and addition, language/font/complexity distribution, train/test separation, and annotation quality control are not sufficiently characterized.

VTEdit is not small: it is comparable in scale to recent video text editing benchmarks. As suggested, we characterize each requested aspect below (Table K) and will add these statistics to the paper. We also report results on each subset, where our method consistently achieves the highest Sen.ACC on both tasks and both languages (Table L).

Scale: VTEdit contains 288 real videos, more than ViTeX-Bench [B] (157 evaluation videos) and comparable to VTE-Bench of SteerVTE (300 real videos). Every trajectory is annotated and evaluated in every frame (48.3 frames on average), giving more than 21,000 frame-level evaluations.

Replacement and addition: VTEdit covers both tasks, with 398 replacement and 42 addition trajectories (Table K), and our method achieves the highest Sen.ACC on both, including addition (0.8454 vs. 0.7542 for the second-best method; Table L).

Language, font, and complexity: most trajectories are Chinese (358 of 440), which is more challenging due to its complex strokes, and our method is the best on both languages (Table L). The text comes from real scenes (e.g., signs, screens, packaging, posters), covering diverse real-world fonts, and varies in length with frequent rotation, perspective distortion, large motion, or scale change (Table K).

Train/test separation: our training set (1,050 clips) and VTEdit are split by source video, so no source video appears in both.

Annotation quality control: after automatic annotation, we review every clip by hand: we correct wrongly recognized original text, fix boxes that do not fit or follow the text, and trim or remove text that is fully occluded or leaves the frame. The detailed procedure is given in Q7 of our response to Reviewer 7qME.

Table K. Statistics of VTEdit.

StatisticValue
Clips / trajectories / frame-level annotations288 / 440 / 21,243
Replacement / addition trajectories398 / 42
Chinese / English trajectories358 / 82
Distinct Chinese characters740
Text length, Chinese / English (characters)1–19 / 2–23
Rotation >10°28.4%
Perspective distortion >10°9.5%
Motion >10% of frame diagonal37.3%
Scale change >1.5×17.5%

Table L. Sen.ACC on the subsets of VTEdit.

MethodReplacementAdditionChineseEnglish
FLUX-Text0.84800.75420.83920.8380
VACE0.16750.02280.09370.4158
VideoPainter0.04230.01750.00000.2143
UniVideo0.00030.00000.00030.0000
Kiwi-Edit0.00000.00000.00000.0000
Seedance0.62240.37300.58870.6396
SteerVTE0.43140.45630.41030.5361
Ours0.95040.84540.94870.9040

[B] ViTeX-Bench: Benchmarking High-Fidelity Video Scene Text Editing. NeurIPS 2026.

AuthorsOpenReview ↗

Kind Reminder to Reviewer J5Ha for the Feedback on Our Rebuttal

Dear Reviewer J5Ha,

Thanks for your time and review. As the discussion period is approaching its end, we would really appreciate some feedback from you on our rebuttal.

There seems to be some misunderstanding regarding the novelty of our method. As clarified in our rebuttal, our techniques are fundamentally different from existing glyph-conditioned and recognizer-supervised methods, including the most relevant video text editing work, SteerVTE: we are the first video text editing method to embed perspective-aligned glyphs directly into every frame of the source video, and we reveal two overlooked properties of recognizer features that motivate our depth-normalized supervision. In a direct comparison, our method substantially outperforms SteerVTE on all metrics. With the newly added temporal metrics and VTEdit statistics, our method achieves the best text stability across frames and the highest accuracy on both tasks and both languages. We would sincerely appreciate it if you could consider updating your rating. If you have any further questions, please do not hesitate to let us know.

Thank you for your time and consideration.

Best regards,

Authors of #3938

AuthorsOpenReview ↗

Follow-up on Our Rebuttal

Dear Reviewer J5Ha,

Thank you again for your time. We notice that Reviewer 7qME has raised the rating, acknowledging that our ablations clarify the contributions of trajectory alignment and depth normalization, and that the direct comparison with SteerVTE fills an important gap in the evaluation. We hope that our clarifications on the novelty of our method and its distinction from SteerVTE have also addressed your concerns. As the discussion period is approaching its end, we would really appreciate your feedback. If you have any further questions, please do not hesitate to let us know. We look forward to your reply.

Best regards,

Authors of #3938

Reviewer_7qMEOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents a method for video scene-text editing. It warps target glyphs along the text trajectory in each frame and uses them to guide a video generation model. It also adds normalized multi-layer OCR feature supervision and introduces VTEdit, a benchmark with 288 real-scene clips and 440 text trajectories. The reported sentence accuracy, background preservation, and user preference are strong. The main open issues are whether the method keeps reliable glyph and temporal quality across complete videos, and how much trajectory alignment and loss normalization each contribute on their own.

Critical strengths
  1. The problem is practical, and the method is easy to understand. The trajectory-aligned glyph reference gives the model explicit information about the target text, location, and shape. The self-reconstruction training setup also avoids the need for separately collected before-and-after editing videos.

  2. The reported results are strong. The full model reaches 0.9408 sentence accuracy and reports good background preservation. The paper also includes useful ablations of the glyph reference, OCR feature supervision, feature depth, and normalization, and evaluates both text replacement and insertion. It reports RAFT-based warping error, multi-frame examples, and a user study that includes temporal consistency, so the remaining issue is the coverage of this evidence rather than its absence.

Critical weaknesses
  1. The paper provides temporal evidence, but it does not fully establish stable text quality across complete clips. It reports Ewarp of 1.5433, four sampled frames, and a user study that includes temporal consistency. However, Kiwi-Edit has Ewarp of 1.5849 despite sentence accuracy of 0, showing that low warping error can also reflect stable failure. I could not access full-length output videos in the review materials. The RAFT protocol is also underspecified: the paper does not clearly state the evaluation region, flow source, occlusion handling, or how outputs with different frame counts are aligned. Frame-averaged OCR and the aggregate user study do not directly measure whether each text trajectory stays correct throughout a clip, and no temporal ablation is reported.

  2. The main contributions are not cleanly isolated. The variant without the rendered reference already reaches 0.9013 sentence accuracy, compared with 0.9408 for the full model, but there is no matched non-aligned rendering control. A closely related specialized method, SteerVTE, is not included in the main comparison. Removing feature-loss normalization also changes both the balance across OCR layers and the total OCR-loss scale, so the gain cannot be assigned to layer balancing alone. Evaluation already uses PP-OCRv6 rather than the PP-OCRv3 recognizer used for training, but a cross-family OCR system or focused human glyph check would provide stronger validation. The target texts were manually checked; the missing detail is how original transcripts, automatic trajectories, drift, occlusion, and incorrect boxes were reviewed or corrected.

Questions for authors
  1. Can the authors provide full-length side-by-side videos for representative replacement and insertion cases, and report trajectory-level or clip-level text correctness and temporal stability?

  2. How exactly is the RAFT warping error computed? Please state the evaluation region, flow source, occlusion handling, aggregation rule, and frame-alignment procedure for every baseline.

  3. Can the authors add a matched non-aligned glyph-rendering control to isolate trajectory alignment? If a direct comparison with SteerVTE is not possible, please explain why and narrow the comparison claim.

  4. What is the exact unnormalized OCR-loss formula and total loss scale? Do the main conclusions hold with a cross-family OCR system or a focused human check of glyph correctness? The target texts were manually checked, but how were original transcripts, automatic trajectories, drift, occlusion, and incorrect boxes reviewed or corrected?

6 comments
AuthorsOpenReview ↗

Response to Reviewer 7qME (Part 1/3)

Thanks for the detailed and constructive comments. We believe that our additional experimental results can well address your concerns. If you have any further questions, please do not hesitate to let us know; otherwise, we would appreciate it if you could consider raising your rating. Below, we first highlight a few key points.

(1) Reviewer may wonder whether our evaluation reliably reflects text correctness over time and across OCR systems. We now provide full-length side-by-side videos in the supplementary material and further report Traj-ACC, which requires the text to be correct in all frames. Our method achieves 0.9159, far above the second-best method (0.4477). Our conclusions also hold with the cross-family DuGuangOCR and a focused human glyph check, where our method remains the best (Sen.ACC of 0.9357; human accuracy of 93.8% vs. 52.8% for the second-best Seedance).

(2) Reviewer may think that our contributions are not cleanly isolated. The two new controls isolate them clearly. In the matched non-aligned control, providing glyphs without trajectory alignment brings no gain, while adding the alignment raises Sen.ACC to 0.9408. In the rescaled control, matching the loss scale without normalization brings no gain either, demonstrating that the improvement of our normalization comes from balancing contributions across depths.

(3) Reviewer may be concerned about the missing comparison with SteerVTE. In a direct comparison using its official model, our method substantially outperforms it on all metrics (Sen.ACC of 0.9408 vs. 0.4337).

Q1. The paper provides temporal evidence, but it does not fully establish stable text quality across complete clips. It reports Ewarp of 1.5433, four sampled frames, and a user study that includes temporal consistency. However, Kiwi-Edit has Ewarp of 1.5849 despite sentence accuracy of 0, showing that low warping error can also reflect stable failure. I could not access full-length output videos in the review materials. Frame-averaged OCR and the aggregate user study do not directly measure whether each text trajectory stays correct throughout a clip, and no temporal ablation is reported. Can the authors provide full-length side-by-side videos for representative replacement and insertion cases, and report trajectory-level or clip-level text correctness and temporal stability?

As suggested, the supplementary material provides full-length side-by-side videos of both replacement and addition, showing that the edited text stays correct and stable throughout each clip. To directly quantify this, we additionally report two trajectory-level metrics: 1) Traj-ACC, the fraction of trajectories whose text is correctly recognized in all of their frames, which requires the text to be both correct and stable, so stable failures such as Kiwi-Edit score 0; and 2) Warp$_{crop},thewarpingerrorwithineachtrajectoryregionusingtheopticalflowofthesourcevideo,followingViTeX−Bench[B].AsshowninTableD,ourmethodachievesbyfarthehighestTraj−ACC(0.9159)andthelowestWarp, the warping error within each trajectory region using the optical flow of the source video, following ViTeX-Bench [B]. As shown in Table D, our method achieves by far the highest Traj-ACC (0.9159) and the lowest Warp_{crop}$ (2.2902). For the temporal ablation, Table E shows that adding the two proposed techniques raises Traj-ACC from 0.7364 to 0.9159 and reduces both warping errors.

Table D. Trajectory-level text correctness and temporal stability on VTEdit. Ours achieves the highest correctness and stability across all metrics.

MethodSen.ACC↑Traj-ACC↑Warp$_{crop}$↓
FLUX-Text0.83930.44776.6234
VACE0.15320.14092.4837
VideoPainter0.03950.03413.5565
UniVideo0.00020.00005.3197
Kiwi-Edit0.00000.00003.7647
Kling O30.18880.00455.1401
Seedance0.59800.43414.4496
SteerVTE0.43370.38643.6991
Ours0.94080.91592.2902

Table E. Traj-ACC and temporal consistency metrics for the ablation variants on VTEdit.

ReferenceSupervisionNormalizationSen.ACC↑Traj-ACC↑EwarpE_{warp}↓Warp$_{crop}$↓
✗✗–0.81540.73642.10812.5837
✓✗–0.91720.88182.06432.3220
✗✓✓0.90130.86591.94372.5395
✓✓✗0.92700.88861.96802.2815
✓✓✓0.94080.91591.54332.2902

[B] ViTeX-Bench: Benchmarking High-Fidelity Video Scene Text Editing. NeurIPS 2026.

AuthorsOpenReview ↗

Response to Reviewer 7qME (Part 2/3)

Q2. The RAFT protocol is also underspecified: the paper does not clearly state the evaluation region, flow source, occlusion handling, or how outputs with different frame counts are aligned. How exactly is the RAFT warping error computed? Please state the evaluation region, flow source, occlusion handling, aggregation rule, and frame-alignment procedure for every baseline.

Thank you for pointing this out. We will add these details to the appendix.

RAFT protocol. For EwarpE_{\mathrm{warp}} in Table 1, we follow the RAFT-based warping-error implementation in EvalCrafter [C]. The evaluation region is the full frame, with all edited videos resized to 480×832 before evaluation. For the flow source, we use consecutive frames of each method's edited video. We estimate forward and backward flow with RAFT and use the forward flow to warp the second frame to the first. For occlusion handling, we exclude pixels that fail the forward–backward consistency check or lie at detected motion boundaries. For aggregation, we average the squared RGB distance over the retained pixels in each frame pair, then over frame pairs within each clip, and finally over clips. We report this mean multiplied by 10310^3.

Frame-alignment procedure. Let FF be the number of source frames.

1) Direct same-frame output. Our method, VACE, UniVideo, and Kiwi-Edit take FF frames and return FF frames. FLUX-Text edits each source frame independently. For VideoPainter, we skip the default 8 fps subsampling to retain all FF frames.

2) Padding and truncation. For SteerVTE, we pad shorter clips to 49 frames by appending source frames in reverse order. For Seedance and Kling O3, we encode all FF source frames at 24 fps to match the output frame rate and apply the same padding when needed to reach the minimum input duration (2 s for Seedance and 3 s for Kling O3). For all three methods, we retain the first FF output frames as the edited clip.

[C] EvalCrafter: Benchmarking and Evaluating Large Video Generation Models. CVPR 2024.

Q3. The main contributions are not cleanly isolated. The variant without the rendered reference already reaches 0.9013 sentence accuracy, compared with 0.9408 for the full model, but there is no matched non-aligned rendering control. Can the authors add a matched non-aligned glyph-rendering control to isolate trajectory alignment?

As suggested, we add a matched non-aligned glyph-rendering control. Specifically, we render the target text into a plain glyph image without trajectory alignment and resize it directly into the bounding rectangle of each text box, while keeping all other settings identical to our full model. As shown in Table F, providing the rendered reference without trajectory alignment does not improve over the no-reference baseline (0.8870 vs. 0.9013 in Sen.ACC). In contrast, incorporating trajectory alignment boosts Sen.ACC to 0.9408 while achieving the best Traj-ACC and Warp$_{crop}$. This cleanly isolates trajectory alignment and demonstrates that the accuracy gain comes from trajectory alignment rather than the rendered reference alone.

Table F. Matched non-aligned glyph-rendering control on VTEdit.

Glyph renderingTrajectory alignmentSen.ACC↑Traj-ACC↑Warp$_{crop}$↓
✗–0.90130.86592.5395
✓✗0.88700.85002.5080
✓✓0.94080.91592.2902

Q4. A closely related specialized method, SteerVTE, is not included in the main comparison. If a direct comparison with SteerVTE is not possible, please explain why and narrow the comparison claim.

As suggested, we now provide a direct evaluation using its official model. As shown in Table G, our method substantially outperforms SteerVTE across all metrics. We will include SteerVTE in Table 1 of the revised paper.

Table G. Direct comparison with SteerVTE on VTEdit.

MethodSen.ACC↑NED↑EwarpE_{warp}↓PSNR$_{bg}$↑SSIM\_{bg}\↑
SteerVTE0.43370.70142.335732.980.9578
Ours0.94080.98621.543336.110.9736
AuthorsOpenReview ↗

Response to Reviewer 7qME (Part 3/3)

Q5. Removing feature-loss normalization also changes both the balance across OCR layers and the total OCR-loss scale, so the gain cannot be assigned to layer balancing alone. What is the exact unnormalized OCR-loss formula and total loss scale?

The unnormalized OCR loss is given by

Lrecw/o=1K∑k=1K1Nk∑i∈S∥ϕk(Ci(V^))−ϕk(Ci(V))∥1,\mathcal{L}_{\text{rec}}^{\text{w/o}}=\frac{1}{K}\sum_{k=1}^{K}\frac{1}{N_k}\sum_{i\in\mathcal{S}}\big\Vert \phi_k\big(\mathcal{C}_i(\hat{V})\big)-\phi_k\big(\mathcal{C}_i(V)\big)\big\Vert _1,

where NkN_k is the total number of feature elements at depth kk over the text crops from frames in S\mathcal{S}, and all other notation follows Eq. (3). Averaged over 128 training samples, Lrecw/o\mathcal{L}_{\text{rec}}^{\text{w/o}} is 0.31, while our normalized Lrec\mathcal{L}_{\text{rec}} in Eq. (3) is 0.37. To separate the cross-depth balance from the total loss scale, we further train a variant with Lrecw/o\mathcal{L}_{\text{rec}}^{\text{w/o}} rescaled to the scale of our Lrec\mathcal{L}_{\text{rec}}. As shown in Table H, this rescaled variant performs on par with the unnormalized one and remains clearly below ours. This shows that matching the loss scale alone brings no gain, and the improvement comes from balancing contributions across depths.

Table H. OCR-loss scale and text accuracy on VTEdit.

VariantLoss scaleSen.ACC↑Traj-ACC↑
w/o normalization0.310.92700.8886
w/o normalization, rescaled0.370.92560.8841
Normalized (Ours)0.370.94080.9159

Q6. Evaluation already uses PP-OCRv6 rather than the PP-OCRv3 recognizer used for training, but a cross-family OCR system or focused human glyph check would provide stronger validation. Do the main conclusions hold with a cross-family OCR system or a focused human check of glyph correctness?

As suggested, we provide both cross-family OCR evaluation and human assessment. First, following AnyText and FLUX-Text, we re-evaluate all methods using DuGuangOCR, an independent recognizer outside the PP-OCR family. As shown in Table I, our method consistently achieves the highest sentence and trajectory accuracy under DuGuangOCR. Furthermore, 4 annotators performed a blinded human glyph check on 100 trajectories sampled across tasks and languages, judging a result as correct only if the target text is correctly rendered in every frame. Our method achieves a human accuracy of 93.8%, clearly above all compared methods, demonstrating that our main conclusions hold across different OCR systems and human perception.

Table I. Text accuracy under PP-OCRv6, DuGuangOCR, and human evaluation on representative methods.

MethodPP-OCRv6 Sen.ACC↑PP-OCRv6 Traj-ACC↑DuGuangOCR Sen.ACC↑DuGuangOCR Traj-ACC↑Human Acc. (%)↑
FLUX-Text0.83930.44770.82650.395543.8
VACE0.15320.14090.15250.1386–
VideoPainter0.03950.03410.03500.0318–
UniVideo0.00020.00000.00020.0000–
Kiwi-Edit0.00000.00000.00000.0000–
Kling O30.18880.00450.15790.0045–
Seedance0.59800.43410.56870.359152.8
SteerVTE0.43370.38640.41250.361434.3
Ours0.94080.91590.93570.879593.8

Q7. The target texts were manually checked, but how were original transcripts, automatic trajectories, drift, occlusion, and incorrect boxes reviewed or corrected?

All clips in both the training set and VTEdit were manually inspected and corrected by the authors using a lightweight local web tool that overlays bounding boxes onto the video. Below are the specific review and correction rules for each aspect, which we will include in the Appendix:

  • Automatic trajectories: Per-frame detections from PP-OCRv6 are associated via box IoU and text similarity. Noisy or fragmented trajectories are discarded directly during review.
  • Original transcripts: Initial transcripts are generated by majority voting across frames, then manually cross-checked against the video frames and corrected if any character is misrecognized.
  • Drift and incorrect boxes: Annotators check box alignment during playback. Overly loose or clipped boxes are fixed by adjusting their corner vertices; severe tracking drift is corrected by re-annotating keyframes and interpolating across intermediate frames.
  • Occlusion: Boxes are retained under slight occlusion. If the text is fully occluded or moves out of frame (leaving partial characters), the clip is truncated at the boundary frame or the trajectory is removed.
  • Addition task trajectories: Initial boxes are manually drawn on keyframes and propagated via planar homography tracking. The tracked boxes are then manually inspected in the tool, with severe drift corrected via keyframe re-annotation.
Reviewer_erwrOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents a video text editing framework formulated as text-conditioned inpainting along trajectories. It introduces two key designs: a trajectory-aligned glyph rendering reference and depth-normalized recognizer feature supervision. The authors also contribute VTEdit, a real-scene benchmark with 440 annotated trajectories.

Critical strengths

Strength:

  1. The video text editing problem is practical, and formulating it as text-conditioned inpainting along trajectories is reasonable.

  2. The proposed methods have a clear motivation, supported by meaningful component comparisons and ablations.

  3. The proposed benchmark makes a valuable contribution to the research community.

Critical weaknesses
  1. My main concern is the fairness of the comparison in Table 1. Although the proposed method achieves better performance, it utilizes more prior knowledge than the baselines. Specifically, it relies on extra bounding-box masks, a more powerful base model (HunyuanVideo 1.5), Glyph-ByT5, and additional training data. These factors make it unclear how much the two proposed techniques actually contribute to the overall performance gains.

  2. The baseline SteerVTE is discussed in the related work, but it is not compared in the experiments.

Questions for authors
  1. Could the authors provide more details for all compared methods, such as their required inputs, training/inference time, additional pre-trained models, and training data?

  2. Could the authors provide experimental results or a direct comparison with SteerVTE?

2 comments
AuthorsOpenReview ↗

Response to Reviewer erwr (Part 1/2)

W1 & Q1. ...the fairness of the comparison in Table 1... it relies on extra bounding-box masks, a more powerful base model (HunyuanVideo 1.5), Glyph-ByT5, and additional training data... how much the two proposed techniques actually contribute... Could the authors provide more details for all compared methods...?

We highlight that the comparison in Table 1 is fair: our method takes the same input information as the compared methods, does not rely on a more powerful base model, and uses neither more prior knowledge nor additional training data. In fact, it uses far less training data than the compared methods. Our superior performance relies on the two proposed techniques. Removing them from our full model drops Sen.ACC from 0.9408 to 0.8154, below the strongest compared method. The details of all compared methods requested in Q1 are listed in Table A and will be added to the Appendix. We address each factor below.

1) The bounding-box masks are not extra prior knowledge, but the same input information as the compared methods. Every method takes the source video and the target text, and is told which text to edit: FLUX-Text, VACE, VideoPainter, and SteerVTE use exactly the same bounding-box masks as ours, and instruction-based methods specify the source text with an instruction such as "change X to Y". With the same masks, these box-based methods still fall clearly behind ours. The difference lies in how this input is used: we warp the glyphs into the text box of every frame by a perspective transform, so that the glyph condition follows the position and perspective of the text throughout the video. This aligned glyph reference is one of the core contributions of our work.

2) HunyuanVideo 1.5 is not a more powerful base model, and Glyph-ByT5 is part of it. HunyuanVideo 1.5 has only 8.3B parameters, fewer than the base models of FLUX-Text, VACE, UniVideo, and SteerVTE (12B to 14B), and Glyph-ByT5 is its native text encoder. Even with Glyph-ByT5, its text rendering ability is limited: on VTR-Bench [A], a benchmark for text rendering in video generation, its word error rate is as high as 0.969 (lower is better), far behind Seedance (0.641). As shown in 4), without the two proposed techniques, this base model, including Glyph-ByT5, does not outperform the compared methods.

3) We use no additional pre-trained models at inference and far less training data. At inference, we use no pre-trained model beyond the base model, and PP-OCRv3 is used only for the training loss. In contrast, VideoPainter, UniVideo, Kiwi-Edit, and SteerVTE all rely on additional pre-trained models at inference, such as FLUX.1-Fill-dev, Qwen2.5-VL, and a PP-OCRv3 glyph encoder (Table A). We train on only 1,050 video clips, about 100 to 1,000 times fewer than FLUX-Text, VideoPainter, Kiwi-Edit, and SteerVTE. In addition, our method runs faster than most open-source compared methods (Table A).

4) The two proposed techniques improve Sen.ACC from 0.8154 to 0.9408. To measure how much the two proposed techniques actually contribute to the overall performance gains, we train a variant without both techniques, with the same base model (including Glyph-ByT5), bounding-box masks, and training data. It reaches a Sen.ACC of only 0.8154, below the strongest compared method. Adding either technique brings a clear gain, and combining them performs best (Table B).

Table A. Details of the compared methods. Inference time is measured for a 49-frame clip at 480×832 with the default sampling steps of each method on a single A800 GPU. N/A: not reported.

MethodRequired inputsBase modelAdditional pre-trained modelsTraining dataTraining timeInference time
FLUX-Textframe + box + caption + target textFLUX.1-Fill-dev (12B)–100K imagesN/A964 s
VACEvideo + box + caption + target textWan2.1-VACE-14B–N/A200K steps, 128× A100702 s
VideoPaintervideo + box + caption + target textCogVideoX-5B-I2VFLUX.1-Fill-dev (12B, first frame)390K clipsN/A272 s
UniVideovideo + edit instructionHunyuanVideo (13B)Qwen2.5-VL-7B500K image pairs + internal data35K steps1201 s
Kiwi-Editvideo + edit instructionWan2.2-TI2V-5BQwen2.5-VL-3B477K video pairsN/A58 s
Kling O3 / Seedancevideo + edit instructionundisclosedundisclosedundisclosedundisclosedN/A
SteerVTEvideo + box + source & target textWan2.1-VACE-14BQwen2.5-VL-3B, PP-OCRv3 (glyph encoder)800K images + 200K clips83K steps, 16× H100260 s
Oursvideo + box + caption + target textHunyuanVideo 1.5 (8.3B)PP-OCRv3 (training only)1,050 clips30K steps, 8× Ascend 910B148 s

Table B. Contribution of the two techniques.

ReferenceSupervisionSen.ACC↑NED↑
✗✗0.81540.9467
✓✗0.91720.9685
✗✓0.90130.9658
✓✓0.94080.9862

[A] VTR-Bench: A Systematic Benchmark for Evaluating Visual Text Rendering in Video Generation. arXiv 2026.

AuthorsOpenReview ↗

Response to Reviewer erwr (Part 2/2)

W2 & Q2. The baseline SteerVTE is discussed in the related work, but it is not compared in the experiments. Could the authors provide experimental results or a direct comparison with SteerVTE?

The SteerVTE authors released their code after our submission. We have since evaluated their released model on VTEdit. As shown in Table C, our method achieves higher sentence accuracy (0.9408 vs. 0.4337), lower warping error, and better background preservation. We will add this comparison to Table 1.

Table C. Comparison with SteerVTE on VTEdit.

MethodSen.ACC↑NED↑EwarpE_{warp}↓PSNR$_{bg}$↑SSIM_{bg}\↑
SteerVTE0.43370.70142.335732.980.9578
Ours0.94080.98621.543336.110.9736

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of the Discussion

Dear Reviewers and Area Chairs,

We sincerely thank all reviewers for their constructive feedback on our work. We are encouraged that the majority of reviewers hold a positive view of our work: Reviewer erwr initially provided a positive assessment, and Reviewer 7qME raised the rating from 2 to 3 after reading our rebuttal, acknowledging that our ablations clarify the contributions of our techniques and that the comparison with SteerVTE strengthens our evaluation.

Regarding the concerns of Reviewer J5Ha, our rebuttal clarifies that our method is fundamentally different from existing glyph-conditioned and recognizer-supervised methods, including SteerVTE: we are the first video text editing method to embed perspective-aligned glyphs directly into every frame of the source video, and we reveal two overlooked properties of recognizer features that motivate our depth-normalized supervision. Our method also substantially outperforms SteerVTE on all metrics and achieves the best text stability across frames.

We believe that our work provides an effective solution for accurate video text editing, and we will publicly release our code, model, and VTEdit to facilitate future research.

We remain available for any further questions until the end of the discussion period. We appreciate your time and consideration throughout this review process.

Best regards,

Authors of #3938

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.