acceptodds
Under review as a conference paper at ICLR 2027

Efficient Scene-Text Refinement via Pixel-Space Grouped Image Generation

Abstract

Generated images often have coherent layouts and visual content, yet local errors in small text and complex characters still limit their readability and practical use. Scene-text refinement requires accuracy, preservation, and efficiency. Existing methods based on variational autoencoders (VAEs) can lose fine strokes during reconstruction, while full-image editing can shift background colors. In this work, we introduce PixText, a model that formulates local text refinement as grouped image generation in pixel space. Conditioned on the original image, PixText generates only the target regions, preserves unedited content, and uses MeanFlow for four-step generation. We further introduce Text Refinement Bench to evaluate text correction and image preservation in generated images. On this benchmark, PixText achieves 83.06% glyph pass and 94.66% exact target-character OCR accuracy, versus 36.77% and 52.31% for the strongest baseline. Its four-step MeanFlow variant retains an 81.84% pass rate with less than one GPU-second of amortized generation cost per image. The model and benchmark will be open-sourced upon formal release.

Primary area:
generative models
Keywords:
diffusion model, text-to-image generation, flow matching model

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_Uk3TOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposed a VAE-free visual text editing and refinement framework. It generates text patches based on the conditions of the original images and pastes the text patches back into the original image for editing/refinement. To reduce visual artifacts introduced by JiT, the paper adopts uniform timestep sampling during continued training to reduce patch-grid artifacts and modifies the x-prediction sampling update to eliminate residual noise caused by incomplete final denoising. Additionally, the paper introduces the Text Refinement Bench for fine-grained text correction, glyph fidelity, and image preservation evaluation.

Critical strengths
  1. The paper demonstrates the effectiveness of VAE-free pixel-space generation for localized scene text editing, supported by large-scale training and extensive experiments.
  2. The paper shows that uniform timestep sampling can mitigate patch-boundary artifacts in pixel-space generation, with improvements demonstrated on both PixText and MiniT2I.
Critical weaknesses
  1. PixText shares a similar task setting with existing methods that perform localized text generation and editing, such as UTDesign. The authors should clarify the differences between these approaches and consider including representative methods in the experimental comparison to better demonstrate PixText's advantages.

  2. Although the architecture supports variable-sized crops, the paper does not clearly establish how text scale affects editing quality and computational cost. The inference-time resizing policy, supported crop-size range, and treatment of large text regions need clarification. Results broken down by text size would help determine whether the reported advantages hold across scales.

Questions for authors
  1. Could the authors include comparisons with existing localized text editing methods, such as UTDesign, or clarify the differences in their task settings and input requirements?

  2. Although CCPE supports variable-sized crops, how does PixText perform across different text scales, particularly for large characters or words? A scale-wise evaluation of editing accuracy and computational cost would be helpful.

  3. How are the erroneous text regions and target glyphs obtained in real-world applications? Given that OCR may fail to detect subtle glyph errors, does the proposed refinement pipeline require manual intervention?

5 comments
AuthorsOpenReview ↗

Response to Reviewer Uk3T(part1)

Critical Weakness 1 and Question 1: Direct comparison with UTDesign and differences in task settings

Reviewer concern. Could the authors include comparisons with existing localized text editing methods, such as UTDesign, or clarify the differences in task settings and input requirements?

Response. We thank the reviewer for suggesting this comparison. UTDesign is a strong and representative localized text-editing method, and its flexible style-reference design makes it an important baseline for our comparison. We have added UTDesign to the related-work discussion and included a formal citation in the revision. We have also evaluated it directly on Text Refinement Bench using the same 1,034 images, 2,638 targets, target boxes, and scoring protocol. For UTDesign, we add up to three neighboring glyphs from the same line as style references.

Table 1. Direct comparison with UTDesign on Text Refinement Bench

MethodOCR Acc ↑CJK-CLIP Pass ↑R@1 ↑FID ↓LPIPS ↓GPU-s/img ↓
UTDesign v1.083.43%56.03%80.29%0.9690.003245.157
PixText Teacher94.66%83.06%92.95%0.340.00104.514
PixText MeanFlow94.58%81.84%92.57%0.380.00110.942

Teacher improves over UTDesign by 11.23 pp OCR, 27.03 pp Pass, and 12.66 pp R@1. MeanFlow improves by 11.15 pp, 25.81 pp, and 12.28 pp, respectively. MeanFlow also uses only four sampling steps and achieves 0.942 GPU-s/img, compared with 28 steps and 5.157 GPU-s/img for UTDesign—an approximately 5.5× lower observed generation cost. MeanFlow has 0.954B parameters, while UTDesign’s transformer alone has approximately 1.108B parameters, excluding its visual encoders, VAE, and LaMa background-repair module.

The scale-wise results clarify where the methods differ. PixText leads overall and on small and medium text. On the combined ≥64 px bin, UTDesign reaches 94.9% OCR, compared with 92.4% for MeanFlow and 92.0% for Teacher, indicating that PixText remains comparable on large characters while providing its largest gains on small and medium text.

The two methods share compatible high-level inputs: both can edit using the original image, target regions, and target text. UTDesign’s style references are also extracted from the original image, so it does not require an additional reference image or ground-truth glyph crop. Their task goals, however, differ. UTDesign targets stylized text replacement and generation in graphic design, including layout adjustment. PixText targets localized error correction after image generation: it repairs a small number of incorrect characters or malformed strokes at their original locations, especially small and structurally complex characters, while preserving the existing layout and image content.

Their conditioning and generation mechanisms also differ. PixText directly uses rendered target glyphs as the content condition, eliminating the need for a separate text encoder, and infers font style and appearance from the surrounding image content. Group JiT uses CCPE to align the source image, target glyphs, and repair regions in one coordinate system, then jointly generates RGB patches containing both the corrected characters and their local background before pasting them back. Both training and inference operate in pixel space without a VAE.

UTDesign instead uses DINOv2 to encode the target glyph and CLIP to encode style-reference crops from the original image. It generates glyphs in latent space, decodes them with a transparent VAE into independently editable RGBA text layers, and composites them onto a background after removing the original text. Thus, UTDesign offers reusable and independently editable text layers, whereas PixText directly optimizes localized correction accuracy, fine-stroke preservation, and fast end-to-end repair.

AuthorsOpenReview ↗

Response to Reviewer Uk3T(part2)

Critical Weakness 2 and Question 2: Accuracy and computational cost across text scales

Reviewer concern. Although CCPE supports variable-sized crops, how does PixText perform across different text scales, particularly for large characters or words? A scale-wise evaluation of editing accuracy and computational cost would be helpful.

Response. We thank the reviewer for this suggestion. We evaluate all seven methods on Text Refinement Bench by target-box short side. OCR is computed per target character, while inference cost is computed per image using the largest target box in that image. Each cell below reports single-character OCR accuracy (%) / amortized inference cost (GPU-s/img). Batch latency is amortized across images under the original evaluation configurations.

For UTDesign, we use the official v1.0 editing weights with 28 steps and CFG 5. Up to three neighboring glyphs from the same line are added as style references: 2,141 targets receive additional references, while 497 fall back to a single reference.

Table 2. OCR accuracy and amortized inference cost by target-box short side

Short side (px)AnyTextAnyText2FLUX-TextTextFluxUTDesignPixText TeacherPixText MeanFlow
Overall25.6 / 4.30748.4 / 8.86852.3 / 16.42028.3 / 14.14283.4 / 5.15794.7 / 4.51494.6 / 0.942
<160.0 / 4.2191.7 / 8.6881.2 / 16.4170.0 / 14.12146.5 / 5.03475.1 / 4.50076.3 / 0.964
16–<240.1 / 4.23812.4 / 8.74013.7 / 16.4145.5 / 14.11476.1 / 5.28595.0 / 4.44894.9 / 0.931
24–<328.1 / 4.30544.8 / 8.88454.9 / 16.42020.7 / 14.17086.4 / 6.81197.1 / 4.52796.4 / 0.943
32–<4030.1 / 4.36873.8 / 8.97881.1 / 16.43137.2 / 14.17490.2 / 7.40997.8 / 4.54698.1 / 0.944
40–<4854.4 / 4.43187.6 / 9.11283.6 / 16.42246.0 / 14.16192.0 / 4.48997.8 / 4.54097.8 / 0.938
48–<5671.1 / 4.38390.9 / 9.01090.1 / 16.41659.5 / 14.12693.4 / 2.27698.3 / 4.56399.2 / 0.943
56–<6485.3 / 4.34793.3 / 8.87893.3 / 16.43164.0 / 14.15597.3 / 3.91093.3 / 4.51292.0 / 0.947
≥6486.2 / 4.22386.2 / 8.70390.2 / 16.41476.0 / 14.10394.9 / 3.18692.0 / 4.49792.4 / 0.944

Two trends emerge. First, PixText’s largest advantage is on small and medium text. Overall, Teacher and MeanFlow exceed UTDesign by 11.23 and 11.15 pp, respectively. Below 16 px, Teacher and MeanFlow achieve 75.1% and 76.3% OCR, compared with 46.5% for UTDesign and 1.7% for the strongest other baseline. At 16–<24 px, they reach 95.0% and 94.9%, compared with 76.1% for UTDesign and 13.7% for the strongest other baseline. PixText also remains ahead in every bin from 24 to 56 px, showing that its main advantage is concentrated where fine strokes are most difficult.

On larger text, PixText remains competitive with UTDesign. At 56–<64 px, UTDesign reaches 97.3%, while Teacher and MeanFlow reach 93.3% and 92.0%. At ≥64 px, UTDesign reaches 94.9%, compared with 92.4% for MeanFlow and 92.0% for Teacher, a difference of only 2.5 pp for MeanFlow. Thus, PixText combines its strongest gains on small and medium text with comparable performance on large characters.

Second, amortized cost remains nearly flat across scales. Teacher stays within 4.177–4.563 GPU-s/img, and MeanFlow stays within 0.931–0.964 GPU-s/img across every bin, including 0.944 GPU-s/img at ≥64 px. These measurements use the largest target per image and the original batch composition, so they describe the evaluated amortized-cost behavior rather than an isolated single-target scaling law.

We additionally test robustness beyond the main training resolution. Using the same PixText checkpoint and inference settings, we upsample the Text Refinement Bench inputs and target regions by 2× and compare performance at the original 1024 scale and at 2048 × 2048.

Table 3. Controlled 2× input-upscaling evaluation

Input scaleOCR Acc ↑CJK-CLIP Pass ↑
1024 main image and targets93.86%81.54%
2048 main image and targets94.88%85.56%
Improvement+1.02 pp+4.02 pp

Although the main-image area during training is primarily capped at 102421024^2, the model successfully performs localized editing at 2048×20482048\times2048. Under this controlled setting, OCR increases from 93.86% to 94.88%, and CJK-CLIP Pass increases from 81.54% to 85.56%.

Overall, the scale-wise results support the paper’s central motivation: PixText provides its largest gains on small and medium text, remains comparable on large characters, retains sub-one-GPU-second amortized MeanFlow inference across all tested size bins, and shows promising robustness when evaluated at 2048 resolution.

AuthorsOpenReview ↗

Response to Reviewer Uk3T(part3)

Question 3: Error detection, target glyph construction, and human involvement

Reviewer concern. How are the erroneous text regions and target glyphs obtained in real-world applications? Since OCR may fail to detect subtle glyph errors, does the proposed refinement pipeline require manual intervention?

Response. We thank the reviewer for raising this practical question. The default PixText pipeline does not require users to manually mark erroneous characters. The target text can be extracted from the user’s generation prompt or supplied by a prompt enhancer. The system automatically proposes repair regions and target text; users may optionally adjust them before generation, but this is not required.

In deployment, an OCR detection head first localizes text regions, and a recognition head produces initial transcriptions. We align these transcriptions with the target text to assign a target string to each region. A transcription mismatch triggers refinement. For character regions whose OCR transcription already matches the target character, CJK-CLIP additionally checks glyph fidelity against the target character. Regions below the character-specific CJK-CLIP threshold are also sent to PixText. This second stage targets cases where OCR reads the character correctly but its strokes are still malformed. Target glyph conditions are then rendered from the aligned target strings, rather than extracted from a reference image.

To evaluate whether the additional verification improves final refinement, we compare two pipelines on Text Refinement Bench: OCR-only error detection and OCR + CJK-CLIP verification. Both use Hungarian matching to assign OCR transcriptions to the target text and share the same PixText repair model. The target text in this experiment is constructed from dataset annotations.

We randomly sample 80 targets with originally incorrect glyphs and 80 with originally correct glyphs. Thirteen reviewers perform an A/B blind evaluation of the two pipelines, with final decisions determined by majority vote.

Table 4. A/B blind evaluation of OCR-only and OCR + CJK-CLIP pipelines

Original targetQuestionsOCR + CJK-CLIP winsTiesOCR baseline winsOCR + CJK-CLIP GSB ↑OCR baseline GSB ↑
Overall1604011731.310.76
Originally incorrect80403911.980.51
Originally correct8007820.981.03

GSB is computed as (wins+ties)/(losses+ties)(\mathrm{wins}+\mathrm{ties})/(\mathrm{losses}+\mathrm{ties}), with wins and losses exchanged when scoring the OCR baseline. On originally incorrect glyphs, the CJK-CLIP pipeline wins 40 comparisons, ties 39, and loses only 1, giving a GSB of 1.98 versus 0.51 for OCR alone. On originally correct glyphs, 78 of 80 comparisons are ties and the remaining 2 favor the OCR baseline, indicating near-parity when no repair is needed. Thus, the additional verification improves final refinement on erroneous glyphs without materially degrading correct ones.

Reviewer_YYSLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses local refinement of small text and complex characters in generated images, where coherent layouts and visual content may still contain local text errors. The authors propose PixText, which formulates local text refinement as grouped image generation in pixel space. Conditioned on the original image, PixText generates only target regions, preserves unedited content, and uses MeanFlow for four-step generation. The paper also introduces Text Refinement Bench to evaluate text correction and image preservation in generated images. On this benchmark, PixText reports 83.06% glyph pass and 94.66% exact target-character OCR accuracy, compared with 36.77% and 52.31% for the strongest baseline. A four-step MeanFlow variant retains an 81.84% pass rate with less than one GPU-second of amortized generation cost per image. The model and benchmark are promised to be open-sourced.

Critical strengths
  1. The problem is practical and well motivated: local text errors in generated images remain an important bottleneck for readability and downstream use, and the paper targets accuracy, preservation, and efficiency simultaneously.

  2. Formulating local text refinement as grouped pixel-space generation is a reasonable direction, as it avoids VAE reconstruction loss of fine strokes and reduces full-image editing effects on background colors.

  3. The reported gains on the proposed benchmark are large, and the four-step MeanFlow variant suggests a favorable accuracy-efficiency trade-off.

  4. The introduction of Text Refinement Bench, if well constructed and released, could facilitate future evaluation of text correction and image preservation in generated images.

Critical weaknesses
  1. Limited methodological novelty and insufficient comparison with the closest work. The trajectory-aligned glyph condition and multi-depth recognizer feature supervision are conceptually close to glyph-conditioned text editing and existing multi-layer OCR supervision methods, especially SteerVTE. The depth-normalized loss is a useful but relatively heuristic refinement. The paper should more clearly articulate its technical distinction from SteerVTE and, if possible, compare directly on the English subset; otherwise, the novelty of “enhanced video text editing” remains limited.

  2. Text addition, background preservation, and baseline fairness are insufficiently validated. Addition samples may be absent from training yet diluted in aggregate results, background metrics are only global averages outside the box, and baseline inputs, prompts, and tuning budgets need clearer specification.

  3. Statistical rigor, benchmark diversity, and efficiency reporting are limited. Results lack variance or significance tests; the user study is small and under-specified, VTEdit is relatively small and imbalanced, and inference time, memory, and sampling-step necessity are not reported.

Questions for authors

Please refer to the weaknesses.

1 comment
AuthorsOpenReview ↗

Response to Reviewer YYSL

Thank you for recognizing in your “Critical Strengths” the practical motivation of our work, the grouped pixel-space generation design, the potential evaluation value of Text Refinement Bench, and the empirical gains and accuracy–efficiency trade-off of the four-step MeanFlow variant.

The “Critical Weaknesses” section evaluates a method and experimental setup that are not those of our paper. The “trajectory-aligned glyph condition,” “multi-depth recognizer feature supervision,” and “depth-normalized loss” in Item 1 are not components of PixText, and VTEdit in Item 3 is not a benchmark used in our paper. PixText addresses local text refinement in static images and is evaluated on Text Refinement Bench and AnyText Bench. The concerns raised in “Critical Weaknesses” do not pertain to our paper, and we are therefore unable to provide a substantive point-by-point response to this section.

We have also added the following during rebuttal:

  • Blinded evaluation with 13 participants: Human-rated glyph correctness is 73.3% for PixText Teacher and 68.3% for MeanFlow, compared with 46.7% for FLUX-Text. We also evaluated CJK-CLIP’s agreement with human judgments.
  • More extensive ablations: These include grouped versus independent generation using the same model weights, pixel-space versus latent-space output, and reconstruction tests with multiple VAEs. The results further support the accuracy and efficiency benefits of grouping and the advantages of pixel-space generation for small-text details.
  • More direct preservation measurements: We separately report interior-background color difference, boundary error, and exterior color difference. Under a unified compositing protocol, Teacher and MeanFlow reduce boundary error by 40.7% and 41.2%, respectively, relative to the best baseline on this metric, further supporting local background and boundary preservation.

Detailed settings and results are available in our responses to Reviewer Kdcd (parts 1–4 and 6). Thank you for your time and consideration.

Reviewer_KdcdOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents PixText for correcting local text errors in generated images. It uses the source image as context and generates only the target regions in pixel space. Glyph images specify the target text, while continuous position encoding and grouped attention align each region with the full image. The paper also introduces a four-step MeanFlow version and Text Refinement Bench, which contains 1,034 images and 2,638 repair targets.

Critical strengths
  1. The problem is practical, and the design fits it well. Generating only the target regions helps preserve the rest of the image, while pixel-space generation is a reasonable choice for small characters and fine strokes.

  2. The reported results are strong. PixText improves Pass from 36.77% to 83.06% and PP-OCRv6 accuracy from 52.31% to 94.66% over FLUX-Text. The four-step version keeps 81.84% Pass while reducing the reported amortized cost from 4.514 to 0.942 GPU-seconds per image. Results on the public AnyText benchmark give additional external support, although the English result trails FLUX-Text.

Critical weaknesses
  1. The paper does not isolate the core contributions of grouped generation and pixel-space output under matched training conditions. It lacks matched comparisons between grouped and independent region generation, and between pixel-space and latent-space output. The paper does include controlled tests of time step sampling, the clean-endpoint update, and CCPE efficiency, so the main gap is specific to the headline architecture choices.

  2. The new benchmark and Pass metric need stronger validation. CJK-CLIP is not yet public, and the reference set, font coverage, and 0.03 threshold are not explained in enough detail. The repair targets were manually checked, but the full data sources and deduplication procedure are not reported. The paper also lacks systematic blinded human validation and region-level preservation results. Full-image FID and LPIPS can be dominated by copied background pixels, and their interpretation depends on whether all methods use the same compositing protocol. The current evidence does not establish that data leakage occurred.

Questions for authors
  1. Under matched data, capacity, and inference settings, how does grouped generation compare with independent region generation and with latent-space output?

  2. How were the CJK-CLIP reference set and 0.03 threshold chosen? Can you report blinded human agreement and error rates across methods, fonts, and character sizes?

  3. What are the exact sources, filtering steps, and deduplication procedures for the training data and Text Refinement Bench? How was overlap with Text Refinement Bench and AnyText Bench checked?

  4. Do all methods receive the same target boxes and text granularity? Can you report preservation separately inside the target regions, around their boundaries, and outside them under the same compositing protocol?

9 comments
AuthorsOpenReview ↗

Response to Reviewer Kdcd (part1)

Reviewer concern. How does grouped generation compare with independent region generation and with latent-space output under matched data, model capacity, and inference settings?

Response. We thank the reviewer for highlighting the need to evaluate grouped generation and pixel-space output separately. We address these two design choices through a fixed-checkpoint grouping ablation and a latent-output variant adapted from the same PixText Teacher, complemented by VAE reconstruction tests on fixed outputs. The grouping ablation shows improvements in both multi-target accuracy and efficiency. The latent-output comparison reveals a pronounced accuracy gap on small text, while the reconstruction tests independently demonstrate the loss of fine glyph detail introduced by VAE compression.

(1) Grouped vs. independent region generation. Both conditions use the same Teacher weights, 25 Euler steps, and guidance settings. We also hold the source-image conditioning, target geometry, glyph inputs, per-target noise, and compositing fixed. In the independent condition, we put each target in its own attention-isolated group and process those groups in the same batch. The outputs are pixel-identical on all 556 single-target images.

Table 1. Grouping ablation on Text Refinement Bench

Evaluation setMetricGroupedBatched independentGrouped − independent, pp [95% CI]
All 1,034 images (2,638 targets)Exact OCR94.43%92.61%+1.82 [1.13, 2.58]
All 1,034 images (2,638 targets)CJK-CLIP Pass82.64%79.87%+2.77 [1.86, 3.69]
478 multi-target images (2,082 targets)Exact OCR95.15%92.84%+2.31 [1.42, 3.23]
478 multi-target images (2,082 targets)CJK-CLIP Pass83.43%79.92%+3.51 [2.34, 4.69]

We compute 95% confidence intervals by paired bootstrap resampling at the image level. Differences are in percentage points (pp).

Table 2. End-to-end latency and peak allocated GPU memory on one H200

Targets per imageGrouped (s / GB)Batched independent (s / GB)Serial independent (s / GB)
114.38 / 2.2914.37 / 2.2914.43 / 2.29
213.68 / 2.2326.12 / 2.9827.17 / 2.23
413.76 / 2.2339.86 / 5.8854.06 / 2.23
813.82 / 2.2370.53 / 17.34108.81 / 2.23
1213.98 / 2.23102.43 / 36.34162.87 / 2.23

At each target count, we use one fixed image for all three modes and report mean latency and peak PyTorch allocated memory over three runs after warm-up. On the 12-target image, batched independent generation takes 7.33× as long and uses 16.27× as much peak allocated memory as grouped generation. Serial independent generation takes 11.65× as long at roughly the same peak memory as grouping.

For multi-target images, these results show that grouping improves text quality with the same Teacher weights, while reducing latency relative to both independent-generation modes and peak memory relative to batched independent generation.

(2) Pixel-space vs. latent-space output. We initialize the latent variant from the Teacher and retain its 32-block GroupJiT backbone, grouping, RGB source-image conditioning, and glyph conditioning. We change the target-region input and output projections to use latents from a frozen FLUX.1-dev VAE. Each latent token covers the same 16 × 16 pixel footprint as a Teacher patch. We continue training the backbone and new projections on the original training data for one epoch. Both the training and held-out denoising losses approach a plateau late in training.

Table 3. Pixel-space Teacher vs. FLUX.1 latent variant on Text Refinement Bench

MetricPixel-space TeacherFLUX.1 latent EMALatent − pixel, pp [95% CI]
Exact OCR94.43%78.66%−15.77 [−18.25, −13.35]
CJK-CLIP Pass82.64%53.71%−28.92 [−31.78, −26.15]

Table 4. Pixel-space vs. latent-space exact OCR by target-box short side

Short sideTargetsPixel-space TeacherFLUX.1 latent EMALatent − pixel, pp [95% CI]
<16 px17574.29%7.43%−66.86 [−74.36, −59.39]
16–31 px1,42396.13%75.26%−20.87 [−23.97, −17.75]
≥32 px1,04095.48%95.29%−0.19 [−1.48, 1.15]

After convergence, the latent variant matches the Teacher on large text: at 32 px and above, exact OCR differs by only 0.19 pp, with the 95% CI including zero. On small text, however, it falls far behind, reaching only 7.43% exact OCR below 16 px and 75.26% at 16–31 px, compared with 74.29% and 96.13% for the Teacher. This sharp size dependence is consistent with the VAE round-trip results in Table 5, where even a single encode–decode pass degrades small glyphs. These results support the conclusion that the codec’s loss of fine detail limits the small-text generation ceiling of the latent-space DiT.

AuthorsOpenReview ↗

Response to Reviewer Kdcd (part2)

(3) VAE reconstruction on fixed outputs. To measure the effect of the codec separately, we apply one encode–decode pass to the same full images produced by the pixel-space Teacher and rescore the same targets. Table 5 reports the decrease in exact OCR / CJK-CLIP Pass, both in pp. Columns are cumulative thresholds based on the short side of the rounded scoring crop.

Table 5. OCR / Pass decrease after one VAE round trip (pp)

VAE<256 px (n=2,634)<128 px (n=2,586)<64 px (n=2,363)<32 px (n=1,575)<16 px (n=172)
SD 1.529.12 / 40.9329.70 / 41.6932.63 / 45.6247.24 / 61.1469.19 / 41.86
FLUX.1-dev1.71 / 2.391.74 / 2.441.86 / 2.712.73 / 3.6214.53 / 13.37
Qwen-Image-Edit-25112.66 / 4.292.75 / 4.373.00 / 4.744.25 / 6.6020.35 / 18.02
HunyuanImage-2.19.91 / 19.0610.13 / 19.4111.17 / 21.2416.57 / 29.6543.02 / 35.47

The FLUX.1-dev round trip alone lowers exact OCR by 14.53 pp and glyph Pass by 13.37 pp below 16 px. This isolates a reconstruction loss on small glyphs with the codec used in the latent variant.

AuthorsOpenReview ↗

Response to Reviewer Kdcd (part3)

Critical Weakness 2 and Question 2: CJK-CLIP reference set, threshold, and blinded human validation

Reviewer concern. How were the CJK-CLIP reference set and 0.03 threshold chosen? Can you report blinded human agreement and error rates across methods, fonts, and character sizes?

Response. We thank the reviewer for prompting a more complete validation of the glyph metric. We have now documented the reference construction and threshold, and conducted a 13-person blinded study on frozen outputs from FLUX-Text, PixText Teacher, and PixText MeanFlow. CJK-CLIP achieves higher overall human agreement than OCR and substantially reduces OCR’s over-acceptance of incorrect glyphs.

(1) Reference set and per-character baseline. For each character, we construct a candidate pool of 500 crops from two sources: held-out source data excluded from PixText training and synthetic glyphs rendered with diverse font files. Using a fixed random seed of 42, we uniformly sample 20 crops without replacement from each pool. The resulting reference set covers 8,189 characters and 163,780 crops, combining source-image appearance variation with controlled font diversity.

For a character yy, we use frozen CJK-CLIP to compute the cosine similarity between each of its 20 reference crops and the target character’s IDS text embedding. Their mean, μy\mu_y, is the character-specific baseline. A test crop II passes when S(I,y)≥μy−0.03S(I,y)\geq\mu_y-0.03. The per-character baseline accounts for different similarity levels across characters, while the tolerance accommodates correct glyphs whose scores vary with font and imaging conditions. All methods use exactly the same references, per-character baselines, and tolerance.

(2) Why a tolerance of 0.03? A correct glyph can score below its character’s reference mean because of legitimate font and imaging variation. We therefore examined the acceptance rates of the reference crops themselves under two fixed rules.

Table 6. Reference-crop acceptance under different CJK-CLIP thresholds

Acceptance ruleAccepted reference cropsAcceptance rate
S(I,y)≥μyS(I,y)\geq\mu_y111,503 / 163,78068.08%
S(I,y)≥μy−0.03S(I,y)\geq\mu_y-0.03148,397 / 163,78090.61%

Using the mean as a hard cutoff rejects 31.92% of the reference crops and is therefore overly strict. A 0.03 tolerance raises reference acceptance by 22.53 pp while retaining a character-specific decision boundary. This diagnostic motivates the tolerance. The blinded human evaluation below further shows that CJK-CLIP with this threshold achieves higher overall agreement with human judgments than OCR.

(3) Blinded protocol and overall human agreement. We sampled 60 target characters, 12 from each of five size bins, without filtering by OCR or CJK-CLIP outcomes. Each target has three frozen outputs, one per method, yielding 180 crops. Thirteen reviewers blinded to method identity labeled each crop as correct, incorrect, or unclear, and we determined the human label by majority vote. Two crops with an unclear majority were excluded from the automatic-metric comparisons, leaving 178 crops.

Table 7. Automatic metrics against blinded human majority labels (n=178)

MetricOCR ExactCJK-CLIP Pass (0.03 tolerance)
Automated acceptance77.0% (137/178)66.3% (118/178)
Agreement with human labels83.1% (148/178)87.1% (155/178)
False acceptance: human-incorrect output accepted41.5% (27/65)21.5% (14/65)
False rejection: human-correct output rejected2.7% (3/113)8.0% (9/113)

Human reviewers judged 63.5% (113/178) of the outputs correct. CJK-CLIP’s 66.3% acceptance rate is close to this human rate, whereas OCR accepts 77.0%. CJK-CLIP improves overall agreement by 3.9 pp and reduces false acceptance by 20.0 pp, nearly halving OCR’s rate, at the cost of a 5.3 pp higher false-rejection rate. This supports Pass as a more conservative measure of glyph correctness rather than mere recognizability.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal Summary and Key Clarifications

Rebuttal Summary and Key Clarifications

As the discussion period draws to a close, we sincerely thank the Area Chair and reviewers for their time and feedback. Below is a brief summary of our rebuttal updates and key clarifications for the final decision.

1. Note Regarding Reviewer YYSL's Review

We thank Reviewer YYSL for the positive remarks on our motivation, pixel-space grouped design, and efficiency in the Critical Strengths section. We noticed that the Critical Weaknesses section (which references trajectory-aligned glyph condition, SteerVTE, enhanced video text editing, and the VTEdit dataset) appears to have been inadvertently pasted from another ICLR 2027 video text-editing submission (Submission 3938, https://openreview.net/forum?id=oC2k9xA1mU), and Questions For Authors refers directly to that section. As there were no paper-specific questions for us to address, we shared a brief summary of our newly added rebuttal experiments in our Nov 14 reply for Reviewer YYSL's reference.


2. Resolution of All Concerns

Reviewers consistently recognize the value of localized small-text refinement, pixel-space grouped generation, and 4-step MeanFlow efficiency. Below is how we addressed all specific questions via new experiments and revisions:

ReviewerConcernResponse & ExperimentsKey Results
Uk3TW1/Q1: UTDesign baselineAdded discussion & direct eval with official UTDesign v1.0 weights.Teacher / MeanFlow: 94.66% / 94.58% OCR, 83.06% / 81.84% Pass (vs. UTDesign 83.43% / 56.03%); MeanFlow ~5.5× faster (0.942 vs. 5.157 GPU-s/img).
W2/Q2: Scale & costScale-wise OCR/cost across all 7 methods (8 bins, <16 to ≥64 px) & 2× upscaling test.<16 px: 75.1%–76.3% (vs. UTDesign 46.5%, others ≤1.7%); MeanFlow stays ~0.94 GPU-s/img; 2× upscaling adds +4.02 pp Pass.
Q3: Auto deploymentClarified automated pipeline (OCR + Hungarian matching + CJK-CLIP) & 13-reviewer blind A/B (160 targets).40–3 wins over OCR-only overall (40–1 on originally wrong targets; 78/80 ties on correct ones).
KdcdW1/Q1(a): Grouped vs. indep.Matched-setting ablation (same weights, steps, noise, compositing).Multi-target: OCR +2.31 pp, Pass +3.51 pp; at 12 targets, 7.33×–11.65× faster, 16.27× less peak memory (2.23 vs. 36.34 GB).
W1/Q1(b) & Follow-up: Pixel vs. latentFLUX.1-dev latent ablation + 4-VAE test; trained latent to 91k–102k steps (convergence) & profiled H200 cost (App. J).At convergence, best latent OCR is 17.71% on <16 px (vs. Pixel 74.29%, +56.57 pp) and 87.70% on 16–31 px (vs. 96.13%), at matched latency (13.80 vs. 13.95 s).
W2/Q2: CJK-CLIP & human evalDetailed reference set (8,189 chars / 163,780 crops), 0.03 threshold, & 13-reviewer blind study (180 crops).CJK-CLIP hits 87.1% human agreement (OCR 83.1%), cuts false acceptance 41.5%→21.5%; human accuracy: PixText 73.3%/68.3% vs. FLUX-Text 46.7%.
W2/Q3: Leakage auditAudited all 2,525,234 training images vs. Text Refinement & AnyText EN/ZH (SHA-256, dHash, PDQ).0 exact or near-duplicate matches, ruling out data leakage risk.
W2/Q4: Fairness & preservationVerified identical boxes/strings across all 6 methods; unified-compositing region metrics.Interior ΔE₀₀ 2.04–2.06 (best baseline 4.13), boundary 2.11–2.13 (vs. 3.59, −41%), native exterior 0.0032 (vs. 1.52).

3. Closing Summary

PixText improves both the readability and visual fidelity of text in existing images. Guided by the source image and target glyphs, it jointly refines multiple text regions directly in pixel space, avoiding VAE stroke degradation on small characters while leaving unedited regions intact. On the full Text Refinement Bench, PixText achieves 94.66% OCR accuracy and an 83.06% glyph pass rate (vs. 83.43% and 56.03% for UTDesign), while the four-step MeanFlow variant retains nearly full quality (94.58% OCR, 81.84% pass) at under 1 GPU-second per image (0.942 vs. 5.157).

Text Refinement Bench further enables comprehensive evaluation of what a faithful edit must preserve: whether the intended text is readable, whether fine strokes are accurately formed, and whether the surrounding image remains unchanged.

We thank the reviewers for their thoughtful questions, which strengthened our experimental validation and clarified the paper's contributions, and the Area Chair for guiding the discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.