acceptodds
Under review as a conference paper at ICLR 2027

Perception or Prior? Legibility-Guided Path Selection for Text Image Restoration in the Wild

Abstract

Text image restoration requires preserving character identity and stroke geometry. Recognition-guided methods exploit character priors but are limited by vocabulary coverage. However, visible stroke patterns can guide reconstruction even for out-of-vocabulary characters, while reliable priors can complement visual evidence under severe degradation. Motivated by this complementarity, we propose GRASP (Glyph Reconstruction via Adaptive Selection of Paths), a framework for legibility-guided selection between visual reconstruction and character-prior completion. A geometry-aware extractor aligns character features in a shared glyph space, where a visual path reconstructs observed structures and a prior path generates character-conditioned glyphs with input-dependent style modulation. A selector uses visual evidence and recognition confidence to choose complete glyphs, with visual fallback for predictions labeled as unknown. The selected glyphs form a structure mask for full-image restoration. We develop a synthetic training pipeline spanning 900 fonts and introduce MText500, a multilingual real-world benchmark. Experiments demonstrate the effectiveness of GRASP across character subsets and degradation levels, including languages absent from full-image restoration training.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Text Image Restoration, Image Restoration

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_uF5kOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents a methodology for text image restoration. Main contribution is a pipeline that takes strength of two approaches. First approach is to have a recognition base method, that exploits or uses character priors to recognize a character in degraded images. Naturally this is limited by the data it is trained to recognize. Second is to have stroke based reconstruction, but this doesn't have an understanding of letters/characters so it may miss strokes not visible in the degraded image. The core contribution is choosing per character between these two strategies. The selected characters then guide a full image restorer. Another contribution is the introduction of MText500, which is a real-world benchmark of 500 images in seven scripts. They report the best OCR accuracy in most settlings, but especially under heavy degradation.

Critical strengths
  1. Relevant problem with good results. The paper shows a real problem, when text is badly degraded, what should the model trust? Visual features or learned priors based on real characters.
  2. The provided method gets best OCR accuracy especially under heavy degradation on both synthetic and real data. Thorough quantatiative comparisons are done.
  3. MText500 is a new benchmark that contains 500 real-world image with seven scripts. It is split by degradation level, and if the characters were seen in training. This could be a useful resource.
Critical weaknesses
  1. Ablations contradict the main contribution. The key idea is "choosing" per character between the two strategies. Both strategies seem like they are already established in prior works, so the selection is the main contribution. However, Tables 3 and 4 are showing that it doesnt add anything substantial. In light degradation visual branch seems to be better than full system. Under heavy degradation the full model is barely better than visual only branch. This is the main reason for the recommendation.
  2. Layout augmentation is not ablated to see how much retraining with layout augmentation improved results. Would training the baselines with this augmentation improve their results substantially?
Questions for authors

Clarifications that could change recommendation

  1. Need better validation for main contribution of selection.

  2. What happens if baseline is retrained on your data with layout augmentation.

9 comments
AuthorsOpenReview ↗

Response to uF5k (Part1/2) : Contribution of GRASP

Dear Reviewer uF5k,

Thank you for recognizing the practical importance of our work. We also appreciate your comments and suggestions. We address your questions point by point below.

Q1. Contribution of GRASP

(1) Contribution of the selection scheme

We agree that both visual reconstruction and prior-based restoration have strong precedents, and our own previous work has also explored both directions. From early CNN-based reconstruction to methods using GAN and diffusion priors, these approaches have substantially advanced natural image restoration. However, the trade-off between fidelity and visual quality has long been a problem for image restoration tasks, especially for highly-structured objects like faces and text, since minor artifacts like distorted strokes can be extremely noticeable.

Existing efforts to preserve both fidelity and visual quality typically fuse the features of two branches (for example, the encoder features delivered by skip connections and the decoder features with generative priors), where the fusion scheme is learned in a data-driven manner on synthetic data and struggles to generalize to complex real-world degradations. Moreover, contrary to the ideal of simultaneously preserving fidelity and visual quality, the fusion of sometimes conflicting features from the two branches often leads to artifacts instead.

GRASP decouples the decision to use a prior from glyph reconstruction. For each character, the visual branch reconstructs a candidate glyph from the image alone, and the prior branch produces a second candidate using the OCR-predicted character identity. A separate selector uses visual features and OCR confidence to choose between the two candidates for that character.

(2) Other contributions

Beyond the selection scheme, another technical contribution lies in the design of layout augmentation. At inference, the selected glyphs form a spatial text mask that guides the restorer. During training, layout augmentation keeps the ground-truth image and its text mask aligned while varying text placement relative to the degraded input. This encourages the restorer to follow the mask. Q2 reports the ablation.

(3) Explanation of Table 3

We agree with the reviewer's reading of Table 3. Under light degradation, Visual-only outperforms GRASP with predicted OCR on all six metrics in both subsets. Tables 3 and 4 were designed to test the intuition behind our selection strategy. We expect Visual to be preferable under light degradation on both Known and Unknown, and under heavy degradation on Unknown, where out-of-vocabulary characters have no matching prior. Under heavy degradation on Known, we expect Prior to help when strokes are missing, while Visual can still be preferable for individual characters whose strokes remain legible. The strong performance of Visual-only under light degradation is therefore consistent with our design.

Input OCR label errors appear to explain much of GRASP's small shortfall in OCR-A under light degradation. With predicted labels, GRASP scores 79.00% versus 79.17% for Visual-only. Correcting only the labels while retaining the original OCR confidence raises GRASP from 81.10% to 81.35% on Light-Known and from 76.87% to 77.07% on Light-Unknown. Both scores then slightly exceed Visual-only (81.33% and 76.99%). Under heavy degradation, the fixed-branch results in Table 4 follow the expected pattern. Prior-only has higher OCR-A on Known, and Visual-only on Unknown. GRASP slightly exceeds the stronger fixed branch in each subset.

One limitation of our original evaluation is that we defined Unknown at the text-line level and pooled the characters within those lines. At each degradation level, the Unknown subset contains 2,120 in-vocabulary and 5,455 out-of-vocabulary (OOV) characters. We should have reported their recognition accuracy separately. We provide that breakdown in our response to Reviewer 3aJ2 (Q2, Table Q2-1). Tables Q2-2 and Q2-3 also show that GRASP retains some robustness when input OCR misclassifies an OOV character as known. Even though this error bypasses the automatic visual fallback, the selector still chooses Visual in 140 of 190 such cases under light degradation and 131 of 222 under heavy degradation.

AuthorsOpenReview ↗

Response to uF5k (Part2/2) : The Effect of Layout Augmentation

Q2. The Effect of Layout Augmentation

Thank you for suggesting that we retrain the baseline on our data. We retrained TiGeSR restorer on our data with layout augmentation. The retrained TiGeSR improves on the original baseline across all six metrics under both light and heavy degradation. OCR-A increases from 76.16% to 76.84% under light degradation and from 64.35% to 65.56% under heavy degradation.

TiGeSR — light degradation

MaskRestorerPSNR ↑SSIM ↑LPIPS ↓DISTS ↓FID ↓OCR-A ↑
TiGeSROriginal21.080.66980.30230.203339.3176.16
TiGeSRw/ Aug21.340.68230.28300.191236.4776.84

TiGeSR — heavy degradation

MaskRestorerPSNR ↑SSIM ↑LPIPS ↓DISTS ↓FID ↓OCR-A ↑
TiGeSROriginal18.220.49430.41140.263154.9964.35
TiGeSRw/ Aug18.430.51420.39080.249551.5065.56

To isolate the contribution of layout augmentation, we also trained the GRASP restorer on the same data with and without layout augmentation, keeping all other training settings fixed.

GRASP — light degradation

MaskRestorerPSNR ↑SSIM ↑LPIPS ↓DISTS ↓FID ↓OCR-A ↑
Oursw/o Aug21.810.71480.26200.182933.8978.44
Oursw/ Aug22.330.74430.23440.159429.3979.00

GRASP — heavy degradation

MaskRestorerPSNR ↑SSIM ↑LPIPS ↓DISTS ↓FID ↓OCR-A ↑
Oursw/o Aug18.190.52530.38510.250948.1568.59
Oursw/ Aug18.640.55130.35950.227743.8969.88

Layout augmentation improves all six metrics for GRASP at both degradation levels. OCR-A rises from 78.44% to 79.00% under light degradation and from 68.59% to 69.88% under heavy degradation. With augmentation, GRASP also outperforms the retrained TiGeSR baseline across all six metrics at both degradation levels.

Thank you for retraining TiGeSR on your data and for the layout augmentation ablation. Retraining improves the baseline, but GRASP still outperforms it, so this resolves my concern about the comparison. I note, however, that the visual-only branch in Tables 3 and 4 also outperforms the retrained TiGeSR. This supports visual reconstruction more than selection, which is the main point of the paper. Still, I consider Q2 resolved by the retrained results.

Reviewer_3aJ2OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces GRASP for text-image restoration. It reconstructs each glyph through two branches: a visual branch that uses observed features and a prior branch conditioned on a predicted character identity. A geometry-aware extractor aligns character features in a shared glyph space. A learned selector chooses one complete glyph, with an explicit visual fallback when recognition predicts an unknown character. The resulting glyph mask guides a TiGeSR-based image restorer. The paper also introduces a 900-font synthetic training pipeline and MText500, a multilingual evaluation set curated from existing scene-text datasets.

Critical strengths
  1. The full system produces consistent improvements in OCR-A on MText500. Tables 1–2 show the best result in all four settings. On the Unknown subsets, the improvements over the strongest listed baseline are 2.62 percentage points under light degradation and 5.36 under heavy degradation. These are useful gains on real-world multilingual images.

  2. The mask substitution experiment supports the value of GRASP’s structure reconstruction. With TiGeSR restoration weights fixed, replacing its masks with GRASP masks improves overall OCR-A from 76.16 to 77.78 under light degradation and from 64.35 to 67.91 under heavy degradation (Tables 3–4). Image-quality metrics do not consistently improve, but the experiment shows that the masks provide better guidance for recognition.

Critical weaknesses
  1. The experiments provide limited evidence that learned selection is preferable to a simpler routing rule. In Table 3, visual-only reconstruction performs slightly better than the full model on all six metrics in both subsets under light degradation. Overall OCR-A is 79.17 for visual-only and 79.00 for the full model. Under heavy degradation, selection does help: overall OCR-A rises from 69.16 to 69.88 in Table 4. However, the main paper does not compare the selector with an OCR-confidence threshold using the same unknown fallback. Such a comparison matters because adaptive selection is a central contribution; the current ablations establish a benefit over fixed branches in one regime, but do not establish the benefit of learning the routing rule.

  2. The reported Unknown-subset results do not isolate recovery of OOV characters. Section 4 states that 72% of characters in SynData’s Unknown subset are outside the vocabulary, while the tables report metrics on complete text lines. These results support restoration of multilingual lines containing OOV characters, but do not show how much of the improvement occurs on the OOV characters themselves. The explicit fallback also depends on recognition predicting an unknown label (Section 3.3), yet the main paper does not report unknown-detection recall or outcomes for OOV characters predicted as known. This leaves an important part of the claimed OOV behavior untested in the presented results.

Questions for authors
  1. How does learned selection compare with a validation-tuned OCR-confidence threshold, using the same branch outputs and unknown fallback? Results under both degradation levels would help establish whether the learned routing mechanism is necessary.

  2. Can you separate recognition accuracy on OOV characters from accuracy on in-vocabulary characters within the Unknown subset? What is the unknown-detection recall, and how does restoration perform when an OOV character is predicted as known? These results would clarify whether the fallback works as intended.

  3. Section 4.2 changes both character labels and recognition confidence in the Full w/ GT OCR experiment. Can you separate the effects of correcting labels and changing confidence? This would help interpret the claimed role of recognition reliability in selection.

3 comments
AuthorsOpenReview ↗

Response to 3aJ2 (Part1/3) : Learned Selector versus Confidence-Threshold Routing

Dear Reviewer 3aJ2,

Thank you for your careful review and suggestions. Your questions helped us clarify how the different components of GRASP contribute to these results.

Q1. Learned Selector versus Confidence-Threshold Routing

Thank you for suggesting an OCR-confidence routing baseline. We changed only routing, keeping the same branch outputs and unknown fallback. The baseline selects Prior at confidence ≥0.68 and Visual below it. We chose 0.68 as the best threshold on mixed light/heavy validation data. For each setting, the Threshold and Ours rows form a pair, with the better value for each metric shown in bold.

SettingMethodPSNR ↑SSIM ↑LPIPS ↓DISTS ↓FID ↓OCR-A ↑
Light-KnownThreshold21.550.73810.24350.163843.5580.95
Light-KnownOurs21.700.74050.24130.161842.4181.10
Light-UnknownThreshold22.940.74790.22740.157241.7376.14
Light-UnknownOurs22.960.74810.22730.156941.5876.87
Heavy-KnownThreshold18.060.53140.37450.231962.5068.88
Heavy-KnownOurs17.920.53110.37150.230561.4569.88
Heavy-UnknownThreshold19.400.57340.34760.224959.6069.82
Heavy-UnknownOurs19.370.57170.34740.224859.5069.87

The learned selector outperforms the threshold baseline across most metrics, with comparable PSNR and SSIM under heavy degradation. For known predictions, the baseline relies on OCR confidence alone. Our selector combines this estimate of recognition reliability with visual cues about degradation severity and surviving stroke detail.

AuthorsOpenReview ↗

Response to 3aJ2 (Part2/3) : IV and OOV Recognition Accuracy within the Unknown Subset

Q2. IV and OOV Recognition Accuracy within the Unknown Subset

Thank you for suggesting that we evaluate in-vocabulary and out-of-vocabulary characters separately within the Unknown subset. We report the three supplementary experiments first (Tables Q2-1 to Q2-3), then summarize what they show about OOV recognition and routing.

We re-evaluated the same SynData Unknown text lines reported in Table 3 (light degradation) and Table 4 (heavy degradation) at the character level. We scored each character individually and reported recognition accuracy separately for in-vocabulary (IV) and out-of-vocabulary (OOV) characters. At each degradation level, 2,120 characters are in the input OCR vocabulary and 5,455 are outside it. A restored character is counted as correct when its recognized identity matches the ground-truth character, including for OOV characters. With GT labels, each OOV character is mapped to <unk> and routed directly to the visual branch. All results are in Table Q2-1.

Table Q2-1. Recognition accuracy (%) for restored IV and OOV characters

MethodOCR LabelsIV LightIV HeavyOOV LightOOV Heavy
OursPredicted77.1265.3857.5447.19
Visual-only-76.8963.9257.7647.22
Prior-onlyPredicted75.3865.2856.1546.03
Prior-onlyGT75.7565.4756.3746.09
OursGT77.9766.2757.7647.22

At each degradation level, we evaluate input OCR on the 5,455 ground-truth OOV characters. A character is correctly detected as OOV when input OCR predicts <unk>. Table Q2-2 reports OOV recall and the number of OOV characters misclassified as known.

Table Q2-2. OOV detection by the input OCR

DegradationCorrectly detected as OOVMisclassified as knownOOV recall (%)
Light5,26519096.52
Heavy5,23322295.93

We next examine how GRASP restores OOV characters when the input OCR misclassifies them as known. We focus on the 190 light-degradation and 222 heavy-degradation cases identified in Table Q2-2. These errors bypass the automatic <unk> fallback. For each character, we record whether the selector chose Visual, then evaluate whether OCR reads the true character from the restored text image for GRASP and Visual-only. Each accuracy is the number of correct final readings divided by the size of that error subset. All results are in Table Q2-3.

Table Q2-3. Branch selection and character recovery on OOV-to-known errors

DegradationOOV misclassified as knownSelector chose VisualGRASP correct, n (%)Visual-only correct, n (%)
Light190140140 (73.68%)152 (80.00%)
Heavy222131131 (59.01%)133 (59.91%)

Conclusion

GRASP recognizes fewer OOV characters correctly than Visual-only only in the subset that the input OCR misclassifies as known. Table Q2-3 shows that GRASP reads 12 fewer of these characters correctly under light degradation (140 versus 152) and 2 fewer under heavy degradation (131 versus 133). These differences account for the overall OOV gaps in Table Q2-1. Both methods read the same number of the remaining OOV characters correctly at each degradation level.

GRASP retains robustness to incorrect input OCR labels. Among the OOV characters missed by input OCR in Table Q2-2, the selector still chooses Visual in 140 of 190 light-degradation cases and 131 of 222 heavy-degradation cases. Table Q2-3 shows that GRASP recovers the true identity in 73.68% of these light cases and 59.01% of these heavy cases.

AuthorsOpenReview ↗

Response to 3aJ2 (Part3/3) : Separating Label Correction from the Confidence Override

Q3. Separating Label Correction from the Confidence Override

Thank you for pointing out that our GT-OCR comparison changed both labels and confidence. We vary the two settings independently. A check mark (✓) indicates the setting used. A dash (—) means we retain the original confidence in the first column or the OCR-predicted label in the second. GT OCR supplies ground-truth character labels. GT maps out-of-vocabulary characters to <unk>. The table reports OCR-A (%) on restored text. Bold marks the higher value for each subset at the same confidence setting.

Use 0.95 confidenceUse GT OCRLight-KnownLight-UnknownHeavy-KnownHeavy-Unknown
——81.1076.8769.8869.87
—✓81.3577.0770.0070.10
✓—81.0976.8569.8669.89
✓✓81.3877.0870.0570.11

Correcting the labels raises OCR-A in all four subsets under either confidence setting. A corrected label can change the glyph prior or trigger the visual fallback for an OOV character. GRASP benefits from more accurate input OCR labels.

Reviewer_bYiDOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes GRASP, which adaptively selects between visual glyph reconstruction and character-prior completion for text image restoration. Experiments on synthetic data and the new multilingual MText500 evaluate its effectiveness across degradation levels and on out-of-vocabulary text.

Critical strengths

For text restoration, the complementary visual and prior branches offer a well-motivated way to preserve visible strokes while exploiting reliable character priors. The reported OCR improvements are good.

Critical weaknesses
  • PP-OCRv6 does not appear to support Korean, Arabic, or Hindi. The authors should verify its language coverage and clarify which recognition models were used to evaluate these languages.
  • The evaluation is conducted on text-line crops, without explicit results for curved or non-horizontal text, leaving the method’s applicability to complex real-world layouts unclear. Please clarify how such cases are handled and provide representative evaluations.
  • Figure references alternate between “Fig.” and “Figure”, this should use a consistent format throughout the manuscript.
Questions for authors

See the Weaknesses

3 comments
AuthorsOpenReview ↗

Response to bYiD

Dear Reviewer bYiD,

Thank you for your careful reply and for updating your review. We appreciate your recognition of how the two branches of GRASP complement each other in preserving visible strokes and reconstructing missing ones. Our responses to your questions are provided below.

Q1. Which recognition models did we use to compute OCR-A for each language?

Thank you for pointing out this error. We incorrectly stated in Section 4 and Appendix E.5 that we used PP-OCRv6 to compute OCR-A for all languages. As you noted, Korean, Arabic, and Hindi fall outside the documented language coverage of PP-OCRv6. We evaluated these languages with the Korean, Arabic, and Devanagari PP-OCRv5 recognizers, respectively. In the revised manuscript, we have corrected both passages to specify which recognizer we used for each language, as shown in the table below.

Language or scriptRecognition model used for OCR-A
ChinesePP-OCRv6_medium_rec
English / Latin-script textPP-OCRv6_medium_rec
FrenchPP-OCRv6_medium_rec
JapanesePP-OCRv6_medium_rec
Koreankorean_PP-OCRv5_mobile_rec
Greekel_PP-OCRv5_mobile_rec
Thaith_PP-OCRv5_mobile_rec
Arabicarabic_PP-OCRv5_mobile_rec
Bangla (Bengali)EasyOCR bengali_g1 with language setting ['bn', 'en']
Hindi (Devanagari)devanagari_PP-OCRv5_mobile_rec

Q2. How does GRASP handle non-horizontal or curved text, and what do the evaluations show?

Thank you for asking how GRASP handles curved and vertical text in real-world images. For every input image, we first detect the text line, warp the detected region, and resize it to the standard line dimensions. Therefore,for curved text, the remaining curvature within the normalized crop is usually mild, so we process its characters in the same way as those in a straight line. GRASP estimates each glyph’s position separately, allowing neighboring characters to follow the residual curve. For vertical text in real-world scenes, we rotate the detected line crop by 90° to make the line horizontal, then process it as unknown text.

MText500 includes both curved and vertical text lines. We have added qualitative examples to the revised supplementary material in Figure S6. These examples show that GRASP restores legible characters in both curved and vertical text lines.

Q3. Can we use a consistent format for figure references?

Thank you for pointing this out. We have revised the manuscript and supplementary material to use “Figure” consistently throughout.

Best Regards,

Authors of ICLR 2027 Conference Submission #265

  • The authors should confirm whether the listed recognizers (ppv5 or ppv6) were used to produce the originally submitted OCR-A results or were changed during rebuttal. If this is only a documentation error?
  • Please clarify how the warping is implemented for curved text.
AuthorsOpenReview ↗

Clarification on OCR Evaluation and Text-Line Warping

Dear Reviewer bYiD,

Thank you for your follow-up questions. We are happy to clarify both points.

Q1. Were the listed OCR recognizers used to produce the originally submitted OCR-A results, or were they changed during rebuttal?

The originally reported OCR-A results were computed using exactly the recognizers listed in our previous response. No recognition models were changed during the rebuttal, and none of the reported OCR-A results were recalculated. The inconsistency was solely a documentation error in the original manuscript.

Q2. How is warping implemented for curved text?

We use PP-OCRv6_medium_det for text-line detection and follow the standard text-line cropping and perspective transformation procedure in the PaddleOCR pipeline. The resulting text-line crops are resized to 128 × 512 before being fed into GRASP. The same detection and transformation procedure is applied to both straight and curved text lines, without any additional curve-specific rectification. As clarified in our previous response, vertical text lines are rotated by 90° to make them horizontal before being processed as unknown text.

We hope these clarifications address your concerns. Thank you again for your careful questions and for giving us the opportunity to clarify these details.

Best regards,
Authors of ICLR 2027 Conference Submission #265

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.