acceptodds
Withdrawn from ICLR 2027

Unified Reinforcement Learning with Progressive Visual Reasoning for Text-Aware Image Restoration

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Text-Aware Image Restoration is inherently challenging because it is a severely ill-posed problem where character strokes are extremely sensitive to distortion. While recent methods integrate Vision-Language Models for guidance, they often suffer from a VLM drift phenomenon where the guidance policy sacrifices OCR accuracy to satisfy the generative biases of a frozen restoration backbone. In this paper, we propose a novel unified reinforcement learning framework that enables the co-evolution of a VLM-based guidance policy and a Diffusion Transformer restoration model. We formulate the restoration process as an active exploration within the rich pre-trained search space of the generative model to find optimal, text-faithful renderings among multiple plausible solutions. To resolve the non-stationary transition dynamics of this joint environment, our framework employs an Asymmetric Hybrid Reinforcement Learning strategy. Specifically, the VLM is optimized via an on-policy approach using Group Relative Policy Optimization, while the Diffusion Transformer is updated via an off-policy approach using DiffusionNFT. Furthermore, we introduce a Progressive Search mechanism that utilizes intermediate denoised estimates and real-time feedback along the denoising trajectory to iteratively refine the textual search space. Extensive experiments on the SA-Text and Real-Text benchmarks demonstrate that our framework achieves new state-of-the-art performance by effectively reducing hallucinations and significantly enhancing functional legibility.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
text-aware image restoration

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_ZM9vOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses image restoration with an emphasis on preserving readable and accurate text. During restoration, a VLM generates textual guidance for a DiT, revising that guidance as it observes intermediate results. The VLM first undergoes SFT on approximately 161,000 examples containing CoT. The two models are then jointly optimized through RL, using GRPO for the VLM and DiffusionNFT for the DiT. Experiments on SA-Text and Real-Text evaluate both text recognition and image quality, with the full method outperforming UniT and TeReDiff on most reported metrics.

Critical strengths
  • Broad empirical evaluation. The paper compares against a large number of text-aware restoration methods. Improvements are observed with ABCNet v2 as well as TESTR, even though ABCNet is not used as the reward model during RL.
  • A practically meaningful question. Recovering an image that looks plausible is insufficient when incorrect characters change the meaning of its text. The paper addresses this concrete limitation of image restoration and evaluates the readability of the recovered text alongside overall image quality.
Critical weaknesses
  • Repeated VLM inference adds a substantial deployment cost. The default restoration process requires multiple calls to a 7B VLM, each generating a reasoning and answer sequence. This additional dependence on a large VLM remains a practical limitation for applications.
  • The benefit of continuing to optimize the VLM after SFT is not convincingly established. This paper explains the degradation under VLM-only RL as "VLM drift", arguing that the VLM sacrifices transcription accuracy to accommodate a frozen DiT's generation biases. However, it lacks a direct comparison of VLM reading accuracy across SFT, VLM-only RL, and joint RL Moreover, in Table 10, joint RL improves the mean F1 over DiT-only RL by just 0.11 percentage points on ABCNet v2 and 0.21 on TESTR. These gains are small relative to the reported variation. Since joint optimization is the paper’s central contribution, stronger evidence is needed to demonstrate the additional benefit of updating the VLM through RL over keeping it frozen after SFT.
Questions for authors
  • What specific capabilities does GRPO improve in the VLM, and is there any evidence that these improvements lead to consistent additional gains in image restoration? For example, comparing the VLM's transcription accuracy and final restoration quality across VLM-only RL, DiT-only RL, and joint RL could help clarify this point.
  • Why is a VLM needed to provide textual guidance, and why is joint training the VLM necessary? Dedicated OCR systems, such as PaddleOCR and TESTR, already provide text recognition capabilities. Why would their outputs not be sufficient as guidance for the DiT, and what additional value justifies using a VLM and further improving it through joint training?
  • What distinguishes the method from UniT beyond joint optimization? UniT already uses a VLM to provide textual guidance and revises that guidance using intermediate OCR feedback during restoration. Please clarify the additional methodological contribution of this paper beyond joint training.
Reviewer_YUrwOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses text-aware image restoration by combining a VLM, GRPO, a Diffusion Transformer, DiffusionNFT, and intermediate denoising feedback into a joint reinforcement learning system. The task is important, and progressively observing intermediate restoration results is intuitively reasonable. However, the theoretical motivation that dynamic prompts lead to a non-Markov environment is invalid or at least insufficiently substantiated; the reward function is inconsistent between the equations and the implementation description; Real-Text is described as a test set while also being used for periodic monitoring, introducing test-set selection bias; and the gains from joint RL over DiT-only are very small and close to the reported variance.

Critical strengths
  1. The research problem has clear value. Text restoration in real images is extremely sensitive to individual character strokes, and conventional perceptual quality metrics cannot fully capture functional readability. Using vision-language feedback to explore candidate restorations therefore has a practical motivation.
  2. Progressive visual reasoning is better aligned with the restoration process than a single static OCR inference performed only before denoising: as intermediate images become clearer, the VLM can correct earlier erroneous judgments based on newly available visual evidence.
  3. The system encompasses VLM supervised fine-tuning, policy optimization, diffusion model updates, and joint rewards, requiring substantial engineering effort.
Critical weaknesses
  1. The positioning relative to the literature contains factual errors, and the most critical comparisons are missing. The paper asserts that existing RL diffusion frameworks all rely on frozen diffusion models and that “the literature still lacks” a viable framework for jointly optimizing an active reasoning policy and the diffusion velocity field. However, PromptRL (2026-02), Think-Then-Generate (2026-01), and UniGRPO (2026-03) have all publicly demonstrated joint RL of reasoning/prompt policies and diffusion or flow models, while ThinkGen has also presented alternating two-model RL; none of these works is cited. The paper also inaccurately describes PromptLoop, publicly released in 2025-10, as concurrent work. These preprints may exempt the authors from the obligation to conduct experimental comparisons, but not from the obligation to correct claims of “being first” or a “gap in the literature.” Experimentally, to support the necessity of the “asymmetric” design, the paper should at least add joint RL with on-policy updates on both sides (GRPO on the text side and FlowGRPO on the image side), alternating RL between the two modules, and a baseline that enables training only of the VLM within the UniT architecture.
  2. The paper claims that dynamic prompts or continuously changing guidance make the environment non-Markov, but this assertion is unconvincing. Once the current intermediate restored image, timestep, low-quality input, and current conditioning are included in the state, the process can still be modeled as a Markov decision process. A non-stationary training environment and non-Markovianity are not the same concept, and the current theoretical motivation conflates the two.
  3. Real-Text is used as the final test benchmark while also being described as held-out data monitored periodically during training. If the monitoring results influence model selection, stopping time, or hyperparameter adjustment, this constitutes indirect use of the test set. The paper must clarify whether an independent validation split exists and report test results again under complete isolation from the training and selection process.
  4. The gains from “joint RL,” which would most directly support the paper's title, are very small. Relative to DiT-only, the cited comparison across multiple random seeds shows gains of approximately +0.11/+0.21, already close to the reported variance. The existing evidence is insufficient to demonstrate that complex two-model joint reinforcement learning is superior to optimizing only the restoration model.
  5. TESTR is incorporated into the training reward and is also used as one of the main evaluation metrics, creating a risk from using the same metric source for training and evaluation. The paper performs cross-evaluation in Appendix D.6 using ABCNet v2, which is not involved in reward computation. This is commendable and partially alleviates the concern; however, a single additional recognizer is still insufficient to demonstrate general improvements in text readability. Human recognition-rate evaluation and validation with more independent OCR systems are still lacking.
Questions for authors
  1. Results for several major baselines are taken from the original papers or existing reports, rather than obtained by reproducing with unified data preprocessing, the same resolution, and the same evaluation implementation. For a task such as text restoration, which is highly sensitive to the OCR pipeline, this undermines the fairness of cross-method comparisons.
  2. There is a substantive contradiction in the reward definitions. Equation (9) describes the shared reward as the combination 0.7 R_vlm + 0.3 R_dit under format gating, whereas other implementation and training descriptions characterize the shared reward as a TESTR-based metric. The paper does not clearly explain how these definitions correspond, when they are used, or which optimizer each applies to, preventing accurate reproduction of the core algorithm.
Reviewer_q9nNOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes an asymmetric joint reinforcement learning framework for text-aware image restoration. The framework jointly optimizes a VLM-based textual guidance policy and a diffusion transformer restoration model. The VLM is updated with on-policy GRPO, while the DiT is optimized with off-policy DiffusionNFT. The paper further introduces a Progressive Search mechanism, where the VLM repeatedly observes intermediate denoising results and updates the textual guidance during the restoration process.

Critical strengths
  1. The paper addresses an important and challenging problem where perceptual restoration quality and character-level correctness must be considered simultaneously.

  2. The CoT-format SFT data and the adversarial hint-correction setting are useful design choices for improving the initial VLM policy.

  3. The reported results on SA-Text Level 1/2 and Real-Text are competitive, although the claims under the most severe degradation level should be stated more carefully.

Critical weaknesses
  1. Equation (9) defines a shared reward that combines VLM word-level recall, a format gate, and a TESTR-based image reward. However, Section 4.1 and Appendix C.3 describe the practical training reward mainly as the TESTR score �0��0�. It is unclear whether the VLM reward and format gate are actually used during joint RL.

  2. The CoT teacher data are generated from static HQ/LQ images and text hints. The paper does not report generating supervision from the actual intermediate images produced by the DiT during its four-step denoising process. During RL, however, the VLM is called independently at every denoising step, and all four outputs receive the same trajectory-level advantage derived from the final image reward. Thus, the paper does not directly establish that the VLM learns accurate step-specific textual correction.

  3. The DiT-only experiment does not show that previously impossible cases become solvable. Setting D improves the average OCR score, but the VLM remains active and continues to provide textual conditions. Therefore, the experiment mainly shows that the DiT can better exploit existing VLM guidance; it does not demonstrate that DiT adaptation can recover samples that were completely unrecognizable before training. Moreover, the gain of joint RL over DiT-only RL is small, especially in the multi-seed evaluation.

  4. The contribution of joint VLM–DiT optimization is not sufficiently isolated. The Progressive Search ablation is relatively convincing, but the additional benefit of jointly updating the VLM and DiT is modest. In Table 10, the improvement of joint RL over DiT-only RL is small compared with the reported variation.

  5. The OCR reward may not adequately constrain overall image fidelity. The training reward is primarily based on text detection and recognition. However, the method is worse than TeReDiff on LPIPS and MANIQA. This raises the question of whether the model improves text recognition at the cost of background fidelity, over-sharpening, or other visual artifacts.

  6. The inference cost and accuracy–latency trade-off need a more complete analysis. With four denoising steps, the method requires multiple VLM calls for every input image. Table 8 shows that using eight steps improves recognition performance, but each additional step introduces another VLM and DiT call.

  7. The scope of the claims is too broad. The SFT data and output format are restricted to Latin/ASCII characters, and the textual hints are limited to at most ten words. Therefore, the current experiments do not establish effectiveness for multilingual text, highly text-dense scenes, or non-Latin scripts. The authors should either narrow the claims or add experiments on multilingual and dense-text datasets. At minimum, the limitation should be emphasized more prominently in the abstract and conclusion.

  8. The SOTA claims should be stated more carefully. The method does not outperform all baselines under all degradation levels. In particular, under SA-Text Level 3, the TESTR average score is lower than UniT. In addition, many baseline numbers are directly adopted from previous work rather than reproduced under a unified training and evaluation protocol.

Questions for authors

Please solve the weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.