acceptodds
Under review as a conference paper at ICLR 2027

UGenIR: Understanding-Conditioned Discrete Generation for Open-World Image Restoration

Abstract

All-in-one image restoration is commonly formulated as a closed-set multi-task regression problem: a single model is trained to map a degraded input to a restored output in one shot. Such regressors often fail under open-world corruptions, where degradation types, severities, and especially their compositions shift beyond the training distribution, making pixel evidence ambiguous and leaving little room for revision. We propose UGenIR, which reframes restoration as understanding- conditioned constrained generation. Given a degraded image, UGenIR first infers an input-consistent semantic description that acts as a language- level prior to constrain global content. Restoration is then performed in discrete token space via MaskGIT-style parallel token completion, where low-confidence tokens are progressively re-masked and refined for effi- cient iterative recovery of fine details. To further improve robustness under distribution shift, we introduce a critic-guided self-correction loop: a lightweight critic predicts spatial error maps during training, and at test time its feedback is converted into token masks to trigger targeted rollback and regeneration on likely-error regions. UGenIR achieves con- sistent gains on multi-task restoration and challenging OOD degradation compositions, improving both perceptual realism and semantic/content consistency with efficient discrete refinement.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Open-World Image Restoration · Discrete Generation · Self- Correction Loop

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_X4zjOpenReview ↗
Rating
1
Confidence
3(−1)
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The manuscript considers all-in-one open world image restoration as a multi-phase synthesis and verification pipeline executed on a unified multimodal model. Built on multiple open source models, the framework integrates these models via a semantic understanding, constrained discrete denoising and critic-guided self-correction four stage close loop. Experiments part demonstrate advancement compared to other AiO methods in multiple no-reference metrics.

Critical strengths
  1. Consistently integrates multiple existing work into a unified pipeline for image restoration usage.
  2. Converting pixel-level error maps predicted by a critic into discrete token-space re-masking masks intuitively performs localized refinement.
Critical weaknesses
  1. Each module belongs to an off-the-shelf component. Show-O, MAGVIT-V2, MaskGIT and Swin-UNet these four part all come from existing methods which present the job simply as tool ensemble. Additional, the "restore-assess-refine" loop has been explored in recent image restoration work, such as RAR (CVPR 26). The overall approach just integrates existing methods and lacks novelty.

2.The method is trained on the GenDS dataset. However, the paper explicitly claims that all baselines are evaluated without fine-tuning on GenDS. Most critically, GenDeg, which proposed the GenDS benchmark is omitted as a baseline. In the GenDeg paper, baselines such as PromptIR, NAFNet and Swin are trained with GenDS which demonstrate drastically improved out-of-distribute performance. The experimental setup is unfair to the other baselines, making the results unmeasurable.

Questions for authors
  1. Since this work relies directly on the GenDS dataset proposed by GenDeg, why are GenDeg and other recent methods (2025/2026) not being compared?
  2. Is it fair to remove all the full-reference metrics in Tables 1 and 3, given that most of these datasets contain paired GT-LQ data?
  3. The inference stage relies on a loop. Could the wall-clock latency (seconds per image) or other efficacy metrics be compared to those of one-shot method on the same computation resource?
Reviewer_wNjVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes UGenIR, an understanding-guided discrete generative framework for open-world image restoration. Built on the pretrained Show-o model, it first generates a textual description of the degraded image’s scene content and degradation information. It then combines image tokens with textual conditioning to produce restored images through MaskGIT-style iterative completion. The authors further introduce a Swin-UNet Critic trained with pixel-error supervision to select among multiple restoration candidates and identify potentially erroneous regions, whose corresponding tokens are remasked and regenerated. The main contribution is the integration of semantic guidance, discrete image generation, and error-guided local self-correction into a unified restoration pipeline. The method is trained on data including GenDS and evaluated on dehazing, deraining, desnowing, low-light enhancement, deblurring, and mixed degradations. The main comparisons report image quality scores and FID, while ablations on the GenDS validation set use metrics including PSNR, LPIPS, and DISTS. The results show that combining semantic guidance with self-correction outperforms either component alone.

Critical strengths

The method combines semantic guidance with spatial feedback to form a clearly defined generative restoration pipeline. It uses both the degraded image and a generated textual description to guide restoration, while predicted error maps support candidate selection and local regeneration. This design provides concrete mechanisms for resolving content ambiguities in degraded images and correcting local errors in generated results, making it a worthwhile direction for further research.

Critical weaknesses
  1. UGenIR is trained on GenDS, whereas the comparison methods use their pretrained checkpoints without adaptation on the same data. In particular, for the real-world desnowing comparison in Figure 4, the paper does not clarify whether each checkpoint was trained for desnowing, so the substantial visual differences may partly reflect differences in task coverage. The absence of relevant baselines, such as the weather-specific version of TransWeather-TUR and MoCE-IR, further limits the representativeness of the comparisons. These issues weaken the evidence that the proposed framework outperforms existing restoration methods and make its claimed advantage in open-world generalization difficult to assess accurately.

  2. The main table and the FoundIR comparison primarily report image quality scores and FID, which cannot directly establish whether the generated results faithfully preserve the original image’s structure and details. Figure 4 also lacks clean reference images, so a visually cleaner result does not necessarily indicate more accurate restoration. Although the GenDS ablations report PSNR, LPIPS, and DISTS, comparisons with the main baselines using these metrics are missing. For a restoration method that relies on semantic priors and regeneration, the lack of systematic evaluation of content fidelity and incorrectly generated details makes it difficult to distinguish actual restoration gains from improvements in visual appearance. This is a key gap affecting my overall assessment.

Questions for authors
  1. Please specify the checkpoints, training data, and degradation tasks used for each baseline, as well as the prompts used by methods requiring text conditioning. In particular, were the models whose outputs closely resemble the input in Figure 4 trained for desnowing? This information is essential for determining whether the visual differences arise from the methods themselves or differences in training task coverage.

  2. Does UGenIR retain its advantage over representative restoration baselines when using the same GenDS training data or controlling for degradation task coverage? Could the authors include relevant methods, such as the weather restoration version of TransWeather-TUR and appropriate task-specific versions of MoCE-IR, to validate the current conclusions?

  3. Please explicitly identify which degradation types, severity levels, or combinations encountered during testing are absent from the training data for both UGenIR and the Critic, and explain how the training and test splits were constructed. Which results support cross-dataset generalization, generalization to unseen degradation types, or generalization to unseen degradation combinations?

  4. For the main test sets with paired clean reference images, could the authors report comparisons with baselines using PSNR, LPIPS, DISTS, and related metrics, and examine whether key structures or details are incorrectly generated? The current no-reference quality scores and FID are insufficient to establish that improvements in visual quality also preserve the original content.

  5. Token-Critic (ECCV 2022) and ITER (AAAI 2024) already use evaluation networks to guide image token resampling or iterative refinement. Please clarify the specific differences in error supervision, feedback mechanisms, and adaptation to restoration tasks, and explain how the existing experiments demonstrate the value of these differences.

Reviewer_ehFGOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes UGenIR, which treats all-in-one image restoration as conditional generation in a discrete token space. Built on the unified model Show-o, it first generates a short text describing the image content and the degradation, trained with Qwen2.5-VL-32B pseudo-captions. It then restores the clean image tokens with MaskGIT-style iterative masked prediction, conditioned on the degraded-image tokens and this text. A separately trained Swin-UNet critic predicts pixel error maps, picks the best of five candidates, and re-masks high-error tokens for one more round of generation. Trained on GenDS at 256×256 and compared with AirNet, PromptIR, InstructIR, DiffUIR and AutoDIR, the method gets the best no-reference quality scores on seven benchmark settings and six FoundIR composite subsets but not consistently the best FID, and a GenDS ablation shows small PSNR and LPIPS gains from the text and the critic.

Critical strengths
  1. Using a critic's error map to re-mask and regenerate only the likely-wrong tokens is a simple and sensible way to do targeted correction in a masked generative model. In the GenDS ablation, combining it with the text description gives consistent gains on all five metrics.
Critical weaknesses
  1. The evaluation does not show that the method restores images. All benchmarks except RealSnow10k have ground truth, yet the main tables (Table 1 and Table 3) report only no-reference quality scores and FID, with no PSNR, SSIM or LPIPS. No-reference scores reward images that look clean and sharp, and a generative model can get such scores by inventing details. FID, the only score in these tables that compares with real clean images, is often much worse than the baselines: 104.57 vs. 57.35 on RainDS raindrop, 136.73 vs. 71.60 on LOLv1, and the worst of all methods on three of six FoundIR subsets. The paper itself states four times that these scores "do not establish faithful recovery of the original image content." All results are also at 256×256 with center crops, and no runtime is reported.
  2. The comparisons are unfair, and the support for the two main components is weak. The baselines use their original checkpoints and are not retrained on GenDS, which the paper admits does not separate the method from the training data. Methods that use semantic or text priors for restoration, such as SUPIR, SeeSR and DA-CLIP, are neither cited nor compared. The paper never says which test sets are unseen, so the "open-world" claim is not tested. In the ablation, the text description alone makes four of five metrics slightly worse. "Self-correction" mixes best-of-5 selection with one round of regeneration, so the gain may come from extra sampling. The critic that judges errors is itself a one-shot regressor, which is the kind of model the paper argues fails under distribution shift.
Questions for authors
  1. Can you report PSNR, SSIM and LPIPS against ground truth on all benchmarks that have it, for all methods?
  2. Which test sets overlap with the GenDS training data, and how does the method do on degradations that are clearly unseen?
  3. How do the baselines perform when retrained on GenDS, and how does UGenIR compare with SUPIR, SeeSR and DA-CLIP?
  4. Can you separate best-of-5 selection from the refinement round, and compare with best-of-5 using random choice or token log-probability at the same compute?
  5. What happens with an empty, a wrong, or a ground-truth description?

Clarifications: 6. What is the tokenizer's reconstruction PSNR on clean images, and what is the runtime per image compared with the baselines? 7. How accurate is the critic's error map on degradations not seen in training?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.