acceptodds
Withdrawn from ICLR 2027

Beyond Synthetic: Real-world Post-Training for Image Restoration with Imperfect Generative Ground Truth

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Generative image restoration has achieved impressive performance, but models trained solely on synthetic data struggle to generalize to real-world scenarios. We propose a degradation-based pre-training and post-training paradigm, in which post-training data are constructed by generating ground-truth for real low-quality (LQ) images using image editing models. However, generated targets usually contain hallucinations, making them unreliable for direct supervision. Our key insight is that effective degradation adaptation does not require every region of a generated target to be fully reliable. Based on this observation, we develop a simple yet effective approach that employs a vision-language model to localize hallucinated regions and masks their gradients during post-training. A localized refinement and verification loop further refines these regions, thereby recovering additional usable supervision. Experiments on old-photo enhancement, user-generated content enhancement, and real-world super-resolution across three representative generative restoration models demonstrate that the proposed framework consistently improves restoration performance in specific real-world scenarios.

Primary area:
generative models
Keywords:
Generative Image Restoration, Generative Image Super-resolution

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_Zun1OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies target-domain post-training for generative image restoration using imperfect generated targets. Starting from real low-quality images, an image editing model generates candidate high-quality targets. A vision-language model identifies semantic hallucinations on an 8-by-8 grid, and the corresponding regions are excluded from spatially aligned training losses. A localized editing and verification loop attempts to correct these regions and recover additional supervision. The paper also presents a controlled synthetic study of broad-degradation pre-training followed by target-degradation post-training, and evaluates the proposed pipeline with SeeSR, OSEDiff, and HYPIR on old-photo enhancement, UGC enhancement, and real-world super-resolution.

Critical strengths
  1. The paper addresses a relevant problem: generated restoration targets can contain hallucinated content, and discarding an entire image may waste useful supervision in otherwise reliable regions. Attempting to retain and repair this supervision is a reasonable direction.
  2. Several qualitative examples show sharper structures and additional details after post-training. The experiments also consider restoration models with different training formulations, suggesting that the pipeline could be applicable beyond a single backbone.
Critical weaknesses
  1. The pre-training experiments do not establish the claimed necessity or sufficiently separate the effect of initialization from the training budget. In Figure 2(a), the model without degradation pre-training steadily approaches the pre-trained model over the reported post-training interval. This supports an advantage in initial performance and early convergence, but does not establish an advantage after convergence. Moreover, the pre-trained model receives 30K additional training steps before the 5K-step adaptation stage. Longer target-only training and comparisons under matched total training budgets are needed. Similarly, Figure 2(b) compares Narrow and Wide pre-training at a single post-training endpoint; the LPIPS difference after adaptation is only 0.001. Without learning curves over a longer adaptation horizon, variation across runs, and an analysis of the allocation between pre-training and post-training, it is difficult to assess how robust this conclusion is to the selected training schedule.

  2. The reliability of the generated supervision, which is central to the method, is not directly established. The paper needs to show that the retained regions are reliable and that refinement improves target fidelity. Downstream improvements after fine-tuning do not by themselves establish either point. Table 3 provides a limited evaluation on 20 real LQ images, but the source and construction of the "hallucination-free references" are unspecified. Its one-round refinement baseline also does not establish the quality of the initial editing-model outputs. There is no reported evaluation of hallucination localization against human annotations, no quantitative analysis of residual errors in regions retained by the mask, and no systematic initial-GGT versus refined-GGT comparison on data with genuine clean references. Figure 3 provides an illustrative example, but a broader set of successful and failed localization/refinement cases is needed.

  3. The ablation setup and evaluation are insufficiently documented. The datasets and experimental protocols used for the ablations are not clearly specified, making the results difficult to assess or reproduce reliably. Metric coverage is also incomplete; in particular, Table 2 reports only no-reference quality scores, leaving the effects of masking and refinement on fidelity and input consistency unclear.

  4. The additional target enhancement is not evaluated separately. The supplementary material mentions HYPIR-based enhancement of the old-photo targets but provides no results with and without this step, making its contribution to the reported improvements unclear.

Questions for authors
  1. Which restoration model, dataset/domain, data split, and training schedule are used in Tables 2 and 3?
  2. Can the authors extend Figure 2 to convergence and provide target-only baselines under matched total training budgets? How sensitive are the Narrow-versus-Wide results to the duration of post-training and to random seeds?
  3. How are the 20 hallucination-free references in Table 3 obtained and verified? Can the initial and refined GGTs be evaluated against genuine clean references on a paired benchmark?
  4. Can the authors evaluate VLM localization against human annotations and quantify the fidelity of regions retained by the mask? Please also provide representative successes and failures showing the initial GGT, detected/masked regions, refined target, and unresolved regions.
  5. Can Table 2 include fidelity or input-consistency metrics alongside quality metrics?
  6. Can the old-photo experiments include an ablation without HYPIR-based target enhancement while holding the remaining processing fixed?
Reviewer_F4xXOpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
3
Soundness
1
Read review
Paper summary

The authors propose a degradation-based pre- and post-training paradigm for generative image restoration by introducing Generated Ground Truth (GGT)—an imperfect GT containing hallucinations. To avoid inaccurate supervision, hallucinated regions are identified and their gradients are masked during training; subsequently, a VLM-guided localized refinement and verification loop is employed to further enhance data efficiency. The problem is relevant, but I find the contribution and supporting evidence insufficient. It is difficult to obtain a ground truth (GT) that is consistent in content under different diffusion-based image restoration methods.

Critical strengths
  1. The paper is clearly written, providing sound reasoning and comprehensive experiments.
  2. The paper proposes a Vision-Language Model (VLM) to localize hallucinated regions, masks out the gradients of these regions during post-training, and subsequently performs image editing on them.
  3. The proposed method achieves competitive quantitative results on most datasets.
Critical weaknesses
  1. Table 1 and Figures 4 and 5 provide performance comparisons of three methods on the Old Photo and UGC datasets. However, it is difficult to draw consistent conclusions from these results, and the paper lacks a detailed explanation, making the claims less convincing.
  2. In the VLM-based hallucination evaluation, an 8×8 patch partition is directly adopted. However, the paper does not mention any ablation study on how denser or sparser partitions affect the performance.
  3. For the ablation study of the refinement loop, the authors rely on only 20 real-world LQ images. This inadequate evaluation scale fails to provide statistically significant evidence, severely undermining the reliability of the reported conclusions. Why does the paper only use the PSNR metric? Other perceptual metrics are not included in the evaluation.
  4. As shown in Table 2, except for a slight improvement in the MUSIQ metric, the effect of the refinement module is not very significant. The authors need to analyze how the proportion of refinement region affects performance.
Questions for authors

For detailed concerns, please refer to the weaknesses.

Reviewer_ETKvOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a pre-training and post-training framework for adapting generative image restoration models to real-world degradations. For post-training, the authors construct training pairs from real low-quality images using generative image editing. Since the generated targets may contain hallucinated content, a vision-language model is used to identify unreliable regions, mask their gradients during training, and guide local refinement and verification. Experiments on old-photo enhancement, user-generated content enhancement, and real-world super-resolution with three restoration models consistently improve both fidelity and perceptual quality. These results suggest that imperfect generated targets can still be useful as supervision for adapting restoration models to real-world data.

Critical strengths
  1. The paper addresses a practical challenge in real-world restoration, where paired supervision for target-domain degradations is often difficult to obtain. A strength of the framework is that it does not require every generated target to be fully reliable. Instead, it retains usable supervision by masking unreliable regions and refining them locally. This makes the proposed post-training approach applicable to real-world domains where clean paired targets are difficult to collect.
Critical weaknesses
  1. Robustness to failures in the editing model and VLM

The proposed framework relies heavily on the external image editing model and VLM. Importantly, both target generation and hallucination verification are based on the same degraded LQ observation. If the LQ image does not contain enough information to recover the underlying clean content, the editing model may produce a plausible but incorrect reconstruction that the VLM cannot reliably reject. Such a sample could then enter post-training as supervision.

This failure case is not characterized in the current ablations. An analysis of hallucination detection errors would therefore be useful, particularly for cases where the LQ input is highly ambiguous. It would also be helpful to report how the final performance changes when different editing models or VLMs are used. These experiments would give a clearer picture of how sensitive the framework is to its external components and under what conditions the generated supervision remains reliable.

  1. Quality of GGT construction

The framework assumes that the regions retained after hallucination filtering are sufficiently faithful to the unknown clean target. Section 5.2 evaluates the refinement pipeline on 20 real LQ images with hallucination-free references, but this experiment mainly shows that the refinement loop can correct problematic regions. It does not fully establish how accurate the initial and refined GGTs are across a larger set of samples, or how often the regions retained for supervision are actually reliable. This matters because downstream improvement alone does not guarantee that the generated targets preserve the true underlying content. If an incorrect reconstruction is perceptually plausible and escapes the hallucination filter, it may still be used as supervision. A larger reference-based evaluation, especially one that measures errors within the retained regions, would more directly test this assumption.

Questions for authors
  1. Robustness. How sensitive is the proposed framework to failures or changes in the image editing model and VLM, particularly when the LQ input is highly ambiguous?

  2. GGT quality. How faithfully do the regions retained for supervision in the initial and refined GGTs preserve the underlying clean content? Could the authors provide a larger reference-based evaluation, particularly for the retained regions?

  3. Scalability. What are the computational cost and average number of editing/VLM calls required to construct GGT at scale?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.