acceptodds
Withdrawn from ICLR 2027

Learned Degradation Guidance: Turning Frozen Image Editors into Controllable Restorers

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Instruction-based image editors have a strong image prior, and classifier-free guidance allows them to trade fidelity for perceptual quality with a single scale. Guidance pushes the output away from a second prediction of the model. In restoration, the model usually makes this second prediction under a generic low-quality prompt. Such a prompt names a degradation but cannot show the one in the image, so the model produces a degradation of its own. We introduce Learned Degradation Guidance (LDG), in which a trained adapter, not a prompt, makes the second prediction. On a frozen editor, a degradation adapter makes an image's own degradation stronger, and a restoration adapter removes it. The two adapters are trained separately, only on images that the editor generates itself, and are combined by guidance at inference. Guidance then pushes the output away from the image's own degradation, and one scale takes the editor from faithful to sharp restoration. With this second prediction, the editor reaches a higher perceptual quality than with one prompted to degrade. In real-world super-resolution tests, LDG with LoRA adapters is closer to the ground truth than the state of the art at its most faithful setting, reaches the highest perceptual quality at larger scales, and at equal or higher quality stays closer to the ground truth in almost every comparison. On real photographs with nine kinds of degradation, such as haze, rain and reflections, it achieves the best average score among the compared methods on a recent benchmark, with one adapter pair per degradation, each trained on about a hundred generated images. Source-attention calibration, an adapter that only reweights the editor's attention with a few thousand parameters, is almost as faithful as LoRA at the faithful end of the scale.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
image restoration, real-world super-resolution, classifier-free guidance, diffusion models, flow matching, image editing models, parameter-efficient fine-tuning, synthetic training data

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_1nDpOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Learned Degradation Guidance (LDG) for controllable image restoration using frozen instruction-based image editors. The method separately trains a restoration adapter and a degradation adapter, where the latter predicts a more severely degraded version of the input. Their predictions are combined through classifier-free guidance to control the fidelity–perception trade-off. Both adapters are trained using a small set of synthetic images generated by the pretrained editor. The paper also introduces Source-Attention Calibration (SAC), a lightweight adaptation mechanism that learns noise-dependent attention biases for different token sources. Experiments cover several real-world super-resolution benchmarks and nine degradation types from RealIR-Bench.

Critical strengths
  1. Interesting and well-motivated degradation-aware guidance. The idea of learning a degradation-specific second prediction rather than relying on a generic negative prompt is intuitive and meaningful. The direction-alignment analysis in Table 7 supports the underlying motivation, and the results on ScreenSR demonstrate noticeable improvements over single-adapter CFG.
  2. Promising few-shot adaptation with extensive experimental evaluation. The method achieves competitive restoration performance using only about 100 self-generated training images per task. The experiments cover multiple restoration benchmarks, degradation types, and pretrained image editors. The proposed SAC also demonstrates promising parameter efficiency, achieving reasonable restoration performance with only a few thousand trainable parameters.
Critical weaknesses
  1. Limited practical gains from the main LDG contribution. Although LDG improves guidance-direction alignment, its practical advantages over single-adapter CFG appear limited and inconsistent. At matched MUSIQ levels (Tables 5–6), LDG achieves only marginal LPIPS improvements on RealSR and DRealSR, sometimes with lower PSNR. On RealIR-Bench, single-adapter CFG achieves a higher FS and removal score at matched input-change magnitude. Moreover, the strong fidelity results at (g=1) do not involve the degradation adapter. Considering the additional adapter training and task-specific adaptation requirements, the benefits of the main contribution are not fully convincing.
  2. Insufficient justification for SAC's design choices. SAC is closely related to existing attention-reweighting methods, such as OminiControl and TACA, making its methodological novelty relatively limited. More importantly, the ablations in Table 5 shuffle or average already-trained SAC biases, which does not establish the necessity of the proposed fine-grained parameterization. Independently training simpler variants, such as head-shared or noise-independent source biases, would provide stronger evidence for SAC's specific design choices.
Questions for authors
  1. Could the authors further clarify the practical benefits of LDG over single-adapter CFG under matched evaluation conditions? The improvements in fidelity at matched perceptual quality appear relatively small or inconsistent on several benchmarks, while the proposed method requires an additional trained adapter. It would be helpful to clarify under which conditions the learned degradation guidance offers a meaningful advantage over prompt-based CFG.

  2. Could the authors provide independently trained ablations for simpler SAC configurations? The current shuffling and averaging experiments modify an already trained SAC adapter, which may not fairly reflect the capacity of simpler designs. Training variants with source biases shared across heads or layers, or without noise-dependent terms, would help establish the necessity of the proposed parameterization.

Reviewer_HY4JOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Learned Degradation Guidance (LDG), which trains separate restoration and degradation-enhancement adapters on frozen image editors and combines their predictions through guidance to control the fidelity–perceptual quality trade-off. Source-Attention Calibration (SAC) is further introduced for parameter-efficient adaptation. Promising results are reported on real-world super-resolution and nine restoration tasks.

Critical strengths
  1. The idea is simple and practical. A second prediction that exaggerates the input's own degradation, instead of a generic low-quality prompt, is a clean way to give a frozen editor a single fidelity–quality control. Adapters train in about 16 minutes on one GPU.
  2. The recipe transfers to a second editor with a different architecture, autoencoder and step count (Kontext), and the authors promise code, data-generation scripts and adapters.
Critical weaknesses
  1. LDG largely builds upon CFG, dual-adapter training, and negative guidance, closely related to Autoguidance, SIMS, and PiSA-SR. Parameter-matched dual-adapter controls are needed to better isolate the methodological contribution.
  2. Degradation-direction alignment is mainly validated on synthetic degradations generated using the same training procedures. Direct evidence that the learned direction accurately captures real-world degradations remains insufficient.
  3. On RealIR-Bench, single-adapter CFG achieves a higher average FS than LDG-LoRA (0.1726 vs. 0.1677) and a higher degradation-removal score at matched input change. This weakens the claim of consistently superior learned guidance.
  4. LDG explores different guidance scales, while adjustable baselines such as PiSA-SR, InvSR, and HYPIR are evaluated mainly with their default settings. Fully tuned fidelity–perception trade-off curves are needed for fair comparison.
  5. Each degradation requires separately trained adapters and a known degradation type. Moreover, guidance can require up to 55 large-model evaluations per image, limiting practicality and leaving unseen composite degradations insufficiently explored.
Questions for authors
  1. Table 13 shows that single-adapter CFG achieves higher FS and better degradation removal at matched input change. What evidence demonstrates that LDG's benefits arise from a more accurate guidance direction rather than stronger modifications?
  2. Degradation-direction alignment is primarily validated on synthetic data. What evidence supports the claim that the learned direction also captures real-world degradations rather than merely fitting predefined synthetic corruption processes?
  3. LDG uses tunable guidance scales, whereas several controllable baselines retain their default settings. How do the current results establish LDG's superiority under comparable perceptual quality and computational budgets?
Reviewer_Gqs9OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes Learned Degradation Guidance (LDG) for adapting frozen image editors to controllable restoration. Clean images generated by the editor are synthetically degraded at two strengths. A restoration adapter learns to recover the clean image, while a degradation adapter learns to strengthen the input's degradation. Their predictions are combined through a CFG-style rule at inference, with a guidance scale controlling restoration strength. The paper also introduces source-attention calibration (SAC), an alternative to LoRA that learns source-specific attention biases. Experiments evaluate super-resolution and nine restoration tasks using FLUX.2 Klein and FLUX.1 Kontext.

Critical strengths
  1. Controlling the fidelity-perceptual quality trade-off is a relevant restoration problem. The method provides a simple inference-time control and demonstrates adaptation using a small set of generated training images on two editing backbones.
  2. The method achieves strong quantitative scores in some settings and produces good qualitative results in several examples.
Critical weaknesses
  1. The writing quality is poor, with insufficient polishing and an uneven narrative flow. The abstract introduces "second prediction" before clearly identifying the two predictions being contrasted, then abruptly adds SAC in its final sentence without explaining how it fits the main contribution. The introduction similarly moves from paired-data requirements and dataset sizes to controllability without a clear transition, and introduces SAC without adequately motivating its role alongside LoRA and LDG. These abrupt shifts make the motivation and relationship between the contributions difficult to follow. Compressed phrasing, such as "One scale from fidelity to perceptual quality" in the Figure 3 caption, also leaves the intended meaning unnecessarily implicit. These issues extend beyond isolated terminology: the paper needs substantial editing to improve clarity, organization, and the flow of its argument.

  2. The comparisons do not convincingly establish a better trade-off than existing controllable restoration methods. Table 1 reports separate fidelity-oriented and quality-oriented settings for LDG, while competing methods are evaluated at their released settings. Reporting both endpoints is reasonable, but favorable scores at different settings do not establish a superior trade-off. Appendix E explicitly states that the controls of PiSA-SR, InvSR, HYPIR, and VOSR were left at their default values. Their achievable trade-off curves should be compared with LDG's curve, particularly because controllability is the central motivation. Figure 3 shows a PSNR advantage, but does not demonstrate LPIPS-MUSIQ dominance over TSD-SR, which has slightly better LPIPS at matched MUSIQ. The matched-quality analysis is useful, but it still compares LDG against individual baseline operating points rather than their attainable frontiers.

  3. The evidence for the specific value of a separately trained degradation branch is mixed. Pushing away from a more degraded prediction is an understandable heuristic, but the argument for improved fidelity relies on alignment assumptions, and improved alignment on synthetic triples does not establish better restoration on real images. At g=1, the degradation adapter is unused, so the strong fidelity results at that setting do not validate the central guidance contribution. Increasing g improves no-reference scores but often reduces reference fidelity; this alone does not distinguish useful recovery from added or invented detail. More importantly, Appendix Table 13 shows that single-adapter CFG achieves a higher RealIR-Bench FS than LDG-LoRA (0.1726 versus 0.1677), and a higher removal score at matched input-change level (1.485 versus 1.361). Although LDG attains higher removal and quality scores at the reported scales, these results do not convincingly establish that the additional degradation adapter offers a preferable trade-off over the simpler alternative. Stronger controlled comparisons and independent perceptual evaluation are needed to justify its added training and inference cost relative to restoration alone.

Questions for authors
  1. Can the authors sweep the available controls of PiSA-SR and other adjustable baselines and compare their fidelity-quality frontiers with LDG under a common evaluation protocol?
  2. What evidence establishes a consistent advantage of the learned degradation branch over single-adapter CFG at matched quality or input-change levels, given the counterexamples in Table 13?
  3. Can human evaluation assess whether the added details at larger guidance scales are preferable and content-preserving relative to g=1 and single-adapter CFG?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.