acceptodds
Withdrawn from ICLR 2027

Learning a Shared Representation for Low-Level Vision via Degradation Understanding

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Low-level vision tasks such as image quality assessment (IQA) and restoration rely on common evidence about image perturbations, yet learned degradation representations are usually architecture- or task-specific. We propose Degradation Understanding as a supervisory task for learning a shared low-level representation. It predicts a hierarchical state over degradation type, composition, parameter keys, and continuous values, exposing a model-agnostic interface without assuming recovery of the true imaging history. To support this task, we construct DU-200K from 160,000 synthetic samples with structured supervision and 40,000 real-captured samples with restoration-grounded feedback. We design a Degradation Understanding vision-language model (DU-VLM) that learns structured prediction via supervised fine-tuning (SFT) and outcome alignment via Group Relative Policy Optimization (GRPO). We evaluate our method across multiple downstream tasks. For IQA task, our representation yields a 29.9% average improvement in SRCC across five datasets. On real-captured data, explicit degradation prompts predicted by our method improve PSNR by up to 1.74 dB over generic text prompts across state-of-the-art image-editing models. On entirely unseen and challenging real-world images, our system remains highly competitive among restoration systems based on diverse degradation-representation paradigms, supporting transfer across tasks, interfaces, backbones, and real scenes.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
MLLM, Low level vision, Degradation Understanding

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_XSnGOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes Degradation Understanding as a structured supervision task for learning representations shared by image quality assessment (IQA) and restoration. A Qwen3-VL-based model predicts degradation types, parameter keys, numerical values, and composition order from RGB, edge, and frequency-domain inputs. The authors construct DU-200K, perform supervised fine-tuning on synthetic degradation annotations, and subsequently apply GRPO using feedback from fixed restoration operators. Validation includes synthetic structured-prediction benchmarks, target-dataset-supervised IQA adaptation, conditioning of frozen restoration models, and real-world restoration evaluation on CleanBench.

Critical strengths
  1. An explicit, reusable degradation interface. Representing degradations through types, parameters, and composition offers a potentially useful interface across otherwise separate quality-assessment and restoration systems. Demonstrating transfer through both hidden features and explicit prompts gives the proposal practical relevance beyond a single restoration architecture.
  2. Matched internal comparisons provide preliminary evidence of useful representation learning. Comparisons among Base Qwen, DU-SFT, and DU-GRPO under the same target-supervised IQA protocol, together with the fixed-backbone DiffBIR comparison, suggest that the training procedure can improve downstream utility. These comparisons are more informative than the unmatched comparisons with external systems.
Critical weaknesses
  1. The Introduction does not sufficiently analyze the limitations of existing work. In particular, the second paragraph describes existing representations as task- or architecture-specific, but does not explain how this limits their use in IQA or restoration. The connection between these limitations and the motivation for the proposed method is therefore unclear.
  2. The reward definitions are internally inconsistent. In Appendix A.5, Eq. 9 defines the paired reward as an average of aesthetic and MSE scores, whereas the adjacent text describes five equally weighted metrics. Eq. 15 specifies an aesthetic-only reward for unpaired samples, while the subsequent text describes averaging multiple available metrics. These conflicting definitions prevent readers from determining the actual training objective and reproducing the alignment procedure.
  3. Degradation understanding on real-captured images is not directly validated. The paper reports downstream restoration results on real images but does not establish whether the model correctly estimates degradation types, parameter keys, numerical values, and composition order in that setting. Better restoration does not necessarily imply a correct degradation diagnosis. A more appropriate validation design is needed.
  4. The IQA comparison appears to use inconsistent supervision protocols. The proposed method trains a separate prediction head using labels from each target dataset. However, the five DeQA-Score results in Fig. 5 exactly match those reported in the original paper, where the model was trained only on KonIQ and evaluated on the other datasets without target-dataset training. Unless these baselines were reproduced under matched supervision, the comparison mixes target-supervised adaptation with cross-dataset generalization. As currently documented, it does not support the headline claim of outperforming DeQA-Score on average.
  5. Writing and figure-quality problems impair clarity and consistency. The abstract contains a broken sentence at “outcome alignment via Group We evaluate,” and the Introduction is difficult to follow in places. Figure 1(a) also contains unclear visual elements, including distorted-looking axes in “Observation Config” and an obscured camera in “Optical Path.” More consequentially, Figure 1 retains labels such as “Cross-dataset IQA” and “Camera Intrinsics” that conflict with the stated evaluation protocol and parameter interpretation.
Questions for authors
  1. How do the task- or architecture-specific representations discussed in the second paragraph of the Introduction limit their use in IQA or restoration? Please explain how these limitations motivate the proposed method.
  2. Which paired and unpaired reward formulas were used to train the reported DU-GRPO checkpoint? Please reconcile Eq. 9 with the adjacent five-metric description, and Eq. 15 with the subsequent description of averaging available metrics. Please specify the normalization procedures and weights.
  3. How well does the model predict degradation types, parameter keys, numerical values, and composition order on real-captured images? Please validate these predictions directly, rather than relying on downstream restoration performance, and provide complete image-level examples explaining how their correctness is assessed.
  4. Were the external baselines in Figure 5 trained using labels from each target dataset, as the proposed IQA heads were? Please specify their training datasets and checkpoints and explain why the DeQA-Score values exactly match its original KonIQ-only evaluation results.
Reviewer_PmgLOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Degradation Understanding (DU) as a supervisory task for learning transferable low-level visual representations. It introduces DU-200K and trains DU-VLM through SFT and GRPO, evaluating its transferability to image quality assessment and restoration. Although promising results are reported, substantial concerns remain regarding methodological novelty, validation of the shared representation, experimental fairness, and the technical soundness of GRPO training.

Critical strengths
  1. Learning degradation representations shared between image quality assessment and restoration is a meaningful research direction with potential broader impact.
  2. Evaluation across five IQA datasets, multiple restoration backbones, and real-world scenarios provides useful empirical evidence for the proposed framework.
Critical weaknesses
  1. The framework mainly combines degradation supervision, SFT, GRPO, and pretrained VLMs. The methodological novelty beyond existing degradation-aware representation learning remains limited.
  2. IQA requires dataset-specific prediction heads, while restoration relies on additional trainable conditioners. These results demonstrate feature utility but do not sufficiently establish a genuinely unified, directly transferable representation.
  3. DU-GRPO uses approximately 1.05M-parameter prediction heads trained on target-dataset labels, without equivalent head-adaptation comparisons for competing IQA representations. The performance advantage may partly reflect supervision differences.
  4. After GRPO, joint accuracy drops from 98.81% to 86.66%, while relative parameter error increases from 0.14 to 0.28. This raises concerns that reward optimization compromises the core degradation-understanding objective.
  5. Eq. (9) uses an undefined �0��0�, while the following text describes an equally weighted five-metric reward. The no-reference reward is also inconsistent with its multi-metric description, raising reproducibility concerns.
  6. Page 3 contains stray “‘latex” text immediately before Section 3. This should be removed, and the manuscript should be proofread.
Questions for authors
  1. The GRPO reward equations are inconsistent with the metric descriptions in Appendix A.5. What exact reward and weights were implemented, and could this discrepancy affect the reported results?
  2. Were the IQA baselines given the same target-dataset supervision and head-training protocol as DU-GRPO? If not, how can the reported gains be attributed specifically to the learned shared representation?
  3. What are the quantitative results of the Type-only, coarse-severity, and shuffled-value ablations described in Section 5.4? Do they establish an independent benefit of continuous degradation parameters over simple category prompts?
Reviewer_3Kw1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper trains a vision-language model to predict structured degradation types, ordered compositions, and numerical parameters using synthetic supervision, followed by restoration-reward GRPO alignment. It evaluates the frozen representation through dataset-specific IQA heads, DiffBIR conditioning, and explicit prompts to image-editing models.

Critical strengths

The matched Base Qwen/DU-SFT/DU-GRPO comparisons provide useful evidence of transfer beyond the handcrafted reward operators. Evaluating both hidden features and explicit prompts is relevant to the proposed shared representation.

Critical weaknesses
  1. The central alignment reward is internally inconsistent (Sec. 4.3; Appendix A.5). Eq. 9 averages aesthetic and MSE scores, but the following text specifies five equally weighted terms: PSNR, SSIM, LPIPS, aesthetic, and CLIP-IQA; the MSE normalization is not defined there. Likewise, Eq. 15 uses only aesthetic quality, whereas Table 5 and its explanation average available no-reference metrics. These alternatives optimize different objectives and imply different overlap with the evaluation metrics, so the reported GRPO improvement cannot be interpreted unambiguously.

  2. The evidence does not establish the value of the quantitative state beyond degradation labels or coarse severity (Sec. 5.4; Table 2). The text describes type-only, coarse-severity, random-parameter, shuffled-value, and shuffled-order controls, but neither Table 2 nor the appendix reports their results. The single mismatched-state row combines several interventions. Consequently, the reported transfer gains do not isolate the numerical values and composition order that distinguish the proposed interface.

Questions for authors
  1. Which exact paired and unpaired reward functions produced the reported DU-GRPO checkpoint? Please reconcile Eqs. 9 and 15 with the accompanying text, specify all normalizations, and identify the metrics shared with downstream evaluation.
  2. Please provide the results of the state-component controls already described in Sec. 5.4, separating correct-type/wrong-value from wrong-type conditions. Do precise values and process order improve over the same predictions reduced to type and coarse severity?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.