- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces GS-IQA, a framework for region-level image quality assessment that jointly predicts distortion locations, categories, and ordinal severity levels. Training follows two GRPO stages: a Glance stage with severity rewards conditioned on correct localization and classification, and a Scrutiny stage that adaptively samples distortion categories for online synthesis based on recent rewards. The authors also introduce Diag-Bench, containing approximately 25K images spanning 12 distortion categories and five severity levels. Experiments demonstrate improvements in structured diagnosis, cross-dataset distortion recognition, and holistic quality prediction after additional fine-tuning.
Critical strengths
- Jointly predicting region–category–severity triplets addresses limitations of global quality scores and provides concrete outputs that can support diagnostic analysis and potentially guide image restoration.
- GS-IQA achieves an F1 of 0.930 and severity accuracy of 0.732 on Diag-Bench, compared with 0.886 and 0.669 for Refine-IQA. The comparison against random online generation is particularly informative: reward-conditioned generation improves severity accuracy from 0.694 to 0.732 while maintaining similar localization performance.
- The paper includes task-adapted baselines, reward and generation ablations, external evaluations, and model-scaling experiments. Splitting pristine source images before synthesis is also an appropriate precaution against source-level leakage.
Critical weaknesses
-
Limited differentiation from closely related work. The paper acknowledges SEAGULL, Refine-IQA, Grounding-IQA, and IQA-Spider, but its novelty is narrower than the broad framing of progressive region-level quality reasoning suggests. In particular, Refine-IQA already combines distortion recognition, severity estimation, grounding, and multi-stage reinforcement fine-tuning. The principal additions appear to be joint multi-region prediction, perception-gated severity rewards, and reward-conditioned online synthesis. Their incremental value should be isolated under matched data.
-
The total reward can favor omitting difficult regions. Eq.5 averages severity rewards over matched regions, contrary to the claim that this normalization prevents selective omission. Consider two ground-truth regions with perfectly predicted boxes and categories. Suppose one severity prediction is correct and the other differs by at least three levels. With valid formatting, the total reward in Eq.6 is If the model omits the difficult region, its F1 reward decreases to , but its severity reward increases to 1: Thus, even the F1 penalty does not always offset the incentive to omit a difficult region.
-
The sampling description is mathematically inconsistent with the reported settings. With rewards and their EMA in , temperature , uniform mixture weight , and 12 categories, Eq.8 implies $ P'_t(c)\ge 0.95\frac{e^{-1}}{11+e^{-1}}+\frac{0.05}{12} \approx0.0349. $ This assumes the EMA is initialized within the reward range. Appendix A.3.3 nevertheless states that sampling probabilities reach 0.004 and remain there.
-
Evaluation and training details leave important attribution questions unresolved. For example, The denominators and conditioning rules for ClsAcc and SevAcc are insufficiently specified, particularly the treatment of missed regions and incorrect categories.
Questions for authors
Please see the weaknesses. I would consider raising my rating if the authors address the concerns.
Thank you for clarifying the sampling probability bound and evaluation protocol. These address my concerns regarding W3 and W4. I would appreciate further discussion of two remaining points:
For W1, do you have an ablation comparing separate predictions of location, category, and severity followed by aggregation against direct joint triplet prediction, under matched data and training budgets? Since the task-adapted baselines also use joint triplet outputs, this comparison would more directly establish the benefit of the joint formulation itself, beyond the proposed reward and sampling mechanisms.
For W2, one possible explanation for the positive overall results is that a sufficiently strong Glance initialization makes this incentive less consequential. Does the gains remain for smaller VLM backbones? In particular, does subsequent training correct difficult severity predictions, or increasingly omit the corresponding regions? Tracking initially misgraded regions across training (whether they are corrected, remain incorrect, or become missed) could help clarify when this incentive materially limits diagnostic performance. Understanding this boundary would better establish the method’s applicability across model capacities and initialization strengths.