- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
ROSA-IQA adapts on-policy self-distillation to learn image-quality descriptions and scores from mean opinion score (MOS) annotations without human-written descriptions. A frozen teacher initialized from the student backbone receives privileged MOS and segmentation-derived crops, and supervises next-token distributions along student-generated trajectories. Disagreement between teacher distributions under factual and counterfactual MOS selects positions for distillation. A continuous scoring head is trained jointly. Using Qwen3-VL-4B trained on KonIQ, the method achieves the highest reported average scoring performance across seven IQA datasets and favorable description evaluations. Direct scoring slightly outperforms reasoning-then-scoring at inference, which the authors interpret as evidence that description supervision acts as a training regularizer.
Critical strengths
- A useful objective and coherent implementation. Learning quality descriptions from existing score annotations reduces dependence on human-written explanations. The pipeline is clearly described, and scoring experiments cover authentic, synthetic, algorithm-induced, and AI-generated image distortions.
- Informative controlled comparisons. Table 3 examines score-only training, off-policy distillation, region guidance, and equal-budget random token selection. Table 4 also shows that direct scoring retains the benefits of joint training without generating descriptions at inference.
Critical weaknesses
- The motivation does not establish a clear bottleneck that the proposed extensions resolve. The paper invokes annotation and training costs, exposure bias, and coarse RL feedback to motivate its adaptation of OPSD. However, the experiments do not clearly identify which limitation constrains existing IQA performance. The on-/off-policy comparison shows a performance difference without establishing exposure bias as its cause, and cost savings are not quantified. This leaves the main additions (MOS/region conditioning and counterfactual token selection) without a strong account of why they are needed. For token selection in particular, comparisons with full-token and random controls support its usefulness but provide limited evidence for the specific selection criterion's advantage over other baseline methods.
- The demonstrated benefits are modest relative to the methodological claims. Since the method learns both scores and descriptions, either output could establish its practical contribution. For scoring, Table 1 improves average SRCC/PLCC over same-backbone IQA-T1 by 0.003/0.014. Table 3's gains over score-only training are concentrated in CSIQ, and token selection adds 0.004/0.003 over full-token distillation with the same region guidance. Description results provide some complementary evidence, but the largest gains concern a confidence criterion defined independently of correctness. Human preferences cover only 12 selected images per dataset across three datasets. Together, these results demonstrate benefits in selected settings without establishing a substantial, broadly supported advance.
Questions for authors
- Which limitation of existing IQA methods is principally responsible for the gains? What evidence distinguishes the proposed explanation from a general benefit of additional description supervision, and supports the particular counterfactual token-selection criterion?
- If reduced cost is a principal benefit, what are the training and direct-scoring inference costs relative to relevant baselines, particularly IQA-T1? Please include teacher computation and region processing.
3 comments
Thank you for the additional experiments and clarifications. The expanded description evaluations and cost analysis strengthen the empirical evidence. However, the scoring gains remain modest and uneven, and the new results do not sufficiently change my assessment of the significance of the methodological contribution. While several concerns have been alleviated, my overall judgment remains unchanged, and I maintain my original rating.
Thank you for taking the time to reply again! We regret that the scoring gains of our model did not fully meet your expectations. However, we would like to note that achieving competitive SOTA performance with a single forward pass, without requiring iterative tool calls or multiple forward passes at inference, is already noteworthy. Comparing our OPSD-based framework with GRPO-only methods (such as Q-Insight and VisualQuality-R1) on score regression may also provide a fairer perspective. Moreover, scoring directly on latent features enables continuous score modeling and fundamentally avoids format or parsing failures commonly encountered in discrete token-level score regression.
On the other hand, generating accurate and reliable quality descriptions is an equally central contribution of this work. In our LLM-as-a-Judge evaluation, ROSA-IQA achieves mean accuracy scores of 5.782 / 6.445 under Gemini 3.8 Flash and GPT 6.1 Sol, outperforming Q-Insight (5.532 / 6.198) and VisualQuality-R1 (5.440 / 6.273). In subjective study, ROSA-IQA attains an overall selection rate of 41.3%, substantially ahead of Q-Insight (16.9%) and VisualQuality-R1 (12.5%).
Finally, we kindly hope you will take into account the novelty of introducing this new training paradigm to IQA, as OPSD offers promising avenues for further meaningful research in privileged-information conditioning and selective token distillation. Thank you again for your thorough review and valuable feedback!
Looking Forward to Further Discussion
Dear Reviewer,
Thank you for your constructive feedback. We have added experiments and clarifications addressing your concerns, responded in detail in our point-by-point reply, and uploaded the revised manuscript. We are eager to hear any further suggestions you may have and hope that the revisions meet with your approval.
We clarified the role of each component. MOS supervises image-level scoring, SAM-2 regions give the teacher local distortion cues, and differences between the teacher distributions under factual and counterfactual MOS guide token selection. We present exposure bias as a motivation for on-policy training and report the on-policy and off-policy results as an empirical comparison. To examine the roles of additional description supervision and quality-sensitive token selection (QS), we compared offline quality-description distillation, generic image-description distillation, and feature distillation. We also added a token analysis, blind assessments of distortion type and severity, and a separate evaluation of region guidance for localization.
We clarified the size and dataset dependence of the scoring gains, added description baselines, and expanded the blinded human preference study to five methods and 150 images. ROSA-IQA received 41.3% of the votes on images selected by description disagreement. We also compared training costs with GRPO, including teacher computation, region encoding, and SAM-2 preprocessing, and clarified how our training procedure differs from IQA-T1's. We reported latency for both description generation and direct scoring. ROSA-IQA takes an average of 0.0378 seconds per image to score directly.