acceptodds
Under review as a conference paper at ICLR 2027

Decoupling Perception and Calibration: Label-Efficient Image Quality Assessment Framework

Abstract

Recent multimodal large language models (MLLMs) have demonstrated strong capabilities in image quality assessment (IQA) tasks. However, adapting such large-scale models is computationally expensive and still relies on substantial Mean Opinion Score (MOS) annotations. We argue that for MLLM-based IQA, the core bottleneck lies not in the quality perception capacity of MLLMs, but in MOS scale calibration. Therefore, we propose LEAF, a Label-Efficient Image Quality Assessment Framework that distills perceptual quality priors from an MLLM teacher into a lightweight student regressor, enabling MOS calibration with minimal human supervision. Specifically, the teacher conducts dense supervision through point-wise judgments and pair-wise preferences, with an estimate of decision reliability. Guided by these signals, the student learns the teacher’s quality perception patterns through joint distillation and is calibrated on a small MOS subset to align with human annotations. Experiments on both user-generated and AI-generated IQA benchmarks demonstrate that our method significantly reduces the need for human annotations while maintaining strong MOS-aligned correlations, making lightweight IQA practical under limited annotation budgets. The code will be released upon publication.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Image Quality Assessment, Knowledge Distillation, Multimodal Learning

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_YGgvOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies label-efficient image quality assessment and argues that pretrained MLLMs already have reasonably strong quality perception, while their main difficulty is aligning predictions with dataset-specific MOS scales. Based on this observation, the paper proposes LEAF, a two-stage framework. In the first stage, an MLLM teacher provides point-wise quality judgments and pair-wise preferences to train a lightweight student without human MOS labels. In the second stage, a small MOS-labeled subset is used to calibrate the student predictions. Experiments on two AIGC and two UGC datasets show that the method can achieve competitive IQA performance with substantially fewer MOS annotations.

Critical strengths

The main idea of separating quality perception from MOS calibration is simple but useful. The paper gives a reasonable motivation that MLLMs can preserve image quality ranking even when their absolute scores are not well aligned with dataset-specific MOS. The experiments cover both AIGC and UGC datasets, and the method performs consistently well in the low-label setting. In particular, the label-free results are strong compared with the reported label-free baselines. The paper also evaluates different teacher models and student backbones, which makes the experimental results more convincing. Another practical advantage is deployment efficiency. The MLLM teacher is only used offline, while inference is performed with the much smaller student model.

Critical weaknesses

My main concern is that the central claim that MLLMs are strong at perception but mainly weak at MOS calibration is demonstrated most directly on AGIQA-3K. It is not clear whether the same behavior holds equally well across other datasets, distortion types, and teacher models. The comparison in the weakly supervised setting is also somewhat difficult to interpret. LEAF uses 10% or 30% MOS, while several competing methods use substantially more labeled data. This supports the label-efficiency argument, but it does not provide a controlled comparison under the same annotation budget. The contribution of pair-wise supervision appears relatively small compared with point-wise distillation, while it requires additional teacher queries. A clearer analysis of the accuracy-cost trade-off would help justify this part of the framework. Finally, the calibration is still dataset-specific. A new dataset or quality distribution appears to require another MOS subset for Stage 2. Cross-dataset calibration or transfer experiments would make the perception-calibration argument stronger.

Questions for authors
  1. The main motivation is that MLLMs already have strong quality perception but are poorly calibrated to MOS. Figure 2 mainly demonstrates this on AGIQA-3K. Does the same pattern hold on AIGIQA-20K, KonIQ-10k, and SPAQ?
  2. How sensitive is LEAF to the choice of teacher? Table 3 compares several MLLMs on AGIQA-3K, but does a weaker teacher consistently lead to weaker results on the other datasets as well?
  3. The weakly supervised baselines often use around 70% MOS, while LEAF uses 10% or 30%. Could you provide comparisons where competing methods are trained with the same 10% or 30% MOS budget?
  4. The gain from pair-wise supervision in Table 2 is relatively small compared with point-wise supervision. Considering that pair-wise supervision requires additional teacher inference, what is the practical accuracy-cost trade-off of this component?
  5. The current method performs calibration separately for each dataset. What happens if the student calibrated on one dataset is directly evaluated on another dataset? A cross-dataset experiment would help show whether Stage 1 learns transferable quality perception.
  6. The results with 10% and 30% MOS are averaged over five random MOS subsets. Could you also report the standard deviation or confidence interval? Some of the differences between methods are fairly small.
  7. How does the performance of the final student compare directly with the MLLM teacher on all four datasets? This comparison would help clarify how much quality information is lost during distillation and how much is recovered by MOS calibration.
  8. In the label-free setting, the student does not use human MOS but still receives dense teacher-generated quality labels. Would “MOS-free” or “human-label-free” be a more precise description than “label-free”?
Reviewer_CgS5OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes LEAF, a two-stage framework for label-efficient image quality assessment. It first distills pointwise quality scores and confidence-weighted pairwise preferences from a frozen InternVL-3.5-8B teacher into a compact visual student. It then fine-tunes the student using a small amount of mean opinion score (MOS) supervision. The central motivation is to separate transferable quality perception from dataset-specific score calibration. Experiments cover four datasets spanning generated and user-generated images, with additional ablations on supervision components, teachers, and student architectures.

Critical strengths
  1. The experiments demonstrate a meaningful benefit from teacher supervision. In Table 2, under the reported 10% MOS setting on AGIQA-3K, the full method improves SRCC/PLCC from 0.759/0.812 for the calibration-only student to 0.841/0.899. This same-student comparison provides useful evidence that the distilled supervision contributes beyond simply fitting a pretrained backbone to the available MOS labels.
  2. The framework offers a practical route to compact deployment. The large teacher is needed offline, while inference uses an 88.6M-parameter student. The reported student latency of 11.10 ms per image on an RTX 4090, together with evaluations across four datasets and multiple architectures, supports the practical usefulness of the training recipe.
Critical weaknesses
  1. The teacher’s supervision history leaves the interpretation of the label budget and evaluation independence unresolved. The official InternVL2.5 report explicitly lists KonIQ-10K in both its pretraining and fine-tuning data mixtures (Tables 4–5), while the InternVL3 and InternVL3.5 reports describe reusing their predecessors’ training corpora. LEAF uses InternVL3.5-8B and evaluates on KonIQ-10K. This lineage does not establish that the specific teacher used KonIQ MOS annotations or encountered LEAF’s test samples. However, it raises a concrete concern requiring clarification. Prior access to benchmark training labels would affect the interpretation of the effective supervision budget; access to test quality labels would threaten evaluation independence. The distinction between direct student MOS supervision and supervision inherited through the teacher is therefore consequential to the paper’s label-efficiency claims.
  2. The experiments do not sufficiently isolate the claimed perception–calibration separation or establish its incremental practical value. Figure 2 trains a new linear head on the teacher’s hidden features. This demonstrates that useful quality information can be extracted with MOS supervision, but does not establish that score-scale mismatch is the dominant limitation of the original teacher. The trainable parameters during second-stage student fine-tuning are also insufficiently specified, leaving unclear how much improvement comes from score calibration, learning a better quality readout, or adapting the representation. Furthermore, although the calibration-only student is a useful control, comparisons with existing representation-based IQA methods under identical MOS budgets are needed to establish the advantage of this recipe. The current evidence supports effective distillation more clearly than a general principle explaining when quality perception is sufficient and only calibration is needed.
Questions for authors
  1. Teacher provenance and evaluation independence: For the exact InternVL3.5-8B checkpoint used, what is known about exposure to KonIQ-10K images, MOS, or quality-derived annotations? Please distinguish image-only exposure from quality supervision, describe checks for overlap with the evaluation split, and identify any remaining uncertainty.
  2. Meaning of calibration: Which parameters are updated during the second stage? At the same MOS budget, how do scalar calibration of the existing score, a new readout on frozen features, and representation fine-tuning compare? These controls would materially clarify the claimed separation.
  3. Label-efficiency comparison: How does LEAF compare with methods such as ARNIQA, CONTRIQUE, or Re-IQA when fitted using the same labeled images and evaluation splits? Please also clarify the exact MOS counts underlying the reported percentages, including labels used for checkpoint selection or hyperparameter tuning.
Reviewer_ajMZOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a label-efficient image quality assessment framework that decouples perceptual knowledge transfer from MOS calibration. The method distills point-wise quality judgments and confidence-weighted pair-wise preferences from an MLLM teacher into a lightweight ConvNeXt student, followed by fine-tuning with a small subset of human MOS annotations. Experiments on four UGC and AIGC benchmarks demonstrate competitive prediction accuracy with reduced annotation requirements.

Critical strengths

The paper addresses the practical challenge of reducing human annotation costs in IQA. The two-stage framework provides a reasonable way to leverage MLLM perceptual knowledge without requiring expensive model deployment.

Critical weaknesses

The central perception–calibration decoupling claim is not sufficiently validated. Figure 2 calibrates a prediction head using MLLM features, while Stage 2 fine-tunes the student model rather than explicitly restricting optimization to score calibration. Consequently, the improvements cannot be clearly attributed to MOS scale calibration rather than additional perceptual representation learning.

Questions for authors

1 Please compare full student fine-tuning with a frozen student backbone and a trainable calibration head to clarify the role of Stage 2.

2 Table 2 shows that adding pair-wise supervision to point-wise distillation improves SRCC only from 0.834 to 0.841, so the practical significance of this component is not clear.

3 The entropy-based confidence in Eq. (5) reflects prediction certainty, but its relationship with actual pair-wise judgment correctness should be quantitatively verified.

4 Please provide comparisons with representative weakly supervised IQA methods trained using the same 10% and 30% MOS budgets.

5 Although Appendix E reports teacher inference latency, the total computational cost of generating point-wise and pair-wise supervision should also be reported.

Reviewer_nhSUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes LEAF, a label-efficient image quality assessment framework that separates teacher-guided quality learning from MOS calibration. In Stage 1, a frozen MLLM teacher provides point-wise quality scores and confidence-weighted pair-wise preferences to train a lightweight student without MOS labels. In Stage 2, the student is fine-tuned using a small MOS-labeled subset. Experiments on two AIGC and two UGC datasets evaluate label-free, 10% MOS, and 30% MOS settings, together with supervision, teacher, student-backbone, pair-budget, and efficiency studies.

Critical strengths
  1. The framework is simple and practically useful. It transfers MLLM quality judgments into an 88.6M student that is much cheaper to deploy, while requiring only a small amount of MOS supervision for final adaptation.

  2. The evaluation is reasonably broad. The paper covers both AIGC and UGC datasets and includes useful component, pair-budget, teacher, student-backbone, and deployment-efficiency analyses. On AGIQA-3K, point-wise teacher supervision clearly improves over calibration-only training, and pair-wise supervision provides an additional gain.

Critical weaknesses
  1. The central perception–calibration claim is not cleanly isolated. In Figure 2 and Appendix B, “calibration” trains a new linear predictor on the MLLM’s hidden representation rather than calibrating its fixed scalar quality score. Likewise, Stage 2 appears to fine-tune the full student model, so it can relearn perceptual features rather than only align the score scale. A comparison among affine/isotonic calibration of fixed scores, head-only calibration with a frozen backbone, and full-model fine-tuning is needed to support the claimed decoupling.

  2. The main label-efficiency evidence has reporting and comparison issues. Figure 4 plots 10% MOS at approximately 0.815/0.879 SRCC/PLCC and 30% at 0.864/0.910, whereas Table 1 and Section 4.4 report 0.841/0.899 and 0.868/0.914; in the figure, these values appear at 20% and 40%. In addition, the 10%/30% results are not compared against representative baselines retrained with the same label budgets and splits, and the five-run results are reported without standard deviations. These issues make the label-efficiency advantage difficult to assess reliably.

Questions for authors
  1. Which MOS-ratio results are correct—Figure 4 or Table 1/Section 4.4? Please provide the exact labeled-image counts, split definition, and mean ± standard deviation for each dataset and ratio.

  2. What performance is obtained when Stage-1 student scores are kept fixed and only an affine or isotonic scalar calibrator is fitted? How does this compare with a frozen-backbone linear head and the current full-model Stage-2 fine-tuning?

  3. How do a calibration-only student and representative supervised IQA baselines perform when retrained on exactly the same 10% and 30% MOS subsets and evaluated on the same splits?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.