- Rating
- 4(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
In this paper, the authors addressed cross-domain blind image quality assessment, which is a valuable topic in the community in recent years. The underlying idea is to leverage the latent quality-aware knowledge (visual anchors) in frozen CLIP backbone and perform quality prediction by projecting image feature to the anchors based on cosine similarity. The idea of training cross-domain BIQA model without using target data and no training is interesting. The authors have done extensive experiments and comparisons to demonstrate the advantage of the proposed model.
Critical strengths
- The proposed method is simple and effective. The quality prior is a one-time closed-form feature operation. An obvious advantage is that it does not need extra training and target-domain data. The directional update accounts for only about 2.5% of total inference latency. The MOS-free KADIS variant consistently exceeds the frozen CLIP baseline on authentic and AIGC datasets.
- The paper provides diagnostics beyond final correlation coefficients. Independently constructed quality directions have a mean cross-dataset cosine similarity of 0.60, compared with about 0.03 for random directions. The image/text branch ablations, random-direction controls, and semantic-task evaluations suggest that the amplification primarily increases quality sensitivity without severely destroying CLIP's semantic structure.
- Extensive experiments and analysis were conducetd.
Critical weaknesses
- The evidence for shared cross-domain structure is not sufficient to support the generality claims. The main geometry analysis uses MOS-derived directions from seven datasets and performs an uncentered SVD on these directions. Although the authors stated that these directions are used only for diagnosis, the analysis still depends on target-domain annotations and does not provide cross-dataset independence tests, confidence intervals, or statistical significance tests.
- A single global direction and fixed linear amplification are the main bottlenecks of the methods. No source is uniformly optimal across targets, gains are weaker for color/contrast distortions and PIPAL, and global pooling can miss localized defects. The KADIS direction also reduces performance on aesthetic quality datasets. The authors could quantify this risk with breakdowns by distortion type, and local versus global defects.
- Some of the state-of-the-art BIQA models, particularly MLLM-based models, are missing in the comparison, such as Q-Align, DepictQA, DeQA-Score, etc. More comparisons should be included, especially models published in the last two years.
Questions for authors
- Please clarify whether the KADIS source bank and each target dataset share content, reference images, or distortion generation. In particular, please explain whether near-duplicate images exist between KADIS and datasets such as KADID or PIPAL, so that potential data leakage during source construction can be ruled out.
- It is better to clarify which baselines in Tables 1 and 2 were rerun with the authors' code and preprocessing, and which numbers were transcribed from prior work? Could the authors report, input preprocessing, and target-data-free protocol, the per-dataset results for frozen CLIP, CLIP-IQA/MA-CLIP, and the main cross-domain methods, together with the mean/standard deviation? This would determine whether the reported gains are statistically meaningful and can be fairly attributed to directional amplification.
- Could the authors provide a genuinely held-out direction evaluation: estimate the shared subspace or direction using only a subset of sources/datasets, then evaluate it on datasets not used in the direction analysis?
- More latest BIQA models, especically MLLM-based one, could be included in the comparisons.
5 comments
W2: Distortion-Specific Performance and Robustness of the Global Quality Direction
We agree that a single global quality direction cannot fully characterize every perceptual degradation. Our method deliberately employs a simple source-derived direction as an approximation to a richer, partially domain-dependent quality structure. This limitation is explicitly discussed in Appendix H, while Appendix A.6 and Appendix F.3 already provide distortion-specific sensitivity and spatial-response analyses. We additionally conducted two complementary experiments to quantify distortion-specific performance and examine whether the global direction depends excessively on particular source degradations.
(1) Distortion-specific performance. We evaluated our original, unchanged global quality direction separately on all 25 KADID-10K distortion types, each containing 405 images. The same KADIS-derived prior, fixed amplification factor, and text calibration are used across all distortion categories, without category-specific adaptation.
Our method achieves positive SRCC improvements over frozen CLIP-IQA on all 25 distortion types, increasing overall KADID SRCC from 0.4739 to 0.6737. The improvements vary across degradation categories. For example, color quantization improves from −0.2132 to 0.4343, while contrast change improves from 0.0877 to 0.1996.
These results demonstrate that directional amplification improves quality discrimination even for challenging color/contrast distortions. However, certain categories remain difficult in terms of absolute performance, consistent with the limitations acknowledged in Appendix H.
(2) Source distortion composition ablation. To investigate whether the global quality direction is dominated by particular degradation types, we constructed additional source quality priors under two settings:
- Single-type: Construct a quality prior using only one of the 25 source distortion types.
- Leave-one-type-out: Construct a quality prior using the remaining 24 types after excluding one source distortion type.
For each configuration, the source quality prior and corresponding source-anchor calibration are independently constructed from the selected KADIS degradations. The frozen backbone, quality prompts, amplification factor, and target evaluation protocol remain unchanged. No target images or MOS annotations are used for source-prior construction.
We evaluated these configurations on three authentic IQA datasets:
| Dataset | Frozen CLIP | Full 25 Types | Single-Type Variants Improving over Frozen |
|---|---|---|---|
| BID | 0.7568 | 0.7873 | 22/25 |
| KonIQ | 0.6452 | 0.7565 | 23/25 |
| LIVEC | 0.7181 | 0.7773 | 22/25 |
Notably, 20 of the 25 single-type quality priors improve performance on all three authentic datasets, suggesting that transferable quality-related information is not exclusive to any particular synthetic degradation.
More importantly, removing any single source distortion type produces a direction highly aligned with the complete direction, with cosine similarities of 0.9929–0.9999. The corresponding absolute SRCC changes remain within 0.0066 across the three authentic datasets. These results demonstrate that the source-derived quality prior is not dominated by any single degradation category and remains stable under changes to source distortion composition.
Q1: Source–Target Independence and Potential Data Leakage
We clarify that our quality prior is constructed exclusively from KADIS, without accessing target images, target reference images, or subjective quality annotations. Source anchors are used to estimate the quality direction; they are not employed as a target-image retrieval bank during inference. Furthermore, KADIS-700k and KADID-10k were explicitly constructed using disjoint sets of pristine reference images[1]. Although the two datasets share synthetic distortion-generation methods, this represents shared degradation categories rather than image-level data leakage.
To provide additional empirical verification, we conducted a reference-image overlap audit involving 100 sampled KADIS source references in 15 different seeds(total 1484) and 281 target reference images (81 from KADID and 200 from PIPAL). We examined exact file identity, pixel-level identity, and perceptual-hash similarity. Near-duplicate candidates were identified using a perceptual-hash Hamming-distance threshold of 6. The audit found zero identical-file pairs, zero pixel-identical pairs, and zero perceptual-hash near-duplicate candidates under this threshold. This audit covers the sampled source references rather than an exhaustive search over the entire KADIS collection. Nevertheless, together with the documented dataset construction protocols, it provides additional evidence supporting source–target independence.
Q2: Baseline Protocols, Reproducibility, and Statistical Reliability
We appreciate the opportunity to clarify the evaluation protocols already documented in Appendix I. As specified in Appendix I.1, Table 1 reports established cross-domain and MLLM baselines using results transcribed from published evaluations. In contrast, the broader training-free baselines in Table 2, including CLIP-IQA and MA-CLIP, are reproduced under the same testing protocol as our method, with per-dataset SRCC/PLCC results explicitly reported. For these controlled evaluations, we consistently use frozen CLIP RN50 with aspect-ratio-preserving preprocessing and no center crop. The source-derived direction, quality prompts, and amplification factor are fixed across target datasets. No target-domain information is used for parameter selection or adaptation. We therefore distinguish published cross-method comparisons in Table 1 from controlled reproduced comparisons in Table 2, rather than assuming that all published baselines share identical preprocessing or implementation settings. The training and target-access requirements of the compared methods are explicitly indicated.
Appendix I.3 (Table 27) already reports SRCC/PLCC standard deviations over 15 independently sampled KADIS reference sets, demonstrating that performance is stable with respect to source-reference selection. To further quantify the reliability of performance improvements, we conducted additional image-level paired bootstrap and paired permutation tests, comparing our complete method with frozen CLIP-IQA on identical test images. Specifically, we perform 1,000 paired bootstrap repetitions, sampling image indices with replacement and computing the SRCC difference between the two methods using the same resampled indices. The 95% percentile CI is calculated from the 2.5th and 97.5th percentiles of the resulting SRCC differences. We additionally perform 1,000 paired permutation trials by randomly exchanging the two methods' predictions within paired image units and recomputing the SRCC difference. Two-sided Monte Carlo p-values are obtained by comparing the absolute randomized differences with the observed absolute difference, using the standard plus-one correction. The results are summarized below:
| Dataset | Frozen SRCC | Ours SRCC | ΔSRCC | 95% Bootstrap CI | p-value |
|---|---|---|---|---|---|
| KonIQ | 0.6452 | 0.7565 | +0.1113 | [0.1033, 0.1191] | 0.001 |
| LIVEC | 0.7181 | 0.7773 | +0.0592 | [0.0422, 0.0764] | 0.005 |
| AGIQA-3K | 0.6517 | 0.7235 | +0.0718 | [0.0584, 0.0847] | 0.001 |
| KADID | 0.4739 | 0.6737 | +0.1998 | [0.1890, 0.2099] | 0.001 |
All four datasets exhibit positive performance differences, with image-level paired bootstrap intervals excluding zero and permutation (p)-values below 0.01. These intervals quantify image-level resampling variability. Together with the 15-source-sampling analysis already reported in Table 27, these findings provide complementary evidence for both source-construction stability and prediction-level performance reliability. Furthermore, Table 3 already separates the image-amplification and text-calibration contributions, confirming that image-side directional amplification accounts for most of the improvement.
[1] Lin, H., Hosu, V., & Saupe, D. (2019). KADID-10k: A Large-scale Artificially Distorted IQA Database.
We sincerely thank the reviewer for recognizing the simplicity, effectiveness, and target-data-free nature of our approach. We appreciate the constructive feedback and have conducted additional experiments to strengthen the geometric evidence, examine distortion-specific robustness, verify source–target independence, and assess statistical reliability. We also clarify several experimental details already documented in the submission.
W1 & Q3: Independent and Statistical Validation of the Shared Quality Structure
We appreciate the reviewer's suggestion to further validate the independence and statistical significance of the observed shared quality structure.
First, we clarify that the seven MOS-derived quality directions in Fig. 4 are constructed exclusively for diagnostic analysis and are never used by our proposed IQA method. Our deployed quality prior is derived solely from MOS-free KADIS, without accessing target images or annotations. Moreover, Appendix C.5 (Fig. 11) already evaluates independently constructed KADIS and KADID directions against the seven-dataset diagnostic subspace, without including either source direction in its estimation.
To determine whether the observed low-dimensional organization generalizes beyond in-sample SVD, we conduct a seven-dataset Leave-One-Dataset-Out (LODO) experiment. For each dataset, we construct a normalized quality direction using the highest- and lowest-MOS 20% of images. In each LODO run, we estimate a shared subspace using uncentered SVD on the other six directions and evaluate the excluded direction without including it in subspace estimation.
Statistical validation. We perform 200 image-level bootstrap repetitions, reconstructing the quality directions and shared subspace in each repetition to obtain 95% percentile confidence intervals (CIs). We additionally perform 200 within-dataset MOS-label permutations to establish a null distribution that preserves image features while disrupting their association with subjective quality.
Table R1. Leave-one-dataset-out validation of the shared quality structure.
PC1 Cosine measures alignment with the leading direction estimated from the other six datasets. PC1/Top-3 Energy denotes the squared projection energy of the held-out normalized direction onto the corresponding subspace.
The results provide three complementary findings:
Consistent out-of-sample alignment. All seven held-out directions show positive PC1 cosine similarities (0.6210-0.7470), with a mean of 0.6968 (95% CI: [0.6737, 0.7011]).
Strong low-dimensional generalization. The first three components explain 75.22% of held-out directional energy on average, substantially exceeding the MOS-permutation null mean of 5.07% (p≈0.005).
Cross-dataset geometric consistency. Independently constructed directions exhibit a mean pairwise cosine similarity of 0.5726 (95% CI: [0.5448, 0.5762]), with 90.86% of their in-sample energy explained by the first three components.
Importantly, the shared structure remains observable even when each target direction is excluded from subspace estimation, demonstrating that the low-dimensional organization is not merely an in-sample SVD effect. The consistent alignment across heterogeneous datasets, together with bootstrap and permutation evidence, further supports the statistical reliability of this observation.