- Rating
- 2
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The authors investigate whether a single linear direction, a rank-axis, in the frozen latent feature space of an IQA model can order images by perceptual quality. They introduce the Rank-Axis Isometry Score (RAIS) to measure whether distances along the axis reflect quality differences, and three measures, Anchor-QEA, ordering rate, Global-QEA, to assess whether the embedding space itself is organized by quality. Their empirical studies find that a single rank-axis often beats the model's fine-tuned head especially at earlier layers, and the embedding space is only weakly ordered by quality. They also show that rank-axes transfer across datasets are well-predicted by covariance-weighted agreement, and local geometry correlates with cross-dataset agreement. Their conclusion is that a single direction is sufficient to encode perceptual quality, but the underlying embedding geometry is not.
Critical strengths
- Introduce a novel perspective of IQA through the lens of rankability and embedding geometry in latent feature representations.
- Propose several novel diagnostic tools, RAIS, Anchor-QEA, ordering rate, Global-QEA, to measure whether distances along the axis reflect quality differences and whether the embedding space itself is organized by quality.
- Have some interesting empirical findings: a simple ridge direction outperforms fine-tuned heads in many cases under the current test conditions; early layers can match last layers; covariance-weighted agreement well explains transfer and outperforms raw angle.
Critical weaknesses
- Despite the paper’s appreciated contributions and interesting observations, its central claim, that a single linear direction is sufficient for quality assessment, remains questionable. It appears surprisingly strong and may be an artifact of the limitations of current IQA benchmarks, which predominantly contain a narrow set of synthetic distortions (e.g., blur, noise, compression) that could induce a simple, one-dimensional quality manifold. By contrast, more sophisticated perceptual experiments often reveal human quality perception to be multidimensional, and subjective judgments are frequently influenced by experimental setup or instructions. Consequently, the conclusion that quality is “linearly separable” may not generalize to more diverse or authentic distortions, or to a broader range of content types.
- The current study focuses on IQA-specific models. It would be interesting to see whether general-purpose vision encoders also exhibit rankability for quality, or whether this is specific to models trained for IQA.
- The results of the paper seem to suggest that the fine-tuned heads are nearly useless, other than overfitting the data. But if the heads were trained on their authors' splits, then the comparison is not completely fair.
Questions for authors
- Your title, “Is One Direction Enough?”, is bold and provocative, but concluding that one direction is sufficient may be too strong, as it could be an artifact of the limitations of current IQA benchmarks. That said, the paper’s real contributions (analyzing latent space structure, introducing novel diagnostic tools, and examining cross-dataset transfer) are significant. Would you consider a major rewrite that frames the findings more cautiously, adjusts this central claim, and discusses the conditions under which a single direction may or may not suffice?
- How does the rankability of IQA-specific models compare to that of generic vision encoders? Would a model not trained for IQA (e.g., DINOv2) also exhibit a strong rank-axis?
- The paper reinforces the idea that deep latent feature spaces hold the key to quality assessment. Similar insights motivated prior IQA work such as LPIPS and deepSSIM, but those methods rely on distance-based or structural similarity comparisons in feature space. In contrast, this paper advocates for a single linear direction. These two views appear to conflict. It would be valuable to add analysis and discussions.
4 comments
W2 / Q2 - Generic encoders.
We added two generic encoders: CLIP ViT-B/16 and DINOv2 ViT-B/14. We extracted their features at the same three layer depths and evaluated them with the same protocol on all eight datasets. The table compares them with the mean of the 9 IQA encoders. All values are means over the eight datasets, weighted by the number of test images.
| R_ρ ℓ_early | R_ρ ℓ_mid | R_ρ ℓ_last | RAIS ℓ_last | Anchor-QEA ℓ_last | |
|---|---|---|---|---|---|
| Mean (9 IQA encoders) | 0.814 | 0.825 | 0.841 | 0.666 | 0.453 |
| CLIP ViT-B/16 | 0.810 | 0.766 | 0.779 | 0.535 | 0.092 |
| DINOv2 ViT-B/14 | 0.807 | 0.740 | 0.626 | 0.381 | 0.050 |
Analysis. At ℓ_early, the generic encoders are closely as rankable as the IQA encoders. CLIP reaches R_ρ = 0.810, and DINOv2 reaches 0.807, against 0.814 for the IQA mean.
The two groups separate with depth. The rankability of the IQA encoders increases from 0.814 at ℓ_early to 0.841 at ℓ_last. The rankability of CLIP decreases to 0.779, and that of DINOv2 decreases to 0.626.
The native embedding space of the generic encoders is not ordered by quality. Their Anchor-QEA stays at or below 0.13 at every layer, while the IQA mean reaches 0.453 at ℓ_last. The revised paper reports these results in §5.1 and Appendix A.4 (Tables 6–7).
W3 - Fairness of the head comparison.
Please refer to our answer to Reviewer DKch (Q3) and to Table 11 of the revised paper for the full comparison.
The rank-axis stays close to the fine-tuned heads and beats them in 52 of the 70 encoder-dataset pairs. Against heads trained on the target dataset, it is better in 13 of 26 pairs, and the head's advantage in the other 13 pairs is small (mean margin −0.043).
Q3 - LPIPS / DeepSSIM vs a rank-axis
LPIPS and DeepSSIM are full-reference approaches, but they address different objectives. While LPIPS[1] measures perceptual similarity, DeepSSIM[2] focuses on structural similarity. Our work instead investigates how perceptual quality information is organized in pretrained visual embeddings. Therefore, we replaced DeepSSIM with DISTS[3] and DeepDC[4] in our quantitative comparison below. To establish an empirical connection with perceptual similarity measures, we compare our rank-axis applied to reference-distorted feature displacements and established FR metrics on the same image pairs.
| SRCC | LIVE | TID2013 | KADID-10k |
|---|---|---|---|
| Rank-axis | 0.960 | 0.841 | 0.897 |
| LPIPS (AlexNet) | 0.914 | 0.731 | 0.823 |
| DISTS | 0.955 | 0.721 | 0.819 |
| DeepDC | 0.945 | 0.706 | 0.834 |
This table is added to the revised paper as Appendix A.11 (Table 29).
[1] Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In IEEE Conference on computer vision and pattern recognition, pp. 586–595. IEEE, 2018.
[2] Keke Zhang, Weiling Chen, Tiesong Zhao, and Zhou Wang. Structural similarity in deep features: Unified image quality assessment robust to geometrically disparate reference. IEEE Transactions on Pattern Analysis and Machine Intelligence, 48(3):2581–2595, 2026.
[3] Keyan Ding, Kede Ma, Shiqi Wang, and Eero P. Simoncelli. Image quality assessment: Unifying structure and texture similarity. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(5):2567–2581, 2022.
[4] Hanwei Zhu, Baoliang Chen, Lingyu Zhu, Shiqi Wang, and Weisi Lin. DeepDC: Deep distance correlation as a perceptual image quality evaluator. IEEE Transactions on Image Processing, 34: 7859–7873, 2025.
W1 / Q1 - Is one direction enough, or an artifact of the benchmarks?
Thank you for this question. We intended to frame the title as a question; however, we updated the paper for clarity.
First, the result is not limited to synthetic distortions. Besides LIVE, TID2013 and KADID-10k, our eight datasets include restoration and GAN outputs (PIPAL), images with authentic distortions (LIVEC, KonIQ-10k, SPAQ) and AI-generated images (AGIQA-3K). The table gives the rankability of the rank-axis at ℓ_last (mean over the 9 IQA encoders) and how much a two-layer MLP on the same features adds:
On authentic and AI-generated images, one direction ranks images about as well as on synthetic ones. This concludes that the single direction performance gains are not an artifact of simple synthetic distortion.
Second, there are cases where a single direction may not fully capture the diversity of IQA latent space. Its relative performance is weakest on TID2013, which encompasses 24 distortion types, and PIPAL, whose test set includes distortion types unseen during training. These are also the datasets where the MLP provides the largest improvements. However, the rank-axis still remains competitive with the MLP, achieving an SRCC within 0.06 of the MLP, demonstrating that the one-dimensional projection captures most of what a nonlinear model can extract from the same features, even under substantial distortion diversity and distribution shift.
Third, we acknowledge that human quality perception is inherently multidimensional and may vary depending on the subjective evaluation protocol. The eight standard IQA benchmarks considered in this work provide a single MOS for each image, thereby allowing us to examine whether this scalar quality score can be captured along a single feature-space direction. Our conclusions are therefore scoped to this evaluation setting. Within this scope, RAIS and Anchor-QEA further extend the analysis beyond ranking consistency by examining spacing and local-ordering in native embedding space. Quality-specific textual descriptions and judgements collected under different subjective study protocols are left for future work as mentioned in conclusion.