- Rating
- 3(+1)
- Confidence
- 2(−2)
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper proposes QAxis for image quality assessment,a simple and interesting IQA pipeline.
Critical strengths
-
QAxis provides a simple and interesting IQA inference pipeline without online language generation.
-
The component ablations compare description consensus, rotation, and ranking individually and in combination. Within the reported setup, their combination performs better than either rotation or ranking alone.
-
Under the KonIQ-to-target protocol, QAxis improves the average PLCC/SRCC over RALI and performs better on several evaluated datasets.
Critical weaknesses
My main concern is the gap between the central interpretation and the evidence supporting it. The reported improvements suggest a useful training approach, but the current experiments leave important alternative explanations unresolved. If more detailed experiments and relevant descriptions are provided in the rebuttal stage, I will raise the score to a positive score.
-
I am not yet convinced that the chosen text direction is special. The axis is found using MOS labels, and the ranking loss then trains image features to follow those labels along the axis. Since the visual encoder and projection layer are trainable, even a fixed random direction might become a useful scoring coordinate. Figure 7 shows that the trained coordinate predicts MOS well, but does not rule out this simpler explanation. Without random-axis, shuffled-label, and ordinary scoring-head controls, I find it hard to attribute the gains to a pre-existing quality structure in the reasoning embeddings.
-
The claim that the axis is shared across datasets seems stronger than the evidence. Every reported axis is learned from KonIQ. The target-dataset scores show that a KonIQ-derived scorer can transfer, but do not show that comparable axes can be recovered from other domains or with less labeled data. Figure 4(b) provides an interesting gradient analysis on KADID, though its fixed distortion setting does not necessarily fix perceived quality across different image contents. I would describe the current result as transfer of one source-trained direction, with the broader shared-axis claim still open.
-
The experiments do not cleanly separate QAxis from the supervision it inherits. The method learns from Q-Insight descriptions, while RALI already uses reasoning descriptions to train a visual scorer. I would find the improvement over RALI more convincing with the same teacher checkpoint, descriptions, backbone, and training budget. The comparison with Q-Insight is useful for scoring accuracy, but QAxis does not provide Q-Insight’s explanations, and the cost of generating its training descriptions is not reported alongside the inference cost.
-
Lack of some more intuitive qualitative analysis, such as the CLIP's image-text similarity heatmap brought by the quality axis, whether it focuses more on the areas that better reflect "quality" or on the image foreground.
Questions for authors
- A comparison with several fixed random axes, a shuffled-label SIR axis, and a learnable scoring head under matched supervision and training conditions would help establish the specific contribution of the proposed direction.
- It is suggested to incorporate the axis results estimated from the synthetic distortion dataset and a smaller training subset. This will help clarify its dependence on KonIQ and the amount of labeled data, as well as whether the axis is domain-specific to the training domain or is common to any quality evaluation domain.
- Attention or attribution visualizations for ordinary alignment, QAxis, and a random-axis variant on the same images would make the proposed interpretation easier to assess.
4 comments
Dear Reviewer,
I hope this message finds you well. As the discussion period is nearing its end less than two days remaining, I want to ensure we have addressed all your concerns satisfactorily. If there are any additional points or feedback you'd like us to consider, please let us know. Your insights are invaluable to us, and we're eager to address any remaining issues to improve our work.
Thank you for your time and effort in reviewing our paper.
Thank you for the additional experiments and clarifications. The axis controls, matched RALI comparison, and local-distortion visualizations meaningfully address several of my concerns. I will increase my score to reflect these improvements, while lowering my confidence because I remain uncertain about the interpretation of the central finding.
In particular, “shared” has been removed from the title, but the introduction still emphasizes a quality axis shared across reasoning and visual representations. My understanding is that the revised claim concerns cross-modal sharing after alignment, together with cross-dataset transferability, rather than a common axis independently recovered across datasets. Making this distinction explicit and consistent throughout the paper would help. The new results strengthen the case for the estimated direction as a useful supervised reference, although they only partially establish the stronger interpretation of a shared quality geometry. Clearer reporting of whether the axis and linear-head controls use the same alignment components would also help isolate the contribution of the proposed axis. Overall, I appreciate the substantial effort and find the revised empirical evidence more convincing.
Thank you for your careful review and recognition of the additional experiments. We have revised the manuscript to distinguish cross-modal sharing after alignment from cross-dataset transferability, and clarified the control settings in Section 4.3 and Table 3.
Thank you again for helping us improve the clarity of our claims.
Comment 1: Random-axis, shuffled-MOS, and learned-head controls (Sec. 4.3, “Effect of the axis choice” ).
Response 1: Thank you for this important observation. We have added multiple random and shuffled-MOS axes , and a learnable linear head trained with InfoNCE and the same ranking loss.
Random and shuffled-MOS axes are consistently less effective than our quality axis. Despite receiving ranking supervision, the learnable linear head also performs substantially worse than QAxis-v1. Ranking constrains relative scores but does not itself identify a quality-related direction in the representation. Our quality axis fixes a semantics-grounded coordinate, allowing ranking supervision to reinforce quality information from reasoning embeddings rather than learn a scoring direction from MOS ordering alone.
Comment 2: Cross-dataset transferability and reduced axis-estimation data.
Response 2: Thank you for this insightful comment. We find evidence supporting cross-dataset transferability. We have revised the manuscript accordingly.
(1) Independent cross-dataset estimation (Section 2.2, Q3).
We independently estimate quality axes from seven datasets. Their directions differ, although their projections remain strongly correlated. For KonIQ–KADID, the first five principal components account for 91.7% of the projection covariance; removing them reduces the projection correlation from 0.859 to 0.479. Projection agreement therefore depends on feature covariance and does not establish identical axis vectors. Accordingly, we revise the corresponding claims to emphasize that QAxis is transferable across datasets.
(2) Reduced axis-estimation data (Section 4.3, “Effect of axis-estimation sample size” ).
We evaluate axes estimated from 20% to 100% of the KonIQ training samples. Using only 20% retains strong cross-dataset performance, with average PLCC/SRCC of 0.805/0.786, supporting robustness to reduced data for axis estimation.
Comment 3: Inherited supervision and computational cost.
Response 3: Thank you for this suggestion. We address the two comparisons separately.
(1) QAxis versus RALI: controlling inherited supervision. (Appendix E.1)
We reproduced RALI using the same settings as our method.
QAxis extracts a transferable quality axis from these descriptions to guide quality-conditioned alignment and ordinal learning, providing an explicit quality reference that enables more effective transfer of quality information from the same supervision.
(2) QAxis versus Q-Insight: computational cost (Section 4.5, “Efficiency Study” ).
QAxis transfers quality information from reasoning descriptions to a lightweight IQA model, providing an explicit quality direction for visual learning and enabling quality prediction without online text generation. We have added a cost analysis separating offline description generation, QAxis training, and inference.
Reason generation is an offline cost, and the reasons can be reused across training runs. QAxis requires only 2% of Q-Insight's inference time, while reason generation is a one-time offline cost.
Comment 4: Spatial sensitivity to quality degradation (Sec. 4.4).
Response 4: Thank you for the suggestion. We have added local-distortion visualizations. We have applied five distortions within predefined local masks.
Figure 10 shows that Grad-CAM for the negative quality score produces stronger QAxis responses in degraded regions. We further quantify the fraction of total CAM response within the distortion mask.
Before distortion, both methods show similar ROI response shares; after distortion, QAxis achieves a higher average share, indicating greater spatial sensitivity to quality degradation beyond foreground saliency alone.
We sincerely thank the reviewer again for the careful review. We would be happy to provide further clarification if needed.