- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes ACQA, a vision-language based framework for image quality assessment. The main idea is to use both semantic content information and quality-related attributes rather than relying only on score regression. ACQA contains a dual-branch VLM, an Attribute Calibration Learning module, and a Quality-Aware Analysis Learning module. The content branch generates image descriptions, while the attribute branch predicts several quality-related attributes. ACAL uses ranking, stability, and disentanglement rewards to improve attribute prediction. QAL further uses content alignment and attribute contribution information to optimize the VLM for quality prediction. Experiments are conducted on synthetic, authentic, and AI-generated image quality datasets.
Critical strengths
The paper considers both semantic content and quality-related attributes rather than directly predicting a quality score. This is a reasonable direction for VLM-based IQA, where semantic information may help explain quality variations across different image domains. The experimental results are strong across several types of IQA datasets. ACQA performs well on authentic, synthetic, and AI-generated images, and the ablation studies show that both the dual-branch design and the proposed ACAL/QAL components contribute to the final performance. The paper also provides relatively detailed component-level analysis. In particular, the effects of the different rewards and the changes in content and attribute information are examined separately.
Critical weaknesses
My main concern is the evaluation protocol. Some baseline results are directly taken from their original papers, but the paper does not clearly establish that all methods use the same data splits and evaluation settings. This makes the performance comparison difficult to interpret. The paper repeatedly claims improved generalization, but most reported results appear to be within-dataset evaluations. A cross-dataset experiment would be more convincing evidence for generalization. The framework is also quite complex. ACAL and QAL introduce several rewards and multiple optimization steps. Although the ablation study shows that most components help, it is still unclear whether the additional complexity is justified by the improvement over simpler alternatives. Finally, the five attributes are manually fixed to contrast, brightness, realism, saturation, and blur. These attributes may not be equally meaningful for all image domains, and some important quality factors are not covered.
Questions for authors
- Could you clarify the exact training/test split used for each dataset? Are the reported results averaged over multiple random splits or runs?
- Some baseline results in Table 1 are taken directly from the original papers. Are these results based on exactly the same dataset splits and preprocessing as ACQA? If not, could you provide a controlled comparison under the same protocol?
- The paper claims improved generalization, but most experiments appear to evaluate each dataset separately. Could you provide a cross-dataset experiment, for example training on one IQA dataset and testing directly on another?
- Why are contrast, brightness, realism, saturation, and blur selected as the five attributes? How sensitive is the method to this choice, especially since “realism” may have different meanings for natural and AI-generated images?
- The ranking reward assumes an exponential quality decay across synthetic distortion levels. How sensitive are the results to this assumption and to the attenuation coefficient ?
- The gain reward uses changes in SRCC and PLCC, while the IQA model is also trained with MOS labels. Could you clarify exactly which split is used to compute these rewards and how you avoid using validation or test information during RL optimization?
- ACQA contains several RL optimization stages and repeated VLM inference. What are the total training cost and inference cost compared with recent VLM-based IQA baselines?
- All main experiments use Qwen 2.5-7B as the VLM backbone. Does the improvement remain with other VLM backbones, or is the method sensitive to the choice of the base model?
Response to Reviewer 6qNS
Dear Reviewer 6qNS,
Thank you for your thoughtful review. In our response, we use 'W' to denote weaknesses and 'Q' for questions. Additionally, we have included extensive experimental results and explanations in the Supplementary Material (SM) to support our responses.
W1. Thank you for your efforts. In response to your comments, we have provided explanations in Q1–Q8 below. In addition, we have supplemented the SM with additional experiments as supporting evidence. We hope these explanations will meet with your approval.
Q1 & Q2. Firstly, we split the training, validation and test sets with a 6:2:2 ratio for each dataset. All results are averaged over ten random splits. However, the difference is that some methods (Q-Align, Compare2Score, Refine-IQA-S2) were trained on large-scale mixed datasets or self-constructed datasets to obtain their pre-trained parameters, which affects the comparison results under the same protocol. Secondly, the experimental results in S_Table 4 of the SM are illustrated to fairly compare ACQA with other methods on the BID, KonIQ-10k, and SPAQ datasets. All methods were trained with the official VLM and the same data-split protocol. ACQA performers well on individual datasets, and the average performance surpasses other methods by at least 2.5% in PLCC.
Q3. The experimental results in S_Table 8 of the SM are illustrated to analyze the performance of ACQA in cross-dataset tasks. The average performance of ACQA surpasses other methods by at least 2.5% in SRCC and 1.3% in PLCC.
Q4. Firstly, during preliminary research, we observed that image distortions across different scenarios are primarily correlated with five selected attributes, which has also been corroborated by methods such as IQA-T1 and ATTIQA, and the reference information is in SM. Secondly, the natural images are mainly degraded by processing-induced unnaturalness. And, the AI-generated images are degraded by implausible textures, illumination, and structure. In our preliminary study, we used a VLM to rate the realism attribute on the SPAQ dataset (over 10,000 natural images) and the AGIQA-20k dataset (over 20,000 images). Researchers observed and validated these ratings, confirming that the VLM’s judgments on realism were largely correct. Finally, the experimental results in S_Table 1 of the SM are illustrated to demonstrate the advantage of five selected attributes over other attributes. With the selected five attributes, ACQA achieves a performance improvement of at least 6.97% on LIVEC and 3.40% on AGIQA-3k.
Q5. We sincerely thank the reviewer for raising this question. Through preliminary experiments, we determined via sensitivity analysis that the optimal value of α is 0.3. The experimental results in S_Table 9 of the SM are illustrated to analyze the sensitivity on α across multiple distorted datasets and specific distortion types. Experimental results demonstrate that setting α to 0.3 ensures the validity of the ranking discrepancy computation with the performance improved by at least 5.44%.
Q6. The gain reward and other rewards are computed on the training and validation sets. We strictly followed a 6:2:2 training/validation/test split, and the experiments across multiple rounds, different random splits were kept independent of one another to avoid data confusion.
Q7. Firstly, S_Figure 1 of the SM has been added to analyze the performance gains and parameter increments of employing different reward mechanisms. Secondly, the experimental results in S_Table 10 of the SM are illustrated to compare the inference costs of ACQA with VLM-based IQA methods. ACQA achieve performance improvement by at least 5.21% at only a small inference cost.
Q8. The experimental results in S_Table 3 of the SM are illustrated to demonstrate the adaptability of ACQA on different VLM backbones (Gemma3-4B, mPLUG-Owl-2, Qwen2-7B, and Qwen2.5-3B). ACQA maintains stable and accurate performance across different VLM models.