- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes RLA-Net, a CLIP-based framework for decomposable image aesthetic assessment. It models aesthetic scores through eight language-defined rubrics and introduces Contrastive Rubric Alignment (CRA) to learn rubric-specific directions from synthetic perturbations, together with Gated Aesthetic Scoring for adaptive rubric aggregation. Experiments on AVA, PARA, TAD66K, and FLICKR-AES show competitive prediction performance, and ablations demonstrate the effectiveness of CRA and routing.
Critical strengths
(1) The paper goes beyond single-score prediction by explicitly modeling rubric states and their contributions, providing a more inspectable aesthetic assessment framework. (2) CRA provides additional rubric-level directional signals without requiring per-image attribute annotations, and improves prediction performance.
Critical weaknesses
(1) Although interpretability is a central motivation, the paper does not verify whether learned rubric states and routing weights agree with human aesthetic judgments. The results show predictive utility but not necessarily human-aligned explanations. (2) Some perturbations affect multiple rubrics simultaneously (e.g., blur affects sharpness and texture). The paper does not sufficiently demonstrate that individual rubric scores capture distinct aesthetic factors. (3) The paper does not compare with MLLM-based aesthetic assessment methods, although they are relevant to the interpretability motivation. Please clarify why these methods are excluded and how RLA-Net differs from them beyond computational efficiency.
Questions for authors
Q1. Have the authors evaluated whether predicted rubric states or routing weights correlate with human judgments of aesthetic factors? Q2. How does the model prevent correlated perturbations from causing entangled rubric representations (e.g., sharpness vs. texture)? Q3. Have the authors considered more fine-grained rubric anchors beyond three levels?
4 comments
Response to Reviewer yjRA - Part II (W3, Q3)
W3. Comparison with MLLM-based aesthetic assessment.
Thank you for suggesting this comparison. As RLA-Net is a CLIP-based model, our original evaluation prioritized closely related methods. MLLM-based methods differ substantially in model scale and computational requirements, and their supervision also varies: ArtiMuse includes aesthetic-caption pretraining [2], whereas Q-Align derives textual rating levels from existing scores [1]. We appreciate the value of a broader comparison and have therefore added both methods:
| Method | Total params (B) | AVA SRCC | AVA PLCC | PARA SRCC | PARA PLCC |
|---|---|---|---|---|---|
| Q-Align [1] | ≈8.2 | 0.822 | 0.817 | 0.923 | 0.940 |
| ArtiMuse [2] | ≈8.0 | 0.827 | 0.826 | 0.936 | 0.958 |
| RLA-Net (ours) | 0.434 | 0.813 | 0.812 | 0.927 | 0.951 |
With 0.434B parameters, approximately 5% of either MLLM's parameter count, RLA-Net achieves competitive correlations: it exceeds Q-Align on PARA and remains within 0.014 of ArtiMuse on both metrics across the two benchmarks. These results are obtained without additional aesthetic-caption or critique supervision. Beyond model size, RLA-Net provides an explicit rubric-conditioned scoring process, jointly learning readable attribute states and their image-dependent use in prediction.
References
[1] Wu et al. Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levels. ICML, 2024.
[2] Cao et al. ArtiMuse: Fine-Grained Image Aesthetics Assessment with Joint Scoring and Expert-Level Understanding. CVPR, 2026.
Q3. Finer-grained rubric anchors beyond three levels.
Thank you for this suggestion. We have extended the anchor-granularity ablation to two, three, and five levels on AVA and PARA. These anchors define semantic reference states rather than discrete output categories: soft assignments to the anchors yield continuous rubric scores.
| No. Levels | AVA SRCC | AVA PLCC | PARA SRCC | PARA PLCC |
|---|---|---|---|---|
| 2 | 0.794 | 0.793 | 0.902 | 0.924 |
| 3 (default) | 0.813 | 0.812 | 0.927 | 0.951 |
| 5 | 0.710 | 0.715 | 0.822 | 0.865 |
Three levels achieve the highest correlations on both datasets. Increasing from three to five levels reduces SRCC/PLCC by 0.103/0.097 on AVA and 0.105/0.086 on PARA, while two levels also perform below the default. Thus, finer-grained anchors do not improve prediction in the tested setting. These results support retaining three levels as a compact semantic reference system that still permits continuous rubric estimates. The additional results and anchor definitions are provided in Section E.11, Table E.12 of the updated supplementary PDF.
Closing: Thank you again for your time, effort, and constructive feedback; we welcome any follow-up questions and will respond promptly.
I appreciate the authors' detailed rebuttal and the substantial additional experiments. The new human-annotated evaluations, MLLM comparisons, and extended ablations address several of my original concerns.
However, I share Reviewer zD3u's concern that the interpretability claim remains insufficiently validated. While the rubric grades show improved alignment with human judgments, the faithfulness of their scoring contributions remains unclear. Moreover, the perturbation analysis on only two images provides limited evidence of rubric specificity.
I am inclined to maintain my initial rating.
Response to Reviewer yjRA
Thank you for the follow-up and for recognizing the human-annotated evaluations, MLLM comparisons, and extended ablations. The human validation in Section E.5 establishes that the rubric grades agree with human judgments.
Regarding the faithfulness of the rubric scoring contributions, we would kindly note that this question was also discussed in our response to Reviewer zD3u, including exact score reconstruction, grade-only readouts, and deletion and edit tests (Sections E.6–E.7). We truly appreciate you raising this point again, as it encouraged us to examine it more closely. We have therefore conducted additional experiments that further strengthen this evidence, and extended the specificity analysis beyond two images. New results are in Sections E.9, E.12, and E.13 of the updated supplementary PDF.
(1) Faithfulness of scoring contributions.
Because the prediction is an exact sum of the rubric terms, we measure the role of the grades directly by intervening on the inputs of the frozen scoring head.
Pathway exchange (Section E.13, Table E.14). For each original–edited image pair, we cross the eight grades and the high-dimensional features of the two images; the signed effects of the two pathways sum to the full logit change.
| Edit | Full change | Grade pathway | Feature pathway |
|---|---|---|---|
| Global blur | 0.550 | 0.240 | 0.330 |
| Desaturation | 0.077 | 0.042 | 0.031 |
| Luminance flattening | 0.280 | 0.145 | 0.145 |
| Brightening | 0.261 | 0.130 | 0.142 |
| Surface smoothing | 0.033 | 0.018 | 0.017 |
| Edge softening | 0.067 | 0.057 | 0.059 |
For all six edits, the grade pathway carries 44–85% of the full change, comparable to or larger than the high-dimensional features.
Single-grade replacement (Table E.15). Replacing one named grade at a time, we compare the logit change caused by the targeted rubric with the mean change caused by the other seven.
| Edit / target rubric | Target | Other seven | Paired difference |
|---|---|---|---|
| Desaturation / color | 0.0312 | 0.0043 | +0.0269 |
| Luminance flattening / contrast | 0.0957 | 0.0129 | +0.0828 |
| Brightening / exposure | 0.0866 | 0.0131 | +0.0735 |
| Surface smoothing / texture | 0.0132 | 0.0025 | +0.0107 |
| Edge softening / sharpness | 0.0049 | 0.0045 | +0.0004 |
For all five edits, the targeted grade changes the score more than the other seven grades on average. For color, contrast, exposure, and texture, the target effect is 5–7 times the seven-rubric mean. Each named grade thus acts on the score in correspondence with its own attribute.
Removing the global residual (Table E.16). With the weights of the full model fixed, we remove only the global residual from the shared context and evaluate on all 19,929 AVA test images:
| Setting | SRCC | PLCC | MAE |
|---|---|---|---|
| Full model | 0.813 | 0.812 | 0.0381 |
| Global residual removed | 0.752 | 0.762 | 0.0562 |
The model retains 92% of its ranking performance, so the prediction does not rest primarily on this bypass. Removing the global residual lowers SRCC to 0.752, whereas removing the grade residual lowers it to 0.648 (Table E.7), so the prediction depends more on the named grades than on this bypass.
(2) Rubric specificity beyond two images.
Ten additional images and three rubric pairs (Figures E.3–E.5). Across sharpness/texture, color/contrast, and exposure/contrast, every joint edit lowers both grades on all twelve images, while every targeted edit lowers its own grade by 0.012–0.080 and changes the paired grade by at most 0.005.
The full AVA test set (Section E.12, Table E.13). On all 19,929 test images:
| Pair (A / B) | Targeted edit | Mean Δ targeted | Mean Δ paired |
|---|---|---|---|
| Sharpness / Texture | Edge softening → sharpness | −0.0236 | +0.0045 |
| Surface smoothing → texture | −0.0477 | +0.0026 | |
| Color / Contrast | Desaturation → color | −0.0767 | +0.0028 |
| Luminance flattening → contrast | −0.0363 | +0.0039 | |
| Exposure / Contrast | Brightening → exposure | −0.0740 | +0.0028 |
| Luminance flattening → contrast | −0.0428 | +0.0038 |
Each targeted edit changes its own grade 5–27 times more than the paired grade. The pattern holds image by image: for the middle half of the test set, every targeted grade decreases by at least 0.0214. Joint edits, which degrade both attributes, lower both grades (Table E.13).
RLA-Net's interpretability is now supported at three levels: the grades agree with human attribute annotations (Section E.5), they carry a substantial and rubric-specific share of the scoring computation (Section E.13), and they respond selectively to their own attributes across three rubric pairs and the full AVA test set (Sections E.9 and E.12). We hope these results resolve the remaining concerns, and we would be grateful if you would take them into account in your final assessment.
Response to Reviewer yjRA - Part I (W1,W2,Q1,Q2)
Response Summary: We are grateful for your careful reading and constructive feedback. We appreciate your recognition of our inspectable rubric-based scoring and the predictive benefits of CRA’s directional guidance without per-image attribute annotations. Your questions have helped us strengthen the evaluation of human alignment and rubric specificity, and clarify our positioning relative to MLLM-based methods. Below, we respond to each weakness and question point by point, supported by additional analyses and experiments. All new results and experimental details are included in the revised PDFs.
W1 and Q1: Alignment with human aesthetic judgments.
We agree that overall-score accuracy alone does not establish human-aligned rubric states. We therefore added two complementary evaluations: public benchmarks for comparison against independently collected human annotations, and an expert-annotated dataset covering all eight rubrics. All methods follow the same zero-shot attribute-evaluation protocol, without additional training or fine-tuning on these evaluation annotations.
1. Public human-annotated benchmarks. We evaluate 393 EVA images and 3,000 PARA images. Their annotation schemes do not match our full eight-rubric system, so we compare to the available, semantically overlapping criteria related to lighting, color, and composition. All methods are evaluated on the same images and reference annotations.
2. Expert annotations covering all eight rubrics. To complement the partial attribute coverage of public benchmarks, we collected annotations for 100 images from 20 aesthetic experts, using criteria semantically aligned with our eight rubrics. We compare all methods under the same zero-shot evaluation protocol, using these annotations exclusively for evaluation.
RLA-Net achieves the lowest MAE and macro RMSE in all three evaluations. The public benchmarks provide evidence of human alignment for overlapping attributes, while the expert study extends this assessment to the complete eight-rubric system. These results directly evaluate rubric-state agreement with human judgments, complementing overall-score accuracy and grade-distribution analyses. More detailed information is provided in Section E.5, Table E.5, and Table E.6 of the updated supplementary PDF.
W2 and Q2: Rubric specificity under correlated perturbations.
Thank you for this question. RLA-Net defines each rubric through distinct language anchors, such as "sharp focus" and "rich texture," with each grade computed over its own rubric levels. CRA constrains the targeted rubric's global image-anchor response; it neither assigns a common degradation target to all rubrics nor requires the others to remain unchanged. Shared encoders and anchors connect this guidance to local grounding during joint training. These mechanisms promote rubric-specific responses while allowing related attributes to change together; they do not impose statistical independence.
We test this on two AVA samples (492626 and 795095), comparing each original with global blur, affecting edge clarity and surface detail; edge softening, primarily affecting sharpness; and surface smoothing, primarily affecting texture. The table reports rubric scores and changes from the originals.
Results. Global blur lowers both scores in both examples. Edge softening lowers sharpness by 0.030/0.012, while texture changes by at most 0.004; surface smoothing lowers texture by 0.055/0.025, while sharpness changes by at most 0.003. Thus, the rubrics exhibit both coordinated responses to shared degradation and differentiated responses to attribute-focused edits. These examples support distinguishable rubric responses consistent with the intended visual changes, rather than an indiscriminate shared degradation score. Visual comparisons are provided in Section E.9, Fig. E.2 of the updated supplementary PDF.