- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper studies photography aesthetic assessment (PAA) and proposes AesAlign, a two-stage framework that decouples aesthetic understanding from score calibration. Stage 1 LoRA-tunes Qwen3.5-2B on 450K MLLM-generated aesthetic analyses (produced by Qwen3.7-Plus under a taxonomy of 4 general dimensions + 3 theme-specific dimensions over 10 themes; these texts also embed the generator's own scores). Stage 2 freezes the model, mean-pools layer-22 hidden states over instruction-token positions from a single forward pass, and fits independent Ridge probes per scoring target on 45K expert-calibrated ratings (5K held out for evaluation). The expert ratings are collected with a coarse-to-fine protocol (10-level banding then 1-10 within band, escalation and trimmed averaging) with reference anchors, shared criterion definitions, and blinding to MLLM outputs. Experiments re-annotate the AADB and PARA test sets under this protocol and report SRCC/PLCC against zero-shot and supervised baselines, plus ablations on backbone, score head, pooling, data scale, and stage combinations. The headline claims are +0.073 SRCC over vanilla score fine-tuning under equal expert-rating budgets, and better cross-dataset generalization.
Critical strengths
S1. The expert-calibrated annotation resource and protocol. The 50K fine-grained ratings (45K calibration + 5K evaluation) collected under a coarse-to-fine protocol with reference anchors, criterion definitions, and blinding to MLLM outputs is a genuine asset for PAA research; the protocol details (Appendix A.2) are above the community's current standard for this task. This is, in my assessment, the paper's core contribution. S2. The sample-efficiency finding. Table 3 shows the Ridge-probe calibration reaching 0.739 SRCC with 450 expert-rated images, exceeding Score SFT trained on all 45K (0.706). Quantifying this supervision asymmetry (cheap text vs. expensive calibrated ratings) is a useful empirical message, and the ablation coverage (Tables 4-5) is generally thorough.
Critical weaknesses
W1. The supervised baselines in Table 2 are under-specified, so the central comparison is not interpretable. The paper states baselines are adapted "following each method's training procedure" on AesAlign-Calibrated, but gives no details: input format, score mapping (e.g., how the 1-100 labels map to Q-Align's discrete buckets), LoRA vs. full fine-tuning, hyperparameters, or convergence criteria. Given that InternVL3-8B degrades after adaptation on AADB (0.684 -> 0.670), I cannot exclude that the supervised baselines are under-tuned, which directly undermines the claim that AesAlign outperforms specialized and general MLLM baselines. This is the paper's most important comparison and it is not reproducible from the manuscript. W2. The method contribution reduces to text-bottlenecked distillation, and the decisive baseline is missing. All 450K analyses (and the scores embedded in them) are generated by Qwen3.7-Plus; the student (Qwen3.5-2B) merely absorbs this via LoRA before a linear probe is calibrated on 45K ratings. The paper never compares against directly probing the teacher's (Qwen3.7-Plus) features with the same Ridge heads — a baseline that would plausibly match or exceed the student, since the text bottleneck can only discard information. Table 5's "no Stage 1" row (0.686) probes the un-adapted 2B, which does not address this. Consequently, what remains beyond distillation is thin: the decoupling claim is further weakened because Stage 1 texts embed the generator's scores, suggesting Stage 2 may partly learn a distribution/format correction rather than aesthetic understanding.
Questions for authors
Q1 (clarification, affects W1): Please specify the exact supervised adaptation protocol for each baseline in Table 2: input format, how AesAlign's 1-100 labels map to each method's output (in particular Q-Align's discrete bucketing), LoRA vs. full fine-tuning, learning rates/epochs, and convergence criteria. Also explain why InternVL3-8B degrades after adaptation on AADB. Q2 (clarification, affects W2): Have you tried fitting the same Ridge probes on the frozen features of Qwen3.7-Plus (the analysis generator) with the 45K ratings? If the student outperforms this teacher-probe baseline, please report it; if not, how do you position the method other than as distillation plus protocol alignment? Q3 (clarification): AesAlign-500K, AADB, and PARA are all Flickr-sourced. Was any cross-dataset near-duplicate removal performed (e.g., SSCD/FAISS retrieval between AesAlign-500K and the AADB/PARA test images)? If not, the cross-dataset SRCC gains in Table 2 may partly reflect image overlap between training and "external" test sets. Q4 (clarification, affects W2): Since the Stage 1 texts embed Qwen3.7-Plus's own scores, can you disentangle how much of the Stage 1 benefit comes from aesthetic understanding versus distribution/format correction — e.g., by re-running Stage 1 on texts with all numerical scores stripped?