- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper adresses the shortcoming of current VLMs to capture fine-grained photographic attributes (brightness, color temperature etc.). The authors adress this issue by creating their own dataset which is then used to LORA fine-tune in first stage the image-encoder of a pretrained VLM (e.g. Qwen3-VL), before instruction-tune the remaining parts (visual merger and LLM). The results show improvements on the provided benchmarks over current proprietary and open source VLM models.
My current recommendation is to reject the paper in its current form. The paper positions itself to provide a threefold of contributions:
- Insight that current VLMs fail to sufficiently capture fine-grained photographic attributes
- A new dataset to support training, with an new evaluation benchmark
- A new trained model CAPTURE
I see the main contribution to be 2), the new dataset which might be useful to the community in the future. However, the provided insight in 1) is limited while the methodological contribution in 3) is lacks sufficient contribution. Finally, the evaluation is very limited.
Critical strengths
- The author created a new dataset enabling a broader task coverage in a single dataset than comparable benchmarks. The dataset has been created using controlled edits and expert edits. The provided vision-language pairs and photographic instruction samples could prove valuable for future methods. I see this as the main contribution of the provided work.
Critical weaknesses
-
Contribution:
a) PDP: The method itself is a simple probing technique, which is fair by itself. However, the discussion is very limited and focussed on the factthat using a model fine tuned on the new dataset, better results are reached. This discussion falls short: The supplementary material actually provides much more insights in which categories the models fail, also the newly finetuned ones. However, no interpretration and discussion is given.
b) PhotoEncoder: The provided framework is a finetuning of Qwen3VL / Intern-ViT encoder. Both the components used (i.e. the Qwen3VL encoders) and the training method (SigLIP losses) are standard to the Qwen3VL model. The loss adaption is straight forward adaption to the properties of the created dataset.
c) Capture: Is a instruction-finetuned Qwen3VL model, no architectural or methodological novelty. -
Lack of baselines and fairness of evaluation:
a) The evaluation focusses exclusively on the provided benchmark, which is of similar distribution than the provided dataset (see question Q1 below). The provided method is not evaluated on other benchmarks (e.g. the ones listed in Tab. 1).
b) Other competitor models have been trained on different data. For fair comparison, one could also fine tune other competitors on the same data. E.g. in Tab. 2 Qwen3.5-9B or Gemma4-31B provide better results than a the default Qwen3VL model. I wouldn't be surprised to see even better results with these backbones. -
Attribution of existing work vs. new work:
a) In several parts of the papers, it is unclear if the provided insight is based on a novel contribution or existing work. E.g. "Photographic Diagnostic Probe (PDP)": In line 126 the authors write "We adapt the PDP" which reads as an adaption of an existing method. The probing method itself seems very standard, but if this exact PDP setup has been used in previous works, these works need to be clearly referenced. b) Listed competitors such as PhotoCritique in Tab. 1 miss references. -
Critical discussion:
a) The authors fail to discuss limitations of their method as well as short comings of the provided new dataset
Questions for authors
- Q1: Could the authors clarify on the details of how the created benchmark relates to the data distribution of the training dataset?
- Q2: Could the authors provide additional results on evaluation of CAPTURE on other benchmarks and / or using the dataset with other backbones
- Q3: While the supplementary material already lists details on dataset creation, the current work misses a critical analysis and discussion of the newly created dataset. More examples, failure cases etc. would be useful to strengthen the contribution.
4 comments
W2 & Q1 & Q2 (Baselines Lackness and Evaluation Fairness)
Reply to 2(a) & Q1 & Q2: Additional Results on Other Benchmarks
We additionally evaluate CAPTURE on ArtiMuse-10K [4]. ArtiMuse-10K is an expert-curated benchmark featuring fine-grained aesthetic analyses across eight dimensions, making it suitable for evaluating models' ability. Among other relevant benchmarks, including AesBench [5] and PhotoCritique [3], our access request for AesBench is still pending, while PhotoCritique is not publicly available.
We use two complementary protocols: (1) the original ArtiMuse evaluation protocol extended to five models, and (2) a broader pairwise evaluation of 17 models with Bradley–Terry (BT) aggregation. In the two protocols, CAPTURE performed well.
Results Analysis. As shown in Tab. A1, CAPTURE achieves the highest win rate. And in Tab. A2, the BT evaluation further confirms its effectiveness, ranking CAPTURE second among 17 models, behind only GPT-5.5 and ahead of ArtiMuse. These results demonstrate that our improvements extend beyond our own benchmark.
Table A1: Comparisons on ArtiMuse-10K using the original artiMuse protocol.
| Aesthetic Attributes | Q-Instruct | Qwen2.5-VL-7B-Instruct | InternVL3-8B | ArtiMuse | CAPTURE |
|---|---|---|---|---|---|
| Composition & Design | 0.6% | 7.0% | 9.8% | 30.7% | 51.9% |
| Visual Elements & Structure | 0.3% | 11.1% | 9.0% | 38.2% | 41.4% |
| Technical Execution | 6.3% | 11.9% | 4.5% | 30.4% | 46.9% |
| Originality & Creativity | 1.0% | 7.4% | 6.2% | 33.7% | 51.7% |
| Theme & Communication | 0.4% | 19.3% | 17.0% | 21.8% | 41.4% |
| Emotion & Viewer Response | 0.7% | 16.7% | 10.9% | 26.3% | 45.4% |
| Overall Gestalt | 1.3% | 8.6% | 8.3% | 39.1% | 42.7% |
| Comprehensive Evaluation | 0.3% | 8.6% | 4.4% | 41.0% | 45.7% |
| Attributes Average | 1.4% | 11.3% | 8.8% | 32.6% | 45.9% |
Table A2: Comparisons of 17 models using pairwise BT ranking.
| Rank | Model | BT rating | 95% CI |
|---|---|---|---|
| 1 | GPT-5.5 | 1182.8 | 1171.1–1192.9 |
| 2 | CAPTURE | 1095.0 | 1085.9–1103.4 |
| 3 | Gemma4-31B | 1021.1 | 1010.6–1031.6 |
| 4 | Kimi-k2.6 | 1018.1 | 1007.3–1028.6 |
| 5 | ArtiMuse | 1000.0 | reference anchor |
| 6 | Gemini-3.1-pro-preview | 995.7 | 985.2–1005.3 |
| 7 | Qwen3VL-8B | 967.1 | 953.3–977.3 |
| 8 | Qwen3.6-27B | 946.6 | 933.9–957.3 |
| 9 | GLM4.6V-9B | 930.4 | 917.1–941.0 |
| 10 | qwen3.6-plus | 905.9 | 894.6–916.6 |
| 11 | Qwen3.5-9B | 905.1 | 893.4–914.4 |
| 12 | LLaVA-OneVision2-8B | 848.7 | 839.1–856.6 |
| 13 | KimiVL-16B | 796.8 | 786.7–806.2 |
| 14 | InternVL3.5-8B | 793.8 | 782.0–804.0 |
| 15 | Llama3.2-Vision-11B | 672.2 | 660.8–682.4 |
| 16 | AesExpert | 586.1 | 571.9–599.4 |
| 17 | Q-Instruct | 556.7 | 544.5–568.5 |
Details of Comparisons. As shown in Tab. R1, we first follow the original ArtiMuse evaluation protocol, extending its four-way response selection to five-way selection by including CAPTURE alongside the four original models. As shown in Tab. R2, to extend our evaluation to 17 models on ArtiMuse, we adopt anonymous pairwise comparisons with Bradley–Terry (BT) aggregation [6,7], avoiding the substantial context demands of jointly comparing 17 long-form responses. We evaluate 7984 common-valid queries. A predefined 17-round round-robin schedule assigns eight disjoint pairs and one bye per query. Qwen3.6-27B judges each pair in both AB and BA orders to mitigate position bias [7]. Wins, ties, and losses are scored as 1, 0.5, and 0, with the two judgments averaged. We fit an L2-regularized BT model, , with , to obtain overall and attribute-specific ratings and rankings [6,8]. Pointwise 95% confidence intervals are estimated using 200 image-cluster bootstrap replicates, preserving within-image dependencies [9].
Regarding Q1, our benchmark evaluates generalization to unseen photographs and scenes. Although the benchmark and training data share the same underlying sources (MIT-Adobe FiveK and PPR10K), they use disjoint source images and scene groups. All photographs, edited variants or scene groups remain in the same partition. This provides a fair evaluation, consistent with established photo-retouching evaluation protocols [10–12].
Reply to 2(b): Using Better Backbones
Our central contribution is to identify and address limitations in the fine-grained photographic perception of vision encoders. We agree that applying our adaptation framework to stronger backbones, such as Qwen3.5-9B or Gemma4-31B, could yield better performance than the current Qwen3VL-based CAPTURE.
We selected Qwen3VL because its vision encoder and compatible pretrained SigLIP2 text encoder are publicly available, enabling our image–text alignment objectives. The same adaptation framework can be transferred to Qwen3.5-9B, which supports a compatible SigLIP2-based alignment interface. We are extending our method to Qwen3.5-9B and plan to release the resulting larger model.
In contrast, Gemma4-31B does not release a weight with a corresponding pretrained contrastively aligned text encoder and readout. Applying our current adaptation protocol would therefore require additional alignment design.
W3 (Attribution of Existing Work)
Reply to 3(a): Existing work vs. new work
PDP builds upon established diagnostic probing methods [13, 14], with a specific focus on the fine-grained visual sensitivity required for photographic applications. To the best of our knowledge, ours is the first systematic probing study that examines photographic attribute discrimination and change-magnitude estimation. This provides a more comprehensive diagnosis of how vision encoders represent subtle tonal and color changes. In comparison, [3] investigates aesthetic feature discriminability through embedding-distance analysis, while [15] probes whether VLM representations encode aesthetic attributes useful for personalized aesthetic-score prediction. We will add this comparison to related works.
Reply to 3(b): Reference
Thank you for the reminder. PhotoCritique has already been cited, namely in line 294, "Qi et al., 2025";
W4 (Lack of Limitations)
Thank you for raising this point. We acknowledge the following limitations of our method and dataset.
Requires a text encoder compatible with the vision encoder. Our current first-stage adaptation uses a compatible text encoder and a readout for image–text alignment. For vision backbones without available compatible components, applying the same adaptation protocol may require additional alignment design.
Coverage of aesthetic preferences. Our dataset builds on FiveK and PPR10K, with aesthetic supervision derived from expert editing records and generated annotations. It may therefore inherit the stylistic preferences represented by these sources and does not encompass the full diversity of photographic styles or individual users' preferences. Broader data collection and more diverse human annotations would help expand this coverage.
Q3 (Dataset examples)
Regarding examples, we already provide additional examples from our dataset on pp. 29–37 of the supplementary material. Through these results, we find that CAPTURE is more sensitive to subtle photographic differences. As shown in Figs. 6–8, it more accurately identifies fine-grained tonal and color attributes and their changes, which are often overlooked by baseline models. Additionally, CAPTURE provides more precise aesthetic judgments. In Figs. 9–10, it identifies specific issues, such as unnatural color casts and imbalanced exposure, while baselines often provide generic assessments and overlook specific photographic issues.
References
[1] Chen T, Kornblith S, Norouzi M, et al. A simple framework for contrastive learning of visual representations[C]//International conference on machine learning. PmLR, 2020: 1597-1607.
[2] Xiao T, Wang X, Efros A A, et al. What should not be contrastive in contrastive learning[J]. arXiv preprint arXiv:2008.05659, 2020.
[3] Qi D, Zhao H, Shi J, et al. The Photographer’s Eye: Teaching Multimodal Large Language Models to See and Critique like Photographers[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2025: 24807-24816.
[4] Cao S, Ma N, Li J, et al. Artimuse: Fine-grained image aesthetics assessment with joint scoring and expert-level understanding[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2026: 15313-15322.
[5] Huang Y, Yuan Q, Sheng X, et al. Aesbench: An expert benchmark for multimodal large language models on image aesthetics perception[J]. arXiv preprint arXiv:2401.08276, 2024.
[6] Chiang W L, Zheng L, Sheng Y, et al. Chatbot arena: An open platform for evaluating llms by human preference[J]. arXiv preprint arXiv:2403.04132, 2024.
[7] Zheng L, Chiang W L, Sheng Y, et al. Judging llm-as-a-judge with mt-bench and chatbot arena[J]. Advances in neural information processing systems, 2023, 36: 46595-46623.
[8] Bradley R A, Terry M E. Rank analysis of incomplete block designs: I. the method of paired comparisons[J]. Biometrika, 1952, 39(3/4): 324-345.
[9] Field C A, Welsh A H. Bootstrapping clustered data[J]. Journal of the Royal Statistical Society Series B: Statistical Methodology, 2007, 69(3): 369-390.
[10] Duan Z P, Zhang J, Lin Z, et al. Diffretouch: Using diffusion to retouch on the shoulder of experts[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2025, 39(3): 2825-2833.
[11] Yang C, Jin M, Jia X, et al. Adaint: Learning adaptive intervals for 3d lookup tables on real-time image enhancement[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 17501-17510.
[12] Ouyang W, Dong Y, Kang X, et al. Rsfnet: A white-box image retouching approach using region-specific color filters[C]//2023 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2023: 12126-12135.
[13] Alain G, Bengio Y. Understanding intermediate layers using linear classifier probes[J]. arXiv preprint arXiv:1610.01644, 2016.
[14] Hewitt J, Liang P. Designing and interpreting probes with control tasks[C]//Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (emnlp-ijcnlp). 2019: 2733-2743.
[15] Ryu K, Yanaka H. What Do Vision–Language Models Encode for Personalized Image Aesthetics Assessment?[C]//Findings of the Association for Computational Linguistics: ACL 2026. 2026: 34146-34167.
We thank Reviewer DzuR for the detailed and constructive feedback. We address the concerns and questions below, grouping closely related points for clarity.
W1 Contributions
Reply to 1(a): PDP Interpretation and Discussion
Thank you for highlighting the need for a more substantive interpretation of the diagnostic results. We will add the following discussion and explanation to the paper.
The problem with native vision encoders is the tension between semantic invariance and photographic sensitivity. In Tab. 4 Qwen3VL achieves only 26.69% accuracy in classification. This suggests that general-purpose visual encoders tend to maintain stable representations of scene content [1,2], whereas photographic understanding requires sensitivity to changes in lighting and color that do not alter the semantics. The representations do not distinguish semantic content from photographic appearance. Our method addresses this limitation by aligning visual representations with photographic attributes while preserving semantic information, improving classification accuracy to 66.21%.
Fine-tuning shifts errors from broad attribute confusion toward fine-grained color-range confusion. As shown in Tab. 19, adapted models achieve 93.9% and 89.2% accuracy on tonal and global color attributes, but only 56.5%, 47.4%, and 60.9% on hue, saturation, and lightness across individual color ranges. Global adjustments generally produce stronger visual signals, whereas color-specific changes are often subtle and localized. Among all classification errors, 62.8% correctly identify the H/S/L dimension but misidentify the hue range, with 78.7% of these errors occurring between adjacent ranges. Such errors may arise from insufficiently localized color representations and ambiguities introduced by discrete hue boundaries.
These findings highlight the need for better attribute disentanglement, region-aware representations, and magnitude estimation. PhotoEncoder represents a step toward fine-grained photographic understanding, while PDP exposes the remaining challenges.
Reply to 1(b-c): Contributions of PhotoEncoder and CAPTURE
Our work advances photographic assistants through three contributions:
Systematically diagnosing vision encoders' limitations in photographic visual representations. Using PDP, we evaluate ten native vision encoders, revealing their limitations in distinguishing tonal and color changes. While prior work [3] has observed limited aesthetic sensitivity in vision encoders, these limitations remain insufficiently characterized.
Developing a framework for fine-grained photographic visual representation adaptation. Photography places particular emphasis on aesthetic quality, where even subtle variations in lighting, tone, and color can significantly affect an image's visual appeal. However, our PDP reveals that existing vision encoders struggle to distinguish such fine-grained variations. To address this, we restructure visual representations to capture photographic attribute states and transitions, enabling finer discrimination. This provides a valuable foundation for photographic analysis and retouching.
Developing a multi-task photographic assistant and dataset. Building on PhotoEncoder, we train CAPTURE, a photographic assistant capable of fine-grained photographic perception, aesthetic analysis, and actionable retouching assistance. By integrating these capabilities into a single model, CAPTURE provides practical value for photographic understanding and retouching workflows.