acceptodds
Withdrawn from ICLR 2027

CAPTURE with PhotoEncoder: Photographic Understanding with Representation Enhanced Vision-Language Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Vision-language models (VLMs) could be powerful tools for understanding and retouching photographs, yet their ability to perceive fine-grained photographic attributes, such as brightness and color temperature, remains limited. Using the **Photographic Diagnostic Probe**, we reveal that existing VLM vision encoders inadequately capture these photographic attributes. To address this limitation, we introduce **PhotoEncoder**, which structures visual representations around photographic attributes and their controlled transitions while preserving pretrained semantic representations. We further construct a dataset of 5M vision-language pairs and 81K instruction-tuning examples derived from editing records to support training. By integrating PhotoEncoder into the language model and applying instruction tuning, we build **CAPTURE** for photographic perception, aesthetic understanding, and retouching assistance. Experiments show that CAPTURE better distinguishes subtle tonal and color changes, enabling more reliable aesthetic judgments and editing decisions.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Vision-Language Models, Visual Representation Learning, Aesthetic Assessment, Image Retouching, Instruction Tuning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_DzuROpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper adresses the shortcoming of current VLMs to capture fine-grained photographic attributes (brightness, color temperature etc.). The authors adress this issue by creating their own dataset which is then used to LORA fine-tune in first stage the image-encoder of a pretrained VLM (e.g. Qwen3-VL), before instruction-tune the remaining parts (visual merger and LLM). The results show improvements on the provided benchmarks over current proprietary and open source VLM models.

My current recommendation is to reject the paper in its current form. The paper positions itself to provide a threefold of contributions:

  1. Insight that current VLMs fail to sufficiently capture fine-grained photographic attributes
  2. A new dataset to support training, with an new evaluation benchmark
  3. A new trained model CAPTURE

I see the main contribution to be 2), the new dataset which might be useful to the community in the future. However, the provided insight in 1) is limited while the methodological contribution in 3) is lacks sufficient contribution. Finally, the evaluation is very limited.

Critical strengths
  1. The author created a new dataset enabling a broader task coverage in a single dataset than comparable benchmarks. The dataset has been created using controlled edits and expert edits. The provided vision-language pairs and photographic instruction samples could prove valuable for future methods. I see this as the main contribution of the provided work.
Critical weaknesses
  1. Contribution:
    a) PDP: The method itself is a simple probing technique, which is fair by itself. However, the discussion is very limited and focussed on the factthat using a model fine tuned on the new dataset, better results are reached. This discussion falls short: The supplementary material actually provides much more insights in which categories the models fail, also the newly finetuned ones. However, no interpretration and discussion is given.
    b) PhotoEncoder: The provided framework is a finetuning of Qwen3VL / Intern-ViT encoder. Both the components used (i.e. the Qwen3VL encoders) and the training method (SigLIP losses) are standard to the Qwen3VL model. The loss adaption is straight forward adaption to the properties of the created dataset.
    c) Capture: Is a instruction-finetuned Qwen3VL model, no architectural or methodological novelty.

  2. Lack of baselines and fairness of evaluation:
    a) The evaluation focusses exclusively on the provided benchmark, which is of similar distribution than the provided dataset (see question Q1 below). The provided method is not evaluated on other benchmarks (e.g. the ones listed in Tab. 1).
    b) Other competitor models have been trained on different data. For fair comparison, one could also fine tune other competitors on the same data. E.g. in Tab. 2 Qwen3.5-9B or Gemma4-31B provide better results than a the default Qwen3VL model. I wouldn't be surprised to see even better results with these backbones.

  3. Attribution of existing work vs. new work:
    a) In several parts of the papers, it is unclear if the provided insight is based on a novel contribution or existing work. E.g. "Photographic Diagnostic Probe (PDP)": In line 126 the authors write "We adapt the PDP" which reads as an adaption of an existing method. The probing method itself seems very standard, but if this exact PDP setup has been used in previous works, these works need to be clearly referenced. b) Listed competitors such as PhotoCritique in Tab. 1 miss references.

  4. Critical discussion:
    a) The authors fail to discuss limitations of their method as well as short comings of the provided new dataset

Questions for authors
  • Q1: Could the authors clarify on the details of how the created benchmark relates to the data distribution of the training dataset?
  • Q2: Could the authors provide additional results on evaluation of CAPTURE on other benchmarks and / or using the dataset with other backbones
  • Q3: While the supplementary material already lists details on dataset creation, the current work misses a critical analysis and discussion of the newly created dataset. More examples, failure cases etc. would be useful to strengthen the contribution.
4 comments

We thank Reviewer DzuR for the detailed and constructive feedback. We address the concerns and questions below, grouping closely related points for clarity.


W1 Contributions

Reply to 1(a): PDP Interpretation and Discussion

Thank you for highlighting the need for a more substantive interpretation of the diagnostic results. We will add the following discussion and explanation to the paper.

The problem with native vision encoders is the tension between semantic invariance and photographic sensitivity. In Tab. 4 Qwen3VL achieves only 26.69% accuracy in classification. This suggests that general-purpose visual encoders tend to maintain stable representations of scene content [1,2], whereas photographic understanding requires sensitivity to changes in lighting and color that do not alter the semantics. The representations do not distinguish semantic content from photographic appearance. Our method addresses this limitation by aligning visual representations with photographic attributes while preserving semantic information, improving classification accuracy to 66.21%.

Fine-tuning shifts errors from broad attribute confusion toward fine-grained color-range confusion. As shown in Tab. 19, adapted models achieve 93.9% and 89.2% accuracy on tonal and global color attributes, but only 56.5%, 47.4%, and 60.9% on hue, saturation, and lightness across individual color ranges. Global adjustments generally produce stronger visual signals, whereas color-specific changes are often subtle and localized. Among all classification errors, 62.8% correctly identify the H/S/L dimension but misidentify the hue range, with 78.7% of these errors occurring between adjacent ranges. Such errors may arise from insufficiently localized color representations and ambiguities introduced by discrete hue boundaries.

These findings highlight the need for better attribute disentanglement, region-aware representations, and magnitude estimation. PhotoEncoder represents a step toward fine-grained photographic understanding, while PDP exposes the remaining challenges.

Reply to 1(b-c): Contributions of PhotoEncoder and CAPTURE

Our work advances photographic assistants through three contributions:

  1. Systematically diagnosing vision encoders' limitations in photographic visual representations. Using PDP, we evaluate ten native vision encoders, revealing their limitations in distinguishing tonal and color changes. While prior work [3] has observed limited aesthetic sensitivity in vision encoders, these limitations remain insufficiently characterized.

  2. Developing a framework for fine-grained photographic visual representation adaptation. Photography places particular emphasis on aesthetic quality, where even subtle variations in lighting, tone, and color can significantly affect an image's visual appeal. However, our PDP reveals that existing vision encoders struggle to distinguish such fine-grained variations. To address this, we restructure visual representations to capture photographic attribute states and transitions, enabling finer discrimination. This provides a valuable foundation for photographic analysis and retouching.

  3. Developing a multi-task photographic assistant and dataset. Building on PhotoEncoder, we train CAPTURE, a photographic assistant capable of fine-grained photographic perception, aesthetic analysis, and actionable retouching assistance. By integrating these capabilities into a single model, CAPTURE provides practical value for photographic understanding and retouching workflows.

W2 & Q1 & Q2 (Baselines Lackness and Evaluation Fairness)

Reply to 2(a) & Q1 & Q2: Additional Results on Other Benchmarks

We additionally evaluate CAPTURE on ArtiMuse-10K [4]. ArtiMuse-10K is an expert-curated benchmark featuring fine-grained aesthetic analyses across eight dimensions, making it suitable for evaluating models' ability. Among other relevant benchmarks, including AesBench [5] and PhotoCritique [3], our access request for AesBench is still pending, while PhotoCritique is not publicly available.

We use two complementary protocols: (1) the original ArtiMuse evaluation protocol extended to five models, and (2) a broader pairwise evaluation of 17 models with Bradley–Terry (BT) aggregation. In the two protocols, CAPTURE performed well.

Results Analysis. As shown in Tab. A1, CAPTURE achieves the highest win rate. And in Tab. A2, the BT evaluation further confirms its effectiveness, ranking CAPTURE second among 17 models, behind only GPT-5.5 and ahead of ArtiMuse. These results demonstrate that our improvements extend beyond our own benchmark.

Table A1: Comparisons on ArtiMuse-10K using the original artiMuse protocol.

Aesthetic AttributesQ-InstructQwen2.5-VL-7B-InstructInternVL3-8BArtiMuseCAPTURE
Composition & Design0.6%7.0%9.8%30.7%51.9%
Visual Elements & Structure0.3%11.1%9.0%38.2%41.4%
Technical Execution6.3%11.9%4.5%30.4%46.9%
Originality & Creativity1.0%7.4%6.2%33.7%51.7%
Theme & Communication0.4%19.3%17.0%21.8%41.4%
Emotion & Viewer Response0.7%16.7%10.9%26.3%45.4%
Overall Gestalt1.3%8.6%8.3%39.1%42.7%
Comprehensive Evaluation0.3%8.6%4.4%41.0%45.7%
Attributes Average1.4%11.3%8.8%32.6%45.9%

Table A2: Comparisons of 17 models using pairwise BT ranking.

RankModelBT rating95% CI
1GPT-5.51182.81171.1–1192.9
2CAPTURE1095.01085.9–1103.4
3Gemma4-31B1021.11010.6–1031.6
4Kimi-k2.61018.11007.3–1028.6
5ArtiMuse1000.0reference anchor
6Gemini-3.1-pro-preview995.7985.2–1005.3
7Qwen3VL-8B967.1953.3–977.3
8Qwen3.6-27B946.6933.9–957.3
9GLM4.6V-9B930.4917.1–941.0
10qwen3.6-plus905.9894.6–916.6
11Qwen3.5-9B905.1893.4–914.4
12LLaVA-OneVision2-8B848.7839.1–856.6
13KimiVL-16B796.8786.7–806.2
14InternVL3.5-8B793.8782.0–804.0
15Llama3.2-Vision-11B672.2660.8–682.4
16AesExpert586.1571.9–599.4
17Q-Instruct556.7544.5–568.5

Details of Comparisons. As shown in Tab. R1, we first follow the original ArtiMuse evaluation protocol, extending its four-way response selection to five-way selection by including CAPTURE alongside the four original models. As shown in Tab. R2, to extend our evaluation to 17 models on ArtiMuse, we adopt anonymous pairwise comparisons with Bradley–Terry (BT) aggregation [6,7], avoiding the substantial context demands of jointly comparing 17 long-form responses. We evaluate 7984 common-valid queries. A predefined 17-round round-robin schedule assigns eight disjoint pairs and one bye per query. Qwen3.6-27B judges each pair in both AB and BA orders to mitigate position bias [7]. Wins, ties, and losses are scored as 1, 0.5, and 0, with the two judgments averaged. We fit an L2-regularized BT model, pij=sigma(betai−betaj)p_{ij}=sigma(beta_i-beta_j), with lambda=0.1lambda=0.1, to obtain overall and attribute-specific ratings and rankings [6,8]. Pointwise 95% confidence intervals are estimated using 200 image-cluster bootstrap replicates, preserving within-image dependencies [9].

Regarding Q1, our benchmark evaluates generalization to unseen photographs and scenes. Although the benchmark and training data share the same underlying sources (MIT-Adobe FiveK and PPR10K), they use disjoint source images and scene groups. All photographs, edited variants or scene groups remain in the same partition. This provides a fair evaluation, consistent with established photo-retouching evaluation protocols [10–12].

Reply to 2(b): Using Better Backbones

Our central contribution is to identify and address limitations in the fine-grained photographic perception of vision encoders. We agree that applying our adaptation framework to stronger backbones, such as Qwen3.5-9B or Gemma4-31B, could yield better performance than the current Qwen3VL-based CAPTURE.

We selected Qwen3VL because its vision encoder and compatible pretrained SigLIP2 text encoder are publicly available, enabling our image–text alignment objectives. The same adaptation framework can be transferred to Qwen3.5-9B, which supports a compatible SigLIP2-based alignment interface. We are extending our method to Qwen3.5-9B and plan to release the resulting larger model.

In contrast, Gemma4-31B does not release a weight with a corresponding pretrained contrastively aligned text encoder and readout. Applying our current adaptation protocol would therefore require additional alignment design.

W3 (Attribution of Existing Work)

Reply to 3(a): Existing work vs. new work

PDP builds upon established diagnostic probing methods [13, 14], with a specific focus on the fine-grained visual sensitivity required for photographic applications. To the best of our knowledge, ours is the first systematic probing study that examines photographic attribute discrimination and change-magnitude estimation. This provides a more comprehensive diagnosis of how vision encoders represent subtle tonal and color changes. In comparison, [3] investigates aesthetic feature discriminability through embedding-distance analysis, while [15] probes whether VLM representations encode aesthetic attributes useful for personalized aesthetic-score prediction. We will add this comparison to related works.

Reply to 3(b): Reference

Thank you for the reminder. PhotoCritique has already been cited, namely in line 294, "Qi et al., 2025";


W4 (Lack of Limitations)

Thank you for raising this point. We acknowledge the following limitations of our method and dataset.

Requires a text encoder compatible with the vision encoder. Our current first-stage adaptation uses a compatible text encoder and a readout for image–text alignment. For vision backbones without available compatible components, applying the same adaptation protocol may require additional alignment design.

Coverage of aesthetic preferences. Our dataset builds on FiveK and PPR10K, with aesthetic supervision derived from expert editing records and generated annotations. It may therefore inherit the stylistic preferences represented by these sources and does not encompass the full diversity of photographic styles or individual users' preferences. Broader data collection and more diverse human annotations would help expand this coverage.


Q3 (Dataset examples)

Regarding examples, we already provide additional examples from our dataset on pp. 29–37 of the supplementary material. Through these results, we find that CAPTURE is more sensitive to subtle photographic differences. As shown in Figs. 6–8, it more accurately identifies fine-grained tonal and color attributes and their changes, which are often overlooked by baseline models. Additionally, CAPTURE provides more precise aesthetic judgments. In Figs. 9–10, it identifies specific issues, such as unnatural color casts and imbalanced exposure, while baselines often provide generic assessments and overlook specific photographic issues.

References

[1] Chen T, Kornblith S, Norouzi M, et al. A simple framework for contrastive learning of visual representations[C]//International conference on machine learning. PmLR, 2020: 1597-1607.

[2] Xiao T, Wang X, Efros A A, et al. What should not be contrastive in contrastive learning[J]. arXiv preprint arXiv:2008.05659, 2020.

[3] Qi D, Zhao H, Shi J, et al. The Photographer’s Eye: Teaching Multimodal Large Language Models to See and Critique like Photographers[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2025: 24807-24816.

[4] Cao S, Ma N, Li J, et al. Artimuse: Fine-grained image aesthetics assessment with joint scoring and expert-level understanding[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2026: 15313-15322.

[5] Huang Y, Yuan Q, Sheng X, et al. Aesbench: An expert benchmark for multimodal large language models on image aesthetics perception[J]. arXiv preprint arXiv:2401.08276, 2024.

[6] Chiang W L, Zheng L, Sheng Y, et al. Chatbot arena: An open platform for evaluating llms by human preference[J]. arXiv preprint arXiv:2403.04132, 2024.

[7] Zheng L, Chiang W L, Sheng Y, et al. Judging llm-as-a-judge with mt-bench and chatbot arena[J]. Advances in neural information processing systems, 2023, 36: 46595-46623.

[8] Bradley R A, Terry M E. Rank analysis of incomplete block designs: I. the method of paired comparisons[J]. Biometrika, 1952, 39(3/4): 324-345.

[9] Field C A, Welsh A H. Bootstrapping clustered data[J]. Journal of the Royal Statistical Society Series B: Statistical Methodology, 2007, 69(3): 369-390.

[10] Duan Z P, Zhang J, Lin Z, et al. Diffretouch: Using diffusion to retouch on the shoulder of experts[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2025, 39(3): 2825-2833.

[11] Yang C, Jin M, Jia X, et al. Adaint: Learning adaptive intervals for 3d lookup tables on real-time image enhancement[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 17501-17510.

[12] Ouyang W, Dong Y, Kang X, et al. Rsfnet: A white-box image retouching approach using region-specific color filters[C]//2023 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2023: 12126-12135.

[13] Alain G, Bengio Y. Understanding intermediate layers using linear classifier probes[J]. arXiv preprint arXiv:1610.01644, 2016.

[14] Hewitt J, Liang P. Designing and interpreting probes with control tasks[C]//Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (emnlp-ijcnlp). 2019: 2733-2743.

[15] Ryu K, Yanaka H. What Do Vision–Language Models Encode for Personalized Image Aesthetics Assessment?[C]//Findings of the Association for Computational Linguistics: ACL 2026. 2026: 34146-34167.

Reviewer_zrpCOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents CAPTURE, a photographic assistant built around PhotoEncoder, an adapted vision encoder for fine-grained tonal and color understanding. A diagnostic probe motivates representation-level adaptation using attribute alignment, edit alignment, content alignment, and feature anchoring. The adapted encoder is integrated into Qwen3VL and instruction-tuned for perception, aesthetic analysis, and Lightroom parameter prediction. The authors construct approximately 5M supervision pairs, 81.5K instruction examples, and a 3.1K-example benchmark. Controlled comparisons show improvements over identical instruction tuning without PhotoEncoder, supported by external aesthetic evaluation and a human retouching-preference study.

Critical strengths
  1. Clear presentation and a coherent approach. The paper clearly connects its diagnostic motivation, encoder adaptation, and downstream photographic tasks. The method and data pipeline are well organized and supported by a helpful overview figure.

  2. Broad evaluation with meaningful controls. PhotoEncoder improves all seven downstream metrics over identical instruction tuning with the original encoder. Loss ablations, two encoder backbones, external evaluation, and a human preference study provide reasonably comprehensive support.

Critical weaknesses
  1. A matched-data encoder fine-tuning baseline is missing. The comparison with instruction tuning alone does not separate the benefit of additional encoder training from that of the proposed PhotoEncoder objectives. A baseline that conventionally fine-tunes the encoder on the same adaptation data with a matched budget, then applies identical instruction tuning, would better establish the method's incremental value.

  2. Loss-weight selection and robustness are unclear. The paper specifies fixed weights without explaining their selection or testing sensitivity. Since edit and content supervision have markedly different effects in the ablations, the robustness of their balance deserves clarification.

  3. Open-ended evaluation relies heavily on generated references. A text-only judge scores answers against model-generated references and rubrics, without independently checking the images. The human study evaluates rendered edits rather than the correctness of critiques, leaving the broader aesthetic-understanding claims less firmly validated.

  4. Held-out evaluation needs clearer documentation. The source-level split is explicit for encoder adaptation, but not equally clear for all instruction-training and downstream benchmark tasks. PDP also reports scores on the validation set used for checkpoint selection, limiting the strength of the independent-generalization evidence.

Questions for authors
  1. How does PhotoEncoder compare with conventional encoder fine-tuning on the same first-stage data and training budget, followed by identical instruction tuning—for example, using direct attribute/change prediction rather than the proposed text-alignment objectives?
  2. How were the loss weights selected, and how sensitive are photographic performance and semantic preservation to the relative weights of edit and content alignment?
  3. Can you clarify source/scene separation across all training stages and benchmark tasks, and whether PDP results are available on a test split independent of checkpoint selection?
4 comments

W1 & Q1 (Conventional encoder fine-tuning baseline)

We thank the reviewer for this valuable suggestion. To compare with conventional supervised encoder fine-tuning baseline, we additionally fine-tune an encoder using direct attribute prediction [1], with the same first-stage data and training budget as PhotoEncoder, followed by identical instruction tuning. This provides a meaningful comparison because it explicitly optimizes the encoder to recognize the same photographic attributes targeted by PhotoEncoder, allowing us to examine whether direct attribute supervision alone is sufficient to achieve comparable downstream performance. The results are reported in Tab. C1.

Table C1: Comparison with matched-data, matched-budget encoder fine-tuning.

Attribute AssessmentTransition AssessmentAesthetic CritiqueStyle ComparisonEditing RecommendationLR Param. PSNRLR Param. SSIM
CAPTURE70.2875.645.340 ±\pm 0.0086.860 ±\pm 0.0114.533 ±\pm 0.02122.050.8879
Direct prediction78.4858.523.650 ±\pm 0.0026.753 ±\pm 0.0104.185 ±\pm 0.02020.530.8634
w/o PhotoEncoder60.4751.584.619 ±\pm 0.0066.071 ±\pm 0.0194.072 ±\pm 0.02320.940.8610

Although direct prediction achieves higher Attribute Assessment accuracy than CAPTURE (78.48% vs. 70.28%), CAPTURE outperforms it across all remaining tasks and metrics, particularly in Transition Assessment (75.64% vs. 58.52%) and Aesthetic Critique (5.340 vs. 3.650). Notably, direct prediction even underperforms the original encoder on Aesthetic Critique. These findings suggest that optimizing attribute classification alone may limit transferability to broader photographic tasks, demonstrating the necessity and effectiveness of our adaptation framework.


W2 (Loss-weight Selection and Robustness)

Thank you for raising this important point. Following your advice, We conduct additional sensitivity experiments by varying λedit\lambda_{\mathrm{edit}} and λcontent\lambda_{\mathrm{content}}, while keeping λattr=1\lambda_{\mathrm{attr}}=1 and λanchor=0.1\lambda_{\mathrm{anchor}}=0.1 fixed.

The results demonstrate that CAPTURE is relatively robust to loss-weight variations, as shown in Tab. C2. Across the tested configurations, Attribute Classification accuracy varies by at most 0.83 percentage points, Magnitude Regression SRCC by 0.004, and Image-to-Text R@1 by 0.37 percentage points. Text-to-Image R@1 exhibits a slightly larger variation of 1.68 percentage points.

Table C2: Loss-weight Selection.

λedit\lambda_{\mathrm{edit}}λcontent\lambda_{\mathrm{content}}Attribute Classification Acc. (%) ↑Magnitude Regression SRCC ↑Image-to-Text R@1 (%) ↑Text-to-Image R@1 (%) ↑
10.166.210.57594.0396.07
0.50.166.780.57794.0394.39
10.566.160.57393.8496.07
20.166.640.57493.6694.58

W3 (Concerns About Evaluation)

Our benchmark evaluates critique correctness using detailed rubrics grounded in human-validated references. To assess the correctness of our generated reference critiques, we recruited five professionally trained photographers to independently evaluate 300 instruction–response pairs against the images and editing evidence. As shown in Tab. C3, the references achieved an overall score of 4.12/5, including 3.75/5 for Visual Grounding and 3.95/5 for Evidence Consistency. These results support the visual correctness and reliability of our references. Building on these references, our rubrics explicitly specify expected visual observations, acceptable semantic equivalents, and conditions for awarding or withholding points, enabling structured evaluation of critique correctness rather than unconstrained text similarity. Moreover, as shown in Fig. 1 and Tab. 4, existing VLM encoders struggle to perceive subtle photographic changes; requiring another VLM to independently interpret the images could introduce additional perceptual errors. Moreover, as shown in Fig. 1 and Tab. 4, existing VLM encoders struggle to perceive subtle photographic changes; requiring another VLM to independently interpret the images could introduce additional perceptual errors. Our Editing Recommendation evaluation further assesses whether photographic diagnoses translate into appropriate corrective actions, while the human study on rendered edits provides complementary validation of their practical effectiveness.

Our improvements also generalize to an independent, expert-curated aesthetic benchmark. On ArtiMuse-10K, as shown in Tab. C3, which provides fine-grained expert aesthetic analyses across eight dimensions, CAPTURE achieves the highest average selection rate (45.9%) under the original evaluation protocol extended to five models, compared with 32.6% for ArtiMuse. These results prove that CAPTURE has a broader aesthetic understanding.

Table C3: Human Evaluation of Generated Photographic Instruction–Response Data (Mean ±\pm Standard Deviation, 1–5 Scale).

AnnotatorQuestion QualityVisual GroundingEvidence ConsistencyOverall
14.92 ±\pm 0.323.67 ±\pm 1.163.88 ±\pm 1.394.16 ±\pm 0.77
24.90 ±\pm 0.474.14 ±\pm 1.164.14 ±\pm 1.274.40 ±\pm 0.74
34.86 ±\pm 0.594.06 ±\pm 1.064.16 ±\pm 1.224.36 ±\pm 0.67
44.66 ±\pm 0.693.63 ±\pm 1.183.90 ±\pm 1.364.06 ±\pm 0.70
53.91 ±\pm 0.613.27 ±\pm 0.733.65 ±\pm 0.893.61 ±\pm 0.50

Details of the Human Evaluation. We uniformly sample 300 generated instruction–response pairs and recruit five professional photographers with formal educational backgrounds in photography and practical experience in the field, ensuring a foundation in basic photography knowledge. Each annotator independently evaluates all 300 pairs using a five-point scale along three dimensions:

  • Question Quality: Whether the instruction is reasonable, clear, and answerable

  • Visual Grounding: Whether the visual judgments and technical descriptions are correct.

  • Evidence Consistency: Whether the explanation is consistent with the image, editing evidence, and recorded operations.

Scores range from 1 (fundamentally incorrect or irrelevant) to 5 (accurate, specific, and well-supported).

Table C4: Comparisons on ArtiMuse-10K using the original artiMuse protocol.

Aesthetic AttributesQ-InstructQwen2.5-VL-7B-InstructInternVL3-8BArtiMuseCAPTURE
Composition & Design0.6%7.0%9.8%30.7%51.9%
Visual Elements & Structure0.3%11.1%9.0%38.2%41.4%
Technical Execution6.3%11.9%4.5%30.4%46.9%
Originality & Creativity1.0%7.4%6.2%33.7%51.7%
Theme & Communication0.4%19.3%17.0%21.8%41.4%
Emotion & Viewer Response0.7%16.7%10.9%26.3%45.4%
Overall Gestalt1.3%8.6%8.3%39.1%42.7%
Comprehensive Evaluation0.3%8.6%4.4%41.0%45.7%
Attributes Average1.4%11.3%8.8%32.6%45.9%

Details of Comparisons. As shown in Tab. C3, we follow the original ArtiMuse evaluation protocol, extending its four-way response selection to five-way selection by including CAPTURE alongside the four original models.

W4 (Held-Out Data Separation and Independent Evaluation)

Reply to Source-Level Separation:

We clarify that instruction tuning and all downstream benchmark tasks follow the same source-image and scene-level partitioning protocol used for encoder adaptation. All edited variants and derived instruction samples associated with the same source image or scene remain within the same partition. This provides a fair held-out evaluation.

Reply Independent PDP Test Evaluation:

We further validate PDP on an independent test set constructed from the test images of ArtiMuse, as shown in Tab. C5. These results demonstrate that the improvements generalize beyond the validation set used for checkpoint selection. Specifically, PhotoEncoder (InternViT) improves classification accuracy from 38.54% to 62.23% and SRCC from 0.366 to 0.495, while PhotoEncoder (Qwen3VL) improves accuracy from 20.68% to 60.88% and SRCC from 0.314 to 0.522. PhotoEncoder consistently outperforms its corresponding original encoder on this independent test set.

Table C5: PDP Results on ArtiMuse-10K test images.

EncoderTest Acc. (%) ↑Test SRCC ↑
Llama3.2-Vision18.780.370
Qwen2.5VL18.920.216
Qwen3VL20.680.314
Qwen3.523.500.311
Qwen3.631.370.290
Gemma418.670.257
LLaVA-OneVision225.240.330
MoonViT24.430.333
GLM4.6V35.230.344
InternViT-300M38.540.366
PhotoEncoder(InternViT)62.230.495
PhotoEncoder(Qwen3VL)60.880.522

Details of the Experiment. For the retained images, we rendered baseline images and controlled, single-parameter Lightroom edits, applying the same 26-attribute measurement and labeling pipeline as the original PDP benchmark. The final test contains 507 source images, 4323 attribute-classification pairs, and 6601 edit-magnitude regression pairs. All models in Tab. C5 were evaluated using their original validation-selected PDP checkpoints. The PhotoEncoder advantage is maintained when evaluation is separated from PDP checkpoint selection.


Reference

[1] Sharir G, Noy A, Zelnik-Manor L. An image is worth 16x16 words, what is a video worth?[J]. arXiv preprint arXiv:2103.13915, 2021.

Thanks for your response. I'll keep my positive score.

Reviewer_YV4bOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces CAPTURE, a photography-oriented vision-language model that adapts its visual encoder to capture tonal and color attributes and editing-induced changes, followed by photographic instruction tuning. Its major contributions include the PhotoEncoder adaptation objectives, the constructed dataset. Evaluations provide evidence that encoder adaptation improves photographic task performance.

Critical strengths
  1. The proposed adaptation is effective. Controlled edits provide targeted supervision for photographic attributes and their changes, rather than relying solely on textual critiques. The effectiveness is furthur validated by evaluations.

  2. Dataset construction. The paper constructed photograph datasets from controlled edits and expert retouching records, spanning attribute perception, aesthetic analysis, and actionable editing. Connecting these complementary forms of supervision is a contribution toward training photography-oriented VLMs beyond generic image understanding.

Critical weaknesses
  1. The methodological contribution is relatively limited given existing difference-alignment methods. The central edit-alignment objective closely follows existing difference-alignment methods, supplemented by established attribute-alignment and feature-preservation objectives. The main downstream comparison uses an unadapted encoder rather than existing adaptation methods trained with the same data budget. Thus the results establish the usefulness of photography-specific adaptation, but do not clearly establish the advantage of the proposed method.

  2. The instruction data and evaluation do not sufficiently establish real-world photographic understanding (diversity, real use cases, etc.). Gemini generates critiques and recommendations from expert-edited results and parameters, but the provided manuscript does not explain how user instructions are constructed and matched to those edits, or how the generated explanations are verified. The model may learn plausible post-hoc explanations rather than faithfully responding to real user intentions. LLM-based scoring and similarity to expert reference images do not resolve this distinction, while the human evaluation is limited.

Questions for authors

Besides questions raised in Critical Weaknesses: How many of the data are human edited? Many images in MIT-Adobe FiveK are just moderately edited, have authors considered more challenging editing data/task from real cases?

2 comments

We deeply appreciate Reviewer YV4b for the detailed and constructive feedback. Our detailed responses can be found below.


W1 (Contributions and Fair Comparisons)

Reply to Limited Methodological Contribution

Our work advances photographic assistants through three contributions:

  1. Systematically diagnosing vision encoders' limitations in photographic visual representations. Using PDP, we evaluate ten native vision encoders, revealing their limitations in distinguishing tonal and color changes. While prior work [3] has observed limited aesthetic sensitivity in vision encoders, these limitations remain insufficiently characterized.

  2. Developing a framework for fine-grained photographic visual representation adaptation. Photography places particular emphasis on aesthetic quality, where even subtle variations in lighting, tone, and color can significantly affect an image's visual appeal. However, our PDP reveals that existing vision encoders struggle to distinguish such fine-grained variations. To address this, we restructure visual representations to capture photographic attribute states and transitions, enabling finer discrimination. This provides a valuable foundation for photographic analysis and retouching.

  3. Developing a multi-task photographic assistant and dataset. Building on PhotoEncoder, we train CAPTURE, a photographic assistant capable of fine-grained photographic perception, aesthetic analysis, and actionable retouching assistance. By integrating these capabilities into a single model, CAPTURE provides practical value for photographic understanding and retouching workflows.

Reply to Using Existing Adaptation Methods as A Baseline

We thank the reviewer for this valuable suggestion. Following the reviewer zrpC's advice, we fine-tune an encoder using direct attribute prediction, with the same first-stage data and training budget as PhotoEncoder, followed by identical instruction tuning. The results are reported in Table B1.

Table B1: Comparison with matched-data, matched-budget encoder fine-tuning.

Attribute AssessmentTransition AssessmentAesthetic CritiqueStyle ComparisonEditing RecommendationLR Param. PSNRLR Param. SSIM
CAPTURE70.2875.645.340 ±\pm 0.0086.860 ±\pm 0.0114.533 ±\pm 0.02122.050.8879
Direct prediction78.4858.523.650 ±\pm 0.0026.753 ±\pm 0.0104.185 ±\pm 0.02020.530.8634
w/o PhotoEncoder60.4751.584.619 ±\pm 0.0066.071 ±\pm 0.0194.072 ±\pm 0.02320.940.8610

Although direct prediction achieves higher Attribute Assessment accuracy than CAPTURE (78.48% vs. 70.28%), CAPTURE outperforms it across all remaining tasks and metrics, particularly in Transition Assessment (75.64% vs. 58.52%) and Aesthetic Critique (5.340 vs. 3.650). Notably, direct prediction even underperforms the original encoder on Aesthetic Critique. These findings suggest that optimizing attribute classification alone may limit transferability to broader photographic tasks, demonstrating the necessity and effectiveness of our adaptation framework.

W2 & Question (Quality of the Annotated Data)

Reply to Concerns About Post-Hoc Explanations and Faithfulness to User Intent

Thank you for your suggestion to add a detailed explanation in the manuscript of how to generate instruction–response data. We will add it to the revised version.

To further validate the quality and reliability of our generated instruction–response data, we conduct an additional human evaluation focusing on Question Quality, Visual Grounding, and Evidence Consistency.

As shown in Tab. B2, the generated instruction–response data achieves an overall mean score of 4.12/5 across five annotators. The high Question Quality score indicates that the generated instructions are generally clear and answerable. More importantly, the Visual Grounding and Evidence Consistency scores suggest that the generated responses are generally supported by image content and consistent with actual human editing records, providing evidence against the concern that they are plausible post-hoc explanations.

Table B2: Human Evaluation of Generated Photographic Instruction–Response Data (Mean ±\pm Standard Deviation, 1–5 Scale).

AnnotatorQuestion QualityVisual GroundingEvidence ConsistencyOverall
14.92 ±\pm 0.323.67 ±\pm 1.163.88 ±\pm 1.394.16 ±\pm 0.77
24.90 ±\pm 0.474.14 ±\pm 1.164.14 ±\pm 1.274.40 ±\pm 0.74
34.86 ±\pm 0.594.06 ±\pm 1.064.16 ±\pm 1.224.36 ±\pm 0.67
44.66 ±\pm 0.693.63 ±\pm 1.183.90 ±\pm 1.364.06 ±\pm 0.70
53.91 ±\pm 0.613.27 ±\pm 0.733.65 ±\pm 0.893.61 ±\pm 0.50

Details of the Human Evaluation. We uniformly sample 300 generated instruction–response pairs and recruit five professional photographers with formal educational backgrounds in photography and practical experience in the field, ensuring a foundation in basic photography knowledge. Each annotator independently evaluates all 300 pairs using a five-point scale along three dimensions:

  • Question Quality: Whether the instruction is reasonable, clear, and answerable

  • Visual Grounding: Whether the visual judgments and technical descriptions are correct.

  • Evidence Consistency: Whether the explanation is consistent with the image, editing evidence, and recorded operations.

Scores range from 1 (fundamentally incorrect or irrelevant) to 5 (accurate, specific, and well-supported).

Reply to the Question

Our editing records are derived from PPR10K and MIT-Adobe FiveK, both of which contain real human-edited results. Our approach inverts relevant knowledge from existing human edits through knowledge inversion [2]. So it can be extended to other human-edited datasets with available Lightroom adjustment parameters, and is not limited to MIT-Adobe FiveK. We use MIT-Adobe FiveK primarily because it is a widely adopted dataset in photo retouching. We can collect more challenging editing data to further expand our dataset.


Reference

[1] Qi D, Zhao H, Shi J, et al. The Photographer’s Eye: Teaching Multimodal Large Language Models to See and Critique like Photographers[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2025: 24807-24816.

[2] Li Z, Sun L, Ming R, et al. Knowledge-Centric Agents for Workflow Generation in ComfyUI[C]//European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2026: 306-323.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Invitation to Further Discussion

Dear Reviewers,

Thank you for your time and valuable feedback. We have provided point-by-point responses to your comments and hope they adequately address your concerns. As the discussion period is approaching its end, we would greatly appreciate it if you could take a moment to review our responses and share your thoughts.

We sincerely welcome further discussion and would be grateful if you could let us know of any remaining concerns or questions, so that we can address them before the discussion period closes.

Thank you again for your time and consideration. We look forward to hearing your feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.