- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper argues that aesthetic preference should be modeled within contextually related image collections (images of the same scene, event or subject, or edited variants of one source image) rather than for isolated images. Its contributions are: (1) ContextAes-40K, 39,033 images organized into 8,069 collections, drawn from RedNote, ETH3D and HPatches (natural photography) and Lightroom (editing); (2) ContextAes-Album, 494 user-curated albums with owner-provided top-5 rankings, used for zero-shot evaluation; and (3) ContextAes, a Qwen3.5-9B model trained in three stages (SkillOpt skill optimization, LoRA SFT, GRPO). ContextAes achieves state-of-the-art results on ContextAes-40K, generalizes strongly across other aesthetic benchmarks, and shows excellent zero-shot performance on ContextAes-Album, demonstrating its ability to capture aesthetic preference within contextually related image collections. The paper is clearly written and the dataset construction is documented in detail.
Critical strengths
- The authors purpose modeling aesthetic preference within contextually related image collections rather than over isolated images.
- They construct ContextAes-40K and ContextAes-Album, two datasets that cover natural photography and image editing and support both training and zero-shot evaluation.
- They propose ContextAes, a Qwen3.5-9B model trained in three stages, and extensive experiments show state-of-the-art accuracy on ContextAes-40K, strong generalization to other aesthetic benchmarks, and excellent zero-shot performance on ContextAes-Album.
Critical weaknesses
- ContextAes-40K is not shown to be an effective training resource beyond the authors' own pipeline, and no non-MLLM model is evaluated on it. Because the dataset is used only to train ContextAes, it remains unclear whether it can support aesthetic preference learning within contextually related image collections in other models.
- Latency and Cost of MLLM-Based Inference ContextAes is built on a 9B parameter MLLM (Qwen3.5-9B) with a three-stage pipeline . In real-time album management, running a 9B model on a smartphone or even a server for thousands of images is computationally expensive compared to traditional aesthetic scorers. The paper does not provide latency benchmarks or on-device deployment tests. While it claims "superior zero-shot performance," it does not address the energy/compute trade-off required for daily use.
Questions for authors
1.What capability or decision does this setting enable that the closely-similar-series setting (FGAesthetics) does not? 2.What is the breakdown of ContextAes-40K by source (images, collections, retained pairs), and how does pairwise accuracy differ across sources, in particular RedNote vs. ETH3D/HPatches?
7 comments
-
W2: Inference efficiency and practical deployment of ContextAes. Thanks for raising this important concern. We acknowledge that directly deploying a 9B-parameter MLLM may introduce considerable computational overhead, particularly for resource-constrained mobile devices. However, our approach is not limited to the 9B model. To explore a more practical deployment solution, we further trained a lightweight 2B-parameter model using the same training methodology and evaluated its performance against larger models. The results are presented below. Both ArtiMuse and Q-Align have been fine-tuned on the same ContextAes-40K training split for a fair comparison.
Model Natural Acc Natural F1 Editing Acc Editing F1 Overall Acc Overall F1 ArtiMuse 67.29% 67.25% 75.92% 75.89% 71.22% 71.19% Q-Align 67.97% 67.93% 77.32% 77.30% 72.23% 72.20% ContextAes (2B) 71.02% 70.77% 75.92% 75.92% 73.25% 73.19% ContextAes (9B) 72.44% 72.44% 78.71% 78.60% 75.30% 75.27% As shown above, the 2B model achieves an overall accuracy of 73.25%, outperforming both fine-tuned ArtiMuse and Q-Align, while maintaining competitive performance compared with the 9B model. Although the smaller model exhibits some performance degradation relative to the 9B version, it provides a more lightweight alternative with a favorable balance between model size and aesthetic preference prediction performance.
To further evaluate its practical deployment feasibility, we have deployed the 2B model on a vivo X500 Pro smartphone and performed on-device inference using its NPU. The model was quantized to W4A16 (4-bit weights and 16-bit computation) and deployed using MediaTek's MTK LLM SDK, with Neuron 10.0.0 serving as the compiler. The measured inference efficiency is summarized below:
Metric Result Average Latency Approximately 0.8 s/sample Inference Power Approximately 2.7 W Estimated Energy per Sample Approximately 2.16 J The measured latency is approximately 0.8 seconds per sample, with an inference power consumption of approximately 2.7 W, corresponding to an estimated energy consumption of 2.16 J per sample.
These results demonstrate that the proposed training methodology can also be applied to smaller MLLMs, enabling a more practical deployment alternative without requiring the 9B model in resource-constrained environments. While the 9B model provides stronger aesthetic preference prediction performance, the 2B variant achieves competitive accuracy and can run directly on a smartphone NPU with low inference latency and power consumption. This provides a feasible trade-off between prediction performance and computational cost for real-world album management applications.
Regarding server-side processing of thousands of images, it is relatively uncommon for a single contextually related collection to contain thousands of images, as such collections are typically associated with specific events, subjects, or contexts. Although a user's entire photo library may contain thousands of images, these images generally span multiple events and contexts and can therefore be organized into smaller contextually related collections before aesthetic preference evaluation. In such cases, the pairwise ContextAes model can be applied to each collection separately.
Nevertheless, for scenarios where an exceptionally large collection needs to be ranked as a whole, we have further developed a score-based variant, ContextAes (Score), to provide a more computationally efficient alternative. Specifically, we attach an MLP-based scoring head to the Qwen3.5-9B backbone and train the model using a pairwise logistic loss. Unlike the pairwise model, which requires autoregressive generation to predict preferences for individual image pairs, the score-based model directly outputs a scalar aesthetic score for each image through the MLP-based head, enabling the ranking of the entire collection based on these scores.
The performance comparison is presented below:
Model Natural Acc Natural F1 Editing Acc Editing F1 Overall Acc Overall F1 ArtiMuse 67.29% 67.25% 75.92% 75.89% 71.22% 71.19% Q-Align 67.97% 67.93% 77.32% 77.30% 72.23% 72.20% ContextAes (Score) 70.35% 70.30% 78.13% 78.12% 73.87% 73.85% ContextAes (Pair) 72.44% 72.44% 78.71% 78.60% 75.30% 75.27%
As shown above, ContextAes (Score) achieves an overall accuracy of 73.87%, outperforming fine-tuned ArtiMuse and Q-Align by 2.65 and 1.64 percentage points, respectively. Although its performance is slightly lower than that of ContextAes (Pair), it provides a more computationally efficient alternative for large-scale image ranking.
To further compare their server-side inference efficiency, we have evaluated ContextAes (Score), ArtiMuse, and Q-Align on a single NVIDIA H20 GPU using the same number of test images and the same maximum input resolution. We tested batch sizes of 1, 4, 8, 16, 32, and 64, with 5 repetitions after 5 warm-up batches. We report the end-to-end (E2E) latency and throughput, together with peak allocated and reserved GPU memory. The results are presented in the following table.
| Batch Size | Model | E2E Latency (ms/img) | E2E Throughput (img/s) | Peak Allocated (GiB) | Peak Reserved (GiB) |
|---|---|---|---|---|---|
| 1 | ContextAes (Score) | 132.20 | 7.56 | 15.73 | 16.02 |
| ArtiMuse | 134.26 | 7.45 | 15.40 | 15.55 | |
| Q-Align | 147.29 | 6.79 | 15.43 | 15.46 | |
| 4 | ContextAes (Score) | 74.83 | 13.36 | 15.84 | 16.26 |
| ArtiMuse | 93.17 | 10.73 | 15.63 | 15.97 | |
| Q-Align | 80.61 | 12.41 | 15.67 | 15.71 | |
| 8 | ContextAes (Score) | 61.33 | 16.31 | 16.00 | 16.53 |
| ArtiMuse | 81.05 | 12.34 | 15.93 | 16.52 | |
| Q-Align | 67.18 | 14.88 | 15.99 | 16.05 | |
| 16 | ContextAes (Score) | 57.19 | 17.49 | 16.32 | 16.79 |
| ArtiMuse | 76.83 | 13.02 | 16.53 | 17.58 | |
| Q-Align | 57.73 | 17.32 | 16.62 | 16.71 | |
| 32 | ContextAes (Score) | 55.19 | 18.12 | 16.96 | 17.95 |
| ArtiMuse | 74.63 | 13.40 | 17.74 | 19.69 | |
| Q-Align | 52.13 | 19.18 | 17.89 | 18.04 | |
| 64 | ContextAes (Score) | 48.90 | 20.45 | 18.24 | 20.12 |
| ArtiMuse | 71.76 | 13.94 | 20.15 | 23.92 | |
| Q-Align | 47.89 | 20.88 | 20.43 | 20.71 |
As shown above, increasing the batch size improves the inference throughput of all three models while reducing the per-image latency. ContextAes (Score) achieves 48.90 ms/image and 20.45 images/s at a batch size of 64, substantially outperforming ArtiMuse in inference speed and achieving throughput comparable to Q-Align. Notably, ContextAes (Score) maintains a relatively low memory footprint as the batch size increases, requiring only 18.24 GiB of peak allocated memory at batch size 64, compared with 20.43 GiB for Q-Align. These results further support its suitability for server-side ranking of exceptionally large image collections.
Overall, these model variants provide complementary deployment options for different application scenarios. The lightweight 2B pairwise model enables on-device inference on smartphones for typical-sized collections, while the 9B pairwise model provides higher prediction accuracy. For exceptionally large collections, ContextAes (Score) offers a more computationally efficient alternative through direct score prediction. Together, these variants provide flexible trade-offs between prediction performance and computational cost for practical album management applications.
The corresponding revisions have been incorporated into Table 15 (Page 31), Table 16 and 17 (Page 32) and Appendix F.2 and F.3 (Page 33, Line 1748--1787), with the modified text highlighted in blue.
To reviewer Qd9F: Thanks for the constructive comments. Responses to specific comments are given below, which have also been incorporated into the revised paper.
W1: Generalizability of ContextAes-40K beyond the proposed model. Thanks for raising this important concern. We agree that the effectiveness of ContextAes-40K should be demonstrated beyond our proposed training pipeline. To this end, we have fine-tuned several models with different architectures on the same ContextAes-40K training split, including conventional visual models (ResNet-50, ViT-B/16, and CLIP-RN50) and aesthetic-specialized models (ArtiMuse and Q-Align). All models have been evaluated on the same held-out test set. The results are presented below:
These results demonstrate that ContextAes-40K supports aesthetic preference learning across different model architectures, rather than being limited to our proposed MLLM-based training pipeline. In particular, conventional visual models can also learn meaningful aesthetic preference representations from the provided training data, while aesthetic-specialized models achieve even stronger performance after fine-tuning. This provides additional evidence of the applicability and utility of ContextAes-40K as a training resource for aesthetic preference evaluation in contextually related image collections.
The corresponding revisions have been incorporated into Table 1 (Page 7) and Appendix E.1.1 (Page 22, Line 1184--1229), with the modified text highlighted in blue.