- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates personalized photo color grading based on pairwise preferences and introduces the PrefLUT method. Users provide pairs of "preferred" and "non-preferred" images, which the system encodes into a reusable user profile. Subsequently, for any new image, the model utilizes this profile to predict a 3D LUT for image editing, eliminating the need for additional optimization or fine-tuning for individual users. Additionally, the paper proposes the PCVP to evaluate the effectiveness of personalized image enhancement and color grading. Comprising five controlled tests that assess quality improvements based on target user preferences and specific query images, this verification protocol establishes a rigorous standard for evaluation in the field of personalization technology.
Critical strengths
Unlike previous methods requiring per-user fine-tuning, PrefLUT introduces an approach that converges to individual user preferences without fine-tuning, thereby enhancing personalization capabilities. Additionally, the PCVP mechanism was designed to verify the differences between individual and group preferences.
Critical weaknesses
-
Lack of user studies, incomplete description of the deployment workflow, and a lack of direct evidence supporting core claims. The paper's key selling point is a personalized editing framework that is reusable and continuously updateable via new feedback; however, it fails to explain how user preferences are generated in practice. For instance, where do the candidate images presented for binary choice come from, for example, random LUTs, preset filters, or the model's own outputs? If candidates are model-generated, are user preferences constrained by the model's output range? More critically, all "feedback-based update" experiments (Fig. 4, Table F1) rely on offline simulations using pre-collected PPSD data; there is no actual user participation in the "selection → profile update → re-editing" closed loop. The feedback pathway involving "user-confirmed manual adjustments" mentioned in the paper lacks any experimental validation. Furthermore, all conclusions regarding personalization rely on proxy metrics like CQS without any human evaluation. Given the marginal gains observed in PCVP (e.g., ΔE00-CQS improvements of ~0.002–0.007), user studies are essential to determine whether these differences are perceptually distinguishable. For a study centered on "personalization," this constitutes a significant evidentiary gap.
-
Personalization claims lack independent verification and are validated on only a single dataset. The primary evidence that the model "truly utilizes user preferences" is the authors' own PCVP metric. Since the PrefLUT training objective , Wrong-User Contrast Loss, directly corresponds to the "Wrong-User" test within PCVP, the claim that the method "uniquely passes PCVP" carries limited weight. It is recommended to include simple yet robust baselines independent of PCVP, such as applying the average changes in brightness, saturation, and color temperature (derived from the "preferred vs. non-preferred" image pairs) to new images, or fitting a single average LUT for each user. If PrefLUT offers only marginal advantages over these methods, the necessity of its complex structure is questionable. Regarding datasets, all personalization experiments were conducted solely on PPSD. The experiments on FiveK and PPR10K utilized data from the target experts directly for training; as such, they represent supervised enhancement for fixed targets rather than learning new users from limited preference data, and thus cannot serve as a secondary personalization test set. It is recommended to adopt an "unseen-expert" protocol by treating the five experts as distinct users and reserving experts not involved in training for testing purposes. The published baselines in Table E2(A) employ this protocol, allowing for direct comparison.
Questions for authors
Mentioned in Weaknesses. User studies are essential for this work, and more verification protocols & datasets are needed.
Thank you for recognizing PrefLUT's personalization and PCVP. We apologize for the late reply, as additional experiments and the user study took time.
(1) User preferences in practice (W1): A user-confirmed edit and its pre-edit image form a preferred/non-preferred pair, so explicit binary selection is optional. This is mentioned in lines 77-78 and 190-197 and reflected in our new user study. Preference pairs need not come from a model. PPSD experiments use pre-collected preferred/non-preferred pairs. In our user study, participants make their own choices on candidates from our PPSD validation-user split and provide manual feedback by adjusting PrefLUT's output (Appendix G, lines 1921-1924 and 1995-2013). Thus, preference collection is not restricted to what a model generates.
(2) Selection → profile update → re-editing (W1): Our new user study completes this process (Appendix G.2, Table G1, lines 1987-2013 and 1999-2008). Users make new choices, PrefLUT updates their profiles with frozen weights, and users judge edits on unseen photographs. Users prefer the updated profiles' outputs over their initial profiles' outputs. We also add matched feedback accumulation with UPE (Appendix F.1, Table F1, lines 1707-1721 and 1729-1774), where PrefLUT shows stronger overall refinement gains and benefits earlier from new feedback. These complementary evaluations show that new choices improve later editing on other images, validating both the practical feedback loop and its effectiveness.
(3) User-confirmed manual adjustments (W1): We add a user study in which participants adjust exposure, contrast, saturation, and temperature and confirm their preferred result (Appendix G.2, Table G1, lines 1987-2013 and 1999-2008). The edited/pre-edit pair updates the profile as defined in Section 3.1 (lines 190-197). On unseen images, manually refined PrefLUT achieves 74.2% wins, 23.3% ties, and 2.5% losses against the initial profile, with all network weights frozen. Thus, users' manual edits improve later automatic editing, providing direct validation of this feedback route.
(4) Perceptually distinguishable differences (W1): We add a blinded user study with 10 participants and 600 judgments across five comparisons, 120 per comparison (Appendix G, Table G1, lines 1921-1939 and 1999-2008). PrefLUT achieves 50.8% wins, 30.0% ties, and 19.2% losses against UPE, and 45.0% wins, 20.0% ties, and 35.0% losses against StarEnhancer. Personal profiles achieve 41.7% wins, 39.2% ties, and 19.2% losses against training-mean profiles. Users more often prefer PrefLUT over both baselines and their personal profile over a population average. This directly supports perceptible personalization benefits without relying on CQS alone.
(5) Independent verification and Wrong-User Contrast Loss (W2): We add an experiment removing the Wrong-User Contrast Loss throughout PrefLUT's entire training chain (Appendix F.2.1, Table F3, lines 1837-1849). PrefLUT still passes all five PCVP tests on all four metrics, exceeding all compared baselines in tests passed. Therefore, full PCVP does not depend on training with this loss.
(6) Average changes and a single average LUT (W2): Following your suggestion, we add average color adjustments and fixed per-user LUTs fitted only on reference pairs (C.2.1, Table E5, lines 1125-1131 and 1621-1644). PrefLUT leads all four CQS and all four direct metrics. Its PSNR is 22.093 dB, versus 20.732 for simple color adjustment and 18.043 for the fixed LUT fitted with the shared PPSD objective. LPIPS also improves to 0.08880 from 0.10095 and 0.25750, respectively. Together with the aggregation and feature-difference ablations (Table 3, lines 433-442), these gains support learning how each user's preferences should guide each image, rather than applying one fixed edit.
(7) Supervised enhancement for fixed targets and an unseen-expert protocol (W2): FiveK and PPR10K provide retouched targets from a small number of experts. Following your suggestion, we add FiveK evaluation that trains on four experts and tests the held-out expert, repeated for all five experts, with 16 references and 500 unseen images (E.2.3, Table E4, lines 1561-1588). Correct profiles outperform wrong-expert and reversed profiles on all five averaged metrics. The training-expert mean is slightly stronger in average fidelity than the held-out expert profile. These expert-target datasets do not directly collect users' pairwise preferences, and the same expert can adjust editing style according to each image's content and intended appearance. PPSD instead directly records many users' preference choices, better reflecting real-world use. PrefLUT demonstrates effective personalization on this dataset. It passes all five PCVP tests on users excluded from training, and our user study further supports its ability to match real participants' preferences. We interpret these findings as support for learning personalization from diverse users' expressed preferences.