acceptodds
Under review as a conference paper at ICLR 2027

GOT-Align: Learning Generative Operation Tokenizers for Preference-Aligned Image Enhancement

Abstract

Photo retouching is inherently subjective: the same photograph can admit multiple plausible renditions, while different users may prefer different appearances. Existing methods either learn toward selected expert retouches or rely on explicit controls and instructions, making it difficult to capture implicit and spatially varying preferences. We propose GOT-Align, a preference-adaptive generative photo retouching framework that models retouching transformations in a shared spatial operation space. Specifically, a Generative Operation Tokenizer (GOT) discretizes bilateral affine transformations into reusable local operation tokens, whose spatial compositions represent diverse retouching transformations while preserving the underlying image content. A conditional autoregressive Transformer then models the image-conditioned distribution over operation-token maps to generate multiple plausible candidates for the same input. For preference adaptation, human or vision-language-model judges compare candidates over local image regions, preserving regional preference evidence that would otherwise be collapsed into a single image-level judgment. These regional preferences are attributed to spatially associated operation tokens and used for GRPO-based policy adaptation. Once adapted to a preference source, the model directly generates preference-aligned retouches for new inputs without requiring per-image controls or textual instructions. Experiments on standard photo retouching benchmarks demonstrate diverse generation and effective adaptation to different preference sources.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Photo Retouching, Generative Operation Tokenization, Autoregressive Modeling, Preference Alignment

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_f3BmOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper presents GOT-Align, a generative image enhancement framework that aims to achieve three goals: generating diverse retouching results from a single input image, supporting spatially varying local adjustments, and adapting the generation process to user preferences through comparative feedback.

The proposed method builds upon bilateral affine grids and introduces a VQ-based Generative Operation Tokenizer (GOT) to learn a discrete codebook of reusable retouching operations. An image-conditioned autoregressive Transformer is then trained to model the distribution of these operation tokens, enabling multiple plausible enhancements for the same input. Finally, the authors introduce a localized preference alignment strategy, where human or VLM judges compare generated results at the regional level. The resulting preferences are mapped to corresponding operation tokens and used to fine-tune the autoregressive predictor through GRPO, allowing the model to generate preference-aligned enhancements without explicit editing instructions.

Critical strengths

1. A clear and intuitive framework for preference-aligned image enhancement.

The overall idea is good. Instead of directly generating enhanced images, the method learns reusable local retouching operations and models their compositions. Using regional preference feedback to guide GRPO is also a reasonable idea, as it connects subjective aesthetic judgments with local editing decisions.

2. A compact representation for generative enhancement.

The VQ-based codebook provides a compact way to represent retouching transformations. By modeling spatially varying operations as discrete tokens, the method can reduce the complexity of generation while sharing a common set of operations across different images. This is a promising direction for efficient enhancement, although the actual efficiency gains still need to be demonstrated.

Critical weaknesses

1. The choice of bilateral grids and the diversity of the learned operation space are not well justified.

I am not fully convinced why bilateral grids are particularly suitable for the proposed generative framework. While they are efficient and support spatially varying transformations, the paper does not explain why they are preferable to other retouching representations in this setting.

More importantly, there is little evidence that the learned operation space actually captures meaningful diversity. GOT learns a codebook of 512 tokens, but it is unclear (1) whether these tokens represent genuinely different operations or contain substantial redundancy, (2) whether different token compositions produce perceptually different results rather than just different bilateral grids, and (3) whether these operations can be reused across different images. The authors claim that their representation enables diverse retouching transformations (Section 3.1), but provide neither quantitative diversity analysis nor visualizations of the editing effects of different operation tokens.

I am also concerned about the lack of explicit regularization or analysis of the generated bilateral grids. Large differences in affine coefficients between neighboring grid cells could lead to unnatural local color transitions. Although bilateral slicing provides some smoothing, it does not necessarily guarantee visually coherent results. The paper does not discuss whether such artifacts occur or whether additional spatial regularization is needed.

2. The benefit of localized preference alignment over global preference optimization is not demonstrated.

One of the main contributions is the proposed regional preference alignment, which maps local judgments to operation tokens for more precise credit assignment. However, the experiments do not show why this is better than conventional image-level preference optimization.

Table 3 only compares GOT with GOT after localized GRPO training. This shows that preference optimization helps, but does not isolate the contribution of localized credit assignment. A much simpler approach, such as assigning one global reward to the entire token sequence and applying standard GRPO, might achieve similar or even better results.

I would expect at least two additional baselines: GOT + Global GRPO, using image-level rewards, and GOT + Regional GRPO, where regional preferences are aggregated into a single reward rather than assigned to individual tokens. Without these comparisons, it is hard to tell whether the proposed localization is actually necessary.

Another concern is the assumed correspondence between image regions and operation tokens. Since the output is generated through a learned decoder and bilateral slicing, a single token can potentially affect multiple regions. Simply projecting superpixel preferences onto spatially corresponding tokens does not necessarily give accurate credit assignment. This assumption needs more justification or experimental evidence.

3. GRPO may compromise the generative diversity that the paper aims to achieve.

The paper repeatedly emphasizes that GOT can generate diverse retouching results, but the autoregressive predictor is directly fine-tuned using GRPO during preference alignment. This could push the model toward a small set of high-reward operations, reducing the diversity of the original generative distribution or even causing mode collapse.

This seems particularly important because the authors claim to achieve both diverse generation and preference alignment. However, there is no analysis of how GRPO affects the learned distribution. Surprisingly, the main comparisons also use greedy decoding, which does not really demonstrate the claimed generative capability.

I would like to see multiple sampled results from the same input, both before and after GRPO, along with quantitative diversity measurements. Otherwise, it is difficult to tell whether GOT-Align still produces diverse enhancements or simply learns to generate a narrower set of preferred results.

4. The experimental evaluation is not fully convincing, especially regarding GT comparisons and personalization.

First, both MIT-Adobe FiveK and PPR10K provide expert-retouched ground truths. The authors select Expert C and Expert A as the reference styles for the respective benchmarks, yet none of the qualitative comparisons include the corresponding GT images. The paper also reports no GT-based metrics, such as PSNR or SSIM, for the complete generative model.

I understand that preference-based evaluation is reasonable for a one-to-many enhancement task. However, comparing against expert-specific baselines using only VLM/human preferences may not be entirely fair. For example, if an expert prefers a darker style, a baseline that faithfully reproduces that style might lose simply because the VLM judge prefers brighter images. This does not necessarily mean GOT produces a better result. Showing the GT images and reporting reference-based metrics would make the comparisons more informative.

Second, I do not think the experiments convincingly demonstrate personalized preference learning. Most results reflect general aesthetic preferences from VLMs or a small group of human evaluators, rather than adaptation to different users with distinct preferences. Why not evaluate on the Personalized Photographic Style Dataset (PPSD) from Learning Personalized Photographic Style from Pairwise User Preferences (CVPR 2026)? This dataset contains pairwise preferences from 767 users and seems much more relevant to the proposed task.

As it stands, the experiments provide stronger evidence for general perceptual quality improvement than for actual user-specific preference alignment.

5. Current writing is poor and hard to follow and lacks of relevant works.

The related work discussion is quite limited, especially for preference-based image enhancement. For example, PerTouch: VLM-Driven Agent for Personalized and Semantic Image Retouching (AAAI 2026) is only briefly discussed, while Learning Personalized Photographic Style from Pairwise User Preferences (CVPR 2026), which addresses a closely related problem, is not mentioned. A clearer comparison with these works would help establish what is actually new in GOT-Align.

The Method section also leaves several important details unclear. For example, how exactly are pairwise regional judgments aggregated into preference scores? How are these scores projected onto operation tokens, especially when their spatial effects may overlap? Equation (3) only defines the token-level advantage, without explaining the complete GRPO objective. The architectures of the tokenizer and autoregressive predictor are also insufficiently described.

These are not minor implementation details, as they are directly related to the proposed contributions. The current description makes it difficult to understand the exact optimization procedure or reproduce the method.

Questions for authors

The following experiments would help address my main concerns and could improve my assessment:

  1. Localized vs. Global GRPO (most important). Please compare the proposed method with GOT + Global GRPO and GOT + Regional GRPO with aggregated rewards, under the same training budget. A clear improvement from token-level credit assignment would substantially strengthen the main contribution.

  2. Generative diversity analysis. Please provide quantitative diversity measurements and multiple sampled results before and after GRPO, along with codebook utilization statistics. This would help verify that the learned operations are genuinely diverse and that preference alignment does not lead to mode collapse.

  3. Personalized preference evaluation. Please consider evaluating on PPSD (CVPR 2026) or a similar user-specific preference dataset. Demonstrating adaptation to different users, especially those with conflicting preferences, would make the personalization claim much more convincing.

  4. GT-based comparisons. Please include the expert GT images in qualitative comparisons and report PSNR/SSIM for the complete model alongside preference scores. This would help clarify whether the improvements come at the cost of fidelity to the target expert styles.

Positive results on the first two points, together with stronger evidence of user-specific adaptation, would be particularly important in reconsidering my assessment.

4 comments

Dear Reviewer f3Bm,

We thank the reviewer for recognizing the intuitive design of GOT-Align, the compactness of its operation representation, and the motivation for learning from regional preference feedback.

W1. Bilateral grids, operation diversity, and spatial coherence

We use bilateral grids to model spatially varying retouching operations. We updated Fig. 1 and Sec. 3.1 to clarify the renderer and added Tables 3 and 6 to quantify inference efficiency and sample diversity. We also added Fig. 6 in the appendix to visualize sampling diversity.

Why bilateral grids. An image-adaptive global 3D LUT selects a color mapping for each input, and all locations within that image share the mapping. Following HDRNet, our renderer slices a bilateral grid using each pixel’s spatial coordinates and an input-derived guidance value. This yields per-pixel affine transforms that can adjust the subject and background differently even when their input RGB values are similar. The spatially varying representation lets us model local editing decisions and associate them with regional preference feedback.

Operation diversity and reuse. Table 6, reproduced in W3, shows that sampled token maps use 89.06%–90.23% of the 512-entry codebook. Pairwise LPIPS measures perceptual differences between retouches produced by different token compositions. We train one codebook per dataset to accommodate differences in photometric distributions, such as brightness and saturation. After Stage 1, the codebook is frozen, and the predictor reuses the same operation vocabulary across images within that dataset.

Spatial coherence. In Stage 1, we train GOT end to end using reconstruction MSE on the final rendered image together with the VQ objective. When a local color transition deviates from the training target, the affected pixels contribute to the reconstruction error, and gradients reach the networks that produce the grid and guidance map. Bilateral slicing also interpolates neighboring affine coefficients, which helps reduce abrupt grid-boundary transitions. We did not observe visible grid-boundary artifacts in the outputs we inspected and used no separate spatial smoothness penalty. In Stage 2, GOT remains frozen, and we train the autoregressive predictor with cross-entropy on the operation tokens.

Inference efficiency. The new Table 3 in Sec. 4.2 reports mean latency over 500 images at 340p on an NVIDIA RTX 4090 with a batch size of one. Timing starts with preprocessed inputs and ends when the predicted retouching transform is ready to apply.

Table 3. Inference efficiency

MethodParameters (M) ↓Latency (s) ↓
AceTone3758.272.386
DiffRetouch840.490.816
GOT-Align54.180.237

W2. Localized credit assignment and region-to-token correspondence

We added Table 4 in Sec. 4.3 to test token-level credit assignment against Global GRPO and Regional GRPO, and added Eq. (3) in Sec. 3.2 to specify the many-to-many projection from regions to token positions.

Global GRPO derives one scalar reward per candidate from whole-image judgments. Regional GRPO pools regional judgments into one scalar reward per candidate. Comparing GOT-Align with Regional GRPO tests the value of retaining the spatial origin of the feedback when both methods use regional judgments. For example, if a candidate receives positive feedback for the subject and negative feedback for the background, GOT-Align can retain both signals at different token positions. Its preference rate exceeds 50% against both baselines on both datasets under all three evaluators.

Table 4. Pairwise preference rate (%) for GOT-Align over each scalar-reward baseline

DatasetBaselineQwenGPT-5.6 LunaHMOS
FiveKGlobal GRPO55.854.654.3
FiveKRegional GRPO58.553.555.4
PPR10KGlobal GRPO52.353.456.2
PPR10KRegional GRPO54.354.855.4

We agree that a token position can affect multiple regions. Let κpt\kappa_{pt} be the renderer’s spatial bilinear interpolation weight between pixel pp and token position tt, and let Sm(n)S_m^{(n)} be the signed preference score for candidate nn in region RmR_m. The new Eq. (3) computes the total interpolation weight BmtB_{mt} linking a region to a token position and uses it to obtain a localized score qt(n)q_t^{(n)}.

q_t^{(n)} = \frac{\sum_{m=1}^{M} B_{mt} S_m^{(n)}}{\sum_{m=1}^{M} B_{mt}}. \tag{3}$$ **A region can contribute to several token positions, and a position can receive weighted feedback from several regions.** Opposing regional scores offset each other according to their interpolation weights. The renderer’s interpolation provides a spatial approximation for assigning credit, and Table 4 measures the effect of using this assignment.

W3. Diversity after GRPO and greedy decoding

We added Sec. 4.4 and Table 6 to compare stochastic samples before and after GRPO. GOT-Align retains substantial diversity in both image space and token space under the tested sampling settings.

The main quantitative comparisons use greedy decoding to produce one reproducible output per input, following the single-output protocol used for the predominantly deterministic baselines and avoiding selection among generated candidates. The separate sampling experiment evaluates the model’s one-to-many generation. All five sampled retouches for each evaluated input remain distinct in both image space and token space after alignment.

Table 6. Sample diversity before and after GRPO

MetricFiveK GOTFiveK GOT-AlignPPR10K GOTPPR10K GOT-Align
Pairwise LPIPS0.090.080.060.06
Token difference rate (%)98.6698.5797.5297.43
Codebook utilization (%)90.2389.6589.0689.06
Codebook entropy (bits)8.178.058.278.21
Codebook perplexity287.22264.35309.16296.45

W4. Evaluation objective, personalization, and PPSD

We use blinded pairwise comparisons to evaluate the full model’s outputs, and Table 5 shows that GOT-Align can adapt to the preferences expressed by different feedback sources.

FiveK and PPR10K provide multiple expert retouches per input. As described in Sec. 4.1, we use Expert C on FiveK and Expert A on PPR10K to standardize the comparison setting, since most baselines are deterministic and produce one retouch per input. In our pairwise evaluation, judges assess overall photographic appearance with method identities hidden. GOT-Align achieves 57.1%–66.1% HMOS across the reported baseline comparisons.

In Table 5, each aligned variant is evaluated against the same pre-alignment GOT model. The Qwen-aligned variant receives a higher preference rate under Qwen evaluation, and the human-aligned variant receives a higher HMOS under human evaluation. This pattern is clearest on FiveK. We plan a follow-up experiment to test whether the model can adapt separately to individual users with conflicting retouching preferences.

Table 5. Pairwise preference rate (%) for each aligned variant over pre-alignment GOT

DatasetAlignment feedbackQwenGPT-5.6 LunaHMOS
FiveKQwen66.762.158.8
FiveKHuman61.364.265.3
PPR10KQwen64.464.558.5
PPR10KHuman62.557.958.6

PPSD provides one preference label per whole-image pair. Our localized alignment requires a preference label for each annotated region. A user may prefer the subject in one candidate and the background in another, while a whole-image label records only the overall choice. Using PPSD with our current localized objective would therefore require additional region-level annotations.

W5. Method details and related work

We revised Sec. 3.2 to explain regional score aggregation and added Eqs. (3) and (5) to specify the projection and training loss. We also expanded Sec. 2.1 to discuss PerTouch and Kim et al.’s work on personalized photographic style. Appendix Sections A.4 and A.5 now provide additional details on the model architectures and training.

Each regional pairwise judgment assigns a weighted positive score to the preferred candidate and an equal negative score to the other. An unreported region contributes zero for that comparison. For each candidate, we normalize the accumulated regional scores by the number of successful comparisons involving that candidate. After projecting the scores with Eq. (3), we subtract the mean across the NN candidates at the same token position to obtain the group-relative advantage.

A_t^{(n)} = q_t^{(n)} - \frac{1}{N}\sum_{j=1}^{N} q_t^{(j)}. \tag{4}

The new Eq. (5) applies this localized advantage in the clipped GRPO loss.

\ell_t^{(n)} = -\min\bigl(\rho_t^{(n)} A_t^{(n)}, \operatorname{clip}\bigl(\rho_t^{(n)}, 1 - \epsilon, 1 + \epsilon\bigr) A_t^{(n)}\bigr) + \beta\widehat{D}_t^{(n)}. \tag{5}

Here, ρt(n)\rho_t^{(n)} is the current-to-rollout policy probability ratio, and D^t(n)\widehat{D}_t^{(n)} is the sampled-token KL estimate relative to the frozen pre-alignment policy. Both the policy and KL terms contribute only at positions with nonzero localized advantage.

Sec. 2.1 now explains how PerTouch translates user instructions into semantic-region controls and uses scene-aware memory, and how Kim et al. (2026) learn personalized photographic style from whole-image pairwise preferences. GOT-Align uses region-level comparisons to adapt a generative policy over reusable local operations.

Q1. Localized vs. Global GRPO

The new Table 4 shows that GOT-Align is preferred over both scalar-reward GRPO baselines on both datasets under all three evaluators. The comparison with Regional GRPO is particularly informative because both methods use regional judgments. GOT-Align preserves their spatial assignment to token positions, while Regional GRPO aggregates them into a candidate-level reward.

Table 4. Pairwise preference rate (%) for GOT-Align over each scalar-reward baseline

DatasetBaselineQwenGPT-5.6 LunaHMOS
FiveKGlobal GRPO55.854.654.3
FiveKRegional GRPO58.553.555.4
PPR10KGlobal GRPO52.353.456.2
PPR10KRegional GRPO54.354.855.4

Q2. Generative diversity

The new Table 6 quantifies the diversity retained after GRPO. Fig. 6 in the appendix shows the diversity of samples generated by GOT and GOT-Align.

In Sec. 4.4, we sample five retouches per input from each model using matched random seeds, a temperature of 1.0, and top-pp of 1.0. Pairwise LPIPS shows only a small decrease after alignment, and codebook utilization stays above 89%. All five samples for each evaluated input remain distinct in both image space and token space.

Table 6. Sample diversity before and after GRPO

MetricFiveK GOTFiveK GOT-AlignPPR10K GOTPPR10K GOT-Align
Pairwise LPIPS0.090.080.060.06
Token difference rate (%)98.6698.5797.5297.43
Codebook utilization (%)90.2389.6589.0689.06
Codebook entropy (bits)8.178.058.278.21
Codebook perplexity287.22264.35309.16296.45

Q3. Personalized preference evaluation

Table 5 supports adaptation to different feedback sources. We plan to test individual personalization with users who have conflicting retouching preferences.

Each variant is compared with the same pre-alignment GOT model. The source-specific pattern is clearest on FiveK. The Qwen-aligned variant scores 66.7% vs. 61.3% under Qwen evaluation, and the human-aligned variant scores 65.3% vs. 58.8% under human evaluation. PPSD provides whole-image preference labels. Using it with our current localized alignment objective would require additional region-level annotations.

Table 5. Pairwise preference rate (%) for each aligned variant over pre-alignment GOT

DatasetAlignment feedbackQwenGPT-5.6 LunaHMOS
FiveKQwen66.762.158.8
FiveKHuman61.364.265.3
PPR10KQwen64.464.558.5
PPR10KHuman62.557.958.6

Q4. Ground-truth comparisons

Our task does not assign a single ground-truth retouch to each input because GOT-Align aims to learn the different aesthetic preferences users have for the same photograph. Multiple retouches of a photograph can be valid for users with different preferences. Existing supervised retouching methods typically train on fixed target edits provided by one or a small number of experts.

We use blinded pairwise comparisons to test whether GOT-Align’s outputs better match evaluators’ aesthetic preferences than the outputs of these methods. As shown in Tables 1 and 2, human evaluators preferred GOT-Align’s outputs in 57.1%–66.1% of pairwise judgments across the reported baseline comparisons (HMOS).

Reviewer_f3BmOpenReview ↗

Good rebuttal

Thank you for the detailed rebuttal and the substantial effort put into revising the paper. I appreciate that the authors have carefully addressed my concerns.

However, I still feel that the original submission was too preliminary and lacked sufficient depth. One remaining concern is that the visual differences between GOT and GOTalign are barely noticeable across the figures, making it difficult to appreciate the practical benefits of the proposed alignment.

While the extensive revisions have certainly improved the paper, they also represent a substantial change from the original submission. I would therefore prefer to maintain my rating of Weak Reject and leave it to the AC to judge whether revisions of this scale are appropriate at the rebuttal stage.

I appreciate the authors' efforts and am happy to defer to the AC's final assessment.

Reviewer_xtLiOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposes GOT-Align, a generative photo-retouching framework designed to produce diverse image enhancements aligned with human or VLM preferences. The authors introduce a Generative Operation Tokenizer (GOT) that represents bilateral affine transformations using a shared vocabulary of discrete local operation tokens. A conditional autoregressive Transformer models the distribution of these tokens to generate multiple retouching candidates. To incorporate preferences, the authors propose a localized preference-alignment strategy that maps region-level comparative feedback onto corresponding operation tokens and optimizes the generative policy using GRPO. Experiments on MIT-Adobe FiveK and PPR10K demonstrate competitive performance against existing retouching methods and improvements following preference alignment.

Critical strengths
  1. The idea is novel and interesting.
  2. The breadth of the baseline comparison is good. Also the authors have reported the VLM-as-a-judge evaluation along with the Human evaluation (10 participants), these two things also helped to understand the improvements across two different evalautor settings.
Critical weaknesses
  1. Poorly written Related Work - Try to include more works related to RLHF, DPO, KTO in Related work section. Although you are not doing experiments for those methods but considering that Preference alignment based optimization is the key theme of the paper, the relevant works should be cited.

  2. In Abstract authors are saying - '''Specifically, a Generative Operation Tokenizer (GOT) .........while preserving the underlying image content.''' - To validate this claim authors need to report additional content-preservation evaluation.

  3. Insufficient ablation studies - The major weakness of the experiment section is lack of relevant ablations. The pipeline contains multiple components, but the reported ablation essentially focuses on comparing the aligned and unaligned models. Other design choices are not systematically evaluated. Some important ablations can be- Codebook size, Token-map resolution, Number of superpixels, Number of sampled candidates, Alternative region-to-token projections.

  4. Unclear distinction between subjective preferences and perceptual image quality - In Introduction, authors are motivating GOT-Align through the subjective nature of photo retouching, emphasizing that different users may prefer different appearances. However, the VLM preference annotation protocol in Appendix A.1 primarily focuses on clear improvements in perceptual quality rather than purely stylistic preferences. This raises the question of whether the proposed framework genuinely learns diverse user-specific aesthetic preferences or mainly optimizes general image quality. I think the authors should clarify this distinction and ideally provide experiments demonstrating adaptation to different, potentially conflicting user preferences.

  5. Insufficient methodological details and reproducibility - Authors need to add more experimental details to make the experiments reproducible.

  6. Limited experimental evaluation beyond pairwise preferences. - Authors are trying to evaluate the human preference using VLMs and Human judge. I think they need to include more metrics that models the human preference such as ImageReward, HPSv1, HPSv2, Laion-asthetic-predictor, Pickscore. Because VLM as a judge and Human judges are also not full proof metrics, so adding these automatic metric will help to cover the more dimensions of the evaluation.

  7. Insufficient evidence for computational efficiency. - The authors are critisizing the other heavy architectures for their computational cost and inference latency but they themselves have not reported the numbers to support this claim. A quantitative comparison of latency, computational requirements, and model size would strengthen this claim.

  8. Human Evaluation Reliability - For Human evaluation, the authors should explicitly report the number of judgments per comparison, participant-level variability, confidence intervals, and the precise HMOS aggregation formula.

Questions for authors

Considering the insufficient experimental validation as one of the major limitations of the current paper, I would appreciate it if the authors could address the concerns highlighted in the weaknesses section, particularly those related to the ablation studies and experimental evidence supporting the key claims. I am open to reconsidering my score if the authors provide convincing evidence or clarifications that adequately address these concerns during the rebuttal.

5 comments

Dear Reviewer xtLi,

Thank you for recognizing the novelty of GOT-Align, the breadth of our baseline comparisons, and the value of evaluating the results with both VLMs and human participants.

W1. Related work

Thank you for pointing out that our discussion of preference alignment was incomplete. We expanded Sec. 2.2 to cover foundational work on preference-based reinforcement learning, RLHF, DPO, and KTO. The added discussion includes the following passage.

Early studies of preference-based deep reinforcement learning trained reward models from human comparisons to guide policy optimization (Christiano et al., 2017). Reinforcement learning from human feedback (RLHF) was subsequently used to align language models with human intent (Ouyang et al., 2022). To simplify the reward-learning and policy-optimization pipeline, Direct Preference Optimization (DPO) directly optimizes policies from preference pairs without a separately trained reward model (Rafailov et al., 2023). KTO explores a different feedback setting, learning from binary desirability judgments without requiring paired preferences (Ethayarajh et al., 2024).

W2. Content preservation

We updated Fig. 1 and Sec. 3.1 to show how GOT preserves the input photograph’s spatial layout by applying local color transforms at the original pixel coordinates. We also added Fig. 6 to Appendix B to illustrate the diversity of sampled retouches.

GOT decodes sampled operation tokens into bilateral grids containing local affine color transforms. Following HDRNet (Gharbi et al., 2017), we slice each grid using pixel coordinates and a guidance map derived from the input. We then apply the resulting coefficients to the input RGB values at those same coordinates.

Appendix B provides complementary evidence on the representation’s ability to reproduce expert retouches. Given an input–retouch pair, GOT reconstructs the target retouch with 34.98 dB PSNR and 0.9507 SSIM on FiveK and 37.31 dB PSNR and 0.9825 SSIM on PPR10K. These metrics quantify reconstruction fidelity to expert-retouched targets.

W3. Design choices and ablations

We added Table 4 in Sec. 4.3 to test token-level reward assignment and Table 6 in Sec. 4.4 to report codebook utilization and sample diversity before and after GRPO.

Our 512-entry codebook has 89.06%–90.23% utilization across GOT and GOT-Align on FiveK and PPR10K. We compute utilization from token frequencies pooled across test inputs and five sampled retouches per input. Both models use most of the entries at the selected codebook size.

Table 6. Sample diversity before and after GRPO

MetricFiveK GOTFiveK GOT-AlignPPR10K GOTPPR10K GOT-Align
Pairwise LPIPS0.090.080.060.06
Token difference rate (%)98.6698.5797.5297.43
Codebook utilization (%)90.2389.6589.0689.06
Codebook entropy (bits)8.178.058.278.21
Codebook perplexity287.22264.35309.16296.45

We use the HDRNet-style 16×16×8×1216\times16\times8\times12 bilateral grid, with a 16×1616\times16 spatial grid, eight guidance bins, and 12 affine coefficients per cell. The 16×1616\times16 operation-token map matches the spatial dimensions of the decoded grid, allowing us to reuse the renderer’s interpolation weights to project regional scores onto token positions in Eq. (3). We empirically selected approximately 40 SLIC superpixels per image and five sampled candidates per input. We will run follow-up ablations on these two settings.

Global GRPO derives one scalar reward per candidate from whole-image judgments. Regional GRPO pools regional judgments into one scalar reward per candidate. GOT-Align maps regional scores to their corresponding token positions. The comparison with Regional GRPO tests the value of preserving spatial correspondence when both methods use regional judgments. GOT-Align’s preference rate exceeded 50% against both baselines on both datasets under all three evaluators.

Table 4. Pairwise preference rate (%) for GOT-Align over each scalar-reward baseline

DatasetBaselineQwenGPT-5.6 LunaHMOS
FiveKGlobal GRPO55.854.654.3
FiveKRegional GRPO58.553.555.4
PPR10KGlobal GRPO52.353.456.2
PPR10KRegional GRPO54.354.855.4

W4. Subjective preferences and perceptual quality

Table 5 shows that GOT-Align adapts to the preferences of its feedback source while also improving perceived retouching quality across evaluators.

Each aligned variant is compared with the same pre-alignment GOT model. On both datasets, Qwen gives the Qwen-aligned variant the higher preference rate, and human evaluators give the human-aligned variant the higher HMOS. The pattern is clearest on FiveK. Under Qwen evaluation, the Qwen-aligned variant scores 66.7% versus 61.3%. Under human evaluation, the human-aligned variant scores 65.3% versus 58.8%. This pattern supports adaptation to the preferences expressed by the feedback source.

GPT-5.6 Luna, which provided no alignment feedback, also prefers both aligned variants over GOT on both datasets. All three evaluators give both variants preference rates above 50%. This agreement indicates that preference alignment can also improve perceptual quality in ways that are recognized across evaluators.

Table 5. Pairwise preference rate (%) for each aligned variant over pre-alignment GOT

DatasetAlignment feedbackQwenGPT-5.6 LunaHMOS
FiveKQwen66.762.158.8
FiveKHuman61.364.265.3
PPR10KQwen64.464.558.5
PPR10KHuman62.557.958.6

W5. Reproducibility

Thank you for pointing out where more implementation detail would improve reproducibility. We revised Sec. 3.1 to clarify the rendering pipeline and added Eqs. (3) and (5) in Sec. 3.2 to specify the regional score projection and the token-level GRPO loss. Sec. 4.1 reports the main training settings, and Appendices A.4 and A.5 provide details on the model architecture and training, respectively.

Sec. 3.1 describes how GOT is trained with image-space reconstruction and vector quantization. The new Eq. (3) uses the renderer’s spatial interpolation weights to project regional preference scores onto operation-token positions. Let RmR_m be superpixel mm, κpt\kappa_{pt} the interpolation weight linking pixel pp to token position tt, and Sm(n)S_m^{(n)} the signed preference score of candidate nn in region RmR_m. We compute the total interpolation weight BmtB_{mt} for each region and use it to obtain the localized score qt(n)q_t^{(n)}.

\tag{3}$$ Eq. (4) subtracts the mean score across candidates at the same token position to obtain the group-relative advantage $A_t^{(n)}$. **The new Eq. (5)** then uses this advantage in the clipped GRPO loss with KL regularization. $$\ell_t^{(n)} = -\min\left(\rho_t^{(n)} A_t^{(n)}, \operatorname{clip}\left(\rho_t^{(n)}, 1-\epsilon, 1+\epsilon\right) A_t^{(n)}\right) + \beta \widehat{D}_t^{(n)}. \tag{5}$$ Here, $\rho_t^{(n)}$ is the ratio of the current policy’s probability to the rollout policy’s probability for the sampled token, and $\widehat{D}_t^{(n)}$ is the sampled-token KL estimate relative to the frozen pre-alignment policy. The parameters $\epsilon$ and $\beta$ control clipping and KL regularization, respectively. **Both the policy and KL terms contribute only at token positions with nonzero localized advantage.** We train GOT and the autoregressive predictor with AdamW and a batch size of 16 for **300 and 100 epochs**, respectively, using learning rates of $2\times10^{-4}$ and $3\times10^{-4}$. GRPO uses a learning rate of $10^{-5}$ with the operation space frozen and runs for **250 steps with Qwen feedback and 50 steps with human feedback.**

W6. Automatic preference metrics

ImageReward, HPS v1/v2, and PickScore were developed for text-to-image (T2I) preference scoring. Most of these metrics require a generation prompt to assess text–image alignment. Our benchmark starts with a source photograph and provides no generation prompt. Applying the prompt-conditioned metrics would require an auxiliary text prompt, and their scores would still provide no direct comparison with the source photograph. We therefore used the prompt-free LAION Aesthetic Predictor V1 to compare retouching results across methods on both datasets and across all five FiveK experts.

LAION Aesthetic Predictor V1 can score the outputs directly. As shown below, the mean scores differ by at most 0.15 across methods on FiveK and 0.04 on PPR10K. The five expert retouches on FiveK also receive similar mean scores, spanning only 0.09. We also consider this metric unsuitable for photo retouching.

MIT-Adobe FiveK. LAION Aesthetic Predictor V1 score

MethodAdaIntLUTwithBGridBPAMGOTGOT-Align
Score ↑5.585.615.505.655.65
ExpertExpert AExpert BExpert CExpert DExpert E
Score ↑5.605.615.655.655.69

PPR10K. LAION Aesthetic Predictor V1 score

MethodAdaIntLUTwithBGridBPAMDiffRetouchGOTGOT-Align
Score ↑6.196.206.176.166.206.19

W7. Inference efficiency

We added Table 3 in Sec. 4.2 to compare inference parameter counts and latency with AceTone and DiffRetouch. GOT-Align uses 54.18M inference parameters and takes 0.237 s per image on average.

We measured all methods on 500 images at 340p using an NVIDIA RTX 4090 and a batch size of one. Timing starts with the preprocessed input and ends when the predicted retouching transform is ready to apply.

Table 3. Inference efficiency

MethodParameters (M) ↓Latency (s) ↓
AceTone3758.272.386
DiffRetouch840.490.816
GOT-Align54.180.237

W8. Human evaluation reliability

We report the number of judgments, the exact HMOS aggregation rule, between-participant variability, and a summary of the per-comparison 95% confidence intervals below. We will incorporate these details into Appendix A.3.

Ten participants each judged 200 matched image pairs per method comparison, yielding 10 judgments per pair and 2,000 judgments per comparison. Method identities were hidden, left–right order was randomized, and participants could select either retouch or indicate a tie.

For methods A and B, let WAW_A and WBW_B be the numbers of judgments favoring A and B, pooled across participants and image pairs. We exclude ties and compute HMOS for A as follows.

HMOSA=100×WAWA+WB.\mathrm{HMOS}_A=100\times\frac{W_A}{W_A+W_B}.

For each method comparison, we compute the standard deviation of preference rates across participants and a 95% confidence interval for HMOS. The table reports the mean between-participant standard deviation and the averages of the lower and upper interval bounds across method comparisons on each dataset.

DatasetMean between-participant SD (percentage points)Mean lower bound of 95% CIsMean upper bound of 95% CIs
MIT-Adobe FiveK5.3755.86%66.38%
PPR10K5.2954.79%65.28%
Reviewer_JBAbOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper discretizes bilateral affine transforms into local operation tokens, uses an autoregressive Transformer to generate diverse retouching candidates, and aligns them with regional human/VLM preferences via GRPO reinforcement learning for instruction-free retouching.

Critical strengths

S1. Bridges continuous spatial editing with discrete Transformer modeling. S2. Uses fine-grained regional feedback instead of noisy global scores.

Critical weaknesses

W1. Heavy modular stitching and lack of fundamental innovation: The overall framework primarily stacks and transfers existing techniques (GOT discretization + autoregressive Transformer + GRPO reinforcement learning). It lacks fundamental underlying algorithmic innovations, giving a strong impression of a piecemeal system with insufficient technical contribution.

W2. Unclear regional mechanics: The paper fails to clarify how image regions are specifically partitioned and how potential preference conflicts across overlapping regions are resolved.

Questions for authors

Expressiveness bottleneck and judge dependency: While bilateral affine transformations preserve image content well, they cannot perform complex semantic edits. Furthermore, the framework relies heavily on the evaluation accuracy of VLM or human judges; spatial ambiguity or bias in regional ratings can directly undermine policy optimization.

3 comments

Dear Reviewer JBAb,

Thank you for highlighting how GOT-Align connects continuous spatial editing with discrete Transformer modeling and uses fine-grained regional feedback.

W1. Technical contribution

The core of GOT-Align is a lightweight generative retouching model that learns a distribution tailored to each user’s aesthetic preferences by mapping their region-level pairwise feedback to the operation tokens that affect those regions, then samples diverse, personalized retouches from that distribution using only the input photo at inference time.

A single photograph can admit several visually appealing retouches. Collecting paired expert retouches is labor-intensive, and training on one target per image captures only one of these plausible renditions. Generative methods trained on collections of high-quality images can produce diverse results, but those collections provide limited guidance about how a particular user would like different regions of a given photograph to look.

GOT-Align learns from choices users can make directly. During preference alignment, a user only needs to choose which of two candidates looks better in a given region. For example, the user may prefer the sky in candidate A and the skin tones in candidate B. Regional comparisons preserve both judgments, which a single whole-image preference would collapse into one. The same framework can also learn from regional comparisons provided by a VLM.

The technical contribution is how regional preferences are assigned to the policy’s operation-token decisions. GOT represents a retouch as a spatial composition of reusable local operations. GOT-Align uses the renderer’s interpolation weights to map each regional judgment to the token positions that influence that region. Different operations within the same candidate can therefore receive different GRPO advantages. Once aligned, the model can sample diverse retouches guided by the learned preferences from a new input photograph alone.

We added Table 4 in Sec. 4.3 to test the effect of this token-level assignment. Global GRPO derives one scalar reward per candidate from whole-image judgments. Regional GRPO uses regional judgments and pools them into one scalar per candidate. The comparison with Regional GRPO tests the value of preserving the spatial correspondence between regional judgments and operation tokens. GOT-Align’s preference rate exceeded 50% against both baselines on both datasets under all three evaluators.

Table 4. Pairwise preference rate (%) for GOT-Align over each baseline

DatasetBaselineQwenGPT-5.6 LunaHMOS
FiveKGlobal GRPO55.854.654.3
FiveKRegional GRPO58.553.555.4
PPR10KGlobal GRPO52.353.456.2
PPR10KRegional GRPO54.354.855.4

We added Table 3 in Sec. 4.2 to quantify inference efficiency. GOT-Align predicts a 16×1616 \times 16 operation-token map and decodes it into a bilateral grid. On an NVIDIA RTX 4090 at 340p with a batch size of one, GOT-Align took 0.237 s per image on average over 500 images. We measured latency from preprocessed input to a retouching transform ready to apply.

Table 3. Inference efficiency

MethodParameters (M) ↓Latency (s) ↓
AceTone3758.272.386
DiffRetouch840.490.816
GOT-Align54.180.237

We also added Table 6 in Sec. 4.4 to evaluate diversity after preference alignment. GOT-Align retains substantial diversity. For every evaluated input, all five sampled retouches remained distinct in both image space and operation-token space.

Table 6. Sample diversity before and after GRPO

MetricFiveK GOTFiveK GOT-AlignPPR10K GOTPPR10K GOT-Align
Pairwise LPIPS0.090.080.060.06
Token difference rate (%)98.6698.5797.5297.43
Codebook utilization (%)90.2389.6589.0689.06
Codebook entropy (bits)8.178.058.278.21
Codebook perplexity287.22264.35309.16296.45

W2. Region partitioning and conflicting feedback

Sec. 3.2 already describes the shared SLIC partition, and we revised it by adding Eq. (3) to explain how we combine conflicting regional scores at a shared token position and Eq. (5) to specify how the resulting advantages enter the GRPO loss.

We apply SLIC once to each input photograph and use the same partition for all its retouching candidates. Each pixel belongs to exactly one superpixel, and we use approximately 40 superpixels per image (Sec. 4.1). A single token position can influence pixels in multiple disjoint superpixels through the renderer’s spatial bilinear interpolation. This is how it can receive feedback from regions with opposing preferences.

Let RmR_m denote the mm-th superpixel, κpt\kappa_{pt} the interpolation weight linking pixel pp to token position tt, and Sm(n)S_m^{(n)} the signed preference score for candidate nn in region RmR_m. The new Eq. (3) sums the pixel-level interpolation weights within each region and uses those sums to compute the weighted score at position tt.

\qquad q_t^{(n)} = \frac{\sum_{m=1}^{M} B_{mt} S_m^{(n)}}{\sum_{m=1}^{M} B_{mt}}. \tag{3}$$ Opposing regional scores **partially or fully offset each other in this weighted average**. Eq. (4) then subtracts the mean across the $N$ candidates at the same token position to obtain the group-relative advantage. $$A_t^{(n)} = q_t^{(n)} - \frac{1}{N}\sum_{j=1}^{N} q_t^{(j)}. \tag{4}$$ **The new Eq. (5)** shows how this localized advantage enters the clipped GRPO loss. $$\ell_t^{(n)} = -\min\left(\rho_t^{(n)} A_t^{(n)}, \operatorname{clip}\left(\rho_t^{(n)}, 1-\epsilon, 1+\epsilon\right) A_t^{(n)}\right) + \beta \widehat{D}_t^{(n)}. \tag{5}$$ Here, $\rho_t^{(n)}$ is the current-to-rollout policy probability ratio, and $\widehat{D}_t^{(n)}$ is the sampled-token KL estimate relative to the frozen pre-alignment policy. **Both the policy and KL terms contribute only at candidate-token positions with nonzero localized advantage.** ### **Q1. Expressiveness of bilateral affine transformations** **Our task is content-preserving photo retouching, which excludes complex semantic edits, and we updated Fig. 1 and Sec. 3.1 to show how the HDRNet-style bilateral renderer implements the spatially varying photometric adjustments needed for this task.** Photo retouching adjusts exposure, contrast, saturation, color balance, and tone while preserving the photograph’s content and structure. The updated **Fig. 1** shows how different samples of operation tokens produce different bilateral affine grids. Following HDRNet, we slice each grid using pixel coordinates and an input-derived guidance map, then apply the resulting affine coefficients to the original pixel colors. This allows the model to adjust the sky tone and the subject exposure separately. Object insertion, object removal, and scene restructuring fall outside the task considered here. ### **Q2. Dependence on human or VLM judgments** **GOT-Align aims to adapt to the aesthetic preferences of different users. To limit ambiguous VLM feedback during alignment, our annotation protocol lets the judge skip regions where it cannot identify a clear preference (Appendix A.1).** The shared SLIC partition gives judges a consistent spatial reference across candidates, so they compare the same region in both retouches. For VLM annotation, we ask the judge to report only regions with a clear local preference. **The VLM can skip individual regions or abstain on the entire pair.** An unreported region contributes zero to that comparison (Sec. 3.2), so an uncertain judgment does not become a forced training label. **Table 5** shows how the same framework adapts to human and Qwen feedback. We compare each aligned variant with the **same pre-alignment GOT model** under Qwen, GPT-5.6 Luna, and human evaluation. **Table 5. Pairwise preference rate (%) for each aligned variant over pre-alignment GOT** | Dataset | Alignment feedback | Qwen | GPT-5.6 Luna | HMOS | | --- | --- | --- | --- | --- | | FiveK | Qwen | 66.7 | 62.1 | 58.8 | | FiveK | Human | 61.3 | 64.2 | 65.3 | | PPR10K | Qwen | 64.4 | 64.5 | 58.5 | | PPR10K | Human | 62.5 | 57.9 | 58.6 | Both aligned variants achieved preference rates above 50% over GOT on both datasets under all three evaluators. On FiveK, **the Qwen-aligned variant scored higher under Qwen evaluation (66.7% vs. 61.3%), and the human-aligned variant scored higher under human evaluation (65.3% vs. 58.8%)**. This pattern is consistent with GOT-Align adapting to the aesthetic preferences expressed by each feedback source.

Dear Reviewer JBAb,

I hope this message finds you well. As the discussion period draws to a close, I wanted to ensure we have addressed all your concerns satisfactorily. If there are any additional points or feedback you'd like us to consider, please let us know. Your insights are invaluable to us, and we're eager to address any remaining issues to improve our work.

Thank you for your time and effort in reviewing our paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.