- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes ToPO, a reweighting strategy of the Diffusion-DPO loss for text-to-image diffusion models. Specifically, it first uses the difference between the frozen reference model's local noise-prediction errors on the two images, together with the cross-attention map of the preferred image, to construct a content-aware spatial weight map and a temporal weight. The spatial and temporal weights are then integrated into the Diffusion-DPO loss, together with an auxiliary ordering loss against the encoded pixel average of the two images. Empirical results on SD-1.5 and SDXL, trained on Pick-a-Pic for 500 updates, show improvements over Diffusion-DPO and other released checkpoints on PickScore, HPSv2, ImageReward, GenEval and T2I-CompBench, together with a 16-participant user study on SDXL. Additionally, ablation studies are conducted on the weight components and the preference scale .
Critical strengths
- The method does not require extra annotations or a reward model. It utilizes the frozen reference model.
- The paper includes sensible controls for the weighting (uniform and shuffled weights) and a matched training run against Diffusion-DPO (Figure 5).
Critical weaknesses
-
The novelty is limited, and the method and its claims are not well supported:
i. (Novelty) Spatial credit assignment for diffusion preference learning (PatchDPO, IAPO, BiDPO), step-aware weighting over denoising time (e.g., SPO, LPO), and cross-attention as a token-to-region link (Prompt-to-Prompt, Attend-and-Excite) are all established, and STAR already allocates credit over both space and time. Spatially reweighted DPO losses also exist in related settings and are not discussed: FocusDPO weights the DPO objective toward focus regions that change across noise levels, and LocalDPO restricts the DPO loss to masked regions. ToPO combines these ideas into a heuristic reweighting.
ii. (Soundness-1) The method is a stack of heuristics (four fixed anchors, a 50/50 prior mix, a Gaussian time kernel, attention averaging, special-token masking and a pixel-midpoint triplet term), with no derivation of why the error gap should be the right allocation, and the pair-dependent weights remove the single implicit reward that justifies DPO. The allocation signal itself, the absolute difference of reference errors, is label-invariant and shows where the reference model treats the two images differently, not why one image is preferred. The appendix propositions (SGD convergence, law of large numbers, a triangle-inequality bound) are standard and do not support the method.
iii. (Soundness-2) The timestep factor also appears to follow the scale of the noise-prediction error rather than the motivation: in the paper's own example (Figure 11), is about at and about 0.02 at , so almost no weight goes to the high-noise steps where the introduction says layout is decided.
iv. (Significance) The multi-method tables compare released checkpoints with different initializations, data and compute, which the paper itself calls descriptive, yet the abstract claims state-of-the-art performance. On SDXL the gains are small or mixed: in the matched study Diffusion-DPO is higher on PickScore and Aesthetic (Table 5), on PartiPrompts ToPO is below Diffusion-DPO on ImageReward and CLIP (Table 7), and on HPDv2 InPO has a higher HPSv2 (Table 9).
-
The matched comparison with Diffusion-DPO is inconsistent and appears to overstate the gain. In Figure 5, the matched Diffusion-DPO run reaches 21.344 on PickScore, 0.2862 on HPSv2 and 0.718 on ImageReward at 500 updates, against 21.542, 0.2903 and 0.812 for ToPO. However, the matched three-seed Diffusion-DPO in Table 5 reports 20.930, 0.2587 and 0.250, which are almost identical to the released Diffusion-DPO checkpoint in Table 1 (20.947, 0.2586, 0.250) and well below the SFT-winners initialization both methods supposedly start from (21.181, 0.2824, 0.642). The ImageReward spread across three training seeds is also 0.001 on SD-1.5 and 0.000 on SDXL, which shows unusually little variation for independent training runs. The paired gains in Table 5 (e.g., +0.563 ImageReward) are therefore several times larger than the matched gain in Figure 5 (+0.094). Likewise, the main-text claim that ImageReward increases from 0.250 to 0.812 compares ToPO, initialized from SFT-winners, with a released Diffusion-DPO checkpoint trained from the base model, so most of that gap comes from the initialization.
[FocusDPO] Jin et al. "FocusDPO: Dynamic Preference Optimization for Multi-Subject Personalized Image Generation via Adaptive Focus." AAAI 2026.
[LocalDPO] Huang et al. "Mind the Generative Details: Direct Localized Detail Preference Optimization for Video Diffusion Models." CVPR 2026.
Questions for authors
- How were the matched Diffusion-DPO numbers in Table 5 obtained, and why do they differ from the matched run in Figure 5 and coincide with the released checkpoint?
- What is the gain of ToPO over Diffusion-DPO trained from the same initialization with the same budget, over several seeds, on all reported benchmarks?
- What is the training cost per update of ToPO compared with Diffusion-DPO?
- Since count and spatial tokens are not up-weighted (Table 10), what explains the large counting gain on GenEval?
- What is the full protocol of the user study, including the number of judgments per comparison and any significance test?
- Does the timestep factor remain informative if the error contrast is normalized by the expected noise-prediction error at each timestep?
- (Suggestion) How does ToPO compare with a simpler spatial weight, such as the pixel or latent difference between the two images?
3 comments
Response part 2
W-iv and Q1. The matched comparison
Thank you for examining these numbers so carefully. Your reading exposed an error in our original description, and we apologize for it. The three-seed table, Table 5 in the original submission and Table 7 in the revision, is not a matched training comparison. It evaluates fixed checkpoints under three generation seeds, and its Diffusion-DPO rows use the officially released weights. This single fact explains each of your observations:
- The Diffusion-DPO scores coincide with Table 1 because both use the same released checkpoint.
- They fall below SFT-winners because the released model was trained from the base model, not from SFT-winners.
- The standard deviations are small because only the generation seed varies, not the training run.
- The gains are larger than in Figure 5 because ToPO starts from SFT-winners while the released model does not.
The matched comparison is Figure 5, where both methods start from the same SFT-winners checkpoint with identical settings. At 500 updates, ToPO reaches a PickScore of 21.542, HPSv2 of 0.2903, and ImageReward of 0.812, against 21.344, 0.2862, and 0.718 for Diffusion-DPO. We have corrected the description of the three-seed table, and the main text now reports this matched gain alongside the released-checkpoint comparison.
Q2. Same initialization and same budget
To strengthen the matched evidence, we trained a second, independent pair of SD-1.5 runs from the same SFT-winners checkpoint with the same data and evaluation, this time recording actual GPU-hours (Section 5.5, Figure 6):
| Method | Updates | GPU-hours | PickScore ↑ | HPSv2 ↑ | ImageReward ↑ |
|---|---|---|---|---|---|
| Diffusion-DPO | 84 | 2.87 | 21.310 | 0.2860 | 0.721 |
| ToPO | 47 | 3.05 | 21.357 | 0.2871 | 0.743 |
| Diffusion-DPO | 251 | 8.48 | 21.397 | 0.2887 | 0.719 |
| ToPO | 140 | 8.99 | 21.539 | 0.2906 | 0.790 |
| Diffusion-DPO | 500 | 16.91 | 21.426 | 0.2895 | 0.736 |
| ToPO | 260 | 16.64 | 21.631 | 0.2912 | 0.815 |
At every comparable budget, ToPO is ahead on all three metrics, and after only 140 updates it already exceeds Diffusion-DPO's final result while using 46.8% fewer GPU-hours. Since this run is independent of Figure 5, the two figures give two matched run pairs that agree. Multiple training seeds on every benchmark did not fit within the rebuttal period; we will add them in the revision.
Q3. Training cost
| Backbone | Method | Seconds per update ↓ | Peak memory per GPU, GiB ↓ |
|---|---|---|---|
| SD-1.5 | Diffusion-DPO | 16.07 | 23.50 |
| SD-1.5 | ToPO | 30.23 | 24.36 |
| SDXL | Diffusion-DPO | 35.49 | 70.94 |
| SDXL | ToPO | 70.45 | 71.01 |
Measured on eight H200 GPUs, ToPO takes 1.88 times longer per update on SD-1.5 and 1.98 times on SDXL, because the frozen reference evaluates both images at four anchors. These passes need no gradients, so peak memory rises by only 0.86 and 0.07 GiB. Per-update cost alone, however, does not determine efficiency: as shown above, ToPO reaches better scores with fewer total GPU-hours. Each update costs more, but it also achieves more.
W-iv. Scope of claims
We agree that our claims should match the evidence, and we have revised the abstract from state-of-the-art to competitive performance. We also acknowledge the mixed SDXL results you note: against the released Diffusion-DPO, ToPO is slightly lower on PickScore and the aesthetic score across generation seeds and on ImageReward and CLIP on PartiPrompts, and InPO has a higher HPSv2 on HPDv2. Appendix S7.4 now discusses this variation. These cases are outweighed by ToPO's broader advantages on SDXL: it leads on most metrics, raises the GenEval average from 0.5917 to 0.6168, and is preferred over Diffusion-DPO by human raters in 63.3% to 70.7% of pairs. Our central claim rests on the matched SD-1.5 comparisons above, where ToPO leads consistently.
Response part 3
W-iii and Q6. The timestep factor
You are right that noise-prediction residuals are naturally larger at low noise, so we tested whether the temporal factor merely follows this scale. We divided the error contrast at each anchor by the expected noise-prediction error at that anchor, keeping everything else fixed (Table 17):
| Variant | PickScore ↑ | HPSv2 ↑ | ImageReward ↑ | Aesthetic ↑ | CLIP ↑ |
|---|---|---|---|---|---|
| ToPO | 21.542 | 0.2903 | 0.812 | 5.361 | 28.393 |
| Error-normalized | 21.499 | 0.2894 | 0.801 | 5.302 | 28.249 |
Normalization scores lower on all five metrics, so removing the error scale does not help. At the same time, the temporal factor is clearly informative, since removing it lowers PickScore by 0.307. The emphasis on low-noise steps also fits our motivation: Pick-a-Pic pairs often differ in fine details, and in our introductory example the decisive difference, a legible "42", forms at late, low-noise steps.
Q4. Count tokens and the counting gain
Thank you for pointing out this apparent discrepancy. The original diagnostic used SD-1.5 pairs generated with the same template and seed, differing only in the attribute word. Even so, SD-1.5 does not always confine the change to that attribute: the layout or background may shift, and the requested count or position may not appear. These side effects mainly masked count and spatial changes, so we rebuilt the diagnostic on before-and-after pairs from AnyEdit and MagicBrush, whose two images differ mainly in the edited attribute:
| Attribute | Target-token weight ratio ↑ | 95% CI |
|---|---|---|
| Count involving one or two | 1.304 | 1.185 – 1.396 |
| Count involving three or four | 1.078 | 1.021 – 1.152 |
| Color | 1.521 | 1.375 – 1.624 |
| Shape | 1.402 | 1.293 – 1.537 |
| Spatial | 1.105 | 1.051 – 1.172 |
All ratios now exceed one. As the two count groups differ, we also compared individual count words:
| Count word | one | two | three | four | five |
|---|---|---|---|---|---|
| Weight ratio ↑ | 1.429 | 1.206 | 1.115 | 1.059 | 1.033 |
Enrichment is strongest for "one", which helps explain the counting gain: a common counting error is duplicating a subject that should appear once (Binyamin et al., CVPR 2025; Kang et al., WACV 2025), and ToPO's strong weight on the singular count word helps avoid it, as Figure 7 illustrates. Consistently, removing token conditioning lowers GenEval Count from 0.4969 to 0.4406.
Q5. User study
Sixteen participants rated all 300 prompts in both comparisons on overall preference, visual appeal, and prompt faithfulness, making a forced binary choice with identities hidden and order randomized. This gives 1,800 judgments per participant and 28,800 in total. Each pair is decided by the 16-vote majority, and an 8–8 split counts as a tie against ToPO.
| Comparison | Dimension | Win / Tie / Loss | Win rate ↑ | 95% CI | Agreement ↑ |
|---|---|---|---|---|---|
| vs. Base | Overall | 198 / 41 / 61 | 66.0% | 60.5 – 71.1 | 0.68 |
| vs. Base | Visual | 220 / 42 / 38 | 73.3% | 68.1 – 78.0 | 0.67 |
| vs. Base | Faithfulness | 202 / 55 / 43 | 67.3% | 61.8 – 72.4 | 0.63 |
| vs. DPO | Overall | 190 / 51 / 59 | 63.3% | 57.7 – 68.6 | 0.64 |
| vs. DPO | Visual | 212 / 34 / 54 | 70.7% | 65.3 – 75.5 | 0.70 |
| vs. DPO | Faithfulness | 196 / 55 / 49 | 65.3% | 59.8 – 70.5 | 0.61 |
Excluding ties, ToPO wins 76.3% to 85.3% of decided pairs, and a two-sided sign test gives p < 1e-16 in every case. Agreement is the raw fraction of votes matching the majority.
Q7. Simpler spatial weights
| Variant | PickScore ↑ | HPSv2 ↑ | ImageReward ↑ | Aesthetic ↑ | CLIP ↑ |
|---|---|---|---|---|---|
| ToPO | 21.542 | 0.2903 | 0.812 | 5.361 | 28.393 |
| Latent difference | 21.553 | 0.2879 | 0.791 | 5.355 | 28.114 |
Replacing the error gap with the latent difference is slightly better on PickScore but worse on the other four metrics. Moreover, a latent difference is a single map shared by all timesteps, so it cannot provide the temporal factor that ToPO derives from the same error gap. We will add a pixel-difference variant in the revision.
We hope this clarifies our contribution and evidence, and we welcome any further questions.
Response part 1
Thank you for the careful and detailed review. We have taken each concern seriously and run additional experiments. We answer each weakness and question below, dividing the response into three parts. Revisions are marked in blue, and tables are cited by their revised numbers.
W-i. Novelty
We agree that spatial credit, step-aware weighting, and cross-attention each have precedents, and we thank you for pointing us to FocusDPO and LocalDPO, which we now discuss in Related Work. The essential difference is not which axes are weighted, but where the local signal comes from. Prior methods obtain it from outside the preference pair: from a reference image, from the way the negative is constructed, or from a trained reward model. ToPO obtains it from the pair itself.
This difference has three consequences.
First, ToPO addresses a harder and more general setting. In BiDPO, LocalDPO, and FocusDPO, the negative is created by editing or corrupting the positive, so the location of the difference is known by construction. PatchDPO and FocusDPO also measure fidelity to a reference image, a goal specific to personalization. In human preference data, neither shortcut exists: the two images are independent samples, there is no reference, and the decisive difference can lie anywhere. ToPO recovers it with the frozen reference model that Diffusion-DPO already uses, so it needs no new data, annotation, or model.
Second, ToPO weights differences rather than coverage. STAR's attention map shows where the prompt is rendered in one sample, regardless of whether two images differ there. ToPO combines two signals: the error gap tells where the images differ, and attention tells which differences concern the prompt. Together they concentrate the update on prompt-relevant differences, which neither signal achieves alone. FocusDPO and LocalDPO do not use prompt tokens at all.
Third, ToPO allocates over time at no extra cost. SPO and LPO must train a step-level reward model and sample candidates online, whereas ToPO's temporal weight comes from the same reference passes that give its spatial weight. STAR, the only prior method weighting both space and time, does so in online reinforcement learning with a reward signal; ToPO does so in offline DPO.
Finally, Prompt-to-Prompt and Attend-and-Excite use attention at inference to edit or guide an image, while ToPO uses it during training to turn pair contrast into token weights. Our contribution is therefore a principle, separating the direction of a preference update, given by the label, from its allocation, given by the pair itself, together with a realization that needs no annotation, reward model, or online sampling.
W-ii. Design choices and the DPO objective
We address the concern that ToPO is a stack of heuristics in three steps.
Every component is tested. Removing the spatial, temporal, or token factor, the special-token mask, or the midpoint term each lowers PickScore, by 0.17 to 0.43 points (Table 3). The revision further tests separability, the number of anchors, the attention source, error normalization, and a latent-difference weight (Tables 16 and 17).
ToPO stays within the DPO framework. Diffusion-DPO already optimizes a surrogate: an ELBO bound with a timestep weighting, evaluated at one sampled step. ToPO keeps this form and only changes the weights inside it. Because these weights are nonnegative and carry no gradient, each pair still pushes the model in the direction of its label. The same idea underlies token-weighted DPO for language models, such as TIS-DPO at ICLR 2025, which we now discuss in Related Work.
Label invariance is intended. The error gap is symmetric so that the allocation can never reverse the direction set by the label. The full weight still uses the label, since tokens are read from the preferred image's attention, and this matters: symmetric and loser-only attention both perform worse (Table 16).
We agree that the appendix propositions are standard; they state basic sanity properties and are presented as supporting analysis, not as a contribution.