- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes PTERA, which replaces deterministic text embeddings with a learnable Gaussian distribution for reward alignment while keeping the diffusion backbone frozen. The authors also introduce PTERA-TTA, which samples from the learned embedding distribution and selects better candidates based on reward feedback at inference time. Experiments on SD1.5, SDXL, and SD3 demonstrate improvements in preference alignment and training efficiency.
Critical strengths
-
The idea is interesting. Learning a distribution over text embeddings provides more flexibility than optimizing a single deterministic embedding, and naturally supports test-time exploration.
-
The method is lightweight and does not require updating the diffusion backbone. The experiments and ablation studies are fairly comprehensive, showing promising improvements on SD1.5 and SDXL with relatively low training cost.
Critical weaknesses
-
The benefit of probabilistic modeling is not entirely convincing. In Table 6, the improvement over Learned-Iso is relatively small, making it unclear how much the learned coordinate-wise variance actually contributes.
-
The gains are less consistent on stronger models. In Table 2, the improvement on SD3 is almost negligible, and PTERA-TTA does not consistently outperform PTERA-mean on SDXL. This raises some questions about the practical benefits of the method on more advanced backbones.
Questions for authors
-
Could the authors further clarify the advantage of learning coordinate-wise variance over using simple isotropic Gaussian perturbations? The differences in Table 6 are quite small, and results across multiple runs would be helpful.
-
Why are the improvements on SD3 much smaller than those on SD1.5 and SDXL? Could this be related to differences in text conditioning, training hyperparameters, or reward optimization?
3 comments
Q2 (W2).
SD3 training configuration. Each SD3 training run takes more than 24 hours in our setup, which limited our initial exploration of architectural and optimization configurations. In the additional experiment, we retained the same PTERA residual-module architecture, whose parameter count is approximately 0.4% of that of each corresponding frozen text encoder. We increased the learning rate from to , changed from to , and reduced the initial reference scale from to . The training reward-query budget remained unchanged at 20K.
| SD3 Benchmark | Method | PickScore ↑ | ImageReward ↑ | HPSv2 ↑ | CLIP ↑ | Aesthetic ↑ |
|---|---|---|---|---|---|---|
| DrawBench | Base | 22.494 | 0.926 | 0.283 | 28.676 | 5.955 |
| DrawBench | PTERA-mean | 22.547 | 0.946 | 0.284 | 28.626 | 5.948 |
| DrawBench | PTERA-sampled | 22.505 | 0.930 | 0.283 | 28.624 | 5.949 |
| DrawBench | PTERA-TTA | 22.573 | 0.981 | 0.284 | 28.598 | 5.968 |
| PartiPrompts | Base | 22.576 | 1.103 | 0.297 | 27.967 | 6.234 |
| PartiPrompts | PTERA-mean | 22.599 | 1.143 | 0.298 | 27.955 | 6.250 |
| PartiPrompts | PTERA-sampled | 22.597 | 1.113 | 0.298 | 27.965 | 6.248 |
| PartiPrompts | PTERA-TTA | 22.620 | 1.167 | 0.302 | 27.868 | 6.274 |
Under the revised configuration, PTERA-mean’s PickScore improvement over the base model increases from 0.002 to 0.053 on DrawBench and from 0.004 to 0.023 on PartiPrompts. PTERA-TTA further improves PickScore over PTERA-mean on both benchmarks. These results suggest that the original near-zero gains were partly limited by the training configuration. Since several hyperparameters were adjusted jointly, we attribute the improvement to the revised recipe as a whole. As the reviewer and Reviewer 8Zv8 both suggest, we attribute the smaller gains on SD3 to two likely factors. First, our architecture choices and training configurations for SD3 have not yet been sufficiently explored. Second, adapting only the text-conditioning network may be inherently limited. With unrestricted capacity, diffusion weight tuning, which takes and as inputs, could be more advantageous. Nevertheless, we consider it an important contribution that, with a very small trainable module, PTERA achieves better efficiency than diffusion weight tuning with competitive performance (Figure 6a and Table 4).
SDXL TTA configuration. We additionally evaluated a larger candidate budget, increasing from 5 to 8 with the selection schedule .
| SDXL | Selection schedule | Candidates | PickScore ↑ | ImageReward ↑ | HPSv2 ↑ | CLIP ↑ | Aesthetic ↑ |
|---|---|---|---|---|---|---|---|
| DrawBench | - | - | 22.999 | 0.773 | 0.284 | 26.799 | 6.215 |
| DrawBench | {50,40,30,20,10} | 5 | 23.000 | 0.707 | 0.283 | 26.687 | 6.104 |
| DrawBench | {50,40,30,20,10} | 8 | 23.017 | 0.751 | 0.284 | 27.000 | 6.116 |
| PartiPrompts | - | - | 23.207 | 0.996 | 0.297 | 27.782 | 6.456 |
| PartiPrompts | {50,40,30,20,10} | 5 | 23.141 | 1.000 | 0.298 | 27.799 | 6.424 |
| PartiPrompts | {50,40,30,20,10} | 8 | 23.300 | 0.991 | 0.300 | 27.619 | 6.520 |
Compared with the original TTA results, PickScore increases from 23.000 to 23.017 on DrawBench and from 23.141 to 23.300 on PartiPrompts. This demonstrates that the original configuration does not exhaust the available test-time gains. Algorithm 1 exposes the selection schedule and candidate count as configurable parameters; the experiment here specifically illustrates the benefit of increasing the candidate budget, at additional inference cost. Thus, TTA provides a configurable performance and computation trade-off, with its practical benefit depending on the chosen search configuration and evaluation metric.
Dear Reviewer oVTT,
Before the discussion period ends, we would appreciate your final look at our revised manuscript. All changes are highlighted in blue. We hope our responses have addressed the reviewer's questions. We would be happy to discuss any remaining concerns or follow-up questions. Thank you.
We appreciate the reviewer’s valuable questions, which helped us clarify and improve our work, and provide our responses below.
Q1 (W1).
Both Learned-Iso and PTERA learn probabilistic conditioning distributions using the preference reward. Their difference lies in the covariance parameterization. Learned-Iso uses a single scalar scale shared across embedding coordinates, whereas PTERA learns a separate variance for each coordinate. This additional flexibility allows PTERA to allocate different exploration magnitudes to different coordinates, rather than scaling perturbations uniformly. Although this provides a more expressive distribution family, expressivity alone does not guarantee better performance under practical optimization.
To assess its empirical contribution, we expanded Table 6 with two additional training seeds, 43 and 44, alongside the original seed 42, and evaluated all variants on both DrawBench and the held-out HPSv2 prompts. Each evaluation uses four generated images per prompt. The expanded table reports PickScore and HPSv2 scores, together with paired 95% bootstrap confidence intervals for ΔPS=PS(PTERA)−PS(variant), separately for each training seed and benchmark.
PTERA achieves the highest mean PickScore among all variants in all six seed–benchmark combinations. Specifically, its PickScore improvements over Learned-Iso are 0.0450, 0.1111, and 0.1230 on DrawBench, and 0.0040, 0.0094, and 0.2149 on the held-out HPSv2 prompts, for seeds 42, 43, and 44, respectively. The corresponding confidence intervals exclude zero for seed 44 on both benchmarks, while the other comparisons retain uncertainty. We therefore interpret these results as evidence of a consistent positive direction in the observed PickScore differences, with modest and setting-dependent gains.