- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents RatioDiff, an offline method for aligning diffusion models with human preferences. It learns preference ratios between denoising transitions, with a reference correction that accounts for their different noisy states. And a Bregman objective is used to preserve the learning signal for pairs that the model ranks incorrectly with high confidence. The method also learns pair weights from the reference model and adjusts regularization during training to limit model drift and preserve denoising quality. Experiments on SD1.5 and SDXL across three prompt benchmarks show improvements in preference scores, visual quality, and text–image alignment.
Critical strengths
- The motivation is clear. The method addresses the need to correct incorrectly ranked pairs while preserving denoising quality. It separates the preference objective from pair weighting and regularization, allowing the contribution of each pair and the strength of regularization to be controlled separately.
- The paper provides extensive theoretical analysis with detailed derivations and proofs. The presentation is clear and easy to follow.
Critical weaknesses
- The baseline comparisons are not convincing enough. Comparisons with methods such as SPO, InPO, and IAPO are missing, making it difficult to assess how competitive RatioDiff is beyond established baselines. Furthermore, some gains over the reported baselines are also small. For example, compared with Diffusion-DPO on PartiPrompt with SDXL, ImageReward increases from 1.0788 to 1.0804 and CLIP from 0.3660 to 0.3671. Without results across multiple seeds or confidence intervals, these small margins provide limited evidence of a reliable advantage.
- The ablations do not show how much the proposed ratio objective contributes to the results. Table 3 only removes adaptive regularization or adaptive preference scaling. There is no ablation of the reference correction and no comparison with DPO using the same weighting and regularization. These comparisons are needed to distinguish the benefit of the core objective from that of the added training components.
Questions for authors
- Add comparisons with SPO, InPO, IAPO or any other baselines under comparable data and compute budgets.
- What does each component contribute to the reported gains? Provide an ablation of the reference correction, a comparison with DPO using the same auxiliary components, and separate tests of the two regularizers.
- Does RatioDiff retain its benefits on flow-based models such as SD3 and FLUX? Support this with experimental results and explain any changes needed to accommodate their prediction parameterizations.