- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper compares binary and clipped-linear CLIP rewards for DDPO fine-tuning of Stable Diffusion 1.5. Across two campaigns, dense rewards reach a CLIP-defined success target earlier; in one campaign, paired comparisons show that the dense-minus-binary trajectory advantage is substantially smaller under OWLv2 and BLIP-VQA. The contribution is a controlled empirical demonstration of evaluator-dependent reward comparisons, explicitly restricted to these automated criteria.
Critical strengths
- The experiments are fairly comprehensive. The paper systematically compares the evaluator used for optimization with external evaluators that are not involved in optimization, and defines evaluator-specific effects as well as paired CLIP-minus-external contrasts. This provides a useful experimental and measurement framework.
Critical weaknesses
- The paper does not introduce a new experimental method; it mainly compares and analyzes existing approaches, and its methodological novelty is limited.
- The external evaluators are not human assessments. OWLv2 has zero-inflated scores, while BLIP-VQA uses only one prompt-specific yes/no question. Neither can be directly equated with genuine compositional correctness.
- Training uses a rolling normalization buffer of 16 rewards to compute advantages for 64 samples. The binary-reward condition is more likely to encounter an all-zero buffer early in training, which may itself create differences in the training signal.
Questions for authors
- What concrete improvement does the paper propose in response to its findings? Without a corresponding intervention, the work feels more like an output of an automated-research process than a complete research contribution.
- The analysis seems relatively superficial. Although the paper shows that reward-induced speed-ups may depend on the evaluator, it does not investigate this finding in greater detail. For example, which task and reward characteristics lead to particular speed-up effects, and how could the findings be used to improve the reward or training process?
Dear Reviewers,
The authors have posted their responses. Could you please read the rebuttal and let us know whether it resolves the questions and concerns raised in your review? If any point remains unclear, feel free to follow up with the authors during the discussion period.
Please update your review and final rating at your earliest convenience.
Thank you very much for your time and contribution.
Best regards, AC