- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Demo-NFT extends DiffusionNFT by replacing a small fraction of native training targets with prompt-matched, high-quality GPT-image-2 demonstrations. Both sources share the existing NFT objective and joint advantage estimation. Demonstration rewards are capped at the highest retained native reward for each prompt before computing advantages.
With 5% demonstrations, all four auxiliary metrics improve over NFT across the four main settings, while task scores improve in two settings and decrease by less than 0.01 in the others. Capping generally improves task accuracy but can reduce auxiliary quality: at 5%, it lowers all four auxiliary metrics on both OCR models. Gains in image quality persist after shared SFT initialization, although GenEval2 task scores decrease, indicating a task–quality trade-off. (Tables 1, 4, and 6)
Critical strengths
S1. Consistent auxiliary-quality gains. Demonstrations improve QJudge, HPSv3, PickScore, and Aesthetic over native-only NFT in all four main settings. Relative to Base, QJudge gains are substantial on GenEval2 but modest on OCR, where demonstrations mainly recover quality lost during NFT. Preserving quality while improving task accuracy remains practically valuable. (Table 1)
S2. Relevant controls and calibration analysis. Comparisons with supervised mixing, multi-reward NFT, and shared-SFT initialization examine plausible alternative explanations. Capping improves task scores in six of eight configurations, with negligible differences in the remaining two. Fixed-input diagnostics help isolate its effects on advantages and gradients, although the overall task–quality benefit is not uniform. (Tables 4 and 6; Appendix E)
Critical weaknesses
W1. The incremental mechanism is not sufficiently isolated from demonstration supervision strength. Demo-NFT retains NFT’s objective, adds external image targets, and introduces reward capping. Simplicity is not itself a weakness, but the strongest evidence establishes the benefit of demonstrations rather than a clear advantage of the proposed integration.
The supervised-mixing comparison remains inconclusive because matching demonstration ratios does not match loss or gradient magnitudes, as Appendix D acknowledges. At a matched 10% ratio, DDRL* exceeds Demo-NFT on all five metrics for SD3.5 GenEval2, whereas Demo-NFT achieves higher task scores in the other three settings. A demonstration-loss coefficient sweep would clarify whether these differences arise from the integration strategy or supervision strength. (Tables 1 and 3; Appendix D)
Additionally, 99.95% of demonstration rewards reach the cap in the SD3.5 OCR 5% diagnostic run. This raises the question of whether demonstration-specific reward evaluation is necessary in that regime. It does not imply equivalence to SFT: joint advantages remain dependent on prompt/batch statistics, and the objective remains NFT. (Appendix E)
W2. The evidence supports specific operating points rather than a generally superior task–quality trade-off. Capping can improve task accuracy while reducing auxiliary quality, and continued demonstrations after shared SFT reduce GenEval2 task scores by 0.0344 on FLUX and 0.0228 on SD3.5. These are meaningful trade-offs rather than uniform improvements. (Tables 4 and 6)
The multi-reward baseline uses equal-weight task/HPSv3 advantages without a mixing-weight sweep. Demo-NFT outperforms this configuration on task score, QJudge, and PickScore, but the baseline achieves higher HPSv3. Alternative weights could narrow these differences. Single-run results without significance testing further limit conclusions from small gains. PickScore also truncates OCR prompts, measuring prefix-conditioned preference rather than full-prompt compliance. (Table 1; Appendices A and B)
Questions for authors
Q1. At a matched demonstration ratio, how sensitive is DDRL* versus Demo-NFT to the demonstration-loss coefficient? A small sweep would help isolate integration strategy from supervision strength.
Q2. Where nearly all demonstration rewards reach the cap, does directly assigning the best retained-native reward produce comparable results?
Q3. How sensitive are the comparisons to the task/HPSv3 mixing weight? Please distinguish superiority over the tested equal-weight configuration from superiority over tuned multi-reward mixtures.
Q4. Can repeated runs or uncertainty estimates establish whether the smaller task and auxiliary-metric differences are reliable?