- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes Self-OPD, a teacher-free OPD framework for flow matching models. At each timestep, it branches the next state into K SDE candidates, rolls each one out with the ODE sampler, and computes advantages against a deterministic self-reference baseline. The velocity field is trained with an all-branch pull-push objective, and multiple rewards are combined by reward-level fusion. Experiments use SD3.5-M on GenEval, OCR, PickScore and HPSv2. In the multi-reward setting, Self-OPD reports GenEval 0.952 and OCR 0.960, against Flow-OPD, DiffusionOPD and DiffusionNFT.
Critical strengths
It needs no task-specific teachers and works with black-box rewards, which makes it simple to use. A single model reports strong GenEval (0.952) and OCR (0.960) in Tab. 2.
Critical weaknesses
- Contributions have limited novelty.The self-reference baseline is similar to DenseGRPO's "step-wise dense reward". DenseGRPO is not cited or compared. Pulling toward the best sample and pushing away from the worst is also close to TAGRPO, and TAGRPO is not cited.
- DiffusionOPD's training times are taken from its paper, but its metrics are not; the results also look like reward hacking. Sec. 4.3 uses "the training times … reported in DiffusionOPD", but Tab. 2 reports GenEval 0.915 for DiffusionOPD versus 0.96 in its paper. Where do these numbers come from? With 0.96, Self-OPD (0.952) does not lead. OPD is meant to "distill their capabilities into a unified student", not to hack one reward. Fig. 3 claims "superior performance in both accuracy and aesthetics", but in "No Liquids Allowed" Self-OPD's people are clearly more distorted than those of the other methods.
Questions for authors
- Where do the DiffusionOPD numbers in Tab. 2 come from? If they were reproduced, please give the setup and the training time of this reproduction. And please report the training cost on the same hardware.
- Please provide evidence beyond the training rewards (e.g., human evaluation of artifacts) to rule out reward hacking.
5 comments
Follow-up on Author Response: Unanswered Same-Hardware Cost, Human Distortion, and Novelty
Thank you for clarifying strict vs. continuous GenEval, which resolves my question on DiffusionOPD's scores. Three concerns remain:
-
Unanswered hardware specs and unfair DiffusionOPD time comparison. Your reply truncated and skipped my original Question 1 request for same-hardware training cost. Across the paper and all threads, GPU model, GPU count, and batch size remain undisclosed, while DiffusionOPD's 97.01h (
max(46.92h GenEval, 33.17h OCR, 85.75h aesthetics) + 11.26h) is copied from its paper. Moreover, comparing this 97.01h against Self-OPD's 48h warm-start (max(27h GenEval, 37h OCR) + 11h) is unfair because the pretraining objectives do not match: DiffusionOPD's 97.01h is bottlenecked by its 85.75h aesthetics teacher, whereas Self-OPD's warm-start omits aesthetics specialist pretraining entirely (excluding the aesthetics teacher, DiffusionOPD requiresmax(46.92h, 33.17h) + 11.26h = 58.18h, shorter than Self-OPD's 90h from-scratch).- Requested: Please report the exact GPU model, GPU count, and batch size, along with an objective-matched DiffusionOPD training cost measured on the same hardware.
-
Human/limb distortion. Your added automatic metrics and user study only measure global aesthetics and task accuracy, missing the severe human distortion I flagged in "No Liquids Allowed."
- Requested: Direct evaluation (e.g., human study on existing outputs) of human body / limb distortion on person-containing prompts across methods.
-
Limited novelty and practicality. Self-OPD is simply an online RL method that ports step-wise branching and pull-push ideas (as in DenseGRPO and TAGRPO) from denoising-MDP RL (DDPO / Flow-GRPO / DanceGRPO) to forward based RL (like DiffusionNFT). Framing it as "teacher-free OPD" is also unpractical: once the teacher is removed and training relies on online reward models, it loses the two key strengths of OPD, namely fast merging (~11h) and merging non-RL specialists (such as SFT models without reward models).
I maintain my rating pending the objective-matched same-hardware DiffusionOPD cost and human distortion evaluation.
Response to Follow-up: Hardware Specs & Cost Analysis, Human Distortion Study, and Method Positioning(1/2)
Thank you for following up and confirming that our clarification on strict vs. continuous GenEval resolves the score-provenance question. Below, we address your remaining concerns directly.
Q1. DiffusionOPD's 97.01h is bottlenecked by its 85.75h aesthetics teacher, whereas Self-OPD's warm-start omits aesthetics specialist pretraining entirely (excluding the aesthetics teacher, DiffusionOPD requires max(46.92h, 33.17h) + 11.26h = 58.18h, shorter than Self-OPD's 90h from-scratch).
A1: First, warm-start checkpoints in Self-OPD serve as student initializers, not teachers, which changes how much pretraining they need. In DiffusionOPD, teachers provide the fixed supervision targets for distillation, so they must be fully trained to convergence. In Self-OPD, warm-start LoRAs only give the student a rough starting point. Our actual OCR and GenEval initializers only score around 0.87 and 0.88, well below DiffusionOPD's teachers. After initialization, the student improves on its own through online rewards and never queries those LoRAs again. If DiffusionOPD used such under-trained checkpoints as teachers, its distillation targets would directly degrade.
Second, Self-OPD drops the external aesthetics teacher, not the aesthetics objective. During joint training, every output in Self-OPD is scored at the same time by its task reward (OCR or GenEval) and two preference rewards (PickScore and HPSv2). By contrast, if we drop the aesthetics teacher from DiffusionOPD to get the 58.18 h figure, DiffusionOPD loses aesthetic supervision completely and only distills OCR and GenEval. Thus, 58.18 h corresponds to a two-objective setup rather than an objective-matched comparison to Self-OPD's full training.
Q2. Unanswered hardware specs and unfair DiffusionOPD time comparison. Please report the exact GPU model, GPU count, and batch size, along with an objective-matched DiffusionOPD training cost measured on the same hardware.
A2: Self-OPD is trained on 8 NVIDIA A800 GPUs with a per-GPU microbatch size of 2. We accumulate one OCR and one GenEval microbatch per step, resulting in an effective global batch size of 32.
For DiffusionOPD, we cited the wall-clock times reported in their paper. Comparing their student-stage GPU-hours (Fig. 3) with wall-clock hours (Table 1) shows they also used 8 GPUs, though their paper does not disclose the exact GPU model or batch size. Because time constraints prevent us from re-running their full teacher-plus-student pipeline during the rebuttal window, we will run and report an objective-matched benchmark on the exact same 8 × A800 machine in the final manuscript.
More importantly, our main goal with Self-OPD is not just to train faster than teacher-based OPD, but to optimize task accuracy and aesthetics on the exact same output image. As noted in Sec. 4.2 of DiffusionOPD, single-task teachers generalize poorly outside their own target domains. Because DiffusionOPD's round-robin recipe (Sec. 3.4) pairs GenEval prompts only with the GenEval teacher and OCR prompts only with the OCR teacher, nothing explicitly forces an OCR or GenEval image to also have high aesthetic quality. Self-OPD addresses this through reward-level fusion, where every GenEval or OCR sample is jointly guided by its task reward plus PickScore and HPSv2. This is why Table 2 reports “Pref. (same test images)”, which measures aesthetic preference directly on the GenEval and OCR test outputs rather than on a separate prompt set.
Q3. Human/limb distortion. Please provide direct evaluation of human body / limb distortion on person-containing prompts across methods.
A3: To directly evaluate this, we added a human-body/limb distortion metric to our 10-volunteer user study (Table S2). Using the same blind, randomized setup across Flow-OPD, DiffusionOPD, DiffusionNFT, and Self-OPD, evaluators rated all person-containing prompts from the 100 held-out test prompts.
Distortion is rated on a 1 to 5 scale (1 means no visible distortion and 5 means severe distortion, so lower is better), checking for extra, missing, or fused limbs, broken joints, and deformed body proportions. Prompts where required people failed to appear were penalized under task accuracy rather than counted as distortion-free. As shown in Table S2, all methods show similarly low average distortion with very small gaps. Self-OPD scores 1.23, which is practically tied with DiffusionOPD at 1.22. While Self-OPD does not systematically worsen human anatomy, occasional background artifacts (such as in Fig. 3's "No Liquids Allowed") can still happen at 512×512 resolution when most spatial capacity goes to rendering large foreground text.
Response to Reviewer PdxZ
Thank you for your detailed review. We address your concerns about novelty, baseline provenance, cost comparisons, and visual artifacts below.
Self-OPD differs substantially from DenseGRPO and TAGRPO in the following respects:
First, the advantage reference differs. Self-OPD labels branches positive or negative relative to the deterministic ODE rollout reward from the same state, rather than a group mean. Standard-deviation normalization rescales these advantages without changing their signs. DenseGRPO normalizes adjacent-state reward gains within trajectory groups, while TAGRPO also uses group-relative advantages.
Second, the update objective differs. Self-OPD performs signed, step-wise regression toward or away from all K branch targets, normalized by the transition variance. DenseGRPO and TAGRPO instead use likelihood-ratio-based, clipped GRPO updates.
Third, the targets and repulsion mechanism differ. Self-OPD uses freshly sampled local branches from one parent state and attenuates repulsion when it would oppose the best branch's direction. TAGRPO uses group-best/group-worst trajectory anchors, cross-sample likelihood ratios, and a memory bank.
We will cite both papers. We could not obtain publicly released official DenseGRPO LoRA weights, so we could not reevaluate DenseGRPO under our evaluation protocol.
Please note that our evaluation uses two GenEval scoring modes, detailed in Appendix D and Fig. 5. Strict scoring requires all prompt sub-requirements to be satisfied, while continuous scoring awards partial credit. Under the same evaluation protocol, DiffusionOPD scores 0.9150 (stri.) and 0.9479 (cont.). Self-OPD's 0.9521 in Table 2 is a strict score, not directly comparable to the 0.96 reported in DiffusionOPD's paper under continuous scoring.
The remaining gap between our continuous score of 0.9479 for DiffusionOPD and its reported 0.96 may reflect differences in scoring settings or evaluation parameters. For example, StepX-OPD (arXiv:2611.04386v1, Table 1 and Sec. 5.1) reports 0.945 for DiffusionOPD using SD3.5-Medium at 512×512, with 40 sampling steps and CFG 4.5, close to our 0.9479 under similar settings. All scores in Table 2 come from our own evaluations under a common protocol.
Table S1 and S2 provide independent evidence against reward hacking as the sole explanation for the observed gains. Table S1 provides evidence beyond the training rewards: Self-OPD leads on ImageReward, Improved Aesthetic Predictor, NIMA, and PaQ-2-PiQ, ranks second on CLIP-IQA, and maintains competitive feature diversity. These results support broader perceptual-quality gains rather than improvements confined to the optimized rewards. In Table S2, Self-OPD achieves the highest mean ratings among all compared methods for both task accuracy and aesthetics.
Table S1. Evaluation of Self-OPD and other methods on additional metrics using the GenEval/OCR test sets. Best scores are bold; second-best are underlined.
Table S2. User study. Ten volunteers rated anonymized, randomly ordered images from four methods on 100 held-out prompts with one image per method per prompt. Task accuracy and aesthetics use 1–5 scales.