- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes STEP, a reward fine-tuning method designed for few-step Gaussian-mixture flow models. The authors address the challenge of reward optimization in highly compressed steps by caching the deployment trajectory and introducing analytic clean-latent probes. A key technical contribution is the combination of a straight-through chain and distance-weighted fusion. This design elegantly decouples the forward and backward passes, maintaining the original sampling trajectory for inference while effectively propagating reward gradients. The method is further stabilized via intermediate rewards and frozen-teacher velocity regularization. The paper evaluates STEP on FLUX.1-dev with 4 NFEs against strong baselines, supported by thorough and convincing ablation studies.
Critical strengths
STEP reuses analytic posterior estimates within sampling intervals to distribute a shared terminal reward over multiple steps, and the probes do not require extra evaluations. This is a practical way to adapt straight-through gradients to few-step GM policies, and it clearly separates forward values from backward paths.
Critical weaknesses
-
Empirical gains are modest and scope is narrow. While STEP improves GenEval and OCR by 0.03 over adapted Flow-GRPO, it still trails LeapAlign on GenEval (0.530 vs. 0.550). The evaluation is limited to a single backbone and four NFEs. More concerning is the missing baseline: we see results for the distilled π-Flow reference, but not for STEP's own base GM initialization before reward training. This leaves the actual improvement over the starting policy unclear.
-
The surrogate gradient and fusion weights lack sufficient validation. The coefficient bound doesn't establish gradient alignment or guarantee improvement of the terminal objective. Comparisons with sampler gradients and weight-sensitivity analysis are lacked.
Questions for authors
-
Appendix C.1 separates the relative fusion weights from the overall scale γ. What evidence supports inverse-distance weighting over uniform fusion with the same total coefficient (c_i = γ/N)? Does the sample-dependent γ offer benefits beyond a fixed overall scale?
-
Table 1 includes DRaFT, but its objective differs from STEP's. With auxiliary losses and teacher regularization matched, how does STEP compare with backpropagation through the actual sampler? Could you compare their update directions and resulting changes in terminal reward, including cases where the surrogate update is detrimental?