- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes SpatioTemporal Adaptive Reward (STAR) Allocation for RL post-training of text-to-image diffusion and flow models. STAR uses text-image attention inside the generative model to construct spatial allocation maps that vary across denoising steps and rollouts, allocates the same group-relative advantage to more relevant latent regions, and applies stronger policy updates to them through a spatially resolved policy objective.
Critical strengths
The paper targets a real problem with a cheap fix. In RL post-training for text-to-image models, one image-level score gets applied equally to every denoising step and image region, even though only part of the image (like the chair in “a red chair”) decides the score. STAR reuses the attention the model already computes to put more weight on prompt-relevant regions, with no extra reward model and only under 2% more training time and 1.56% more memory, so it’s easy to drop into existing pipelines.
The paper also worked out why the obvious approach fails: multiplying weights into the reward gets averaged away, since the probability ratio is shared across positions. Their stop-gradient trick leaves the loss and clipping unchanged but lets each position keep its own weight in the gradient. Gains appear across three different reward types (rule-based, OCR, learned preference), so it doesn’t look tied to one setup.
Critical weaknesses
The evidence doesn’t show where the gains come from, or that attention does what the paper claims. Improvements are small and near saturation (GenEval 0.95 → 0.9759, PickScore 23.31 → 23.60), with no multiple seeds or error bars. The ablations only vary STAR’s own design choices; key controls are missing (random or shuffled spatial weights, uniform up-weighting, spatial-only vs. temporal-only, sensitivity to λ). The paper also never shows real attention heatmaps or coefficient maps across denoising steps (Figure 1 is only a schematic), so it is unclear whether the maps localize prompt-relevant regions. Per-step heatmaps, a quantitative overlap check with detection or segmentation masks, failure cases, and same-prompt, same-seed comparisons of Flow-GRPO vs. Flow-GRPO+STAR would help.
The evaluation is narrow, and the theory is disconnected from the experiments. The only baseline is Flow-GRPO, on a single backbone (SD3.5-M), with no comparison to related credit assignment methods (e.g., B2-DiffuRL) and no image quality, diversity, or human evaluation. Theorems 3 and 4 rely on assumptions that are never tested, and Theorem 5 only gives a first-order gain for small step sizes, so the theory adds little real support.
Questions for authors
- Are the reported numbers (GenEval 0.9759, OCR 0.9757, PickScore 23.60) from a single run or averaged over seeds? Could you report mean ± std over several seeds for both Flow-GRPO and STAR under identical training budgets? If the gaps fall within seed noise, my assessment would change substantially.
- Could you compare STAR against (a) random or shuffled spatial coefficients, (b) constant uniform up-weighting (e.g., all positions multiplied by 1.25, the mean of C), (c) spatial-only and temporal-only variants, and (d) different values of λ? Which of these recovers most of the improvement?
- Were Flow-GRPO and STAR trained with identical hyperparameters, steps, and reward evaluations? Are the Flow-GRPO numbers (GenEval 0.95, OCR 0.92, PickScore 23.31) reproduced in your setup or taken from the original paper? The OCR baseline in particular seems low.
- Can you empirically check the assumptions behind Theorems 3-4, for instance whether the per-position gradient projections a_{b,u} correlate with H_{b,u}? Otherwise, what supports the claim that the gain comes from attention-gradient alignment?
- The paper says STAR applies to SDXL and Qwen-Image style backbones, but only SD3.5-M is tested. Is there any evidence on another backbone? How sensitive is the method to the text-unit extraction (LLM-based for OCR vs. the deterministic rule)?