acceptodds
Under review as a conference paper at ICLR 2027

FlowSAPO: Semantics-Aware Regional Optimization for Visual Generation

Abstract

Online reinforcement learning aligns diffusion and flow models using rewards evaluated on complete images or videos, while policy optimization acts on denoising transitions over a latent grid. A single global clipping statistic discards the spatial distribution of local likelihood changes, whereas clipping each latent position separately does not coordinate related positions. We introduce FLowSAPO, which combines exact positionwise likelihood ratios with regional surrogate weights constructed from the model's own text attention. Connected, overlapping regions are built during rollout and frozen during replay. Within each region, local log ratios are aggregated with inverse coverage weights and exponentiated before clipping, while the reward and scalar advantage remain unchanged. Under Gaussian transitions with shared covariance at a fixed state, we show that the global statistic depends on local divergences only through their total. We also show that the regional surrogate gradient at the rollout policy is proportional to the local gradient for any stored cover; away from that point, the cover affects both exponential weights and shared clipping decisions. Across the reported image and video comparisons, FLowSAPO improves over the local objective, including GenEval attribute binding from 0.86 to 0.96 and All Frame OCR on Wan2.1 from 0.339 to 0.427. It also raises VideoReward visual quality from 2.302 to 3.901 and motion quality from 0.811 to 1.596 compared with flowgrpo. These results support regional optimization as a useful complement to positionwise clipping for visual generation.

Primary area:
generative models
Keywords:
Image Generation, Video Generation, GRPO, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_h8AmOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Because the sampler's transition density is diagonal-Gaussian, the log-density change decomposes exactly over latent positions, so the paper asks which positions should share a PPO clipping decision: FlowGRPO clips one global geometric-mean statistic that is blind to sign-cancelling spatial patterns, while pure positionwise ("1×1") clipping keeps local information but never coordinates related positions. FlowSAPO retains the exact 1×1 branch and adds a regional one — during rollout the model's own text attention grows connected, overlapping regions of similar-attending cells under a noise-dependent size cap, each region's local log-ratios are summed with inverse-coverage normalization (an exact conservation identity) and clipped at an area-dependent width, regions are frozen during replay, and the two branches mix at 0.75/0.25 under a shared scalar advantage; App. H analyses all three clipping scopes under a fixed-state Gaussian model.

Critical strengths
  • The theoretical appendix is disciplined about its own scope to an unusual degree, which makes it trustworthy rather than decorative
  • Experimental results indicates the effectiveness of the proposed method.
Critical weaknesses
  • A single training seed per setting, with the checkpoint picked at peak reward on the very quantity being optimized. App. B.4: "We fix the base training seed to 3407 for all experiments"; "we select the checkpoint with the highest evaluation reward for the training objective"; "we therefore do not require paired confidence intervals or estimates of variability across independent training runs". That rule inflates exactly the deltas in dispute, and several entries in the same tables run the other way: SD3.5 UnifiedReward 3.382 versus DiffusionNFT's 3.423, and on VBench both Object Class (0.749) and Background Consistency (0.973) fall below the untrained Wan2.1 base (0.772, 0.984). Comparability compounds it — FlowGRPO is run at clip width 1e-4 (images) / 1e-3 (videos) while 1×1 uses 0.10, and neither crossover is reported, so "global versus local" is partly a comparison of band widths.

  • No semantics-free control for a method titled "semantics-aware".

Questions for authors

Please run 2–3 independent training seeds for at least SD3.5/PickScore and Wan VQ+MQ, reporting mean ± std for FlowGRPO, 1×1 and FlowSAPO, and say whether the selected checkpoint would change under selection by final-step reward. This is the single addition that would move my rating.

7 comments
AuthorsOpenReview ↗

Response to Weakness 1 (Part 1): Training Seeds and Checkpoint Selection

Thank you for the detailed comments on training variability and checkpoint selection. We have completed three independent training runs for every method in both requested settings and extended the SD3.5 training comparison. FlowSAPO improves all six DrawBench metrics and all eight reported VBench dimensions over 1×1 in the three-seed averages, and remains ahead of both baselines at the extended SD3.5 training endpoint.

1. Three independent training seeds.

We trained FlowGRPO, FlowSAPO (1×1), and full FlowSAPO with seeds 3407, 2025, and 2026 in both SD3.5 Medium with PickScore training and Wan2.1-T2V 1.3B with VQ/MQ training. Repeated runs keep each method’s training configuration, budget, and evaluation protocol fixed.

Tables R1a–R1b report mean ± standard deviation across independent training runs. These results are now included in Appendix L.3 (Tables 9 and 10).

Table R1a. SD3.5 Medium trained with PickScore and evaluated on DrawBench. Higher is better.

MetricFlowGRPOFlowSAPO (1×1)FlowSAPO
PickScore23.411 ± 0.04723.425 ± 0.14823.793 ± 0.051
CLIPScore0.280 ± 0.0020.276 ± 0.0040.278 ± 0.003
HPSv2.10.304 ± 0.0040.292 ± 0.0040.316 ± 0.004
Aesthetic5.946 ± 0.0735.939 ± 0.1226.080 ± 0.004
ImageReward1.291 ± 0.0221.267 ± 0.0281.339 ± 0.023
UnifiedReward3.328 ± 0.0403.208 ± 0.0233.331 ± 0.063

Table R1b. Wan2.1-T2V 1.3B trained with VQ/MQ and evaluated on VBench. Higher is better.

DimensionFlowGRPOFlowSAPO (1×1)FlowSAPO
Subject Consistency0.971 ± 0.0050.979 ± 0.0030.983 ± 0.004
Background Consistency0.970 ± 0.0020.971 ± 0.0020.973 ± 0.002
Motion Smoothness0.987 ± 0.0010.986 ± 0.0010.987 ± 0.002
Aesthetic Quality0.641 ± 0.0060.642 ± 0.0080.654 ± 0.005
Imaging Quality0.654 ± 0.0080.658 ± 0.0080.661 ± 0.007
Object Class0.743 ± 0.0060.736 ± 0.0100.754 ± 0.008
Multiple Objects0.599 ± 0.0070.594 ± 0.0080.615 ± 0.007
Color0.899 ± 0.0050.891 ± 0.0070.914 ± 0.007

FlowSAPO also achieves higher means than FlowGRPO on five of six image metrics and seven of eight video dimensions, with motion smoothness tied at the reported precision. Its PickScore gain over FlowGRPO is 0.382, while its observed standard deviation is 0.051; its VBench standard deviations range from 0.002 to 0.008. The performance advantage is retained when training variability is measured directly.

2. Checkpoint selection and performance later in training.

We apply the same optimized-reward selection criterion within matched training budgets to our reproduced comparisons, including DiffusionNFT (Appendix B.4). We aim to demonstrate both higher rewards with fewer optimizer updates and a sustained advantage as training continues. We therefore extended the SD3.5 runs of FlowGRPO, 1×1, and FlowSAPO from 4,320 to 8,640 updates and report the subsequent evaluation curves.

FlowSAPO at 4,320 updates already exceeds FlowGRPO’s reward at 8,640 updates, and remains ahead of both baselines at the extended endpoint. The curves are available through this anonymous visualization link and are included in Appendix L.2 (Figure 18).

FlowSAPO also remains ahead of FlowGRPO at the final displayed evaluations in Figure 1(b) for Wan VQ/MQ, Figure 4(a) for Wan OCR, and Figure 8 for FLUX PickScore. These results substantiate the gains through independent training repetitions and sustained performance at fixed training endpoints.

AuthorsOpenReview ↗

Response to Weakness 1 (Part 2): Metric Comparisons and Clipping Widths

Thank you for examining the metric-level comparisons and clipping configurations. We clarify the reported metric differences and provide additional clipping-width experiments. The new tests retain FlowSAPO’s advantage over 1×1 on all six DrawBench metrics at base widths of 0.2 and 0.7, showing that the gain extends beyond the default setting of 0.1.

1. Metric-specific comparisons.

For SD3.5 UnifiedReward in Table 1, FlowSAPO scores 3.382 versus DiffusionNFT’s 3.423, while exceeding the base model, FlowGRPO, and 1×1 at 3.094, 3.338, and 3.220. On FLUX.1 [dev], FlowSAPO outperforms DiffusionNFT on all six reported metrics, including UnifiedReward, 3.578 versus 3.568. The UnifiedReward gap therefore occurs in the evaluated SD3.5 configuration, while the FLUX comparison shows gains across all six metrics.

For Wan in the original Table 3, Background Consistency and Object Class are lower than the base model, but both exceed FlowGRPO under the same VQ/MQ reward: 0.973 versus 0.970 and 0.749 versus 0.741. FlowSAPO improves seven of eight reported VBench dimensions over FlowGRPO and six over the base model, alongside higher VQ and MQ rewards. These comparisons evaluate optimization methods under the same prescribed reward, with VBench providing an independent assessment.

2. Clipping definitions and sensitivity.

FlowGRPO’s clipping widths, 10−410^{-4} for images and 10−310^{-3} for videos, apply to a global weight obtained by averaging log-density changes across the latent space. Our width of 0.1 applies to exact positionwise density ratios, defining the local interval [0.9, 1.1]. The numerical widths therefore operate on different statistical scales. Moreover, changing a global threshold cannot recover local changes that cancel during global aggregation.

We preserve FlowGRPO’s baseline formulation and configuration. Its maintainer also reported that replacing the global mean with a sum remained unstable after adjusting the clipping coefficient (official discussion).

We additionally tested FlowSAPO with base clipping half-widths of 0.2 and 0.7, keeping the regional area exponent fixed at 0.7. As shown in Table R2, both wider settings retain the advantage over 1×1 on all six metrics, with PickScores of 23.824 and 23.834, close to the default 23.828. These controls are now included in Appendix L.5 (Table 12), with reference scores in Appendix L.4, Table 11.

Table R2. Clipping-width controls on SD3.5 Medium with PickScore training and DrawBench evaluation. All results use training seed 3407. Higher is better.

Method and clipping settingPickScoreCLIPScoreHPSv2.1AestheticImageRewardUnifiedReward
FlowGRPO, global width 10−410^{-4}23.4380.2800.3046.0241.3103.338
FlowSAPO (1×1), local width 0.123.5610.2720.2946.0381.2773.220
FlowSAPO, base width 0.123.8280.2750.3156.0841.3543.382
FlowSAPO, base width 0.223.8240.2740.3106.0991.3223.351
FlowSAPO, base width 0.723.8340.2780.3146.0551.3443.329

The improvement over 1×1 is retained at all three tested FlowSAPO widths. The default full-versus-1×1 comparison also uses the same local clipping width of 0.1, directly supporting the additional contribution of the regional branch.

AuthorsOpenReview ↗

Response to Weakness 2: A Semantics-Free Region Control

Thank you for calling for a direct test of semantic grouping. We have added a random connected-region control and a static SAM3 alternative on SD3.5 Medium, trained with PickScore and evaluated on DrawBench. Table R3 shows that random grouping underperforms 1×1 on all six metrics, whereas FlowSAPO improves all six and outperforms both regional alternatives.

Random regions without semantic member selection. At every rollout step, we record the original region size at each attention-cell seed. We then grow a replacement region from the same seed by uniformly sampling its eight-connected frontier until the recorded size is reached. Coverage is recomputed, and the objective and clipping rules are retained. This preserves connected aggregation, region sizes, and their area-dependent clipping widths while removing attention-guided member selection.

SAM3 as an alternative semantic grouping. We obtain SAM3 regions from each completed rollout image and reuse them at every denoising timestep. This provides an external semantic grouping method to compare with our regions derived from the generator’s text attention at each sampled state.

Both experiments and their results have been incorporated into Appendix L.4 (Table 11) of the revised manuscript.

Table R3. Region-construction controls on SD3.5 Medium with PickScore training and DrawBench evaluation. All results use training seed 3407. Higher is better.

MethodPickScoreCLIPScoreHPSv2.1AestheticImageRewardUnifiedReward
FlowGRPO23.4380.2800.3046.0241.3103.338
FlowSAPO (1×1)23.5610.2720.2946.0381.2773.220
Random connected regions23.3420.2690.2856.0171.2273.046
SAM3 regions23.5230.2730.3086.0591.3113.223
FlowSAPO23.8280.2750.3156.0841.3543.382

Across all six metrics, the ordering is FlowSAPO > SAM3 regions > random regions. Yet random regions remain below the positionwise baseline on every metric, showing that adding a regional loss with matched sizes and clipping widths is insufficient to obtain the gains. SAM3 improves over random grouping and raises PickScore above FlowGRPO, while our attention-derived construction further reaches 23.828 and exceeds SAM3 on all six metrics.

These controls directly establish the practical importance of region membership: semantic grouping is beneficial in this comparison, and our construction delivers the strongest results among the tested alternatives using the generator’s own attention.

Reviewer_U4sKOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper introduces FlowSAPO, which combines positionwise clipping of exact local likelihood ratios with a semantics-aware regional objective for fine-tuning diffusion and flow models. The model’s own text attention groups nearby positions with similar attention patterns into connected, overlapping regions. Each region combines its members’ local log-ratios into one weight for a shared clipping decision, while the terminal reward and scalar advantage remain unchanged. The paper provides a Gaussian analysis of regional grouping and reports improvements over positionwise clipping and FlowGRPO across image and video generation tasks, including attribute binding, OCR, and visual and motion quality.

Critical strengths
  1. FlowSAPO combines position-wise clipping of exact local likelihood ratios with connected, overlapping regions constructed from the model’s own text attention. This directly addresses the lack of coordination between related positions, while keeping the terminal reward and scalar advantage unchanged and requiring no additional semantic annotations.
  2. The reported image and video comparisons support regional optimization as a useful complement to position-wise clipping, including improvements in GenEval attribute binding from 0.86 to 0.96 and All Frame OCR from 0.339 to 0.427. These comparisons substantiate the practical benefit of adding the regional objective.
Critical weaknesses

Positionwise clipping with exact local likelihood ratios is well motivated. However, the necessity of semantics-aware region construction beyond local measurement is insufficiently justified. The regional branch introduces several interacting design choices, and the ablations do not clearly isolate their contributions, leaving the main source of improvement and the need for the added complexity unclear.

Questions for authors

The method shows substantial gains on tasks requiring fine details, such as text rendering. Could the authors provide qualitative comparisons with and without the regional branch on text rendering to clarify its contribution and justify its necessity? If this point is clearly addressed, I would consider raising my score.

2 comments
AuthorsOpenReview ↗

Response to Weakness: Identifying the Source of Regional Gains

We appreciate the reviewer’s careful assessment and constructive suggestions for evaluating the contribution of the regional branch. Our new ablations on SD3.5 Medium, using PickScore training and DrawBench evaluation, establish three findings. First, regional coordination adds a consistent benefit beyond local clipping: FlowSAPO improves all six metrics over the 1×1 baseline. Second, the comparisons support semantic grouping and favor our attention-derived construction: size-matched random regions underperform 1×1, SAM3 regions improve over random grouping, and FlowSAPO outperforms both alternatives on all six metrics (Table R1), using the generator’s own text attention without an external segmentation model. Third, the advantage is retained under the tested optimization controls: dividing the mixed policy loss by 1.75 or separately increasing the base clipping half-width to 0.2 or 0.7 still yields improvements over 1×1 on all six metrics (Appendix L.5, Table 12). These results substantiate both the regional branch’s contribution and our choice of region construction.

1. Controlling loss scale and clipping width. The mixed policy objective has a rollout-point gradient factor of 1.75 relative to the positionwise objective. We therefore trained with (0.75Lf+0.25Lr)/1.75(0.75L_f+0.25L_r)/1.75, keeping the learning rate fixed and adding the reference penalty after this normalization. As shown in Table R1, this variant reaches a PickScore of 23.872, compared with 23.561 for 1×1, and also improves all five evaluation-only metrics. The regional benefit is thus retained after removing this gradient-scale factor.

We further increased the base clipping half-width from 0.1 to 0.2 or 0.7. Both settings remain above 1×1 on all six metrics, with PickScores of 23.824 and 23.834, respectively. These normalization and clipping-sensitivity experiments are now reported in Appendix L.5 (Table 12).

2. Examining which positions are grouped. Our random-region control retains every attention-cell seed and its original region size, but selects members by randomly growing an eight-connected region. Coverage is recomputed, and the original loss and clipping rules are retained. Despite matching region sizes and their area-dependent clipping widths, this control scores below 1×1 on every metric. This result shows that simply introducing connected regional aggregation is insufficient to obtain the improvement.

We also tested a semantic alternative: SAM3 regions extracted from the final generated image and reused throughout denoising. These regions outperform random grouping on all six metrics, but remain below FlowSAPO on every metric. The comparison supports content-aware grouping and favors our regions derived from text attention at each sampled state over the tested static segmentation alternative. Both comparisons have been incorporated into Appendix L.4 (Table 11).

Table R1. Region-construction and loss-scaling controls. All results use SD3.5 Medium, PickScore training, DrawBench evaluation, and training seed 3407. Higher is better for all metrics.

MethodPickScoreCLIPScoreHPSv2.1AestheticImageRewardUnifiedReward
FlowGRPO23.4380.2800.3046.0241.3103.338
FlowSAPO (1×1)23.5610.2720.2946.0381.2773.220
FlowSAPO23.8280.2750.3156.0841.3543.382
FlowSAPO: mixed policy loss ÷ 1.7523.8720.2790.3136.0801.4093.351
Random connected regions23.3420.2690.2856.0171.2273.046
SAM3 regions23.5230.2730.3086.0591.3113.223

These results give a concrete justification for the added regional design: its gains over local measurement persist under the tested changes to optimization scale and clipping, while replacing its region assignments with size-matched random groups removes the benefit. FlowSAPO also exceeds the SAM3 alternative across all six metrics. Its grouping signal comes from the generator's own rollout text attention, requiring neither an external segmentation model nor additional semantic annotations.

AuthorsOpenReview ↗

Response to Question: Text Rendering With and Without the Regional Branch

We thank the reviewer for this concrete suggestion. We provide an anonymous visualization link for direct inspection and have incorporated the qualitative comparisons into Appendix L.1 (Figures 16 and 17) of the revised manuscript.

The figures compare FlowGRPO, FlowSAPO (1×1), and full FlowSAPO across 12 text-rendering prompts, using Wan2.1-T2V 1.3B trained with the OCR reward. The 1×1 variant retains exact local ratios and positionwise clipping but omits the regional branch, so the middle and right columns directly compare text rendering without and with regional optimization.

The examples show specific character errors that remain under positionwise optimization. In Figure 16, the 1×1 variant renders “FRESH” as “FRESSH” and “MINT” as “MNT”, whereas full FlowSAPO renders both target words correctly. In Figure 17, the 1×1 variant repeats “E” in “LEMON,” producing “LEEMON,” while full FlowSAPO correctly renders “LEMON.” The full model also preserves the requested white chalk lettering and green crate lettering in the “FRESH” and “LEMON” examples.

These comparisons complement the All-Frame OCR improvement from 0.339 for 1×1 to 0.427 for full FlowSAPO in Figure 4. Together, the qualitative and quantitative results demonstrate that the regional branch improves complete-word rendering beyond positionwise clipping, reducing omitted and repeated characters in the illustrated examples.

Reviewer_wvDoOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper propose FlowSAPO, which proposes semantics-aware regional clipping for online reinforcement learning of diffusion and flow models. It combines exact positionwise likelihood ratios with overlapping regions derived from rollout text attention, allowing related latent positions to share clipping decisions. Experiments on image and video generation show improvements in preference alignment, compositional generation, video quality, and text rendering.

Critical strengths
  • FlowSAPO identifies limitation of global clipping in visual policy optimization and addresses it with exact local likelihood ratios plus attention-derived regional coordination. This is a general objective-level contribution rather than a task-specific heuristic.
  • Evidence is broad and consistent across image and video backbones, multiple reward objectives, compositional generation, and OCR. Improvements over both FlowGRPO and the positionwise 1×1 variant support the value of the regional branch.
Critical weaknesses
  • The experiments do not isolate whether the gains come from semantically meaningful grouping. Fixed-patch ablations are insufficient to exclude improvements caused by overlapping aggregation or altered clipping behavior; random or shuffled-attention regions would provide a more decisive control.

  • Results use a single training seed. Paired generation seeds reduce evaluation variance but do not address the potentially substantial optimization variance of online RL, limiting confidence in the reported improvements over strong baselines.

Questions for authors
  • Could the authors clarify what evidence isolates semantic grouping as the source of improvement, rather than overlapping aggregation or modified clipping behavior? Are results available for random regions, shuffled attention, or another grouping signal?

  • Proposition 4 implies that the regional branch scales the rollout-point gradient by ((1−λ)+λb=1.75)((1-\lambda)+\lambda b=1.75) under the default setting. How much of the gain remains after matching the gradient scale or effective learning rate?

  • All reported runs appear to use one training seed. Could the authors provide any evidence of optimization stability across seeds or clarify why paired evaluation seeds are sufficient to characterize online-RL training variance?

  • Since regional weights are not density ratios, generally have expectation below one, and may lose the lower clipping boundary for large regions, how should the regional objective be interpreted relative to PPO, and what stability safeguards are essential?

4 comments
AuthorsOpenReview ↗

Response to Weakness 1: The Contribution of Semantic Region Construction

We thank the reviewer for this constructive suggestion, which helped us strengthen the experimental evidence for our semantic region construction. We added two controls on SD3.5 Medium with PickScore training and DrawBench evaluation: size-matched random connected regions and SAM3-based semantic regions. As shown in Table R1, FlowSAPO outperforms both controls and the positionwise 1×1 baseline on all six metrics, demonstrating the added value of our attention-derived region construction.

(1) Random connected regions with matched sizes.

Every attention cell remains a region-growth seed. At each rollout step, we record each seed's original attention-derived region size, then grow a random region from the same seed by uniformly sampling its current eight-connected frontier until that size is reached. Coverage is recomputed, while the loss and clipping rules are retained. This preserves region sizes and their area-dependent clipping widths while changing member selection.

(2) SAM3 regions from the final image.

We apply SAM3 to each sample's final generated image and reuse the resulting semantic regions across all denoising timesteps. This provides a static semantic alternative to FlowSAPO's regions derived from text attention at each sampled state.

Table R1. Region-construction controls on SD3.5 Medium with PickScore training and DrawBench evaluation. All results use training seed 3407. Baseline scores are from the original Table 1. Higher is better for all metrics.

MethodPickScoreCLIPScoreHPSv2.1AestheticImageRewardUnifiedReward
FlowGRPO23.4380.2800.3046.0241.3103.338
FlowSAPO (1×1)23.5610.2720.2946.0381.2773.220
Random connected regions (Mismatch)23.3420.2690.2856.0171.2273.046
SAM3 regions23.5230.2730.3086.0591.3113.223
FlowSAPO23.8280.2750.3156.0841.3543.382

Random regions perform below 1×1 on all six metrics, showing that matched region sizes and clipping widths alone do not reproduce the gains. SAM3 regions outperform random regions on all six metrics and improve PickScore over FlowGRPO from 23.438 to 23.523. FlowSAPO further reaches 23.828 and exceeds SAM3 on all six metrics. These results support content-aware grouping and favor our rollout-attention construction over the tested alternatives.

We have added both region-construction controls and their results to Appendix L.4 (Table 11) of the revised manuscript.

AuthorsOpenReview ↗

Response to Weakness 2: Training Stability Across Multiple Seeds

We thank the reviewer for highlighting the importance of evaluating training variability. We have completed three independent training runs for each of FlowGRPO, FlowSAPO (1×1), and FlowSAPO in both image and video settings. The three-seed averages retain FlowSAPO's advantage over both baselines, directly addressing the concern about reliance on a single training run (Tables R2a–R2b).

The image experiments use SD3.5 Medium with PickScore training and DrawBench evaluation. The video experiments use Wan2.1-T2V 1.3B with VideoReward VQ/MQ training and VBench evaluation. Within each setting, repeated runs of each method use the same training configuration, budget, and evaluation protocol. All results below are mean ± standard deviation across three independent training seeds: 3407, 2025, and 2026.

Table R2a. Three-seed results on DrawBench. Higher is better for all metrics.

MetricFlowGRPOFlowSAPO (1×1)FlowSAPO
PickScore23.411 ± 0.04723.425 ± 0.14823.793 ± 0.051
CLIPScore0.280 ± 0.0020.276 ± 0.0040.278 ± 0.003
HPSv2.10.304 ± 0.0040.292 ± 0.0040.316 ± 0.004
Aesthetic5.946 ± 0.0735.939 ± 0.1226.080 ± 0.004
ImageReward1.291 ± 0.0221.267 ± 0.0281.339 ± 0.023
UnifiedReward3.328 ± 0.0403.208 ± 0.0233.331 ± 0.063

Table R2b. Three-seed results on VBench. Higher is better for all metrics.

DimensionFlowGRPOFlowSAPO (1×1)FlowSAPO
Subject Consistency0.971 ± 0.0050.979 ± 0.0030.983 ± 0.004
Background Consistency0.970 ± 0.0020.971 ± 0.0020.973 ± 0.002
Motion Smoothness0.987 ± 0.0010.986 ± 0.0010.987 ± 0.002
Aesthetic Quality0.641 ± 0.0060.642 ± 0.0080.654 ± 0.005
Imaging Quality0.654 ± 0.0080.658 ± 0.0080.661 ± 0.007
Object Class0.743 ± 0.0060.736 ± 0.0100.754 ± 0.008
Multiple Objects0.599 ± 0.0070.594 ± 0.0080.615 ± 0.007
Color0.899 ± 0.0050.891 ± 0.0070.914 ± 0.007

The benefit of the regional branch persists in the three-seed averages: FlowSAPO improves over 1×1 on all six DrawBench metrics and all eight reported VBench dimensions. Compared with FlowGRPO, it achieves higher means on five of six image metrics and seven of eight video dimensions, with motion smoothness tied at the reported precision. Its PickScore improvement over FlowGRPO is 0.382, while its observed standard deviation is 0.051; its VBench standard deviations range from 0.002 to 0.008.

These repeated-training results directly quantify optimization variability and substantiate FlowSAPO's gains in both image and video generation. We have incorporated the three-seed DrawBench and VBench results into Appendix L.3 (Tables 9 and 10, respectively) of the revised manuscript.

AuthorsOpenReview ↗

Response to Questions on Semantic Grouping, Gradient Scale, and Optimization Stability

Q1: Evidence for semantic grouping.

The controls in our W1 response and Table R1 show that matching region sizes and clipping widths alone does not reproduce FlowSAPO's gains. Static SAM3 regions extracted from the final image outperform random connected regions and improve PickScore over FlowGRPO, supporting the value of semantic grouping. FlowSAPO further exceeds both controls and the 1×1 baseline on all six metrics, demonstrating the added value of our attention-derived region construction. These comparisons are now included in Appendix L.4 (Table 11) of the revised manuscript.

Q2: Gradient scale and normalization.

FlowSAPO retains exact positionwise ratios, preserving local likelihood changes that can cancel under global geometric averaging. The regional branch introduces an additional gradient-scale difference because its loss averages over NaN_a regions rather than NfN_f positions. As shown in Appendix H.2, with Nf/Na=4N_f/N_a=4 and a regional mixing weight of 0.25, the mixed policy objective has a gradient factor of 0.75+0.25(Nf/Na)=1.750.75+0.25(N_f/N_a)=1.75 relative to the full 1×1 objective at rollout.

We directly controlled for this factor by dividing the mixed policy loss, 0.75Lf+0.25Lr0.75L_f+0.25L_r, by 1.75 before adding the reference penalty, while keeping the learning rate unchanged. As shown in Table R3, the rescaled variant achieves a PickScore of 23.872, compared with 23.828 for default FlowSAPO and 23.561 for 1×1. It also remains above 1×1 on all five evaluation-only metrics. The regional branch's advantage therefore persists after this gradient-scale normalization. We have added this experiment and its implementation details to Appendix L.5 (Table 12) of the revised manuscript.

Table R3. Gradient-scale control on SD3.5 Medium with PickScore training and DrawBench evaluation. All results use training seed 3407. Higher is better for all metrics.

MethodPickScoreCLIPScoreHPSv2.1AestheticImageRewardUnifiedReward
FlowGRPO23.4380.2800.3046.0241.3103.338
FlowSAPO (1×1)23.5610.2720.2946.0381.2773.220
FlowSAPO23.8280.2750.3156.0841.3543.382
FlowSAPO: mixed policy loss ÷ 1.7523.8720.2790.3136.0801.4093.351

Q3: Optimization stability across training seeds.

Paired generation seeds reduce evaluation noise but do not characterize training variability. As reported in our W2 response and Tables R2a–R2b, we have completed three independent training runs for each method in both image and video settings and report mean ± standard deviation.

The three-seed averages preserve FlowSAPO's advantage over 1×1 on all six DrawBench metrics and all eight reported VBench dimensions. Relative to FlowGRPO, FlowSAPO achieves higher means on five of six image metrics and seven of eight video dimensions, with motion smoothness tied at the reported precision. These results directly address training variability beyond the original single-seed comparisons and are now included in Appendix L.3 (Tables 9 and 10) of the revised manuscript.

Q4: Density ratios, clipping, and stability.

FlowGRPO uses a geometrically averaged global weight rather than the exact joint density ratio. GRPO-Guard reports that shifts in this weight and variance differences across timesteps can cause unbalanced clipping, leaving positive-advantage updates insufficiently constrained.

FlowSAPO retains exact positionwise density ratios before numerical guarding. Its regional weights aggregate coverage-corrected local log-ratios without additional normalization by region area; their departure from exact density ratios comes from correcting overlapping contributions. This preserves local likelihood changes while allowing related positions to share clipping decisions. We use a local clipping width of 0.1, compared with the 10−410^{-4} global width of the FlowGRPO image baseline.

Region growth is capped at 64 attention cells, limiting aggregation extent and regional clipping widths. Although the regional lower boundary can become inactive, positionwise clipping remains. Figure 7 supports the chosen cap, Figure 8 shows steadier reward improvement on FLUX, and Figure 9 shows fewer transitions meeting the same negative-advantage global clipping condition.

Additionally, increasing the base clipping half-width from 0.1 to 0.2 or 0.7, with the area exponent fixed at 0.7, yields PickScores of 23.824 and 23.834, respectively, close to the default 23.828. Both variants remain above 1×1 on all six metrics, supporting robustness at the tested clipping settings. These sensitivity results are now included in Appendix L.5 (Table 12) of the revised manuscript.

Area_Chair_UnQ1OpenReview ↗

rebuttal response

The rebuttal period is quickly approaching the end. Please go over the authors’ rebuttal and revision, and add acknowledgment and further comments if needed.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revisions for the Area Chair and Reviewers

Dear Area Chair and Reviewers,

We thank the Area Chair for overseeing the discussion and all three reviewers for their careful reading and concrete suggestions. Your feedback helped us test which part of FlowSAPO produces the gains and how well those gains hold across training runs. We have incorporated the following results into the revised manuscript.

Contribution of semantic grouping. On DrawBench, random connected regions matched for size and clipping width scored below positionwise clipping (1×1) on all six metrics. Static SAM3 regions improved on random grouping across all six, and FlowSAPO’s regions, built from the generator’s own text attention at each sampled state, outperformed both alternatives on every metric. The advantage over 1×1 also persisted after dividing the mixed policy loss by 1.75 and, in separate runs, increasing the clipping half-width from 0.1 to 0.2 or 0.7 (Appendices L.4–L.5, Tables 11–12).

Effect on text rendering. We added side-by-side comparisons across 12 video prompts. For example, the 1×1 model renders “FRESH” as “FRESSH” and “MINT” as “MNT”, while full FlowSAPO spells both correctly. These examples illustrate the All-Frame OCR improvement from 0.339 to 0.427 reported in Figure 4. View the comparisons in Figures 16–17, Appendix L.1.

Independent training seeds. We trained each of the three methods with three independent seeds in both the image and video settings. FlowSAPO achieved higher means than 1×1 on all six DrawBench metrics and all eight reported VBench dimensions. Compared with FlowGRPO, it led on five of six and seven of eight, respectively. Mean PickScore was 23.793 ± 0.051 for FlowSAPO and 23.411 ± 0.047 for FlowGRPO (mean ± standard deviation, Appendix L.3, Tables 9–10).

Checkpoint selection and training efficiency. We extended all three SD3.5/PickScore runs from 4,320 to 8,640 updates. At 4,320 updates, the original reporting endpoint, FlowSAPO already exceeded FlowGRPO’s evaluation reward at 8,640. When all three methods reached 8,640 updates, FlowSAPO remained ahead of both baselines. View the training curves in Figure 18, Appendix L.2.

We thank the Area Chair and all three reviewers for the time and care you devoted to our work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.