- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces MeanFlowAdvantage (MFA), a signed advantage-weighted least-squares objective for reward fine-tuning of average-velocity generators. Unlike conventional flow-matching models, MeanFlow directly predicts the interval-average velocity , which is used for few-step generation. However, existing reward fine-tuning methods typically operate in instantaneous-velocity or -prediction space, creating a mismatch between the training objective and the representation deployed at inference time.
To address this issue, the authors construct an induced prediction using a shared, stop-gradient MeanFlow derivative correction. This formulation enables signed advantage-based reward optimization in prediction space while ensuring that rollout and reference regularization directly constrain the deployed average-velocity maps. The paper further provides a theoretical analysis of the resulting objective and evaluates the method on text-to-image generation and DNA promoter design.
Critical strengths
1. A practically meaningful problem with a representation-aware formulation.
The paper addresses an important challenge in reward fine-tuning of few-step generative models. Since MeanFlow directly uses interval-average velocities for inference, applying conventional prediction-space objectives without accounting for this representation may introduce a mismatch between optimization and generation. The proposed formulation addresses this issue by applying reward supervision in prediction space while explicitly regularizing the underlying average-velocity maps. This is a reasonable and well-motivated design choice that preserves the native few-step sampling mechanism of MeanFlow.
2. Promising empirical results across two different application domains.
The experiments cover both text-to-image generation and DNA promoter design, demonstrating the applicability of the proposed objective beyond a single domain.
On SD3.5-Medium, MFA improves all eight reported metrics over MeanFlowNFT under matched evaluation settings. In particular, Aesthetic Score increases from 5.928 to 6.300 (+0.372), while HPSv3 improves from 13.676 to 13.933 (+0.257). Notably, neither of these metrics is directly optimized during training, suggesting that the improvements are not restricted to the training rewards.
The DNA promoter experiments further demonstrate the flexibility of the framework in both teacher-free on-policy RL and teacher-guided reward-graded distillation. In the distillation setting, MFA achieves a Sei profile MSE of 0.0467, compared with 0.0570 for MeanFlowNFT, corresponding to approximately an 18% reduction. These results strengthen the empirical support for the proposed formulation.
Critical weaknesses
1. Limited methodological novelty relative to AdvantageFlow and MeanFlowNFT.
The methodological novelty appears somewhat limited, as the proposed framework primarily combines existing techniques from AdvantageFlow and MeanFlowNFT.
Specifically, AdvantageFlow has already introduced the signed advantage-weighted least-squares objective with rollout and reference regularization. Meanwhile, MeanFlowNFT has established how to apply reward-based fine-tuning to average-velocity generators through an induced instantaneous-velocity predictor derived from the MeanFlow identity.
The proposed MFA objective largely follows the AdvantageFlow formulation (Eq. 8), while adopting the MeanFlow-based prediction transformation already explored by MeanFlowNFT. The main technical distinction lies in using a shared, detached derivative correction so that the regularization terms reduce to direct differences between average-velocity predictions (Eqs. 7–10).
Although this construction provides a useful representation-aligned formulation, the core optimization objective and prediction-space transformation are both closely related to prior work. Consequently, the contribution appears more incremental than fundamentally novel, and the paper could more clearly articulate the methodological advances beyond integrating these existing components.
2. A substantial gap between the theoretical assumptions and practical implementation.
The theoretical guarantees rely on strong assumptions that may not hold in practical training. In particular, Assumption A3 requires
which assumes that the rollout model is already a calibrated MeanFlow fixed point under its own sampling distribution.
This assumption is central to the population-level interpretation in Theorem 1 and the associated analysis. Theorem 2 further characterizes exact sampling under an idealized self-consistent fixed-point condition.
However, the practical implementation involves finite-step sampling, EMA-based rollout models, LoRA parameterization, finite-difference derivative estimation, and sample-dependent adaptive scaling. These factors introduce discrepancies relative to the assumptions of exact corrections, calibrated rollouts, and unrestricted function optimization used in the theoretical analysis.
The experimental results also indicate that the learned model does not fully satisfy the idealized flow-map consistency condition. As reported in Section 5.1 and Figure 4, the relative semigroup residual increases from 0.40 at two sampling steps to 0.76 at 32 steps, suggesting that the theoretical fixed point is only approximately attained.
Although the analysis offers useful intuition, the gap between its idealized assumptions and the actual training procedure limits the extent to which the theoretical guarantees can explain the observed practical behavior.
3. Relatively modest improvements on several evaluation metrics.
Although MFA outperforms MeanFlowNFT on all eight reported metrics under matched evaluation settings, the improvements on several metrics are relatively small.
For example, ImageReward increases only from 1.446 to 1.451 (+0.005), CLIPScore from 0.296 to 0.297 (+0.001), and OCR from 0.627 to 0.631 (+0.004). Similarly, HPSv2 improves only from 0.327 to 0.330 (+0.003).
These gains are substantially smaller than those observed for Aesthetic Score and HPSv3. Therefore, while the consistent numerical improvements are encouraging, their practical significance is less convincing for several metrics. The reported results support improvements over the baseline, but the overall empirical advantage appears uneven across different evaluation criteria.
Questions for authors
See weaknesses