- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper introduces a novel training-free fast sampling method for the flow-matching generative models. Based on the idea of exponential integrator, the author splits the velocity field into the affine velocity field and the residual to the true velocity field, integrate the affine velocity field exactly, and put the residual into an integral with Adams-Bashforth approximation. Theoretical derivations demonstrated several nice properties of the proposed method. Comprehensive experiments in unconditional, class-conditional, and text-conditional generations show the effectiveness of the method under different circumstances.
Critical strengths
This paper provides a novel idea that splits the velocity field into the Gaussian affine velocity field part and its residual part, and then apply the exponential integrator to the splitted form. Rigorous derivations are provided inside of the paper. The experiments are comprehensive and covers different types of generation tasks.
Critical weaknesses
See the questions below.
Questions for authors
The paper is solid in general. Here are some of my questions:
- For the text-conditional generations, how are the distribution mean and covariance matrix estimated? More precisely, it's mentioned inside Appendix E.2 that it's done by "a self sampling for 2000 template prompts and 25 samples for each prompt", how are these prompts selected/created?
- Would the estimation of mean and covariance matrix largely affect the final generation quality in the text-conditional case? For example, when a new prompt is given for text-conditional generation, does the generation quality of the image relates to how closely the new prompt is related to the estimation template prompts?
Thank you for the positive feedback
Thank you for the careful review and the constructive comments.
Calibration prompts and moment estimation (Q1). Since the SANA training set is not available, we estimate the target statistics from model self-samples. We construct 2,000 template prompts by independently sampling a subject, a style, and a modifier uniformly from 20 subjects, 10 styles, and 10 modifiers. This produces 1,263 distinct prompts, none of which appears in MJHQ-30K.
For each template prompt, we generate 25 samples using the released SANA pipeline with 20 sampling steps and CFG scale w=4.5, giving 50,000 latent samples in total. As described in Appendix B, the pooled mean m1 is estimated by averaging all 50,000 latents, while the within-condition covariance Sˉw is estimated by averaging the unbiased sample covariances of the 25 samples for each template prompt. We do not store a separate conditional mean for each prompt. Instead, at sampling time, the first network evaluation provides an endpoint prediction x^1,w, from which we estimate the trajectory-specific affine center as m1+α(x^1,w−m1). The per-template sample means are used only during calibration to fit the scalar α, whose fitted value is 0.261 on SANA. We have added the details of the template prompt generator to Appendix E.2.
Sensitivity to calibration prompts (Q2). We examined both the amount of calibration data and the coverage of the template prompts. First, we re-estimated the calibrated statistics using different numbers of conditions and self-samples per condition. The following table reports FID for AREX-C-AB2 on SANA:
Here, dof=Ccal(K−1) is the effective degrees of freedom of the within-condition covariance estimate, and d=8192 is the latent dimension. The covariance estimate is sensitive to the amount of calibration data. When dof<d, the estimate is rank-deficient, and generation quality can deteriorate substantially. With 2,000 conditions and 10 samples each, the degrees of freedom reach approximately 2.2d, and the FID is already within 0.1 of the full-calibration result at both NFEs. Thus, reliable covariance estimation requires enough samples, but the full 50,000-sample calibration is not necessary to obtain similar FID in this experiment. In contrast, the fitted α is relatively stable across calibration subsets, varying from 0.257 to 0.262 compared with 0.261 using the full calibration set.
Second, we examined whether generation quality depends on how closely the evaluation prompts match the calibration templates. We grouped MJHQ-30K categories by their coverage in the templates before evaluating the results. We evaluated each group on disjoint prompts from the remaining 29,000 MJHQ-30K samples, using approximately 2,000–3,000 images for each subgroup FID comparison. The smaller evaluation sets lead to relatively high absolute FID values, so comparisons are most meaningful between samplers within the same group.
AREX-C-AB2 improves over DPM-Solver-v3 in all three groups. At NFE 4, the FID improvements are 2.19, 3.55, and 3.51 for the covered, partly covered, and uncovered groups, respectively. At NFE 8, the corresponding improvements are 0.90, 1.45, and 0.88. Thus, the relative improvement does not systematically decrease as template coverage decreases. This is consistent with the construction of AREX-C. The covariance and residual profile are shared across prompts, while the affine center is estimated separately for each new prompt using its first network evaluation. The shared covariance determines the affine dynamics around this center, and the residual profile adjusts the numerical integration. Therefore, the method does not require a new prompt to closely match one of the calibration templates.
Overall, these results show that sufficient calibration samples are important for accurately estimating the covariance, while category-level similarity to the calibration prompts is not necessary for the observed gains on MJHQ-30K.