acceptodds
Under review as a conference paper at ICLR 2027

Moment Z-Flow: Learning Path Moments for One-Step Generation

13 views

Abstract

One-step generation requires a representation of finite-time transport that can be predicted in a single network evaluation. We introduce Moment Z-Flow, which jointly learns normalized temporal moments of a velocity path and combines them through a normalized generating-function readout. Training combines local moment supervision with cocycle consistency across intervals. A complex exponential representation organizes the moments into a weighted cocycle under interval composition. The complex coordinate controls temporal amplitude and phase; the deployed readout is exactly implementable as a fixed real-linear combination and preserves constant velocity. At 100k updates on CIFAR-10, our MeanFlow-based implementation achieves FID-50K 3.54, compared with 3.84 for the MeanFlow reference. Activating the moment readout improves the same-checkpoint center FID from 4.33 to 3.54 and from 4.43 to 3.68 across two training seeds. Without cocycle supervision, the two-seed mean FID is 4.04, compared with 3.61 for the full method. The sCM + Z-Flow extension achieves a reported CIFAR-10 FID of 3.58 at 100k updates. On ImageNet-256, Z-Flow achieves a one-step FID of 2.44 at 400k training steps. These results support joint path-moment prediction and calibrated readout as a practical approach to fast generative transport.

Primary area:
generative models
Keywords:
flow matching;one step generation; generative modeling

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_HoJ1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Moment Z-Flow jointly predicts temporal moments of a velocity path for one-step generation. It trains these predictions using local differential identities and cocycle consistency across intervals, then combines the outputs through a normalized real-linear readout.

On CIFAR-10, the method achieves FID 3.54 versus 3.84 for MeanFlow at 100k updates. Two-seed experiments support the benefit of the active readout and cocycle supervision. The paper also reports one-step FID 2.44 on ImageNet-256.

Critical strengths
  • The path-moment formulation provides a coherent extension of average-velocity prediction. The local identities connect the moment outputs to tractable supervision, while the cocycle relates predictions across intervals. The analysis clearly distinguishes moment-estimation error from readout bias and does not claim that higher moments improve an already exact interval average.

  • The within-checkpoint and cocycle comparisons provide useful evidence. The active readout improves over the center readout in both training seeds. With the representation and readout fixed, cocycle supervision reduces mean FID from 4.04 to 3.61. These controls support the usefulness of output combination and interval-consistent training, while preserving one backbone evaluation at inference.

Critical weaknesses
  • The experiments do not isolate the contribution of higher moments from cocycle supervision. The cocycle ablation retains all moment heads in both conditions. Since the zeroth moment also satisfies an interval-composition identity, a zeroth-moment model with the same consistency training is an important missing control. The current results establish that cocycle helps the complete method, but leave the additional value of the proposed higher-moment representation unclear.

  • The advantage over MeanFlow is not established under comparable training compute or paired training seeds. The reported FID improvement from 3.84 to 3.54 comes with 17.3% more training GPU-hours. The two-seed readout comparisons support active versus center readout, but do not provide a corresponding two-seed comparison against native MeanFlow. In addition, seed 0 shares evaluation noise with readout screening. The independent confirmation is helpful, but the evidence for a stable advantage over the baseline remains limited. The ImageNet result lacks matched comparisons that could resolve this concern at larger scale.

Questions for authors
  • Do any existing results compare a zeroth-moment model with the same cocycle supervision against the full moment model? This would help determine whether higher moments provide an advantage beyond interval consistency.

  • Do existing training curves or runs allow a compute-matched, paired-seed comparison with MeanFlow? Please also clarify the readout search range and whether the selected parameter was fixed before the independent confirmation evaluation.

2 comments

We thank the reviewer for identifying the controls needed to separate the representation, readout, and training-budget effects. The new evidence addresses these separately: the full extension improves over K=0 + cocycle, positive gains occur across two training seeds at approximately matched budgets, and fixed-q improvements survive independent confirmation.

1. Incremental value of the full moment extension.

At 100k updates:

ModelSeed 0 FIDSeed 1 FID
Native MeanFlow3.853.91
K=0 + cocycle3.723.80
Full K=3, fixed q=-1+0.5i3.573.70

Each entry averages two FID-50K evaluations on noise disjoint from screening, with common evaluation noise, reference statistics, and preprocessing across methods. The gains over K=0 + cocycle are 0.15/0.10 FID.

The control retains the EMA-teacher construction, detached subinterval targets, split distribution, optimizer, and global cocycle schedule. Both use the order-averaged objective in Eqs. (10)–(11). The zeroth-order coefficient is γj\gamma_j for K=0K=0 and γj/4\gamma_j/4 for K=3K=3. This matches the global weight and normalization convention.

These results support the full extension beyond the specified K=0 control, evaluating the combined moment supervision and readout rather than isolating each component. The submitted no-cocycle scores, 4.01/4.07, further motivate the combined method. Center-versus-active addresses the complementary question of how the learned moment outputs contribute at inference.

2. Paired training seeds at approximately matched budgets.

The supplementary comparison uses the same independent confirmation protocol. We used the original GPU-hour totals as approximate targets.

Target GPU-hoursSeedMeanFlow steps / FIDFull steps / FID
48.150100000 / 3.8585252 / 3.72
48.15198226 / 3.9284326 / 3.84
56.480117301 / 3.78100000 / 3.57
56.481115219 / 3.8398915 / 3.72

Both seeds improve at both target budgets. All runs use eight H100 GPUs, FP32, and batch 1024. Supplementary timing includes synchronized training computation, optimizer/EMA updates, and communication; startup, checkpoint I/O, and evaluation are excluded. Thresholds are fixed before FID evaluation, and the first checkpoint reaching each threshold is selected. q is not retuned.

For seeds 0 and 1, we paired native MeanFlow and the full model by matching their initialization seeds and training random streams. Each reported FID averages two evaluations of the same checkpoint.

The submitted scores are preserved. Full-model seed-1 FIDs 3.68 and 3.70 use the same 100k checkpoint under original and new evaluation protocols; 3.72 in the compute table uses the earlier 98915-update checkpoint.

3. Fixed and selected readouts survive independent confirmation.

The original seed-0 final evaluation shared noise with screening. Our records cover only part of the original search and do not show when we fixed the readout.

We hold the originally reported q=-1+0.5i fixed before the new confirmation. Separately, we screen 35 points with Re(q) in {-2,-1.5,-1,-0.5,0,0.5,1} and Im(q) in {0,0.25,0.5,0.75,1}. Only seed-0 screening selects q*=-1+i. Its coefficients are frozen before independent confirmation and transferred unchanged to seed 1; seed-1 grid evaluations do not select another winner.

Full-model readoutSeed 0 FIDSeed 1 FID
Center4.354.44
Fixed q=-1+0.5i3.573.70
Selected q*=-1+i3.553.69

The fixed readout improves over MeanFlow by 0.28/0.21 and over K=0 + cocycle by 0.15/0.10. Readout selection adds only a small gain. Thus, the main improvement remains with a fixed q, independent evaluation noise, and no per-seed retuning. These controls strengthen the CIFAR-10 evidence while keeping the ImageNet result separate from matched-baseline claims.

We appreciate your careful review. For the new comparison in Table 5, we fixed the original readout before evaluation and used noise disjoint from screening. The full model improved on both native MeanFlow and the zeroth-order cocycle control in both seeds. We paired native MeanFlow and the full model by initialization seed and training random stream for seeds 0 and 1. The full model had lower FID in both pairs at both approximate training budgets (Table 9). We hope these additions address your concerns. We’d appreciate hearing your updated view and would welcome a reconsideration of the rating if you find the new evidence persuasive.

Reviewer_JEU2OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes an extension of MeanFlow by jointly modeling higher-order moments of the velocity trajectory in addition to the average velocity field. These moments are combined through a linear readout during inference, which aims to provide a better estimate of the finite-time transport. The authors further introduce a cocycle consistency loss to regularize the moment predictions and a complex-valued readout strategy. Experiments on CIFAR-10 show that the proposed approach can achieve improvements over MeanFlow.

Critical strengths

The theoretical framework of the paper is generally sound and clearly formulated. The modeling of higher-order moments appears to be mathematically valid, and the proposed variance reduction argument is reasonable under the assumption that different moment predictions provide sufficiently independent estimates. The experiments on CIFAR-10 demonstrate that the proposed approach can improve upon MeanFlow to some extent.

Critical weaknesses
  1. I feel that this paper is somewhat incomplete, especially in terms of experimental evaluation. The authors do not compare their method with the current state-of-the-art few-step generation methods, which is a common practice for evaluating a new generative modeling approach. Instead, the experiments mainly compare against the original MeanFlow on CIFAR-10, which is insufficient to demonstrate the effectiveness of a new generation algorithm. The ImageNet results also appear to be reported without a sufficiently comprehensive evaluation or analysis. Overall, I have concerns about the completeness and maturity of the experimental validation, which is an important issue for a main conference submission.

  2. Regarding the comparison with existing methods, several works have attempted to improve MeanFlow after its introduction, such as Improved MeanFlow [1]. As a method aiming to improve MeanFlow, it is important for this paper to compare with these existing improvements. If the authors consider their contribution to be orthogonal to these approaches, it would be more convincing to integrate the proposed method into these stronger MeanFlow variants and demonstrate whether further improvements can be achieved. However, the paper does not provide such experiments or discussions.

  3. I am concerned that the readout coefficients introduced in this paper may be somewhat heuristic or sensitive. From the experimental results, the choice of the readout parameter has a substantial impact on performance: inappropriate coefficients may fail to improve the results or even degrade performance. However, the paper does not clearly explain how this parameter is selected or whether there is a principled criterion for choosing it. Such sensitivity makes the effectiveness of the proposed readout strategy less convincing. The authors should provide more analysis on the robustness of the readout selection and clarify whether the improvement comes from the proposed complex generating-function formulation or simply from finding a favorable linear combination of moment outputs.

[1] Geng, Zhengyang, et al. "Improved mean flows: On the challenges of fastforward generative models." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2026.

Questions for authors

I would like to know how many readout coefficients (or complex parameters qq) were explored during the selection process, and among these candidates, how many actually improve over the MeanFlow baseline. More importantly, is there any observable pattern in the effective readout coefficients, or how sensitive is the performance to the choice of qq? In my opinion, introducing a complex-valued readout parameter is a relatively novel design choice, and therefore a more comprehensive analysis of its robustness and selection principle is necessary.

2 comments

Thank you for suggesting a comparison with iMF and for asking how sensitive the results are to the readout.

1. Comparison with improved MeanFlow. We adapted the iMF objective to our unconditional CIFAR-10 SongUNet (Table 5).

ConditionSeed 0 FIDSeed 1 FID
iMF-objective baseline3.403.49
K=0 + cocycle3.353.43
K=3 + cocycle, fixed active readout3.243.32
Full model at approximate baseline compute3.333.41

At 100k updates, the full model improves on the adapted baseline by 0.16 and 0.17 FID and on the zeroth-order cocycle control by 0.11 in each seed. The last row uses 85,714 and 85,581 updates, chosen by proportional cost conversion to approximate the baseline budget (Table 10). The gains at those checkpoints are 0.07 and 0.08 FID.

At h=0h=0, the SongUNet reads dedicated auxiliary velocity channels to supply a detached JVP direction. For moment kk, we hold the lower endpoint fixed and divide the velocity residual by k+1k+1. The auxiliary loss regresses ϵ−x\epsilon-x with weight 1.0 relative to the adaptively normalized center loss (p=1p=1, offset 0.01). Training uses eight GPUs, a microbatch of 64 per GPU, and two accumulation steps. Training cost includes the boundary pass. Sampling uses only the moment outputs from one full-interval call (Appendix F.2).

For comparison, Table 13 reports published ImageNet-256 results alongside ours.

MethodNFETraining budgetFID
Shortcut XL [1]1250 epochs10.60
MeanFlow XL/2 [2]1240 epochs3.43
IMM XL/2 [3]21.2M updates, batch 40967.77
iMF XL/2 [4]1800 epochs, batch 10241.72
Moment Z-Flow1400k updates, batch 2562.44

Architectures, guidance, and budgets differ across these systems. For example, guided IMM uses two calls, and published iMF uses a modified 610M-parameter Transformer. All five systems train generators from scratch and use pretrained latent tokenizers. The CIFAR-10 experiment tests the moment extension under the improved objective on the same SongUNet backbone.

2. Readout sensitivity. Our records cover only part of the original search, so its total candidate count and win rate are unknown. For the new study, we screened all 35 combinations of real parts {-2, -1.5, -1, -0.5, 0, 0.5, 1} and imaginary parts {0, 0.25, 0.5, 0.75, 1}. We used the same noise, reference statistics, and preprocessing within each seed (Table 11).

FID-5K criterionSeed 0Seed 1
Better than MeanFlow25/3523/35
Better than K=0 + cocycle21/3519/35
Better than own center30/3529/35

The gains extend to nearby settings on both axes. On seed 0, holding the real part at -1 and varying the imaginary part from 0.25 to 0.5 to 0.75 gives FID-5K of 12.63, 12.58, and 12.64. Holding the imaginary part at 0.5 and moving the real part to -0.5 or -1.5 gives 12.73 and 12.70. MeanFlow scores 12.95 on the same noise.

3. Comparison with real-linear readouts. A fixed qq gives a real-linear readout. We also searched over real coefficients satisfying a0+a1/2+a2/3+a3/4=1a_0+a_1/2+a_2/3+a_3/4=1. Each family had 35 candidates with ∥a∥2≤10\|a\|_2\le 10 and ∥a∥1≤20\|a\|_1\le 20. The qq family also required ∣d3(q)∣≥0.05|d_3(q)|\ge 0.05. The real family included the center and fixed-qq readouts plus 33 projected Gaussian directions at log-spaced radii from 0.1 to 10. We rejected candidates outside the norm bounds before computing FID. The two searches match in size and norm bounds, though their coverage depends on how candidates are drawn (Appendix F.3).

We kept the original q=−1+0.5iq=-1+0.5i fixed throughout confirmation. For each search family, we selected coefficients using seed-0 FID-5K alone and carried them unchanged into confirmation on both seeds. Each confirmation score averages two FID-50K evaluations on noise disjoint from screening (Table 12).

ReadoutScreening FID-5KConfirmation seed 0 / 1
Fixed q=-1+0.5i12.583.57 / 3.70
Selected q*=-1+i12.553.55 / 3.69
Selected real-linear12.543.56 / 3.71

The original fixed readout captures most of the gain. Selecting qq improves confirmation FID by only 0.02 and 0.01 across the two seeds. Under this search budget, the two-parameter qq family and the three-degree-of-freedom real family give similar results.

References: [1] arXiv:2410.12557v3; [2] arXiv:2505.13447v1; [3] arXiv:2503.07565v2; [4] arXiv:2512.02012v2.

Thank you for your suggestions. We tested the moment extension with a CIFAR-10 adaptation of the improved MeanFlow objective and saw gains over the adapted baseline in both seeds (Table 5). On readout sensitivity, our new 35-point study found that several settings around the original choice improved on MeanFlow. The original fixed readout also performed close to the selected readouts on fresh evaluation noise (Appendix F.3). Table 13 now puts the ImageNet result alongside published methods and their training budgets. We hope this addresses your concerns about the evaluation. We’d welcome your updated thoughts and would be grateful if you’d consider whether the revised paper warrants a higher rating.

Reviewer_ZCfDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper extends the interval-average velocity of MeanFlow to a set of normalized temporal moments along a transport trajectory. It derives a local differential identity for each moment and an interval composition identity. It uses local regression and an EMA based cocycle loss to train a shared network. For inference, the predicted moments are combined through a normalized linear readout. The coefficients are parameterized by a complex number. The method still uses one network evaluation and the experiments consistantly show the efforts.

Critical strengths

(1) The mathematical formulation is clear about what the method does and does not guarantee. In particular, Eq. (13) separates moment-estimation error from the bias introduced by the nonzero readout parameter. The appendix A.4 gives a concrete example of that bias, and the appendix D.1 explains why the actual weighted, detached training update is not simply optimization of the exact hierarchy residual. These distinctions make the theoretical claims easier to assess.

(2) The comparisons give useful evidence that the readout matters in practice. Its improvement over the center readout appears in both training seeds in Table 6. Table 3 separately shows that cocycle supervision improves the full model in both seeds. The extra outputs add very few parameters and preserve 1-NFE sampling.

Critical weaknesses

(1) It is unclear how much the higher moments help. Without cocycle supervision, the model performs worse than native MeanFlow. A (K=0) baseline with the same cocycle loss is needed to separate the benefit of higher moments from interval consistency. The center-versus-active comparison does not answer this question.

(2) The experimental section needs more detail. In particular, the ImageNet initialization and training setup are not sufficiently clear, and an equal-compute comparison with MeanFlow would make the reported gains easier to assess.

Questions for authors

(1) How does a (K=0) model trained with the same cocycle loss perform? This would help clarify how much of the gain comes from the higher moments.

(2) Could you clarify the ImageNet training setup, especially whether the model is trained from scratch or initialized from a pretrained checkpoint?

(3) Does the improvement over native MeanFlow hold when both methods receive the same training compute budget?

2 comments

We thank the reviewer for recognizing the distinction between exact moment identities, the training surrogate, and readout bias. The added controls show that the full moment extension improves over zeroth-order cocycle training in both seeds, with positive results also observed at approximately matched training budgets.

1. The full moment extension improves over zeroth-order consistency.

At 100k updates, independent confirmation gives:

ModelSeed 0 FIDSeed 1 FID
Native MeanFlow3.853.91
K=0 + cocycle3.723.80
K=3 + cocycle, fixed q=-1+0.5i3.573.70

The full extension improves over K=0 + cocycle by 0.15 and 0.10 FID. Each entry averages two FID-50K evaluations on noise disjoint from screening. Methods share evaluation noise, reference statistics, and preprocessing within each comparison.

The K=0 control retains the EMA-teacher intermediate-state construction, detached subinterval targets, split distribution, optimizer, and global cocycle schedule. Both models use the order-averaged objective in Eqs. (10)–(11). The zeroth-order term has coefficient γj\gamma_j for K=0K=0 and γj/4\gamma_j/4 for K=3K=3. Thus, the global weight and normalization rule are shared.

This comparison supports the full extension beyond the specified K=0 control; it evaluates the combined moment supervision and readout rather than isolating each component. Together with the submitted no-cocycle results (4.01/4.07), it identifies the combination of moment learning, interval consistency, and readout as the effective method. The submitted full-model scores 3.54/3.68 remain unchanged; 3.57/3.70 are independent confirmation scores for the same checkpoints.

2. ImageNet generator initialization and training are now explicit.

The DiT-XL/2 generator starts from random weights, without a pretrained generator checkpoint. The 400k updates cover the entire joint moment-and-cocycle training stage. The zero-initialized output projection predicts 16 latent channels: four for m0m_0 and four for each of three residual heads. The parameterization mk=m0/(k+1)+(t−r)ρkm_k=m_0/(k+1)+(t-r)\rho_k enforces the diagonal ratios for bounded residuals.

The fixed stabilityai/sd-vae-ft-mse VAE is pretrained; “from scratch” refers to the generator. There is no generator pretraining cost, and external VAE pretraining is excluded from generator-training accounting.

The other Appendix B settings are unchanged: batch 256, AdamW at 1e-4, betas (0.9,0.95), zero weight decay, 10k warmup, and EMA 0.9999. Evaluation uses 50k class-balanced samples, seed 42, and the ADM evaluator/reference. At sampling scale 1.0, no unconditional prediction is mixed into the class-conditional output. Sampling requires one generator call; VAE decoding is excluded from NFE. Appendix B now includes the generator initialization details.

3. Positive results at approximately matched training budgets.

The supplementary comparison below uses the same independent confirmation protocol. We used the original GPU-hour totals as approximate targets.

Target GPU-hoursSeedMeanFlow steps / FIDFull steps / FID
48.150100000 / 3.8585252 / 3.72
48.15198226 / 3.9284326 / 3.84
56.480117301 / 3.78100000 / 3.57
56.481115219 / 3.8398915 / 3.72

Improvements are 0.13/0.08 at the lower budget and 0.21/0.11 at the higher budget. MeanFlow receives additional updates at the higher budget, while the full model receives fewer at the lower budget. q is fixed throughout.

Runs use eight H100 GPUs, FP32, and batch 1024. Supplementary timing accumulates synchronized training-step time, including computation, optimizer/EMA updates, and communication, but excluding startup, checkpoint I/O, and evaluation. The first checkpoint reaching each prespecified threshold is selected independently of FID. The full model had lower FID in both seed pairs at both target budgets.

Thank you again for your feedback. Table 5 now includes the zeroth-order cocycle control you suggested. At 100k updates, the full model with a fixed readout improves on that control by 0.15 and 0.10 FID across the two seeds. It also retains an advantage over native MeanFlow at both approximate training budgets (Table 9). For ImageNet, Appendix B now clarifies that we trained the generator from random initialization for all 400k updates and used a fixed pretrained VAE. We hope these revisions address your concerns. We’d appreciate your updated thoughts and, if the new results have strengthened your view of the paper, your consideration of a higher rating.

Comments on the paper

1 comment

We thank the reviewers for suggesting the controls and comparisons below. We have incorporated the results from our responses into the revised manuscript.

  1. Higher moments and zeroth-order consistency (Section 5.7, Table 5). At 100k CIFAR-10 updates, native MeanFlow obtains FIDs of 3.85 and 3.91 for seeds 0 and 1. Adding cocycle supervision at K=0K=0 gives 3.72 and 3.80. The full K=3K=3 model with the fixed readout q=−1+0.5iq=-1+0.5i gives 3.57 and 3.70. Each score averages two FID-50K evaluations on noise disjoint from screening. The gains of 0.15 and 0.10 over the zeroth-order control measure the combined contribution of higher-moment supervision and the active readout. The original full-model scores of 3.54 and 3.68 come from the same checkpoints under the original evaluation protocol.

  2. Training compute (Appendix F.1, Table 9). We report update counts and FID for both methods at target budgets of 48.15 and 56.48 GPU-hours. The full model improves over native MeanFlow by 0.13 and 0.08 FID at the lower budget, and by 0.21 and 0.11 at the higher budget. Appendix F.1 explains how we timed training and selected checkpoints before FID evaluation. We paired native MeanFlow and the full model by initialization seed and training random stream for seeds 0 and 1. The GPU-hour totals from the original runs serve as approximate budget targets.

  3. Transfer to the improved MeanFlow objective (Section 5.7 and Appendix F.2, Tables 5 and 10). In our unconditional CIFAR-10 SongUNet adaptation of the iMF objective, the full model reaches 3.24 and 3.32 FID at 100k updates, compared with 3.40 and 3.49 for the adapted baseline. It improves over the corresponding K=0K=0 cocycle control by 0.11 in each seed. Earlier checkpoints retain gains of 0.07 and 0.08 at approximate baseline compute.

  4. Readout sensitivity and selection (Appendix F.3, Tables 11–12). In a separately specified 35-point grid, 25 settings on seed 0 and 23 on seed 1 outperform native MeanFlow on matched FID-5K evaluations. We selected coefficients on seed 0 alone, froze them before independent FID-50K confirmation, and transferred them unchanged to seed 1. The fixed readout gives 3.57 and 3.70, the selected qq gives 3.55 and 3.69, and the selected real-linear readout gives 3.56 and 3.71. Appendix F.3 also explains why the available records cannot establish the complete original search budget.

  5. ImageNet initialization and comparisons (Section 5.8, Appendix B, and Table 13). The DiT-XL/2 generator starts from random weights, and all 400k updates are part of joint moment-and-cocycle training. The VAE is fixed and pretrained. Appendix B gives the initialization and evaluation details for the reported one-step FID-50K of 2.44. Table 13 places this result alongside Shortcut, MeanFlow, IMM, and iMF, with generator training budgets and network evaluations shown explicitly. Architectures and guidance settings differ across these published systems.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.