acceptodds
Under review as a conference paper at ICLR 2027

Plain Flow Maps with Analytic Heads

11 views

Abstract

Flow maps learn direct jumps between arbitrary points along diffusion ODE trajectories via differential objectives. This requires differentiating the entire model with respect to time, incurring severe overheads. We show that this is unnecessary. We introduce plain flow maps, which retain a standard diffusion backbone and expose the differentiated time only to a lightweight analytic head. We show that this head can be as simple as a single layer featuring time-adaptive spatial and channel mixing, yet remains highly effective. We introduce two parameterizations, secant and velocity, allowing terminal-time differentiation to be resolved solely within the head via closed-form derivatives and integrals, respectively. This design completely bypasses explicit differentiation tools and their associated drawbacks. Extensive experiments demonstrate remarkable performance on from-scratch ImageNet and text-to-image training.

Primary area:
generative models
Keywords:
generative models, diffusion models

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 4 reviews, above 85% of the others.
Reviewer_6RZHOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Plain Flow Maps with Analytic Heads, which bypass expensive full-backbone differentiation with respect to terminal time by restricting required derivative computations to a lightweight analytic head. The paper demonstrates strong results on ImageNet and text-to-image generation, with substantial improvements in training efficiency.

Critical strengths
  1. This paper reduces the training budget by bypassing expensive full-backbone differentiation. The overall theory is sound, and I find no fatal errors in the proofs.

  2. The experiments clearly demonstrate the validity of the proposed methods.

Critical weaknesses

While I do not have major concerns with this paper, I have some minor concerns:

  1. The flow-map composition consistency across different time intervals is not explicitly evaluated. The paper does not explicitly evaluate the flow-map composition property across different time intervals. Since the predicted trajectories are not guaranteed to be consistent under imperfect training, additional experiments would strengthen the theoretical justification.

  2. The computational overhead of the analytic bilinear head, particularly for Plain-Velocity, deserves further analysis. Plain-Velocity requires a relatively complicated Fourier-based integration procedure involving 769 temporal evaluation points. A detailed computational comparison with Plain-Secant would help justify this additional complexity.

  3. Numerical verification of the closed-form derivatives and integrals would strengthen the paper's reliability. Although the analytic derivatives and integrals appear mathematically correct, the paper lacks numerical comparisons against automatic differentiation and high-precision integration. Such verification would increase confidence in the implementation's correctness and numerical stability.

Questions for authors

Please refer to Weaknesses.

5 comments

We thank the reviewer for the positive assessment of our method and for suggesting these additional validations.

W1. Flow-map composition consistency.

We agree that exact temporal differentiation/integration does not guarantee exact composition for a finitely trained model. To investigate whether restricting terminal-time conditioning to a lightweight head limits the representation of arbitrary-time transitions, we evaluate XL/2 models using 1,024 samples.

For t>s>rt>s>r, we measure the discrepancy between the direct transition Fθ(xt,t,r)F_\theta(x_t,t,r) and the composed transition Fθ(Fθ(xt,t,s),s,r)F_\theta(F_\theta(x_t,t,s),s,r). We select t∈0.25,0.5,0.75,1t\in\\{0.25,0.5,0.75,1\\} and r∈0,0.25,0.5,0.75r\in\\{0,0.25,0.5,0.75\\} with r<tr<t, yielding 10 distinct intervals. Each interval is evaluated at three intermediate times, s=(1−α)t+αrs=(1-\alpha)t+\alpha r, where α∈0.25,0.5,0.75\alpha\in\\{0.25,0.5,0.75\\}, resulting in 30 time triples.

Table 5 in the revised manuscript reports the composition consistency errors (relative RMSE, %):

MethodComposition RMSE (%) ↓
MeanFlow-XL/216.94
SoFlow-XL/214.53
TVM-XL/28.05
Plain-Secant-XL/2 (Ours)7.36
Plain-Velocity-XL/2 (Ours)7.46

Both Plain-Secant and Plain-Velocity achieve the lowest composition errors among the evaluated methods, outperforming TVM (8.05%) with errors of 7.36% and 7.46%, respectively. These results provide direct empirical evidence that restricting terminal-time conditioning to a lightweight head does not introduce a noticeable consistency degradation relative to the evaluated baselines. The comparable results of Secant and Velocity further support the effectiveness of our shared analytic head architecture across both parameterizations.

W2. Computational overhead of Plain-Velocity.

We thank the reviewer for highlighting the computational overhead of Plain-Velocity's Fourier-based integration. To address this concern, we provide comprehensive benchmarks at both the system level (training and inference) and the head level. Since both parameterizations share the same standard DiT backbone and bilinear head architecture, their main computational differences arise from the terminal-time operations. Our results demonstrate that the additional integration overhead remains small at the system level.

i) Training efficiency. We have added Plain-Velocity to the training efficiency comparison in Table 3 of the revised manuscript:

MethodBatch sizeTime (s) ↓Mem (GB) ↓
DiT-XL/22560.1726.49
MeanFlow-XL/2†2560.8146.73
SoFlow-XL/22560.2627.04
TVM-B/22560.3119.37
TVM-XL/22560.9571.44
Plain-Secant-XL/2 (Ours)2560.2427.60
Plain-Secant-XL/2 (Ours)10240.7268.45
Plain-Velocity-XL/2 (Ours)2560.2427.69
Plain-Velocity-XL/2 (Ours)10240.7368.77

Plain-Velocity achieves nearly identical training efficiency to Plain-Secant. At batch size 256, both require only 0.24 s/step, with nearly identical peak memory usage. Under the 80GB GPU memory constraint, both support a maximum feasible batch size of 1024, with step times of 0.72 s and 0.73 s, respectively. Relative to TVM-XL/2, both parameterizations achieve approximately 4× faster training steps and 61% lower peak memory at batch size 256.

ii) Inference efficiency. We additionally benchmark inference latency and peak GPU memory, with the results included in Table 4 of the revised manuscript:

MethodNFETime (ms) ↓Mem (GB) ↓
DiT-XL/2113.392.587
226.712.587
MeanFlow-XL/2113.682.594
227.072.594
SoFlow-XL/2113.762.594
227.482.594
TVM-XL/2121.542.599
243.122.599
Plain-Secant-XL/2 (Ours)114.282.610
228.482.612
Plain-Velocity-XL/2 (Ours)115.212.613
229.952.613

Both variants achieve inference speeds close to standard DiT, while being approximately 1.4–1.5× faster than TVM-XL/2 at 1–2 NFE. Unlike TVM, which introduces additional architectural modifications for training stability, our framework retains the standard DiT backbone without such modifications. Despite its FP32 analytic integration, Plain-Velocity incurs only 5–7% additional inference latency compared with Plain-Secant, with nearly identical memory consumption.

iii) Head-level forward-and-backward cost. To isolate the computational overhead of Fourier-based integration, we benchmark the forward-and-backward computation of both analytic heads under the default configuration.

MethodTime (ms) ↓Peak memory (MB) ↓
Plain-Secant head (BF16)3.630166.3
Plain-Velocity head (FP32)3.859200.4

Plain-Velocity incurs only 6.3% additional head-level forward-and-backward latency, while peak memory increases by approximately 20.5%, primarily reflecting the FP32 spectral integration. Importantly, the 769-point Fourier grid evaluates only lightweight temporal modulation factors, without repeated backbone evaluations or repeated full attention-head evaluations.

These results explain why the additional integration complexity incurs only minor system-level overhead. Our unified analytic head architecture supports both closed-form differentiation and integration while retaining computational efficiency comparable to Plain-Secant. Detailed benchmarking protocols are provided in Appendix C.2 of the revised manuscript.

W3. Numerical verification of analytic derivatives and integrals.

We thank the reviewer for this valuable suggestion. To verify the correctness and numerical stability of our analytic implementations, we conduct numerical comparisons against automatic differentiation (AD) and independent high-precision numerical integration under FP32 and BF16.

i) Plain-Secant derivative accuracy. We compare our analytic terminal derivative against head-only AD and an FP64 AD reference.

PrecisionAnalytic vs. AD (%) ↓Analytic vs. FP64 (%) ↓AD vs. FP64 (%) ↓
FP324.231×10−64.231\times10^{-6}3.343×10−53.343\times10^{-5}3.342×10−53.342\times10^{-5}
BF160.1670.5330.526

Under FP32, our analytic derivative agrees almost perfectly with AD, with a relative error of only 4.231×10−6%4.231\times10^{-6}\%. Both methods exhibit nearly identical errors relative to the FP64 reference. Under BF16, the analytic and AD implementations remain comparably accurate (0.533% vs. 0.526% relative to FP64), indicating that our explicit differentiation introduces no appreciable numerical degradation beyond mixed-precision rounding.

ii) Plain-Velocity integral accuracy. We compare our closed-form integral against an independently computed FP64 numerical reference using 128-node Gauss–Legendre quadrature, across different integration gaps.

Gap δ\deltaFP64 (%) ↓FP32 (%) ↓BF16 (%) ↓
11.504×10−101.504\times10^{-10}0.23325.15
0.53.128×10−93.128\times10^{-9}0.20132.07
0.14.173×10−94.173\times10^{-9}0.0656.462
10−210^{-2}3.912×10−93.912\times10^{-9}0.0520.705
10−310^{-3}4.051×10−94.051\times10^{-9}0.3950.509
10−410^{-4}3.142×10−93.142\times10^{-9}0.5890.675

The FP64 closed-form implementation agrees with the independent quadrature reference to within 4.2×10−9%4.2\times10^{-9}\%, providing strong numerical validation of the analytic integration. Under FP32, relative errors remain below 0.6% across all tested intervals. In contrast, BF16 exhibits substantially larger errors, reaching 32.07%, due to its sensitivity to floating-point rounding and cancellation among Fourier terms.

These findings support our implementation choice to evaluate the complete Plain-Velocity integral head in FP32. As demonstrated by our computational benchmarks in W2, this precision requirement introduces only minor practical overhead. We have added these results in Appendix B.6 in the revised manuscript.

Reviewer_NbmNOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces Plain Flow Maps, an efficient generative modeling framework that confines terminal-time conditioning and explicit differentiation to a lightweight analytic head, eliminating the need for time differentiation through the diffusion backbone. Specifically, the authors propose two trajectory parameterizations, Plain-Secant and Plain-Velocity, along with a harmonic bilinear head that enables exact temporal differentiation and integration. Experiments on ImageNet-256 and from-scratch text-to-image generation demonstrate strong few-step generation performance, while significantly improving training speed and memory efficiency.

Critical strengths
  1. Novel and efficient architectural design. The paper decouples terminal-time conditioning from the diffusion backbone and confines explicit temporal differentiation to a lightweight analytic head, effectively eliminating the computational overhead and engineering complexity associated with full-backbone JVPs in conventional flow-map training.

  2. Elegant and mathematically grounded analytic head. By combining harmonic time embeddings, time-adaptive channel mixing, and linear attention, the proposed head enables exact temporal differentiation and integration while maintaining sufficient representational capacity.

  3. Well-motivated design and comprehensive ablations. The layer-wise derivative analysis and terminal-time injection experiments provide convincing evidence for restricting terminal-time conditioning to the final head.

Critical weaknesses
  1. Insufficient comparisons with stronger baselines and limited performance attribution. Although the proposed method achieves competitive generative performance, iMF and α-Flow report better FID scores in the supplementary material but are omitted from the main comparison table.

  2. Lack of direct evaluation of flow-map composition consistency. Although the composition property is fundamental to flow maps, the paper primarily evaluates generation quality without explicitly measuring consistency across different time intervals. Furthermore, the ImageNet experiments mainly focus on 1–2 NFE, leaving generalization to more sampling steps and arbitrary time intervals insufficiently explored.

  3. Limited analysis of the practical efficiency and numerical stability of Plain-Velocity. Although the proposed analytic integration is mathematically exact, its implementation involves multiple Fourier components and requires FP32 computation to mitigate numerical errors.

Questions for authors

See weakness.

4 comments

We thank the reviewer for recognizing the architectural contribution and analytic formulation. We address the three concerns below.

W1. Stronger baselines and performance attribution.

We agree with your suggestion. In the revised manuscript, we have included α\alpha-Flow and iMF in the main Table 2.

Our primary motivation is to make flow-map training more practical through architectural simplification. While source-differentiated methods such as iMF and the MeanFlow stage of α\alpha-Flow still require full-network JVPs during training, our terminal-time separation entirely eliminates backbone JVPs. Unlike TVM, our method retains a standard DiT backbone without specialized stabilization modifications or JVP-compatible kernels. Compared with TVM-XL/2, Plain-Secant achieves approximately 4× faster training steps and 61% lower peak memory, while improving the reported 1/2-NFE FIDs from 3.29/2.80 to 2.89/2.40.

We will refine our claims to emphasize simplicity and efficiency gains, together with competitive few-step generative quality, rather than claiming absolute SOTA FID across all existing methods.

W2. Composition consistency and generalization to more sampling steps.

We agree that endpoint FID alone does not directly evaluate arbitrary-time flow maps. To address this concern, we conduct two complementary evaluations: composition consistency across different time intervals and generation quality with increased sampling steps.

i) Composition consistency. To investigate whether restricting terminal-time conditioning to a lightweight head limits the representation of arbitrary-time transitions, we evaluate XL/2 models using 1,024 samples.

For t>s>rt>s>r, we measure the discrepancy between the direct transition Fθ(xt,t,r)F_\theta(x_t,t,r) and the composed transition Fθ(Fθ(xt,t,s),s,r)F_\theta(F_\theta(x_t,t,s),s,r). We select t∈0.25,0.5,0.75,1t\in\\{0.25,0.5,0.75,1\\} and r∈0,0.25,0.5,0.75r\in\\{0,0.25,0.5,0.75\\} with r<tr<t, yielding 10 distinct intervals. Each interval is evaluated at three intermediate times, s=(1−α)t+αrs=(1-\alpha)t+\alpha r, where α∈0.25,0.5,0.75\alpha\in\\{0.25,0.5,0.75\\}, resulting in 30 time triples.

Table 5 in the revised manuscript reports the composition consistency errors (relative RMSE, %):

MethodComposition RMSE (%) ↓
MeanFlow-XL/216.94
SoFlow-XL/214.53
TVM-XL/28.05
Plain-Secant-XL/2 (Ours)7.36
Plain-Velocity-XL/2 (Ours)7.46

Both Plain-Secant and Plain-Velocity achieve the lowest composition errors among the evaluated methods, outperforming TVM (8.05%) with errors of 7.36% and 7.46%, respectively. These results provide direct empirical evidence that restricting terminal-time conditioning to a lightweight head does not introduce a noticeable consistency degradation relative to the evaluated baselines. The comparable results of Secant and Velocity further support the effectiveness of our shared analytic head architecture across both parameterizations.

ii) Generalization to more sampling steps. We additionally extend the ImageNet-256 evaluation from 1–2 NFE to up to 128 NFE, using the existing 300K-step EMA checkpoints of both XL/2 models.

NFEPlain-Secant FID ↓Plain-Velocity FID ↓
12.892.92
22.402.43
42.012.05
81.851.86
161.821.79
321.771.75
641.741.78
1281.781.77

Both parameterizations exhibit substantial improvements as the number of sampling steps increases. Plain-Secant achieves its best FID of 1.74 at 64 NFE, while Plain-Velocity reaches 1.75 at 32 NFE. Beyond 32 NFE, performance largely stabilizes with only minor fluctuations, rather than deteriorating under repeated flow-map composition.

Together, these results demonstrate that our lightweight-head architecture not only achieves competitive composition consistency across different time intervals but also supports effective multi-step generation beyond the originally evaluated 1–2 NFE regime. This provides additional empirical support for the ability of both parameterizations to represent and compose finite-time transitions. We have added these results in Table 5 and Appendix Figure 9 in the revised manuscript.

W3. Plain-Velocity efficiency and numerical stability.

Since Plain-Velocity shares the exact same backbone and only differs in the integration of the lightweight analytic head, its system-level costs are similar to Plain-Secant. Our new experiments demonstrate that this overhead is remarkably small in practice.

i) Training efficiency. We have added Plain-Velocity to the training efficiency comparison in Table 3 of the revised manuscript:

MethodBatch sizeTime (s) ↓Mem (GB) ↓
DiT-XL/22560.1726.49
MeanFlow-XL/2†2560.8146.73
SoFlow-XL/22560.2627.04
TVM-B/22560.3119.37
TVM-XL/22560.9571.44
Plain-Secant-XL/2 (Ours)2560.2427.60
Plain-Secant-XL/2 (Ours)10240.7268.45
Plain-Velocity-XL/2 (Ours)2560.2427.69
Plain-Velocity-XL/2 (Ours)10240.7368.77

Remarkably, Plain-Velocity achieves nearly identical training efficiency to Plain-Secant. Both also achieve a maximum feasible batch size of 1024 under the 80GB GPU memory constraint. These results demonstrate that the additional Fourier-based integration and its FP32 computation introduce negligible system-level training overhead. Both parameterizations retain approximately 4× faster training steps and 61% lower peak memory than TVM-XL/2 at batch size 256.

ii) Inference efficiency. We additionally benchmarked low-NFE inference latency and peak GPU memory against related methods, with results added to Table 4:

MethodNFETime (ms) ↓Mem (GB) ↓
DiT-XL/2113.392.587
226.712.587
MeanFlow-XL/2113.682.594
227.072.594
SoFlow-XL/2113.762.594
227.482.594
TVM-XL/2121.542.599
243.122.599
Plain-Secant-XL/2 (Ours)114.282.610
228.482.612
Plain-Velocity-XL/2 (Ours)115.212.613
229.952.613

Both variants achieve inference speeds close to standard DiT and are approximately 1.4–1.5× faster than TVM-XL/2 at 1–2 NFE. Unlike TVM, which introduces additional architectural modifications to stabilize terminal-differentiated training, our framework retains the standard DiT backbone without such modifications. This architectural simplicity preserves near-DiT inference efficiency. Despite its FP32 analytic integration, Plain-Velocity incurs only approximately 5–7% additional inference latency compared with Plain-Secant, with nearly identical memory consumption.

iii) Numerical accuracy of closed-form integration. As discussed in Appendix B.5, the bilinear Velocity integral involves products of three Fourier-valued factors, whose numerical cancellation can amplify rounding errors under BF16. Following the reviewer's suggestion, we compare the closed-form integral against an independent high-precision FP64 numerical integration reference across different time intervals.

The following table reports relative integral errors (%):

Time gap δ\deltaFP64 (%) ↓FP32 (%) ↓BF16 (%) ↓
11.504 × 10⁻¹⁰0.23325.15
0.53.128 × 10⁻⁹0.20132.07
0.14.173 × 10⁻⁹0.0656.462
10⁻²3.912 × 10⁻⁹0.0520.705
10⁻³4.051 × 10⁻⁹0.3950.509
10⁻⁴3.142 × 10⁻⁹0.5890.675

The FP64 closed-form implementation agrees with the numerical reference to extremely high precision, supporting the correctness of our analytic derivation and implementation. Under FP32, relative errors remain below 0.6% across all tested intervals, whereas BF16 errors reach 32.07%, demonstrating the sensitivity to floating-point cancellation.

These findings empirically justify our choice to evaluate the complete Plain-Velocity integral head in FP32. Importantly, the training and inference benchmarks demonstrate that this precision requirement introduces only minor practical overhead, since the integration is confined to the lightweight head rather than the backbone.

Detailed benchmarking protocols are provided in Appendix C.2, and the numerical verification is included in Appendix B.6 of the revised manuscript.

Reviewer_NbmNOpenReview ↗

Official Comment by Reviewer NbmN

Thanks to the authors for their response, which has addressed some of my concerns. Therefore, I will maintain my original acceptance score.

We sincerely thank the reviewer for taking the time to carefully consider our rebuttal and additional experimental results. We greatly appreciate your constructive feedback and continued support. Your suggestions have been valuable in helping us strengthen the experimental evaluation and improve the overall presentation of our work.

Reviewer_3EEKOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Plain Flow Maps, a framework for efficient few-step generative modeling that avoids explicit terminal-time differentiation through the entire diffusion backbone. The key idea is to separate source-state representation learning from terminal-time conditioning. Specifically, the authors employ a standard diffusion backbone conditioned only on the source state and source time, while introducing the terminal time exclusively through a lightweight analytic head.

The authors develop two parameterizations, Plain-Secant and Plain-Velocity, and design an analytic bilinear head using harmonic time embeddings, channel modulation, and linear attention. This allows terminal-time derivatives and integrals to be computed analytically without full-backbone Jacobian-vector products.

Experiments on ImageNet-256×256 and text-to-image generation demonstrate promising generation quality, with substantial improvements in training efficiency compared with previous terminal-differentiation methods. Additional analyses investigate terminal-time conditioning, head architectures, and model scalability.

Critical strengths

Strengths

  1. The paper is well-written and presents a simple yet effective idea. The motivating studies in Figures 3 and 4 provide useful insights into terminal-time conditioning and help justify the proposed architecture.
  2. The analytical formulations are elegant and well-developed. In particular, the bilinear head provides a practical way to retain sufficient modeling capacity while enabling analytical differentiation and integration.
  3. The experimental results are promising. The method demonstrates strong generation quality on both ImageNet and text-to-image benchmarks, while Plain-Secant achieves substantial improvements in training speed and memory efficiency.
Critical weaknesses

Weaknesses

  1. The comparisons with recent methods are not sufficiently comprehensive in the main paper. In particular, Table 2 omits iMF-XL/2 and α-Flow-XL/2, although both methods are included in Appendix Table 9 and achieve better FID scores than the proposed method. For example, iMF-XL/2 achieves FIDs of 1.72/1.54 at 1/2 NFEs, compared with 2.89/2.40 for Plain-Secant-XL/2. I understand that the main contribution of this paper is training efficiency rather than solely generation quality. Nevertheless, a more comprehensive comparison of the quality-efficiency trade-off against these stronger baselines would make the experimental claims more convincing.
  2. Although the paper demonstrates improvements over TVM, it remains unclear how much of the improvement in generation quality can be attributed specifically to the proposed analytic head. Around Lines 353–358, the authors mention several differences in training configurations, including CFG-scale conditioning and changes to the time-sampling distribution. These factors may also affect generation quality. I would appreciate a more controlled comparison under identical training configurations. Additionally, for Plain-Secant, it would be useful to compare analytical differentiation with automatic differentiation restricted to the same lightweight head. This would help clarify how much of the efficiency improvement actually comes from analytical computation rather than simply removing terminal-time conditioning from the backbone.
  3. The computational efficiency of Plain-Velocity is not sufficiently evaluated. According to Appendix B.3, its analytical integration uses a Fourier grid containing 769 points under the default configuration. Furthermore, Appendix B.5 mentions that the bilinear integral must be evaluated in FP32 to avoid numerical errors. However, Table 3 only reports the training efficiency of Plain-Secant. I am concerned that the additional analytical integration may introduce non-negligible computational overhead. The authors should provide training time, memory consumption, and inference latency for both parameterizations to better demonstrate their practical efficiency.
  4. The experimental evaluation does not sufficiently examine whether the proposed architecture accurately represents flow maps between arbitrary time points. One of the central motivations of the paper is to learn transitions between arbitrary source and terminal times, and Equation (4) introduces the composition property of flow maps. However, the experiments primarily evaluate the final generation quality using a small number of sampling steps. It would be useful to directly evaluate trajectory approximation errors and composition consistency across different time intervals. Such analyses would provide stronger evidence that restricting terminal-time conditioning to a lightweight head does not introduce important representational limitations.
  5. The text-to-image experiments demonstrate promising performance, but they lack a controlled comparison using the same backbone and training recipe. As described around Lines 423–430, the authors modify the PixelGen architecture and adopt a different VAE. Consequently, it is difficult to determine how much of the improvement comes from the proposed method rather than other architectural and training choices. A comparison with a standard flow-matching baseline trained under identical settings would further strengthen the conclusions.

Errors

  1. Around Lines 302–305, the authors refer to (F_\theta(X_t,t,t,c,w)) as the local prediction used by the training objective. However, according to Equations (14) and (18), (F_\theta(X_t,t,t,c,w)=X_t). I believe the authors intend to refer to (f_\theta(X_t,t,t,c,w)), which appears in Equation (21).
  2. The DPG-Bench score of SANA-Sprint is reported as 81.10 in Table 4 but 81.08 in Appendix Table 11. These values should be consistent.
Questions for authors

Questions

  1. Around Lines 302–306, the authors state that both objectives require only a single backbone evaluation. However, Equation (22) also involves an EMA teacher evaluated at the predicted terminal state. Does the statement refer only to the student backbone? I would appreciate a more precise explanation of the number of backbone evaluations required during training.
  2. How sensitive is the analytic bilinear head to the harmonic frequency configuration? The paper uses a fixed harmonic representation, but does not provide an ablation on the number of frequencies or the frequency scale. I would be interested in understanding how these choices affect generation quality, numerical stability, and computational efficiency, particularly for Plain-Velocity.
6 comments

Thank you for the constructive feedback. And we address the raised concerns as follows.

W1. Baselines and positioning.

We agree with your suggestion. In the revised manuscript, we have included α\alpha-Flow and iMF in the main Table 2.

Our primary motivation is to make flow-map training more practical through architectural simplification. While source-differentiated methods such as iMF and the MeanFlow stage of α\alpha-Flow still require full-network JVPs during training, our terminal-time separation entirely eliminates backbone JVPs. Unlike TVM, our method retains a standard DiT backbone without specialized stabilization modifications or JVP-compatible kernels. Compared with TVM-XL/2, Plain-Secant achieves approximately 4× faster training steps and 61% lower peak memory, while improving the reported 1/2-NFE FIDs from 3.29/2.80 to 2.89/2.40.

We will refine our claims to emphasize simplicity and efficiency gains, together with competitive few-step generative quality, rather than claiming absolute SOTA FID across all existing methods.

W2. Controlled head comparison and analytical computation.

i) Controlled comparison of head architectures. We would like to clarify that Table 1, Figure 6 (left), and Appendix Figure 8 already provide a controlled comparison between the standard DiT head and our bilinear analytic head. Both use identical backbones, training objectives, CFG conditioning, time sampling, optimization settings, and training budgets. Across the B/2, M/2, L/2, and XL/2 scales, our bilinear head achieves 6.3%, 7.7%, 7.7%, and 8.0% lower FID, respectively, than the standard DiT head. Thus, the training adaptations do not confound this comparison, directly supporting the effectiveness of our proposed head design.

ii) Analytic differentiation vs. head-only autodiff. To isolate the computational benefit of analytical differentiation from architectural decoupling, we additionally benchmark analytical differentiation against head-only automatic differentiation (AD) using the identical bilinear head.

MethodTime (ms) ↓Memory (MB) ↓
Head-only AD7.66161.90
Analytic (Ours)3.68166.51

Our analytic implementation achieves a 2.08× speedup in head-level forward-and-backward computation over AD, with only a minor memory overhead.

Importantly, our contribution consists of three complementary aspects: (1) Architectural decoupling, which eliminates full-backbone terminal-time JVPs; (2) Analytic differentiation, which provides an additional 2.08× head-level speedup over AD; and (3) Unified analytic design, where a single analytically tractable head architecture supports both closed-form differentiation for Plain-Secant and exact integration for Plain-Velocity. While AD can compute derivatives, it does not directly provide the definite integrals required by Plain-Velocity. Our unified design enables both parameterizations using standard tensor operations and reverse-mode backpropagation, without relying on JVP-specific autodiff support, simplifying implementation and improving portability across computational backends.

We have included this comparison in the revised appendix B.6.

W3. Plain-Velocity efficiency.

We thank the reviewer for highlighting the missing efficiency measurements for Plain-Velocity. Since Plain-Velocity shares the exact same backbone and only differs in the integration of the lightweight analytic head, its system-level costs are similar to Plain-Secant.

i) Training efficiency. We have added Plain-Velocity to the training efficiency comparison in Table 3 of the revised manuscript:

MethodBatch sizeTime (s) ↓Mem (GB) ↓
DiT-XL/22560.1726.49
MeanFlow-XL/2†2560.8146.73
SoFlow-XL/22560.2627.04
TVM-B/22560.3119.37
TVM-XL/22560.9571.44
Plain-Secant-XL/2 (Ours)2560.2427.60
Plain-Secant-XL/2 (Ours)10240.7268.45
Plain-Velocity-XL/2 (Ours)2560.2427.69
Plain-Velocity-XL/2 (Ours)10240.7368.77

Remarkably, Plain-Velocity achieves nearly identical training efficiency to Plain-Secant. Both also achieve a maximum feasible batch size of 1024 under the 80GB GPU memory constraint. These results demonstrate that the additional Fourier-based integration and its FP32 computation introduce negligible system-level training overhead. Both parameterizations retain approximately 4× faster training steps and 61% lower peak memory than TVM-XL/2 at batch size 256.

ii) Inference efficiency. We additionally benchmarked low-NFE inference latency and peak GPU memory against related methods, with results added to Table 4:

MethodNFETime (ms) ↓Mem (GB) ↓
DiT-XL/2113.392.587
226.712.587
MeanFlow-XL/2113.682.594
227.072.594
SoFlow-XL/2113.762.594
227.482.594
TVM-XL/2121.542.599
243.122.599
Plain-Secant-XL/2 (Ours)114.282.610
228.482.612
Plain-Velocity-XL/2 (Ours)115.212.613
229.952.613

Both variants achieve inference speeds close to standard DiT and are approximately 1.4–1.5× faster than TVM-XL/2 at 1–2 NFE. Unlike TVM, which introduces additional architectural modifications to stabilize terminal-differentiated training, our framework retains the standard DiT backbone without such modifications. This architectural simplicity preserves near-DiT inference efficiency. Despite its FP32 analytic integration, Plain-Velocity incurs only approximately 5–7% additional inference latency compared with Plain-Secant, with nearly identical memory consumption.

W4. Composition consistency and trajectory approximation.

We thank the reviewer for suggesting a direct evaluation of finite-interval flow-map accuracy. To investigate whether restricting terminal-time conditioning to a lightweight head limits the representation of arbitrary-time transitions, we evaluate XL/2 models using 1,024 samples.

(i) Composition consistency. For t>s>rt>s>r, we measure the discrepancy between the direct transition Fθ(xt,t,r)F_\theta(x_t,t,r) and the composed transition Fθ(Fθ(xt,t,s),s,r)F_\theta(F_\theta(x_t,t,s),s,r). We select t∈0.25,0.5,0.75,1t\in\\{0.25,0.5,0.75,1\\} and r∈0,0.25,0.5,0.75r\in\\{0,0.25,0.5,0.75\\} with r<tr<t, yielding 10 distinct intervals. For each interval, we evaluate three intermediate times, s=(1−α)t+αrs=(1-\alpha)t+\alpha r, where α∈0.25,0.5,0.75\alpha\in\\{0.25,0.5,0.75\\}.

(ii) Trajectory approximation. We compare direct flow-map predictions against numerical solutions of each model's induced ODE, defined by its diagonal velocity field vθ(x,u)=fθ(x,u,u)v_\theta(x,u)=f_\theta(x,u,u), using the adaptive Dormand--Prince (\texttt{dopri5}) solver. We report the average approximation error over the same 10 intervals (Traj. avg.) and the error over the complete generation interval from t=1t=1 to r=0r=0 (Traj. 1→0).

Table 5 in the revised manuscript reports relative RMSE (%) for composition consistency and trajectory approximation.

MethodComp. ↓Traj. (avg.) ↓Traj. (1→0) ↓
MeanFlow-XL/216.9420.3449.23
SoFlow-XL/214.5318.0443.49
TVM-XL/28.0521.0653.44
Plain-Secant-XL/2 (Ours)7.3614.2434.92
Plain-Velocity-XL/2 (Ours)7.4614.4335.30

Both Plain-Secant and Plain-Velocity achieve the lowest composition errors among the evaluated methods, improving upon TVM's 8.05% with errors of 7.36% and 7.46%, respectively. More importantly, both variants exhibit substantially lower trajectory approximation errors than the baselines, both averaged across different intervals and over the complete 1→01\to0 generation interval.

These results demonstrate that restricting terminal-time conditioning to a lightweight head does not compromise finite-interval consistency or agreement with the model's induced continuous dynamics. The consistently close performance of Secant and Velocity further supports the effectiveness of our unified analytic head architecture for both parameterizations.

W5. Text-to-image control.

We agree that the PixelGen and VAE changes prevent a controlled attribution of the text-to-image gains. A matched flow-matching baseline requires a separate full training run with the same modified backbone, VAE, data and recipe, which we cannot complete within the rebuttal period. We will explicitly present these experiments as a demonstration of scalability and applicability.

Q1. Backbone evaluations. The statement refers to one gradient-tracking backbone evaluation, whose source features are shared by the local velocity, finite-time map and terminal derivative. Eq. (22) additionally evaluates the EMA at the predicted terminal state; Eq. (21) also requires an unconditional EMA evaluation at the source for CFG. Thus, the general guided case uses one gradient-tracking and two no-gradient target evaluations. We have corrected the wording and distinguish loss-target EMA from evaluation EMA in the revised manuscript.

Reviewer_6eSJOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Plain Flow Maps, which confine terminal-time dependence to a lightweight analytic head while retaining a standard diffusion backbone. Two parameterizations are introduced: Plain-Secant predicts interval-averaged velocities and uses analytic differentiation, while Plain-Velocity predicts future instantaneous velocities and uses closed-form integration. The head combines harmonic time embeddings, linear channel modulation, and softmax-free spatial mixing to avoid full-backbone terminal-time differentiation. Validation includes head ablations, scaling studies, and efficiency measurements on ImageNet, together with from-scratch text-to-image generation. The results demonstrate competitive few-step generation quality, with substantial training speed and memory improvements for Plain-Secant over TVM in the reported benchmarks.

Critical strengths
  1. A simple architectural solution to a central bottleneck in terminal-differentiated flow-map training. The derivative decomposition and delayed-injection study support moving terminal-time computation out of the backbone. The Secant and Velocity parameterizations and the harmonic analytic head implement this idea through closed-form calculations, avoiding full-backbone terminal-time JVPs or finite differences while retaining a standard diffusion backbone and conventional computation tools.
  2. Plain-Secant demonstrates a practically meaningful quality–efficiency trade-off. Plain-Secant achieves approximately 4× faster training steps and 2.6× lower memory than TVM-XL/2 at the same scale and batch size, while also reporting better FID. The scaling results, head ablations, and text-to-image experiments further support its practical value.
Critical weaknesses
  1. The main-text positioning of relative generative performance remains incomplete. Main Table 2 omits stronger baselines included in Appendix Table 9. The paper reports 1/2-NFE FID scores of 2.81/2.16 for α-Flow and 1.72/1.54 for iMF, compared with 2.89/2.40 for Plain-Secant. State-of-the-art FID is not required for the paper to be valuable. However, the main text should include these results and account for differences in training settings to clarify the relative advantages in quality, efficiency, and simplicity.

  2. The framework-wide efficiency claim is supported primarily by Secant, while the actual cost of Velocity remains unknown. Plain-Velocity is one of the two central parameterizations, but Table 3 reports neither its runtime, memory, nor maximum feasible batch size. The current results therefore support the system advantages of Secant, but are insufficient to establish a unified efficiency advantage across the entire framework.

Questions for authors
  1. Can the authors clarify the baseline selection criteria for main Table 2 and include α-Flow and iMF in the main comparison, or limit the generative-quality claims to what the presented direct comparisons support? Could the authors also distinguish more clearly between the advantages in generative quality, training efficiency, and architectural simplicity?
  2. Can the authors compare training cost and quality with α-Flow and iMF under matched training and evaluation protocols where feasible, or clearly identify the factors preventing direct comparison? Should the main text explicitly position the contribution around efficiency and simplicity?
  3. Please report its training and low-NFE inference runtime, peak memory, and maximum feasible batch size relative to Plain-Secant. What is the error of the closed-form integral relative to a high-precision reference under FP32 and BF16?
  4. Were the delayed-injection study in Figure 4 and the final FID results obtained from single training runs? Could the authors provide multi-seed variability at least at B/2 scale, so that the 5.96–6.33 injection-position differences and the small Secant–Velocity gap can be interpreted reliably?
3 comments

Thank you for recognizing the architectural contribution and practical value of Plain Flow Maps.

W1, Q1 & Q2. Baseline selection, comparison and positioning.

We agree with your suggestion. In the revised manuscript, we have included α\alpha-Flow and iMF in the main Table 2.

Our primary motivation is to make flow-map training more practical and accessible through architectural simplification. Existing flow-map methods based on continuous-time differential objectives, including MeanFlow, iMF, α-Flow (in its MeanFlow stage), and TVM, rely on full-backbone JVPs, introducing substantial computational overhead and implementation complexity. TVM further requires specialized architectural designs for stabilization. In contrast, by leveraging terminal differentiation and confining terminal-time dependence to a lightweight analytic head, our approach completely eliminates full-backbone JVPs, retains a standard DiT backbone, and supports standard backpropagation while remaining compatible with off-the-shelf acceleration tools such as FlashAttention. This architectural simplification translates into practical benefits: compared with TVM-XL/2, Plain-Secant achieves approximately 4× faster training steps and 61% lower peak memory, while improving the reported 1/2-NFE FIDs from 3.29/2.80 to 2.89/2.40.

Our experiments therefore primarily focus on established flow-map formulations, with TVM as the closest terminal-differentiation reference and original MeanFlow as the foundational source-differentiation baseline. We largely follow TVM's training budget, with the adaptations detailed in the manuscript. α-Flow introduces additional objective improvements and curriculum learning, while iMF further modifies the architecture and training objective. These improvements were developed around source-differentiated MeanFlow training. Adapting these improvements to our terminal-differentiated formulation requires separate development and validation, which we leave to future work.

We will refine our claims to emphasize architectural simplicity and efficiency gains, together with competitive generation quality in the revised manuscript.

W2 & Q3. Plain-Velocity cost and integral accuracy.

We thank the reviewer for highlighting the missing efficiency measurements for Plain-Velocity. Since Plain-Velocity shares the exact same backbone and only differs in the integration of the lightweight analytic head, its system-level costs are similar to Plain-Secant. Our new experiments demonstrate that this overhead is remarkably small in practice.

i) Training efficiency. We have added Plain-Velocity to the training efficiency comparison in Table 3 of the revised manuscript:

MethodBatch sizeTime (s) ↓Mem (GB) ↓
DiT-XL/22560.1726.49
MeanFlow-XL/22560.8146.73
SoFlow-XL/22560.2627.04
TVM-B/22560.3119.37
TVM-XL/22560.9571.44
Plain-Secant-XL/2 (Ours)2560.2427.60
Plain-Secant-XL/2 (Ours)10240.7268.45
Plain-Velocity-XL/2 (Ours)2560.2427.69
Plain-Velocity-XL/2 (Ours)10240.7368.77

Remarkably, Plain-Velocity achieves nearly identical training efficiency to Plain-Secant. Both also achieve a maximum feasible batch size of 1024 under the 80GB GPU memory constraint. These results demonstrate that the additional Fourier-based integration and its FP32 computation introduce negligible system-level training overhead. Both parameterizations retain approximately 4× faster training steps and 61% lower peak memory than TVM-XL/2 at batch size 256.

ii) Inference efficiency. We additionally benchmarked low-NFE inference latency and peak GPU memory against related methods, with results added to Table 4:

MethodNFETime (ms) ↓Mem (GB) ↓
DiT-XL/2113.392.587
226.712.587
MeanFlow-XL/2113.682.594
227.072.594
SoFlow-XL/2113.762.594
227.482.594
TVM-XL/2121.542.599
243.122.599
Plain-Secant-XL/2 (Ours)114.282.610
228.482.612
Plain-Velocity-XL/2 (Ours)115.212.613
229.952.613

Both variants achieve inference speeds close to standard DiT and are approximately 1.4–1.5× faster than TVM-XL/2 at 1–2 NFE. Unlike TVM, which introduces additional architectural modifications to stabilize terminal-differentiated training, our framework retains the standard DiT backbone without such modifications. This architectural simplicity preserves near-DiT inference efficiency. Despite its FP32 analytic integration, Plain-Velocity incurs only approximately 5–7% additional inference latency compared with Plain-Secant, with nearly identical memory consumption.

iii) Numerical accuracy of closed-form integration. As discussed in Appendix B.5, the bilinear Velocity integral involves products of three Fourier-valued factors, whose numerical cancellation can amplify rounding errors under BF16. Following the reviewer's suggestion, we compare the closed-form integral against an independent high-precision FP64 numerical integration reference across different time intervals.

The following table reports relative integral errors (%):

Time gap δ\deltaFP64 (%) ↓FP32 (%) ↓BF16 (%) ↓
11.504 × 10⁻¹⁰0.23325.15
0.53.128 × 10⁻⁹0.20132.07
0.14.173 × 10⁻⁹0.0656.462
10⁻²3.912 × 10⁻⁹0.0520.705
10⁻³4.051 × 10⁻⁹0.3950.509
10⁻⁴3.142 × 10⁻⁹0.5890.675

The FP64 closed-form implementation agrees with the numerical reference to extremely high precision, supporting the correctness of our analytic derivation and implementation. Under FP32, relative errors remain below 0.6% across all tested intervals, whereas BF16 errors reach 32.07%, demonstrating the sensitivity to floating-point cancellation.

These findings empirically justify our choice to evaluate the complete Plain-Velocity integral head in FP32. Importantly, the training and inference benchmarks demonstrate that this precision requirement introduces only minor practical overhead, since the integration is confined to the lightweight head rather than the backbone.

Detailed training/inference efficiency benchmarking protocols are provided in Appendix C.2, and the numerical verification is included in Appendix B.6 of the revised manuscript.

Q4. Training variability.

We thank the reviewer for this suggestion. The original Figure 4 and final FID results were obtained from a single training run per configuration. To assess the robustness of our findings, we conducted three independent training runs at the B/2 scale.

For the delayed-injection study, given the substantial cost of TVM training (even TVM-B/2 is slower per step than our Plain-Secant-XL/2, as shown in Table 3), we selected two representative configurations: i=0i=0: 6.186±0.0436.186\pm0.043 and i=12i=12: 6.250±0.0586.250\pm0.058 (mean ±\pm std). The small difference relative to the observed run-to-run variability supports our finding that restricting terminal-time conditioning to the head preserves comparable generative quality.

For Plain-Secant and Plain-Velocity, the B/2 one-step FIDs are 5.045±0.0215.045\pm0.021 and 5.024±0.0285.024\pm0.028, respectively. Their mean difference is small relative to the seed variability, supporting our conclusion that both parameterizations achieve comparable quality.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.