- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Plain Flow Maps with Analytic Heads, which bypass expensive full-backbone differentiation with respect to terminal time by restricting required derivative computations to a lightweight analytic head. The paper demonstrates strong results on ImageNet and text-to-image generation, with substantial improvements in training efficiency.
Critical strengths
-
This paper reduces the training budget by bypassing expensive full-backbone differentiation. The overall theory is sound, and I find no fatal errors in the proofs.
-
The experiments clearly demonstrate the validity of the proposed methods.
Critical weaknesses
While I do not have major concerns with this paper, I have some minor concerns:
-
The flow-map composition consistency across different time intervals is not explicitly evaluated. The paper does not explicitly evaluate the flow-map composition property across different time intervals. Since the predicted trajectories are not guaranteed to be consistent under imperfect training, additional experiments would strengthen the theoretical justification.
-
The computational overhead of the analytic bilinear head, particularly for Plain-Velocity, deserves further analysis. Plain-Velocity requires a relatively complicated Fourier-based integration procedure involving 769 temporal evaluation points. A detailed computational comparison with Plain-Secant would help justify this additional complexity.
-
Numerical verification of the closed-form derivatives and integrals would strengthen the paper's reliability. Although the analytic derivatives and integrals appear mathematically correct, the paper lacks numerical comparisons against automatic differentiation and high-precision integration. Such verification would increase confidence in the implementation's correctness and numerical stability.
Questions for authors
Please refer to Weaknesses.
5 comments
W2. Computational overhead of Plain-Velocity.
We thank the reviewer for highlighting the computational overhead of Plain-Velocity's Fourier-based integration. To address this concern, we provide comprehensive benchmarks at both the system level (training and inference) and the head level. Since both parameterizations share the same standard DiT backbone and bilinear head architecture, their main computational differences arise from the terminal-time operations. Our results demonstrate that the additional integration overhead remains small at the system level.
i) Training efficiency. We have added Plain-Velocity to the training efficiency comparison in Table 3 of the revised manuscript:
| Method | Batch size | Time (s) ↓ | Mem (GB) ↓ |
|---|---|---|---|
| DiT-XL/2 | 256 | 0.17 | 26.49 |
| MeanFlow-XL/2† | 256 | 0.81 | 46.73 |
| SoFlow-XL/2 | 256 | 0.26 | 27.04 |
| TVM-B/2 | 256 | 0.31 | 19.37 |
| TVM-XL/2 | 256 | 0.95 | 71.44 |
| Plain-Secant-XL/2 (Ours) | 256 | 0.24 | 27.60 |
| Plain-Secant-XL/2 (Ours) | 1024 | 0.72 | 68.45 |
| Plain-Velocity-XL/2 (Ours) | 256 | 0.24 | 27.69 |
| Plain-Velocity-XL/2 (Ours) | 1024 | 0.73 | 68.77 |
Plain-Velocity achieves nearly identical training efficiency to Plain-Secant. At batch size 256, both require only 0.24 s/step, with nearly identical peak memory usage. Under the 80GB GPU memory constraint, both support a maximum feasible batch size of 1024, with step times of 0.72 s and 0.73 s, respectively. Relative to TVM-XL/2, both parameterizations achieve approximately 4× faster training steps and 61% lower peak memory at batch size 256.
ii) Inference efficiency. We additionally benchmark inference latency and peak GPU memory, with the results included in Table 4 of the revised manuscript:
| Method | NFE | Time (ms) ↓ | Mem (GB) ↓ |
|---|---|---|---|
| DiT-XL/2 | 1 | 13.39 | 2.587 |
| 2 | 26.71 | 2.587 | |
| MeanFlow-XL/2 | 1 | 13.68 | 2.594 |
| 2 | 27.07 | 2.594 | |
| SoFlow-XL/2 | 1 | 13.76 | 2.594 |
| 2 | 27.48 | 2.594 | |
| TVM-XL/2 | 1 | 21.54 | 2.599 |
| 2 | 43.12 | 2.599 | |
| Plain-Secant-XL/2 (Ours) | 1 | 14.28 | 2.610 |
| 2 | 28.48 | 2.612 | |
| Plain-Velocity-XL/2 (Ours) | 1 | 15.21 | 2.613 |
| 2 | 29.95 | 2.613 |
Both variants achieve inference speeds close to standard DiT, while being approximately 1.4–1.5× faster than TVM-XL/2 at 1–2 NFE. Unlike TVM, which introduces additional architectural modifications for training stability, our framework retains the standard DiT backbone without such modifications. Despite its FP32 analytic integration, Plain-Velocity incurs only 5–7% additional inference latency compared with Plain-Secant, with nearly identical memory consumption.
iii) Head-level forward-and-backward cost. To isolate the computational overhead of Fourier-based integration, we benchmark the forward-and-backward computation of both analytic heads under the default configuration.
| Method | Time (ms) ↓ | Peak memory (MB) ↓ |
|---|---|---|
| Plain-Secant head (BF16) | 3.630 | 166.3 |
| Plain-Velocity head (FP32) | 3.859 | 200.4 |
Plain-Velocity incurs only 6.3% additional head-level forward-and-backward latency, while peak memory increases by approximately 20.5%, primarily reflecting the FP32 spectral integration. Importantly, the 769-point Fourier grid evaluates only lightweight temporal modulation factors, without repeated backbone evaluations or repeated full attention-head evaluations.
These results explain why the additional integration complexity incurs only minor system-level overhead. Our unified analytic head architecture supports both closed-form differentiation and integration while retaining computational efficiency comparable to Plain-Secant. Detailed benchmarking protocols are provided in Appendix C.2 of the revised manuscript.
W3. Numerical verification of analytic derivatives and integrals.
We thank the reviewer for this valuable suggestion. To verify the correctness and numerical stability of our analytic implementations, we conduct numerical comparisons against automatic differentiation (AD) and independent high-precision numerical integration under FP32 and BF16.
i) Plain-Secant derivative accuracy. We compare our analytic terminal derivative against head-only AD and an FP64 AD reference.
| Precision | Analytic vs. AD (%) ↓ | Analytic vs. FP64 (%) ↓ | AD vs. FP64 (%) ↓ |
|---|---|---|---|
| FP32 | |||
| BF16 | 0.167 | 0.533 | 0.526 |
Under FP32, our analytic derivative agrees almost perfectly with AD, with a relative error of only . Both methods exhibit nearly identical errors relative to the FP64 reference. Under BF16, the analytic and AD implementations remain comparably accurate (0.533% vs. 0.526% relative to FP64), indicating that our explicit differentiation introduces no appreciable numerical degradation beyond mixed-precision rounding.
ii) Plain-Velocity integral accuracy. We compare our closed-form integral against an independently computed FP64 numerical reference using 128-node Gauss–Legendre quadrature, across different integration gaps.
| Gap | FP64 (%) ↓ | FP32 (%) ↓ | BF16 (%) ↓ |
|---|---|---|---|
| 1 | 0.233 | 25.15 | |
| 0.5 | 0.201 | 32.07 | |
| 0.1 | 0.065 | 6.462 | |
| 0.052 | 0.705 | ||
| 0.395 | 0.509 | ||
| 0.589 | 0.675 |
The FP64 closed-form implementation agrees with the independent quadrature reference to within , providing strong numerical validation of the analytic integration. Under FP32, relative errors remain below 0.6% across all tested intervals. In contrast, BF16 exhibits substantially larger errors, reaching 32.07%, due to its sensitivity to floating-point rounding and cancellation among Fourier terms.
These findings support our implementation choice to evaluate the complete Plain-Velocity integral head in FP32. As demonstrated by our computational benchmarks in W2, this precision requirement introduces only minor practical overhead. We have added these results in Appendix B.6 in the revised manuscript.
We thank the reviewer for the positive assessment of our method and for suggesting these additional validations.
W1. Flow-map composition consistency.
We agree that exact temporal differentiation/integration does not guarantee exact composition for a finitely trained model. To investigate whether restricting terminal-time conditioning to a lightweight head limits the representation of arbitrary-time transitions, we evaluate XL/2 models using 1,024 samples.
For t>s>r, we measure the discrepancy between the direct transition Fθ(xt,t,r) and the composed transition Fθ(Fθ(xt,t,s),s,r). We select t∈0.25,0.5,0.75,1 and r∈0,0.25,0.5,0.75 with r<t, yielding 10 distinct intervals. Each interval is evaluated at three intermediate times, s=(1−α)t+αr, where α∈0.25,0.5,0.75, resulting in 30 time triples.
Table 5 in the revised manuscript reports the composition consistency errors (relative RMSE, %):
Both Plain-Secant and Plain-Velocity achieve the lowest composition errors among the evaluated methods, outperforming TVM (8.05%) with errors of 7.36% and 7.46%, respectively. These results provide direct empirical evidence that restricting terminal-time conditioning to a lightweight head does not introduce a noticeable consistency degradation relative to the evaluated baselines. The comparable results of Secant and Velocity further support the effectiveness of our shared analytic head architecture across both parameterizations.