- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This manuscript is about how to make the generation of flow-based generative models satisfy constraints while preserving distributional fidelity. This work learns the terminal constraint into the local dynamics of a flow map without any rollout, and keeps this alignment when the number of sampling steps changes, by proposing a simulation-free terminal alignment for any-step flow map generation.
The interesting and novel part is the map-based adjoint. The classical Pontryagin adjoint needs the intermediate state , the endpoint , and the Jacobian of the remaining-time flow; this work replaces all of them with the learned map: , , then one VJP gives , which is injected into autograd through a zero-forward-value surrogate loss. The cost per update is fixed and independent of the number of ODE steps.
Critical strengths
- The problem is well framed and well illustrated in Table 1, i.e., training-time constraint alignment, any-step generation, and no ODE rollout during training are required at the same time, which has not been addressed before.
- The map-based adjoint is simple and cheap. It replaces the intermediate state, endpoint, and remaining-time Jacobian of the Pontryagin adjoint with the learned map and one VJP, so it can be plugged into any flow model with fixed cost per update.
- The ablation is well designed.
Critical weaknesses
All six benchmarks are either 2D toy distributions or small MLPs on POD latents of at most 144 dimensions. So I was wondering how the method performs in high-dimensional settings.
Questions for authors
- It seems that VFM and Direct map give almost same numbers everywhere. Could the authors explain why the noise adapter of VFM seems to have no effect here?
- Ours, VFM, and Direct map start from the FMM–LSD parent, while ODE-Adjoint, Flow-GRPO, and FM-RE start from FM (Euler), which is necessarily poor at small . Could the authors add an ODE-Adjoint baseline on the same FMM–LSD parent?
- Compared with FMRG, which uses the Jacobian of the remaining-time map for inference-time guidance, this work moves the same quantity from inference time to training time. Could the authors clarify the increment over FMRG?
- Could the authors discuss the performance in high-dimensional settings, or the limitation there?
6 comments
Response (Part 2)
C2
(Q1) [...] VFM and Direct map give almost same numbers everywhere. [...] explain why the noise adapter of VFM seems to have no effect here?
Response.
VFM and Direct map both train the flow map through the terminal penalty. In our evaluated VFM models, the additional source adapter contributes little to the reported metrics. The inspected Six Disks checkpoint learns a source distribution very close to the base Gaussian. Replacing the adapter by the identity while keeping the trained map fixed has little effect on violation across all four tested tasks, with a modest SWD benefit from the adapter on Six Disks.
-
Joint training and learned source distribution. Our submitted VFM implementation applies terminal feedback to both the adapter and map, with KL regularization of the adapted source toward the Gaussian prior (Appendix C.2). We retain the KL coefficient of used in the public VFM implementation (Mammadov et al., 2026). For the inspected Six Disks checkpoint, the adapted Gaussian has a mean-vector norm of , mean standard deviation , and KL divergence from the base Gaussian. The learned source therefore differs little from the original source distribution at this checkpoint.
-
Effect of replacing the adapter. We hold the trained map and base draws fixed and replace only the learned source adapter by the identity at deployment. Across eight uniform step counts, mean violation changes from to on Six Disks, from to on SMD9, and from to on Stokes. Dynamic Stall remains approximately . SWD changes from to on Six Disks, to on SMD9, and to on Stokes; Dynamic Stall changes by less than . Six Disks uses one training seed; the other tasks use three, with means over test conditions and step counts, using independent source draws. This replacement test confirms that the learned adapter adds little to the measured violation reduction beyond the trained map. Its small additional contribution helps explain why VFM and Direct map have similar scores here.
The comparison with and without the adapter is now reported in revised Appendix D.3, Table 17.
Response (Part 3)
C3
(Q2) Ours, VFM, and Direct map start from the FMM–LSD parent, while ODE-Adjoint, Flow-GRPO, and FM-RE start from FM (Euler), which is necessarily poor at small K. [...] add an ODE-Adjoint baseline on the same FMM–LSD parent?
Response.
We added an ODE-feedback control with the same FMM–LSD parent, loss components and learned-map sampler as Ours, using eight Euler steps for training feedback to match the submitted ODE-Adjoint rollout budget. The new comparison shows a trade-off: Ours achieves lower violation at every tested step count, while ODE feedback achieves lower SWD and has a longer observed alignment-training time.
-
Controlled comparison. The submitted ODE-Adjoint starts from FM (Table 5, p. 24). The new control and Ours start from the same FMM–LSD checkpoint and use the same network, FM/direct/LSD terms, feedback coefficient of 1, optimizer and 10,000-update budget. Only the terminal-feedback estimator changes: the control differentiates through an eight-step Euler rollout instead of using the map adjoint. At deployment, both models compose learned maps. The sampling step count below therefore refers to map evaluations, not the eight Euler steps used during training.
-
New paired results. The following Six Disks comparison uses one alignment seed and 4,096 paired validation draws per schedule at . Violation is reported in units of .
Steps Ours violation ODE feedback violation Ours SWD ODE feedback SWD 1 1.641 1.809 0.10322 0.08487 4 0.429 1.686 0.11249 0.08407 16 0.362 0.679 0.11146 0.07900 64 0.370 0.577 0.11088 0.07846 Averaged over these schedules, Ours reduces violation from to , while SWD increases from to . Thus the constraint benefit persists with a matched parent and sampler, with a trade-off in distributional fidelity.
-
Training cost. The recorded 10,000-update alignment runs take min for ODE feedback and min for Ours, a ratio of , excluding the common parent pretraining. These are observed wall times under shared GPU use. The map adjoint supplies terminal feedback without the sequential integration required by the ODE control.
Revised Appendix D.6 reports the matched comparison and alignment times (Tables 25–26). Appendix D.8 gives the separate training-stage costs.
Response (Part 1)
Thank you for the constructive review and for recognizing the value of a simple, efficient map-based adjoint for training-time alignment and the ablations supporting its role. Below, we respond point by point to your concerns, with additional comparisons that clarify baseline fairness, the contribution relative to prior work, and performance at larger latent dimensions.
Response.
The submission includes a 512-dimensional latent generator for Dynamic Stall, and we now extend the evaluation to learned dimensions 1,024 and 2,048. At both larger dimensions, Ours improves violation and SWD over same-parent Direct controls across all three training seeds and tested step counts.
Submitted scale. The submission uses learned dimensions 144, 128 and 512 for SMD9, Stokes and Dynamic Stall (Table 2). Dynamic Stall decodes to 98,304 physical coordinates; its submitted latent generator is a width-128, depth-3 MLP. Learned and decoded dimensions are distinct.
New larger-state comparison. Across three training seeds, Direct and Ours start from the same FMM parent within each rank, with the same backbone and training budget. Paired tests cover 12 conditions and K=1,2,4,8,16,32,64,128. The codecs are fitted on the same training data.
Entries show mean ± sample SD across three training seeds, after averaging the 12 conditions and eight step counts within each seed. At each larger rank, Ours improves both metrics over Direct in all 24 seed–step comparisons, reducing mean violation by 9.5% at rank 1024 and 12.4% at rank 2048. The SWD improvements are small. Thus the within-rank benefit over Direct persists, while increasing rank does not improve the absolute generation metrics.
Computation and memory at larger dimensions. Direct and Ours have identical trainable parameter counts at each rank; the frozen POD basis is stored separately. On one H200, the following measurements use the final checkpoints from one training run per method and batch size 64. Paired entries are Direct/Ours.
Times are medians over repeated measurements, excluding optimizer updates and data preparation. Peak allocated memory includes the model, codec and runtime buffers. The frozen basis accounts for much of the memory growth. Avoiding an ODE rollout removes dependence on its integration length, while evaluating larger maps and differentiating through their decoders still carries a cost.
These results show that alignment improves violation and SWD over Direct at larger POD dimensions, with increased memory use as the model and decoder grow.
Revised Appendix D.7 now reports the reconstruction comparison, three-seed generation results, tolerance attainment, and measured model, codec and computation costs (Tables 29–31). Revised Sec. 7 now discusses the measured state-size scaling and the role of model capacity and decoder storage.