acceptodds
Withdrawn from ICLR 2027

Gradual Fine-Tuning for Flow Matching Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Fine-tuning flow matching models is a central challenge in settings with limited data, evolving distributions, or computational constraints. While recent work has produced significant advances, particularly in the area of reward-based fine-tuning, current methods fail to demonstrate both theoretical correctness and strong empirical results in terms of stability, efficiency, and accuracy. In this work, we propose Gradual Fine-Tuning (GFT), a simple yet principled framework for fine-tuning flow generative models using samples from the target distribution. For stochastic flows, GFT defines a temperature-controlled sequence of intermediate objectives that smoothly interpolate between the pretrained and target drifts, provably converging to the true target as the temperature approaches zero. Empirically, GFT significantly improves convergence stability, while maintaining or improving generation quality, inference efficiency, and generation diversity compared to other fine-tuning methods. When extended to reward-based fine-tuning, GFT outperforms Adjoint Matching on several metrics while achieving over an order of magnitude reduction in training time. Our results position GFT as a simple yet theoretically grounded and practically effective alternative for scalable adaptation of flow matching models under distribution shift.

Primary area:
generative models
Keywords:
Generative Modeling, Flow Matching, Fine-Tuning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_ujkwOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper raised the issue of instability when naively fine-tuning flow-matching(FM) models with a dataset in a new domain, and proposes Gradual Fine Tuning(GFT). Specifically, they introduce a KL divergence loss with a reference trajectory distribution, which primarily regulates the training but is eased as training progresses. This KL divergence loss is known to be implemented by a velocity L2 regression. The authors apply their proposed method to different scenarios of data-based fine-tuning/reward-based fine-tuning/MeanFlow distillation, where GFT consistently reached a similar generative performance to naive FM fine-tuning with more monotonic training and less deviation from the reference distribution.

Critical strengths
  • Compared to naive fine-tuning, the proposed method achieved better trajectory length and diversity at comparable performance levels.
Critical weaknesses
  • The regularization term proposed by GFT ultimately reduces to an L2 loss between the current and old velocities, which is equivalent to KL divergence. Such KL regularization has already been widely adopted in most existing diffusion/flow fine-tuning methods, with explicit KL loss[1, 2] or in their training objective derivation[3, 4].

    • It is also well established and widely exploited that the KL divergence between path measures or sampling transition distributions can be expressed in terms of the squared L2 difference between velocities(flow matching) or noise predictions(diffusion)[1, 3].

    • Figure 16 and Table 3 seems like naive MF+EMA appears to yield faster training(Figure 16) with more stability(Table 3) than MF+GFT.

    • Although the authors claim the instability of naive fine-tuning, no experimental scenarios were provided where it actually fails; while naive fine-tuning exhibits some fluctuations, it ultimately reaches a much lower FID substantially faster without collapsing. The "monotonic and stable" training behavior of GFT appears to be because it's "slower".

  • The only task where GFT clearly outperforms the baseline was reward fine-tuning, which showed poor comparison with an inappropriate baseline:

    • The authors compare GFT against AM[3] when their training settings are fundamentally different(data-based vs. reward-based). Moreover, as it faithfully implements memoryless SOC, AM is one of slower reward fine-tuning methods[5, 6].
    • The authors argue that GFT had to be combined with ReFL[7] because AM is a first-order method, where they should instead have considered a zeroth-order method such as GRPO-based methods[2, 6].
    • If the paper claims GFT+ReFL as their proposed method, the appropriate baseline should be vanilla ReFL.
    • Despite emphasizing training stability as a major contribution, the authors additionally applied EMA on GFT+ReFL only.

[1] DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Models, NeurIPS 2023

[2] Flow-GRPO: Training Flow Matching Models via Online RL, NeurIPS 2025

[3] Adjoint Matching: Fine-tuning Flow and Diffusion Generative Models with Memoryless Stochastic Optimal Control, ICLR 2025

[4] Value Gradient Guidance for Flow Matching Alignment, NeurIPS 2025

[5] DiffusionNFT: Online Diffusion Reinforcement with Forward Process, ICLR 2026

[6] TempFlow-GRPO: When Timing Matters for GRPO in Flow Models, ICLR 2026

[7] ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation, NeurIPS 2023

Questions for authors
  • What feature does the "path length increase" metric measure? If it is intended to quantify the preservation of the pre-trained dynamics, does Figure 2(left) indicate naive fine-tuning is preferable with a value closer to 0? Would it not be more appropriate to report the KL divergence as the difference between vθv_\theta and vrefv_{\text{ref}} instead?

  • Could the authors report the hyperparameters used to obtain the experimental results(e.g., learning rate, batch size, beta schedule, and LoRA configuration)? These factors significantly affect training dynamics, yet the paper does not clarify whether they were matched to the baselines, which makes it skeptical that standard flow matching is actually unstable.

4 comments

We thank the reviewer for their detailed and thorough review, and in particular for their suggestions on related work and baselines, which have substantially improved both our experiments and how we position our contribution. We have uploaded a revised PDF, and refer to its figure and section numbers throughout.

Novelty of the regularizer

We agree that expressing the path-space KL as a squared difference of drifts or noise predictions is well established, and that KL regularization toward the pretrained model is widely used in diffusion and flow fine-tuning. We do not claim this regularization itself as a contribution, and we will revise the writing to make this clear and to cite more of the relevant prior work.

The novelty of GFT is the role of the regularizer, and its implementation. In prior works ([1, 2, 3, 4]), the KL coefficient is fixed and the fine-tuned model targets a single reward-tilted distribution pθ0exp⁡(r/α)p_{\theta_0}\exp(r/\alpha), where the regularization coefficient sets the trade-off between reward and deviation from the pretrained model. In GFT, the target is given by samples, and the coefficient is annealed to zero. The regularizer therefore does not change the endpoint, which coincides with the CFM target as β→0\beta \to 0 (Theorem 4.3). Instead, it shapes how fine-tuning reaches that endpoint. This setting motivates the analysis we provide, including

  • a closed-form minimizer at each temperature, which form a continuous sequence between the pretrained and target vector fields (Theorem 4.1),
  • bounds on the deviation of this minimizer and its terminal distribution from the target as a function of β\beta (Theorem 4.3),
  • a simulation-free regression objective that admits arbitrary source–target couplings, including OT, and extends to few-step models (Sections 4.3 and 4.5).

We will revise the introduction of the paper to present the contribution as annealed regularization for sample-based fine-tuning, together with its analysis, rather than the KL regularizer itself.

Instability of standard fine-tuning and hyperparameters (Question 2)

All CFM and GFT runs within each setting use identical learning rates, batch sizes, optimizer settings, number of epochs, and LoRA configurations. The only difference between them is the cooling schedule and regularization term. We will add the full configuration to Appendix O. GFT and CFM share the same fine-tuning target when β=0\beta = 0 (Theorem 4.3), so GFT's benefit lies in reaching convergence with high stability, rather than reaching a lower final FID. However, we do agree that standard CFM fine-tuning reaches a given FID faster than GFT in the cross-domain setting.

We have added an experiment to demonstrate one of the clear failure modes of standard CFM fine-tuning (Figure 5). With a very small fine-tuning dataset, standard CFM memorizes the training data almost entirely, with a near-copy rate of approximately 97%. GFT with a final temperature of zero reaches the same near-copy rate at a later stage of fine-tuning, so this benefit does not come from slower training alone. Instead, a nonzero final temperature regularizes substantially reduces memorization, and Figure 5 demonstrates that the rate of memorization can be reliably controlled by tuning the tuning schedule.

In settings where CFM does not fail outright, its instability still has a practical cost. The empirical KL divergence between the fine-tuned and pretrained models (Figures 2 and 3, right panels) shows that under full fine-tuning, CFM's generated distribution swings sharply toward and away from the pretrained model throughout training. In contrast, GFT departs from the pretrained model gradually, without restricting capacity, and ends at an FID comparable to CFM's, with a lower KL divergence.

In part, GFT's stability does come from its slower adaptation to the target distribution. However, this behavior is a direct consequence of the method's design, and it differs from naively slowing down CFM, such as through a reduced learning rate. This would leave the regression target unchanged, whereas GFT changes the target itself, regressing onto intermediate fields vβ∗v^{\ast}_\beta that lie between the pretrained and target vector fields. This provides control that a learning rate does not. The final temperature sets the endpoint and its FID/path-length trade-off (Figure 12), and the schedule determines how far the model has departed from the pretrained dynamics at each stage of training, as reflected in the KL curves.

Reward fine-tuning baselines

We agree with these points, and we will revise both the experiments and how we present them.

  • We have added an image-scale comparison against ORW-CFM-W2 (Fan et al., 2025b) in the online fine-tuning setting (Figure 6). ORW-CFM-W2 is the closest existing method to reward-based GFT, it is also a zeroth order, online, reward-weighted flow matching method that regularizes the fine-tuned model toward the pretrained dynamics, but its regularization coefficient is fixed throughout fine-tuning. The reward of ORW-CFM-W2 increases as its regularization coefficient decreases, with a best final reward of approximately 0.39 across the tested values $\lbrace 1, 0.8, 0.5, 0.1\rbrace $. GFT improves more slowly early in fine-tuning, while its regularization is strong, but reaches a higher final reward of approximately 0.48. Due to time and computational constraints, we have not yet been able to compare against other baselines, but we place to add Flow-GRPO and DPOK as baselines in the revision.
  • We compare against Adjoint Matching because it shares GFT's KL-regularized, theoretically grounded formulation. We additionally use this comparison as an opportunity to showcase the natural extension of GFT to the reward-based fine-tuning setting (Section 4.6). However, we agree that this comparison is not sufficient on its own, and have added comparison to other reward-based fine-tuning methods.
  • The comparison to ReFL is given in Appendix N.4 (Figure 9). ReFL alone achieves higher ImageReward and HPS than GFT (+ReFL), but its CLIPScore declines during training, while GFT (+ReFL) maintains it. The contribution of the GFT term is therefore preserving prompt alignment, not increasing reward, and we will revise the paper to make this clear.
  • We agree that applying EMA only to GFT (+ReFL) confounds the comparison, and will rerun all methods with matched EMA in the revision.
  • Our comparison to AM targets the setting where the reward is differentiable, where first order methods such as AM are the natural baseline. GRPO-based methods (Flow-GRPO, TempFlow-GRPO, DiffusionNFT) target the more general setting where the reward need not be differentiable, and estimate advantages by sampling groups of images for each prompt, which introduces substantial computational overhead. A fair comparison with these methods therefore requires matching total compute, and we will add a compute-matched comparison against GRPO-based methods in the revision. In the zeroth order setting, our comparison against ORW-CFM-W2 (above) provides a direct baseline with the same structure as GFT.

Given these points, we will present reward fine-tuning as a demonstration that GFT extends to the reward setting (Section 4.6), rather than as a headline result, and will revise Figure 1 and the abstract accordingly.

Mean Flow Results

We agree that adding EMA to standard MeanFlow fine-tuning yields greater stability and faster convergence than MF-GFT without EMA. However, we view EMA and GFT as complementary rather than alternative methods. EMA averages weights over training, which smooths the evaluated model without changing its training target, while GFT changes the target itself. Our results show that their effects compose. MF-GFT roughly halves the instantaneous variance of standard MF both without EMA (44.1 vs. 87.2) and with EMA (16.3 vs. 34.4), and with EMA it converges fully monotonically (Spearman ρ\rho −1.00-1.00 vs. −0.93-0.93) at a comparable rate (Table 3). We agree that standard MF with EMA reaches a given FID sooner (Figure 18). We will revise Section 5 to compare methods with and without EMA, and to present GFT as a complement to EMA.

Path length metric (Question 1)

Path length measures the kinetic energy of generated trajectories, Ex0∫01∥vθ(xt,t)∥2dt\mathbb{E}_{x_0}\int_0^1 \Vert v_\theta(x_t, t)\Vert ^2 dt (Appendix O), reported as the percent change relative to the pretrained model. We use it as a proxy for inference efficiency, as for a given source and target, lower kinetic energy corresponds to straighter trajectories, which can be integrated accurately with fewer steps. A value closer to 0 is therefore not the goal. Lower values indicate more efficient generation, so in Figure 2 (middle), GFT produces shorter paths than both CFM variants. We agree that path length does not measure preservation of the pretrained dynamics, and we will revise our wording in Section 5.2 accordingly.

Following the reviewer's suggestion, we have added the KL divergence between the fine-tuned and pretrained models as a direct measure of deviation (Figures 2 and 3, right panels). Under full fine-tuning, CFM departs from the pretrained model immediately and its KL divergence fluctuates sharply, while GFT departs gradually as β\beta is annealed. In the in-domain setting, LoRA GFT reaches FID comparable to full CFM while remaining much closer to the pretrained model, indicating that much of full CFM's deviation is unnecessary for accuracy.

I thank the authors for their responses to address my concerns. However, I still believe the novelty and practicality of the proposed method are limited. While the authors argue for the benefit of GFT based on the training fluctuations in naive SFT or existing methods, I think that training stability is a virtue only when it leads the resulting model to achieve final performance or convergence speed better than(or at least comparable to) that of the original method.

I will maintain my score in the current stage.

We thank the reviewer for their thorough feedback and for engaging with our responses. We agree that GFT requires more training to reach the final accuracy of standard fine-tuning, and that this cost should be stated clearly. We believe that in many practical settings the option to trade adaptation speed for a controlled transition from the pretrained model is valuable. We will make this trade-off, and the settings in which it is worthwhile, more explicit in future versions of this work.

Reviewer_HuNAOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes GFT to mitigate training instability that arises during the fine-tuning of flow matching. GFT applies regularization that constrains deviations from the velocity field of a pretrained model, and the strength of this regularization is then gradually reduced, allowing the model to adapt to the target distribution. Experiments show improved training stability and demonstrate the broader applicability of the method.

Critical strengths
  • GFT is a simple method that incorporates annealed regularization into existing CFM frameworks. It has been shown to reduce fluctuations in FID scores during training across various datasets. Its key advantages lie in its simple implementation and its potential to enhance the stability of the fine tuning process.

  • The authors demonstrate that GFT+ReFL achieves higher ImageReward and HPS compared to Adjoint Matching. This is significant, as it yields useful results for text-to-image fine tuning.

Critical weaknesses

W1: The practical benefits of improved stability are unclear. There is no consistent FID improvement, and in some cases, GFT adapts more slowly or the final FID is worse. Therefore, the claim that stability is improved while maintaining generation quality is not sufficiently supported.

W2: The statement that Adjoint Matching is incompatible with full fine-tuning and LoRA requires further clarification regarding its scope of applicability.

Questions for authors

Q1: Given a fixed training budget, what is the advantage of GFT in terms of the time required to reach a final or target FID score? How should the FMoW results be interpreted in light of the claim that generative quality is maintained?

Q2: Could the author clarify whether this incompatibility is a general limitation of Adjoint Matching or a constraint specific to a particular implementation? This would help in understanding the scope of the paper's claim.

1 comment

We thank the reviewer for their constructive feedback, particularly on demonstrating the practical value of training stability, which has motivated several of the new experiments described below. We have uploaded a revised PDF, and refer to its figure and section numbers throughout.

Weakness 1 / Question 1:

We agree that our results do not show a consistent FID improvement over baseline methods. GFT is not designed to improve final FID at convergence, but rather matches the minimizer of standard CFM fine-tuning when β=0\beta = 0 as shown in Theorem 4.3. We concede that at a fixed training budget, CFM generally converges faster than GFT. However, GFT's advantage is not speed to a target FID, but transitioning from the pretrained model to the fine-tuning target in a controlled way, which provides increased training stability. We will be happy to revise the abstract and introduction sections to state this precisely.

We would also like to note that we have updated Figure 19 after some additional tuning of the cooling schedule, and full fine-tuning with GFT now matches the FID of CFM on the FMoW dataset in the cross-domain setting.

To show the practical benefits of GFT more directly, we have added the following results:

  • Memorization under limited data (Figure 5). With a small fine-tuning dataset, standard CFM exhibits clear memorization, with a near-copy rate of approximately 97%. GFT with a nonzero final temperature substantially reduces memorization, and we show that this can be reliably controlled by tuning the cooling schedule.
  • Divergence from the pretrained model (Figures 2 and 3, right panels).
    • Cross-domain: Full CFM moves away from the pretrained model immediately, and its KL divergence swings widely throughout training. GFT stays close to the pretrained model early on and departs gradually as β\beta is annealed, with smaller fluctuations. It ends at an FID comparable to CFM's, but with a lower KL divergence from the pretrained model. LoRA CFM limits the deviation by restricting capacity, but as a result it cannot match full fine-tuning's FID under this large shift (Figure 3). GFT controls the deviation without restricting capacity.
    • In-domain: Full CFM fine-tuning also shows wide fluctuations in empirical KL from the pretrained model in this setting, while both LoRA variants reach comparable FID with KL close to zero. This indicates that much of full CFM's deviation is unnecessary for accuracy. Although the fluctuations of LoRA CFM fine-tuning are significantly smaller than those of full CFM fine-tuning, LoRA GFT removes these fluctuations almost entirely in the in-domain setting.

Since CFM's FID fluctuates, its final checkpoint is often not the best, so reaching CFM's best FID in practice requires frequent held-out evaluation and checkpoint selection. Additionally, GFT retains more of the target's diversity on Camelyon17 and RxRx1 (Table 4), produces shorter paths (Figures 2, 3, and 19), and allows explicit control of the FID/path-length trade-off through the final temperature (Figure 12). For one-step models, MF-GFT roughly halves the instantaneous variance of standard MeanFlow fine-tuning, with and without EMA (Table 3).

Weakness 2 / Question 2:

We agree that this claim is misleading as currently stated. Adjoint Matching defines the control as the difference between the fine-tuned and pretrained drifts, in which case nonlinear fine-tuning methods cannot be applied directly to the control (Appendix D). However, this is a definition rather than a structural constraint, and as implemented the AM loss only requires evaluating both models, enabling the use of full and parameter-efficient fine-tuning methods. We will remove this claim from the main body of the paper, and clarify in Appendix D. The distinctions we intend to draw between GFT and AM are the following:

  • Training cost. Each AM training step simulates trajectories under a memoryless noise schedule and solves the lean adjoint ODE backward along them. GFT is simulation-free, where each step is a regression on interpolated samples that requires one additional forward pass of the frozen pretrained model. In our 2D experiments, an AM epoch takes about 17× longer than a GFT epoch (Table 2).
  • Few-step generation. AM's objective is defined over full SDE trajectories. Extending it to few-step models such as MeanFlow, which parameterize average velocities rather than an instantaneous drift, would require a nontrivial reformulation. GFT's regression objective extends directly (Section 4.5, Appendix M).
Reviewer_V9EsOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Gradual Fine-Tuning (GFT) for better fine tuning pretrained flow matching models. It introduces a regularization term that penalizes deviation from the pretrained dynamics and gradually reduces its weight. The paper provides theoretical analysis and evaluates extensions to few-step generation and reward-based fine-tuning. The experiments demonstrate the advantages of the proposed method.

Critical strengths
  1. The method is easy to implement and addresses a practical problem in fine tuning pretrained flow models.

  2. The experiments show improvements in training stability and computational efficiency.

Critical weaknesses
  1. Under exact optimization and sufficient model capacity, setting β=0\beta=0 should recover the target process rather than cause collapse. The paper should distinguish mode collapse from legitimate concentration toward the target distribution.

  2. Theorem 4.3 describes the limit of ideal minimizers as β→0\beta\to0. It does not establish that training follows these minimizers under given arbitrary cooling schedule. How far the learned model under certain β\beta away from the corresponding optimal model vβ∗v_\beta^\ast value remains unclear.

  3. The practical approximation requires clearer qualification. The main experiments approximate the importance weight by wθ≈1w_\theta\approx1, which is a strong condition for most realistic scenarios. Although ablations support this choice empirically, the exact objective’s guarantees do not automatically apply to the resulting approximate training procedure.

Questions for authors
  1. Under what conditions does the annealed training procedure track the ideal minimizers? How sensitive is performance to the cooling rate?

  2. What's the advantages of the proposed method compared with the guidance-based methods that provide training-free guidance to the pretrained model?

2 comments

We thank the reviewer for their careful reading of our paper, and for their thoughtful questions which have helped us clarify our theoretical results. We have uploaded a revised PDF, and refer to its figure and section numbers throughout.

Weakness 1:

We agree that under exact optimization and sufficient model capacity, setting β=0\beta = 0 recovers the target process. This is the limiting behavior established by Theorem 4.3, whose bounds vanish as β→0+\beta \to 0^+. We do not claim that standard CFM fine-tuning collapses in this setting, and we agree that "collapse" should be used more carefully in the main body of the paper. Specifically, Section 4.2 previously used the wording "preventing premature collapse toward the target", which wrongly suggests that convergence toward the target distribution is itself undesirable. We have revised this wording, and will reserve "mode collapse" for failure to cover target modes, as distinct from legitimate concentration toward qq.

Consistent with Theorem 4.3, CFM and GFT recover both target modes with comparable coverage in our experiments on synthetic 2D data (Figure 10, Table 2). Mode collapse occurs only for the reward-based methods. ORW-CFM-W2 recovers only one mode, while Adjoint Matching shows higher covariance error (Table 2) and decreasing diversity throughout training (Figure 10). The reward optimized in these experiments is r(x)=log⁡q(x)−log⁡pθ0(x)r(x) = \log q(x) - \log p_{\theta_0}(x), making the tilted distribution pθ0exp⁡(r(x))p_{\theta_0}\exp(r(x)) equivalent to qq, meaning that these results reflect under-coverage rather than legitimate concentration. We will state the reward explicitly in Appendix O.2 to make this clear.

Table 4 reports Vendi scores relative to the target data's own diversity, where legitimate concentration toward qq should match qq's own diversity. CFM retains 77.9% (Camelyon17) and 62.9% (RxRx1) of target diversity, compared with 88.4% and 81.6% for GFT, and the two are comparable on FMoW (54.5% vs. 53.1%). Since both methods fall below the target's diversity, the gap reflects incomplete coverage of qq rather than concentration toward it. This is consistent with Theorem 4.3, which concerns population minimizers. Under a finite dataset and training budget, our results suggest that the gradual transition helps GFT retain coverage.

We additionally report new memorization metrics for image-scale fine-tuning in Figure 5. With a small fine-tuning dataset, CFM reaches a near-copy rate of approximately 97%. GFT with a final β\beta value of 0.0 reaches the same memorization level, but increasing the final temperature reliably and consistently reduces memorization.

Weakness 3:

We will revise the paper to make it clear to what extent the exact objective's guarantees apply to the practical approximation. The practical objective has a closed-form minimizer, which is exactly the GFT minimizer (8) studied in Theorems 4.1 and 4.3. With wθ=1w_\theta = 1, the practical objective is

12EZ∼π, t∼U[0,1], Xt∼Pq(⋅∣Z)[∥σt−1(vθ(Xt,t)−vq(Xt,t∣Z))∥2+β∥σt−1(vθ(Xt,t)−vθ0(Xt,t))∥2],\frac{1}{2}\mathbb{E}_{Z \sim \pi,\ t \sim \mathcal{U}[0,1],\ X_t \sim \mathbb{P}_q(\cdot | Z)}\bigg[ \Vert \sigma_t^{-1}(v_\theta(X_t, t) - v_q(X_t, t|Z))\Vert ^2 + \beta \Vert \sigma_t^{-1}(v_\theta(X_t, t) - v_{\theta_0}(X_t, t)) \Vert ^2 \bigg],

with all vectors evaluated at (Xt,t)(X_t, t). This expectation is taken under a fixed measure that does not depend on θ\theta, so the objective can be minimized pointwise. Conditioning on Xt=xX_t = x, the integrand is

12EZ[∥σt−1(vθ(x,t)−vq(x,t∣Z))∥2∣Xt=x]+β2∥σt−1(vθ(x,t)−vθ0(x,t))∥2.\frac{1}{2}\mathbb{E}_Z \bigg[ \Vert \sigma_t^{-1}(v_\theta(x, t) - v_q(x, t|Z))\Vert ^2 \Big| X_t = x \bigg] + \frac{\beta}{2}\Vert \sigma_t^{-1}(v_\theta(x, t) - v_{\theta_0}(x, t)) \Vert ^2.

With Σt=σtσt⊺\Sigma_t = \sigma_t \sigma_t^\intercal, setting the gradient with respect to vθv_\theta to zero gives

Σt−1[(vθ(x,t)−EZ[vq(x,t∣Z)∣Xt=x])+β(vθ(x,t)−vθ0(x,t))]=0.\Sigma_t^{-1} \bigg[ (v_\theta(x, t) - \mathbb{E}_Z[v_q(x, t|Z) | X_t = x]) + \beta(v_\theta(x, t) - v_{\theta_0}(x, t)) \bigg] = 0.

By the marginalization property of the conditional target, EZ[vq(x,t∣Z)∣Xt=x]=vq(x,t)\mathbb{E}_Z[v_q(x, t|Z) | X_t = x] = v_q(x, t), so

v∗(x,t)=11+βvq(x,t)+β1+βvθ0(x,t),v^{\ast}(x, t) = \frac{1}{1+\beta}v_q(x, t) + \frac{\beta}{1+\beta}v_{\theta_0}(x, t),

which is the exact GFT minimizer (8). In addition, the bounds in Theorem 4.3 are stated in L2(Pq)L^2(\mathbb{P}_q), the same measure the practical objective integrates over, meaning that the unweighted objective directly controls the quantity that appears in our convergence bounds.

We agree that the practical objective is no longer the two-term KL objective (7), and that their values differ away from the optimum because the expectation is taken under Pq\mathbb{P}_q rather than Pθ\mathbb{P}_\theta. With this change, the path-space KL formulation serves as a motivation for the objective rather than an exact identity. We will revise Section 4.3 to accurately reflect this.

Weakness 2 / Question 1:

We agree that Theorem 4.3 characterizes the ideal minimizers rather than the training trajectory, and we will clarify this in the revision. The distance from the minimizer vβ∗v^{\ast}_\beta is controlled by the training loss. As shown in our response to Weakness 3, the practical objective is quadratic in vv under the fixed measure Pq\mathbb{P}_q, so its excess loss is exactly the distance to the minimizer:

L~β(vθ)−L~β(vβ∗)=1+β2∥vθ−vβ∗∥L2(Pq)2.\tilde{\mathcal{L}}_\beta(v_\theta) - \tilde{\mathcal{L}}_\beta(v^{\ast}_\beta) = \frac{1+\beta}{2}\Vert v_\theta - v^{\ast}_\beta\Vert ^2_{L^2(\mathbb{P}_q)}.

Applying the Gronwall argument of Theorem 4.3 to the learned field gives

W22(pθ,q)≤e2Lθ+1[41+βΔL~β(θ)+2(β1+β)2C′],W_2^2(p_\theta, q) \leq e^{2L_\theta + 1}\bigg[ \frac{4}{1+\beta}\Delta \tilde{\mathcal{L}}_\beta(\theta) + 2 \left( \frac{\beta}{1+\beta} \right)^2C' \bigg],

so the learned model's error separates into optimization error and regularization bias.

Intuitively, the minimizers vβ∗v^{\ast}_\beta move continuously from vθ0v_{\theta_0} toward vqv_q as β\beta decreases. A sufficiently gradual schedule therefore poses a sequence of nearby problems, each warm-started from the previous solution. Our existing results support this:

  • Varying only the final temperature βmin⁡\beta_{\min} produces final FID and path length that are ordered consistently with βmin⁡\beta_{\min}, and βmin⁡=0\beta_{\min} = 0 recovers the final FID and path length of CFM (Appendix P.1, Figure 12). This is the behavior expected if training reaches vβmin⁡∗v^{\ast}_{\beta_{\min}}.
  • Linear, exponential, and sigmoid schedules decrease β\beta at different rates throughout training, yet reach very similar final FID and path length, differing mainly in how quickly they get there (Appendix P.2, Figure 13). Within this range, the cooling rate affects the trajectory but not the endpoint.
  • GFT's FID decreases close to monotonically during training (Spearman ρ\rho between −0.76-0.76 and −0.97-0.97 in Tables 1, 5), which is consistent with following a smoothly moving sequence of targets rather than lagging behind and catching up.

Question 2:

We see training-free guidance as a related but distinct problem. Guidance steers a fixed pretrained model at inference time toward samples favored by a guidance signal. Fine-tuning instead changes the model itself so that it generates from a new target distribution. GFT has two main advantages in the settings we consider:

  • Inference Cost. Guidance shifts the cost of computation from training to generation. Gradient-based guidance requires differentiating the guidance function at each sampling step. Methods such as D-Flow (Ben-Hamu et al., 2024) backpropagate through the entire ODE solve multiple times for every sample. A model fine-tuned with GFT generates at exactly the cost of the base model, and also remains compatible with few-step generators (Section 4.5).
  • Large Distribution Shifts. Since the pretrained parameters are unchanged, guidance steers among samples the pretrained model can already produce. When the target lies largely outside that set, as in our cross-domain experiments, stronger guidance cannot recover the target without pushing samples away from the model's learned manifold. Fine-tuning updates the model to represent the new distribution directly.

We agree that training-free guidance has real advantages, such as removing additional training time and maintaining the exact pretrained model. When the shift is small and inference cost is acceptable, guidance may be a natural choice. The two approaches are also complementary, since a GFT-fine-tuned model can itself be guided at inference time. We will expand the discussion in Section 2 to make this distinction explicit.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.