Gradual Fine-Tuning for Flow Matching Models

Fine-tuning flow matching models is a central challenge in settings with limited data, evolving distributions, or computational constraints. While recent work has produced significant advances, particularly in the area of reward-based fine-tuning, current methods fail to demonstrate both theoretical correctness and strong empirical results in terms of stability, efficiency, and accuracy. In this work, we propose Gradual Fine-Tuning (GFT), a simple yet principled framework for fine-tuning flow generative models using samples from the target distribution. For stochastic flows, GFT defines a temperature-controlled sequence of intermediate objectives that smoothly interpolate between the pretrained and target drifts, provably converging to the true target as the temperature approaches zero. Empirically, GFT significantly improves convergence stability, while maintaining or improving generation quality, inference efficiency, and generation diversity compared to other fine-tuning methods. When extended to reward-based fine-tuning, GFT outperforms Adjoint Matching on several metrics while achieving over an order of magnitude reduction in training time. Our results position GFT as a simple yet theoretically grounded and practically effective alternative for scalable adaptation of flow matching models under distribution shift.