acceptodds
Under review as a conference paper at ICLR 2027

A Flow Matching Algorithm for Many-Shot Adaptation to Unseen Distributions

Abstract

While generative modeling has achieved remarkable success on tasks like natural language-conditioned image generation, enabling model adaptation from example data points remains a relatively underexplored and challenging problem. To this end, we propose Function Projection for Flow Matching (FP-FM), an algorithm that directly conditions generation on samples from the target distribution. FP-FM learns basis functions to span the velocity fields corresponding to a set of training distributions, and adapts to new distributions by computing a simple least-squares projection onto this basis. This enables efficient generation of samples from diverse target distributions without additional training at inference time. We further introduce multiple variants of FP-FM that provide a trade-off in expressivity and compute by enriching the coefficient calculation, e.g., by making the coefficients dependent on time. FP-FM achieves competitive precision and recall relative to baselines across synthetic and image-based datasets, with especially strong gains on unseen distributions, while requiring significantly lower compute.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
flow matching, many-shot, generalization, transfer

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_PLxyOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Function Projection for Flow Matching (FP-FM), a framework for adapting flow-based generative models to unseen target distributions using samples from the target distribution as context. The key idea is to learn a set of basis functions that span the velocity fields associated with training distributions. Given samples from a new target distribution, FP-FM estimates the corresponding velocity field by solving a least-squares projection problem over the learned basis, without requiring gradient-based fine-tuning at inference time. The paper introduces three variants—Static, Temporal, and Dynamic FP-FM—where the projection coefficients are respectively fixed, time-dependent, or state-and-time-dependent, providing different trade-offs between expressivity and computational cost. The method is evaluated on 2D synthetic data, MNIST, and ImageNet under training-distribution and unseen-distribution settings. The results show that the more expressive FP-FM variants, particularly Dynamic FP-FM, achieve competitive or improved precision, recall, and FID on unseen distributions, while avoiding per-target gradient-based fine-tuning.

Critical strengths
  1. The proposed formulation is interesting and conceptually clean. Representing distribution-specific velocity fields using a shared set of learned basis functions provides an interesting function-space perspective on adapting flow-matching models. The Static, Temporal, and Dynamic variants also provide a clear progression in expressivity and computational cost.

  2. Given context samples from a new target distribution, the method obtains its coefficients through least-squares projection rather than updating model parameters. The experiments demonstrate substantial computational savings over per-target fine-tuning while retaining competitive performance in several settings.

Critical weaknesses
  1. The ablation study does not sufficiently validate the method in the most important ImageNet setting. The paper studies the number of context shots and basis functions primarily on MNIST. However, these factors are particularly important for understanding the practicality and scalability of FP-FM on ImageNet. For example, an ImageNet evaluation across different context sizes (e.g., 8/16/32/64/128) would clarify how many target examples are actually required before FP-FM becomes effective.

  2. The baseline comparison is insufficient to establish the competitiveness of FP-FM for modern sample-based generative adaptation. The main baselines are unconditional/conditional flow matching, classifier/distribution guidance, and fine-tuning (LoRA for ImageNet). While these baselines provide useful reference points, they do not sufficiently cover recent approaches that also perform example-conditioned or context-based generative adaptation.

  3. The strong performance of Dynamic FP-FM is potentially confounded by its tendency to reconstruct the provided context examples. The authors explicitly observe that Dynamic FP-FM “largely reconstructs the images provided in context, introducing only minor visual perturbations,” and later acknowledge that it tends to generate samples similar to the context. This raises an important question regarding whether its strong precision/FID primarily reflects successful modeling of the underlying target distribution or a form of context reconstruction. Although the appendix provides a nearest-neighbor-based overfitting analysis, more comprehensive diversity and memorization evaluations would be necessary to clarify this issue.

  4. The evaluation provides limited evidence for adaptation to substantially unseen distributions. In the main ImageNet experiment, the model is trained on 900 ImageNet classes, while the remaining 100 classes are treated as unseen targets. Moreover, FP-FM receives 128 context images from each target class at inference time. It remains unclear how well the proposed basis functions generalize under larger distribution shifts or when substantially fewer target samples are available. This is particularly relevant because the paper acknowledges that the method is expected to perform poorly in few-shot regimes.

Questions for authors

Please see the Critical Weaknesses.

1 comment

The ablation study does not sufficiently validate the method in the most important ImageNet setting. .... However, these factors are particularly important for understanding the practicality and scalability of FP-FM on ImageNet. For example, an ImageNet evaluation across different context sizes (e.g., 8/16/32/64/128) would clarify how many target examples are actually required before FP-FM becomes effective.

Context imagesLoRATemporalDynamic
8236.7290.7160.6
16231.6265.6148.7
32224.1235.2139.7
64222.7220.4134.2
128222.4212.8130.2

We run your desired ablation on Imagenet. We find that Temporal FP-FM outperforms LoRA finetuning between 32 and 64 context images and beyond. We have added a discussion of these results to the appendix.

We additionally ablate the number of basis functions, k, on imagenet for Temporal FP-FM. We focus on Temporal as it is the best performing variant on ImageNet.

K80k steps
64247.6 / .156 / .318
128242.0 / .174 / .301
256240.2 / .163 / .333
512240.7 / .159 / .327

We find that Temporal FP-FM is not too sensitive to the number of basis functions. The difference between 64 and 512 basis functions is only about 7 FID. We have added a discussion of these results to the appendix.


The baseline comparison is insufficient to establish the competitiveness of FP-FM for modern sample-based generative adaptation. The main baselines are unconditional/conditional flow matching, classifier/distribution guidance, and fine-tuning (LoRA for ImageNet). While these baselines provide useful reference points, they do not sufficiently cover recent approaches that also perform example-conditioned or context-based generative adaptation.

We create one additional baseline similar to a set-based autoencoder. It first encodes each image into a latent space using a VAE. Then it passes these latents through a MLP and averages the results, outputting a class representation. This representation is then used as input to a conditional model. We find the following comparison to temporal FP-FM:

MethodSeen (TD)Unseen (US)
Set-conditioned DiT-B/2197.7 / .384 / .107230.0 / .271 / .090
Temporal FP-FM207.6 / .289 / .335212.8 / .252 / .354

If you have any other baselines in mind, we are happy to run anything that fits into the setting.


The strong performance of Dynamic FP-FM is potentially confounded by its tendency to reconstruct the provided context examples. ... Although the appendix provides a nearest-neighbor-based overfitting analysis, more comprehensive diversity and memorization evaluations would be necessary to clarify this issue.

As mentioned in the paper, the expressivity of Dynamic FP-FM allows it to overfit to the provided samples, and therefore it tends to (usually) reproduce these samples with added noise or visual artifacts. For image domains, this is likely unsatisfactory, which is why we suggest practitioners to use Temporal FP-FM in line 439. However, there may be other domains where practitioners prefer fidelity over diversity, in which case it may find uses. Please also see the discussion with reviewer fsZk, where we revised the discussion over overfitting in the appendix.

The evaluation provides limited evidence for adaptation to substantially unseen distributions. In the main ImageNet experiment, the model is trained on 900 ImageNet classes, while the remaining 100 classes are treated as unseen targets. Moreover, FP-FM receives 128 context images from each target class at inference time. It remains unclear how well the proposed basis functions generalize under larger distribution shifts

MethodTraining classesHeld-out animals
Real images (floor / ceiling)94.9 / .685 / .67854.7 / .704 / .714
Temporal FP-FM220.9 / .292 / .308281.8 / .065 / .284

We retrain a model on all non-animal imagenet classes and evaluate on animals. This is quite a large distribution shift because many animals have idiosyncratic visual features, such as fur texture and eyes. Unsurprisingly, temporal FP-FM has higher FID on the heldout animals than on the training classes. Qualitatively, we observe the model can recreate the textures but fails with global structure. For example, for a dog class, it outputs something that looks like a big ball of fur but fails to correctly create four legs and a head. We have added this ablation to Appendix E.6.

Reviewer_xMyVOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies the problem of adapting a generative model to generate samples from a target distribution given by a number of samples at inference time. To this end the authors propose learning a set of basis functions general enough to approximate any velocity field in the flow matching framework. These basis functions together with the given examples from the target distribution are used to approximate the target velocity with a linear combination of basis functions and hence generate new samples from the target distribution. The authors call the method of learning and adapting these basis function FP-FM (Function Projection for Flow Matching) and instantiate it in three variants: static, temporal and dynamic (depending on what the coefficients in linear combination depend on). Unlike prior methods, such as conditional, guided generation, or fine-tuning, FP-FM allows to generate samples from target distributions with good accuracy and moderate cost. The method was tested on toy (Arc), small scale (MNIST) and large scale (ImageNet) data.

Critical strengths
  1. The method is simple and relatively well-presented. Three variants are well-motivated.
  2. The ability of generating samples from unseen distributions given only a handful of examples and cheap adaptation is impressive.
Critical weaknesses
  1. The ImageNet numbers are quite high for FID. Is this due to the fact the FID was calculated using only 128 samples or is this due to underfitting? Since FID may be noisy on that amount of samples, it might be useful to redraw the example sets and report the FID bands. In general, this could also be useful for evaluations on the other datasets.
Questions for authors
  1. Could the authors expand their analysis with training time complexity of FP-FM compared to the baselines?
  2. Is stop-grad applied to the coefficients in the approximate velocity field during training or are the gradients allowed to pass through the coefficients? What is the effect of either choice?
  3. Could the authors provide FID bands under resampling the example sets?
2 comments

The ImageNet numbers are quite high for FID. Is this due to the fact the FID was calculated using only 128 samples or is this due to underfitting? Since FID may be noisy on that amount of samples, it might be useful to redraw the example sets and report the FID bands. In general, this could also be useful for evaluations on the other datasets

Evaluation (FID ↓)TemporalLoRADynamic
128 images/class (5 reference redraws)213.6 ± 0.1222.8 ± 0.2130.5 ± 0.1
128 images/class (3 context redraws)211.1 ± 0.2222.9 ± 0.7130.8 ± 1.0
512 images/class172.5185.193.9

FID is sensitive to the number of samples. To address this, we run three different ablations. We first compute FID over 128 generated images relative to 5 sets of 128 reference images. We find that the variance is low, meaning that our metric is not very sensitive to the reference images. Next we compute FID of 3 sets of 128 generated images (using different context images) relative to 128 reference images. Again we find that std deviation is low, meaning that our metric is not very sensitive to the context images. Lastly, we increase the number of generated images to 512 and compare against 512 reference images. FID is indeed lower because we used a larger number of images, but the overall trends are the same.

TLDR: Increasing the number of images to compute FID indeed changes the magnitude but not the trends.


Could the authors expand their analysis with training time complexity of FP-FM compared to the baselines?

We find that the Conditional, Temporal FP-FM, and Dynamic FP-FM all take ~2.92 steps per second, so the difference in training complexity is negligible. This is because the main expense of all methods is data processing and a forward pass.


Is stop-grad applied to the coefficients in the approximate velocity field during training or are the gradients allowed to pass through the coefficients? What is the effect of either choice?

No, stop grad is not applied. We previously experimented with this but found that stop grad performs slightly worse by about 4 FID.

Reviewer_xMyVOpenReview ↗

Reponse to the Rebuttal

Thank you for additional clarifications. I would suggest increasing the number of samples for FID to thousands where data and compute allow. Nevertheless, I remain positive about the paper and therefore keep my original score.

Reviewer_fsZkOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces Function Projection for Flow Matching (FP-FM), a method for adapting a pretrained flow-matching model to distributions unseen during training, using a an held-out set of examples from the new distribution. FP-FMT learns a set of basis functions representing velocity fields and computes their coefficients using least-squares. The authors presents three variants of their approach: Static, Temporal, and Dynamic. The methods are evaluated on synthetic 2D distributions, MNIST, and ImageNet.

Critical strengths
  1. Intuitive approach with a clear presentation . The paper is well written and easy to follow. The proposed approach is intuitive and provides a perspective on adapting flow-matching models to unseen distributions through a learned basis of velocity fields. The different variants are well motivated.

  2. Efficient adaptation without training. The ability to adapt to new distributions without additional training is appealing. The experiments demonstrate promising results on low-dimensional distributions and MNIST.

Critical weaknesses
  1. Limited effectiveness on high-dimensional image generation. My main concern is that, despite the intuitive formulation, the proposed method does not appear to work particularly well on realistic, high-dimensional data. In Figure 3, the generated ImageNet samples generally fall into two categories: either they closely resemble an image from the context set, with only minor changes, or they have relatively poor visual quality. The authors also acknowledge this behavior in Section 5. While Dynamic FP-FM obtains the strongest quantitative results, I am not convinced that these improvements demonstrate successful generative modeling rather than a tendency to reproduce the provided examples. The Temporal variant appears less prone to copying, but its generated images are often of relatively low quality. Overall, I find the qualitative results unconvincing, especially given that generating new samples from unseen distributions is the central goal of the paper.

  2. Quantitative results do not match the qualitative results. While Table 3 shows improvements in precision, recall, and FID on ImageNet, I find the qualitative results much less convincing. Many of the generated images either closely resemble examples from the context set or have poor visual quality. I am therefore not sure how much the reported improvements reflect actual generation quality. Also, FID is computed using only 128 generated and 128 reference images per class, which raises concerns about the reliability of these results. Overall, I am not convinced that the method is effective for high-dimensional image generation.

Questions for authors
  1. Memorization vs. generation. In Figure 3, many of the images generated by Dynamic FP-FM look almost identical to images from the context set. How much of the reported performance comes from copying these images rather than generating new ones? The nearest-neighbor analysis in Appendix E.3 is helpful, but I would like to understand how these results relate to the clear similarities shown in Figure 3. What distance metric was used, and could the authors provide a more detailed analysis of this behavior?

  2. Generation quality. The images generated by Temporal FP-FM often look quite poor, while Dynamic FP-FM seems to mostly reproduce the context images. Can the authors provide more examples showing that the method can generate new, realistic images that are different from the context? I think this is important for understanding whether the method is useful in practice.

  3. ImageNet evaluation. The FID scores in Table 3 are computed using only 128 generated and 128 reference images per class. Given the small number of images, how reliable are these results? It would be useful to see whether the conclusions hold with more evaluation samples.

1 comment

Memorization vs. generation. In Figure 3, many of the images generated by Dynamic FP-FM look almost identical to images from the context set. How much of the reported performance comes from copying these images rather than generating new ones? The nearest-neighbor analysis in Appendix E.3 is helpful, but I would like to understand how these results relate to the clear similarities shown in Figure 3. What distance metric was used, and could the authors provide a more detailed analysis of this behavior?

As mentioned in the paper, the expressivity of Dynamic FP-FM allows it to overfit to the provided samples, and therefore it tends to (usually) reproduce these samples with added noise or visual artifacts. For image domains, this is likely unsatisfactory, which is why we suggest practitioners to use Temporal FP-FM in line 439. However, there may be other domains where practitioners prefer fidelity over diversity, in which case it may find uses.

The nearest-neighbor statistic in Appendix E.3 was computed with Euclidean distance between Inception-V3 pooled features. These features are designed to be invariant to layout and fine detail, that’s likely why they understated the similarity visible in Figure 3. We repeated the analysis in the VAE latent space. For Dynamic FP-FM, \tau falls to 0.44 and essentially all generations lie below the 5th percentile of the real-image distances, confirming that Dynamic FP-FM largely reconstructs its context. Temporal FP-FM has \tau = 1.02 with 4.9% of generations below the 5th percentile (5% expected for real images). So its samples are as far from the context as a fresh photograph of the class. The LoRA baseline behaves the same way (\tau = 1.01, 5.3%). We have improved the discussion in the appendix to address this comment.


Generation quality. The images generated by Temporal FP-FM often look quite poor, while Dynamic FP-FM seems to mostly reproduce the context images. Can the authors provide more examples showing that the method can generate new, realistic images that are different from the context? I think this is important for understanding whether the method is useful in practice.

In the supplementary material, you can find hundreds of generated images. We disagree that the Temporal FP-FM images are low quality. Compare the Temporal FP-FM and finetune images in Figure 3, which use the same underlying architecture. These images, sampled from the same noise, are extremely similar. Since finetuning is the gold standard for adaptation, this demonstrates Temporal FP-FM yields useful images. We also see this quantitatively, where Temporal FP-FM has similar FID as finetuning on the unseen support datasets despite a much lower cost.

Note that these experiments are proof of concept, and use a DIT small architecture. Generating industry grade images would require a larger model+more compute and is beyond the scope of this work.


ImageNet evaluation. The FID scores in Table 3 are computed using only 128 generated and 128 reference images per class. Given the small number of images, how reliable are these results? It would be useful to see whether the conclusions hold with more evaluation samples.

Evaluation (FID ↓)TemporalLoRADynamic
128 images/class (5 reference redraws)213.6 ± 0.1222.8 ± 0.2130.5 ± 0.1
128 images/class (3 context redraws)211.1 ± 0.2222.9 ± 0.7130.8 ± 1.0
512 images/class172.5185.193.9

To address this, we run three different ablations. We first compute FID over 128 generated images relative to 5 sets of 128 reference images. We find that the variance is low, meaning that our metric is not very sensitive to the reference images. Next we compute FID of 3 sets of 128 generated images (using different context images) relative to 128 reference images. Again we find that std deviation is low, meaning that our metric is not very sensitive to the context images. Lastly, we increase the number of generated images to 512 and compare against 512 reference images. FID is indeed lower because we used a larger number of images, but the overall trends are the same.

TLDR: Increasing the number of images to compute FID indeed changes the magnitude but not the trends.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.