acceptodds
Withdrawn from ICLR 2027

Localized Subspace Source Adaptation for Flow Matching

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Spatially varying local subspaces are not captured by the global statistics commonly used to adapt Flow Matching (FM) sources. We study which target information should be retained in the initial distribution without changing the probability path or FM objective. We introduce Localized Subspace Source Adaptation (LSSA), a source-only adaptation that represents target regions with multiple anchors and orients Gaussian-mixture components using local PCA projectors. LSSA retains dominant regional directions without reproducing local covariance magnitudes, while a shared variance budget controls total scale and an isotropic residual preserves ambient variation. Across structured toy distributions, scientific data, CLIP and HyCoCLIP embedding generation, LSSA consistently improves distributional fidelity, and competitive in MNIST, over standard Gaussian and statistic-specific source baselines. It achieves the lowest SWD and MMD in the evaluated scientific and embedding-generation settings and improves over the Gaussian in both pixel and VAE-latent MNIST generation. These results show that explicitly modeling localized subspace structure is an effective source-design principle in the evaluated structured and representation spaces for FM.

Primary area:
generative models
Keywords:
Source Adaptation, Generative Modeling

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_UyerOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper designs a new source (noise) distribution for flow matching (FM) that adapts to the data distribution. The goal is to show that FM using this new source outperforms the standard choice (i.e., normal centered with unit variance) and the other source constructions recently proposed in the literature. The proposed approach consists in splitting the ambiant space into KK neighborhoods chosen according to data points, and then choosing a Gaussian Mixture noise distribution. In this GMM, the covariance of each mode is estimated with a truncated PCA on the corresponding neighborhood of the data, and the weights of the mixture are defined with the density of data points. The novelty compared to previous work is to adapt the source noise locally in space (i.e., the PCA part).

Critical strengths
  • The idea is simple to follow
  • The numerical results are reasonably good
  • The numerical cost of the method to estimate the source is probably not very high
Critical weaknesses
  • The contribution is limited: there are a lot of papers in the literature that try (with more or less success) to design a better source noise. The use of both anisotropic Gaussian noise whose covariance captures the local geometry of the data manifold, and of isotropic GMMs is already in the literature. The novelty here is, in a nutshell, to mix the two ideas.
  • There are lots of hyperparameters to this source design. In my opinion, there is a low probability that it will be adopted in practice in some future works.
Questions for authors

Main questions

  1. About the theoretical claims (l. 216 and Remark 1): I had trouble to follow what you wanted to say. First, what exactly would be the goal of establishing this bound / what does it tell about your method? Secondly, I could not understand several notation, please fix them and clarify the statements: are AKA_K and aka_k the same? What does PAk(x)P_{A_k(x)} mean? (l 247). Did you mean P⋆(ak(x))P^\star(a_k(x))? Similarly, what is PiP_i here compared to Pi⋆P_i^\star? Also an orthogonal projector is always 1-Lipschitz, so why do you need to make that assumption?

  2. I find there is a contradiction between the known fact that having a source distribution too close to the data distribution is usually harmful [1], and the fact that your approach is exactly trying to fit at the best the source on the data. Actually, from Figure 1, we see that your source is very similar to the data. What in your opinion explains this apparent contradiction? Is it only due to the isotropy added via β<1\beta < 1? How does your method behave when β=1\beta=1?

  3. Section 3.3: why not use the Voronoi cells as neighborhoods. What happens if you do? Could you also reexplain the difference between the final neighborhood and the Voronoi cells.

  4. In the appendix, you say you use both independent and Hungarian coupling (which I guess here is exactly the minibatch OT coupling) while for instance you use independent coupling for Adapting Noise baseline. The latter was initially proposed with the minibatch OT coupling. For fairness, could you compare all methods for independant and minibatch OT, for instance of MNIST?

  5. Coud you scale to a bigger image dataset (Cifar10, Celeba64...)?

  6. Is it necessary that the source has zero mean or the method could work without recentering (i.e., mi=aim_i=a_i)?

  7. Does the proposed method improve over standard FM for unimodal (1-class) data?

Minor remarks

  • The first sentence of the abstract is not clear, we don't understand what the spatially varying subspace refer to. In general, both with the abstract and introduction, it took me time to understand what you meant by subspaces even though I know the topic. Consider revising the paper for reaching a broader audience.
  • When introducing FM, [3] should also be cited.
  • The reference [2] is missing

[1] : Lee, Kwanseok Kim, and Joonseok Lee. Is there a better source distribution than gaussian? Exploring source distributions for image flow matching. Transactions on Machine Learning Research, 2026

[2] Luo, Gaoxiang and Cole, Frank and Zhang, Sihang and Wan, Yuxiang and Lu, Yulong and Sun, Ju, Flow Matching for Multimodal Distributions, CVPR 2026

[3] Albergo, Michael S. and Vanden-Eijnden, Eric, Building normalizing flows with stochastic interpolants, ICLR 2023

Reviewer_Rt2POpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

LSSA changes the source distribution of flow matching to a data-derived Gaussian mixture. K-means anchors and regional frequencies locate components; local PCA supplies their principal subspaces. A common covariance budget retains orientations but discards local eigenvalue magnitudes, with isotropic residual noise. The paper studies protein conformations, conditional CLIP/HyCoCLIP embedding generation, and MNIST, with controlled tests of localization, anisotropy, spectrum, scale, anchor count, and projector assignment.

Critical strengths

S1. The experiments test the proposed insight directly. Tables 5 and 8 separate localization, anisotropy, and correct anchor-projector correspondence under a matched second-moment budget. ProteinConformers SWD improves from 0.3007 for Local-Iso and 0.2768 for shuffled projectors to 0.2374 for correct assignments. Table 6 further tests whether adding local spectrum or scale helps, and Table 7 varies anchor resolution. These controls provide substantive evidence that localized orientation matters, beyond merely showing that a fitted mixture beats a Gaussian.

S2. The construction is simple and its claims are appropriately scoped. Equations (9)-(12) retain the ordinary linear path and FM loss while providing a tractable low-rank-plus-isotropic sampler. Remark 1 explicitly distinguishes representation error from generation error and acknowledges its assumptions. The reported distributional improvements extend across several representations, with retrieval and image-quality checks. The authors accurately acknowledge the exceptions in MNIST pixel FID and CLIP T2I recall rather than claiming universal superiority.

Critical weaknesses

W1. The precision and practical scale of the improvement remain uncertain. Tables 1-8 report no variability across source fitting or generator runs. The large ProteinConformers gains are persuasive, but several retrieval differences are small, such as CLIP I2T R@1 of 10.18 versus 9.96, and should not be treated as established rankings. Repeated projector/spectrum and correct/shuffled controls would make the central statistical conclusion more secure. Source preprocessing time, storage, and sampling cost are also unreported, so the practical tradeoff of local PCA and low-rank mixture sampling is not yet quantified.

W2. The contribution needs clearer positioning and a verifiable specification. MM-FM already fits a GMM source with mode-dependent coupling [R1], and CCVFM builds a local-PPCA mixture for a velocity-space surrogate [R2]. These are not equivalent algorithms: LSSA's distinctive evidence concerns projector-only transfer under a shared budget, which Table 6 partially addresses. This distinction should be explicit. Separately, the supplied 11-page PDF lacks the referenced Appendices A-D, leaving the allocation between c and R, actual coupling, and K/k/r settings unavailable. This limits the present review; it is not evidence that the authors failed to submit those materials or that the reported controls are invalid.

[R1] Luo, Cole, Zhang, Wan, Lu, and Sun (2026). Flow Matching for Multimodal Distributions. [R2] Wang, She, and Su (2026). Coreset-Induced Conditional Velocity Flow Matching.

Questions for authors

Q1. Please identify the separate calibration of c and R, actual coupling, K/k/r values, and definitions of the spectrum/density controls, together with the splits, training budgets, solver/NFE, and metric estimators in the referenced material.

Q2. What is the uncertainty across independently fitted sources and generator runs for the main gains and decisive Table 6/Table 8 controls? Which of the small retrieval improvements remain supported?

Q3. How does the projector-only construction relate to the fitted-mixture sources in MM-FM and CCVFM, and which part of Table 6 isolates that distinction? What preprocessing, storage, and sampling overhead accompanies the improvement?

Reviewer_Ays2OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes Localized Subspace Source Adaptation (LSSA), a method for improving Flow Matching by replacing the standard Gaussian source with a Gaussian mixture that incorporates local geometric information from the target distribution. Specifically, the authors first use K-means to identify representative regions of the target distribution, with mixture weights determined by the proportion of training samples assigned to each region. They then apply local PCA to estimate the dominant subspace directions around each anchor and use the resulting projection matrices to construct the covariance of each Gaussian component. The proposed method preserves local directional structure while maintaining a shared variance budget, without modifying the standard Flow Matching objective or probability path.

Critical strengths

The paper presents an interesting approach to adapting the source distribution in Flow Matching by incorporating local geometric information from the target data. Rather than modifying the training objective or probability path, it provides a simple way to construct a geometry-aware source distribution.

Critical weaknesses
  1. The proposed method constructs its source distribution directly from the training data, using K-means anchors, empirical mixture weights, and local PCA directions. This raises concerns about whether the resulting Gaussian mixture may overfit the observed data structure and affect the diversity or mode coverage of generated samples. However, the paper does not investigate the robustness of the generation process to imperfect source estimation, particularly when certain regions of the target distribution are insufficiently represented in the training data.

  2. The proposed method requires K-means clustering, nearest-neighbor search, and local PCA estimation to construct the Gaussian mixture source. These preprocessing steps may introduce substantial computational and memory overhead for large-scale, high-dimensional datasets. However, the paper does not provide a quantitative analysis of these costs or how they scale with dataset size and dimensionality, leaving the computational practicality of the method for large-scale generative modeling unclear.

Questions for authors
  1. Since the proposed Gaussian mixture source is constructed directly from the target data, its probability mass may substantially overlap with the target distribution. However, the paper does not analyze how this source–target overlap affects the learned Flow Matching velocity field. Does LSSA actually simplify the transport dynamics compared to a standard Gaussian source, or could it introduce ambiguous transport directions? An analysis of velocity magnitudes, conditional velocity variance, and trajectory curvature under matched coupling settings would help clarify this issue.

  2. Could the authors discuss whether the proposed method remains effective for more complex data distributions, such as those encountered in natural image generation? In particular, it is unclear whether the K-means and local PCA-based source construction can adequately capture the local geometric structure of such distributions.

  3. Could the authors report the computational and memory overhead of constructing the proposed Gaussian mixture source, including K-means clustering and local PCA estimation? In particular, how do these costs scale with dataset size, dimensionality, and the number of mixture components?

  4. How does the quality of the constructed Gaussian mixture source affect the diversity and mode coverage of generated samples? In particular, if the mixture fails to adequately represent certain regions or modes of the target distribution, does this negatively affect the final generation quality? An analysis under imperfect source estimation would help clarify the robustness and generalization of the proposed approach.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.