acceptodds
Withdrawn from ICLR 2027

FlowSeed: Decoupled Source Learning for Cross-Modal Generation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Flow-based generative models typically start from condition-independent Gaussian sources, leaving source initialization largely unexplored. We show that source initialization substantially affects generation quality, while jointly learning condition-dependent sources and generators can undesirably entangle source learning with generative modeling. We therefore propose FlowSeed, a decoupled framework that learns condition-dependent sources independently of the image generator. FlowSeed first encodes paired cross-modal correspondence in an aligned representation space, from which independently trained flows recover compatible source structures. For text-to-image generation, a lightweight text flow provides condition-dependent initialization for the aligned semantic component, while a token-wise FlowEdit-style strategy bridges it to the image generation flow. By decoupling source learning from generation, FlowSeed preserves the efficient and diverse generation paradigm of standard Flow Matching while enabling more effective condition-aware initialization. Experiments across multiple representation spaces consistently demonstrate improved generation performance over standard Flow Matching and learned-source baselines. Moreover, FlowSeed supports plug-and-play adaptation of pretrained generators to new conditioning modalities without retraining the generator.

Primary area:
generative models
Keywords:
Generative Model, Flow Matching, Source Learning.

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_EHkAOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper in inscribed in a growing body of works that attempts to improve FM by pairing targets with an ‘optimal’ source distribution. Instead of simultaneously learning a flow and an unsupervised transformation on the initial distribution, the authors propose to decouple the process. The goal is to obtain condition dependent sources to improve prediction alignment with the guiding condition.

Critical strengths

Experiments are consistent with claims, and the idea is original and fills a gap in the existing literature. Decoupling is shown to improve both image-text consistency and, somewhat surprisingly, overall image quality.

Conditioning modality transferability shown in section 5 is an interesting application, with great potential to domains outside image/text.

Presentation is good. The submission is clearly written (except for section 4) and structured, and the progression of ideas is natural and readable.

Critical weaknesses

The motivation is rather light. The advantages of decoupling are shown with a toy experiment, but a theoretical analysis is missing.

Ablations are missing, and section 4 is hard to follow, with unstated assumptions and unjustified design choices. Why concatenating aligned and unaligned image representations in section 4? How can you guarantee that the better performance shown in table 1 is not due to increased expressivity of the higher dimensional space you are using? In my opinion this is the weakest point of the manuscript, and I'm willing to raise my score if this point is addressed.

Questions for authors

Consider the naïve decoupling approach, where we use the Representation Autoencoder paradigm with CLIP or SigCLIP as encoder. Given a text conditioning signal, we encode it, revert the flow, and obtain an initial sample z0. Have you compared against this or a similar baseline?

Can you show FID, precision and recall for figure 1b? What does D_x stand for in equation (9)?

I’m confused as to what motivates the factorized image representation in section 4. In Reg a single global token is added, but here its many more right? This raises the question, given that decoupling as explained in section 3 could be achieved with a single representation, why increse the number of dimensions? Is it possible to train FlowSeed without an unaligned component? Do you have experiments on this?

Any intuition on what does the push-forward measure under your method look like?

1 comment
AuthorsOpenReview ↗

Response to Reviewer EHkA.

Thank you for recognizing the originality and conditioning-transfer potential of FlowSeed. We directly address your main concern: whether the gain comes from decoupled source learning or increased representation capacity.

W2 / Q3: Dimensionality, factorization, and component contributions.

In DINOv2/SigLIP, we add one global semantic token alongside the original 256 local DINO tokens, as in REG. The global component provides an aligned interface for source transfer; local tokens retain spatial detail and stochastic variation. The following controls separate their roles. A/B use the same data, backbone design, and training protocol; B-D share one frozen image-flow checkpoint, representation dimensions, decoder, and conditioning. All entries are re-evaluated for this control.

VariantRepresentationInitialization/samplingGenEvalDPGFID
A: Standard FMLocal onlyGaussian66.271.92.90
B: Standard FMGlobal + localGaussian67.672.82.77
C: Decoupled sourceGlobal + localText-flow inversion; no FlowEdit71.574.32.79
D: Full FlowSeedGlobal + localToken-wise FlowEdit73.075.02.81

Adding the global representation gives +1.4 GenEval points (B-A). Learned initialization alone adds +3.9 GenEval/+1.5 DPG (C-B), before FlowEdit adds +1.5/+0.7 (D-C). Thus, D improves over B by 5.4/2.2 points at fixed dimensionality and with the same image generator. FID changes from 2.77 to 2.81, showing a small image-quality trade-off for the semantic improvement. Each FID uses its indicated sampler, including the full sampler for D.

The unaligned component is also not required for applicability. Table 1's single-space CLIP setting gives Standard FM/FlowSeed GenEval 58.4/64.7, DPG 61.2/70.8, and FID 3.45/3.10, without concatenating an unaligned representation. Both methods share the DALL-E 2-style decoder within that setting. These results establish single-space feasibility; the matched controls above isolate contributions in the main setting.

Q1: Comparison with the proposed naive decoupling baseline.

We tested your baseline in the single-space CLIP setting. Both variants start from the same text embedding and use the same forward image flow, decoder, and forward sampling budget, without FlowEdit. Only the inverse mapping changes:

Inverse mapping for initializationGenEvalDPGFID
Image-flow inversion (naive)59.162.43.58
Independent text-flow inversion63.268.53.18

Independent text-flow inversion improves GenEval/DPG by 4.1/6.1 points and reduces FID by 0.40. This isolates its benefit from FlowEdit and representation expansion. Both use 50 inverse and 50 forward steps, totaling 100 velocity-field evaluations; this matches evaluation counts, not the computational cost of the differently sized flows.

Q2: FID, precision/recall, and the meaning of DxD_x.

We supplemented the capacity study at 80M and 160M, spanning the CSFM optimum and subsequent degradation. Sample counts, reference images, and evaluators are identical.

MethodCapacityFIDPrecisionRecall
CSFM80M2.850.730.55
CSFM160M3.160.690.48
FlowSeed80M2.810.750.58
FlowSeed160M2.740.760.60

CSFM loses both fidelity and coverage beyond 80M. FlowSeed instead improves FID, precision, and recall while its prompt-following scores increase (Table 2). This supports scalable source learning beyond semantic metrics alone. In Eq. (9), DxD_x is the x-modality reconstruction decoder used during representation alignment.

W1 / Q4: Mechanism, assumptions, and push-forward.

Jointly updating a learned source changes both interpolated states and sample-level velocity targets x1−z0x_1-z_0. We hypothesize that this interaction explains the observed scaling limitation. FlowSeed removes this optimization coupling: source learning leaves the standard image-FM objective and learned image vector field fixed.

For explicit two-stage initialization, let zc=Tc(Etext(c))z_c=T_c(E_{\rm text}(c)). Then q0(⋅∣c)=δzc⊗N(0,Ilocal)q_0(\cdot\mid c)=\delta_{z_c}\otimes\mathcal N(0,I_{\rm local}) and p_{\rm FS}(\cdot\mid c)=(G_c)_{\\#}q_0(\cdot\mid c). Semantic initialization is condition dependent while local randomness remains. FlowEdit uses a separate velocity-difference sampler; exact equivalence to inversion is not assumed.

For the toy's unconditional flows, identical aligned marginals, Gaussian priors, independent couplings, and FM paths define the same population objective. This motivates compatible transports; practical alignment is supported empirically, not guaranteed pointwise. Together, the matched controls and naive baseline directly support a source-learning contribution independent of added dimensions or FlowEdit.

Reviewer_qPbNOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The authors present FlowSeed, a framework for conditioned source learning in diffusion models. By independently learning the conditioned source distribution and the generation process, FlowSeed improves generation quality and allows for scaling of the source-flow network without diminishing returns. This suggests a potential impact to the wide field of diffusion model generation by providing a new pareto-optimal method for noise sampling.

Critical strengths

S1: The framework proposed by the manuscript can be deployed without the need for retraining on any pretrained class-conditional generator. The qualitative results reported by the authors (Fig. 4, 9, 10) show impressive improvements in perceptual quality and semantic consistency of the generations compared to unconditioned outputs when FlowSeed is applied on pretrained models.

Critical weaknesses

W1: The paper's presentation is poor. The absence of a structured Introduction and Related Work section makes it hard for the reader to correctly identify the state of the art, the problem motivation and the setting in general. This could be especially problematic for readers that are not completely familiar with the field. In general, the authors have structured the paper's narrative in the initial sections so that the current solutions in the field of source initialization and conditioning are shown to have shortcomings, so that their method's necessity is motivated. This choice of narrative does not discuss the broader state of the art in the field of generative models, which leads to poor grounding of the manuscript with respect to the state of the art. This is also apparent by the small size of the provided bibliography.

W2: The evaluations reported by the authors are somewhat inconsistent and lack statistical rigor in some places. For instance, the gaps reported in Fig. 1a are reported without standard deviations, confidence intervals or statistical significance tests; given the absence of seed counts in the reported results, it is unclear whether the claim that the choice of source initialization actually matters to generation quality, rather than the result simply coming from the natural variance emerging from the noise sampling process. Similarly, Fig. 2b shows the failure of previous source network approaches to scale, but the authors do not report FlowSeed together with these baselines, so it is not clear whether the scaling capabilities are recovered by decoupled flow conditioning or not. In addition, the qualitative results reported in the main paper and supplementaries are not very diverse; while they show a wide range of text conditionings, they mostly show the same underwater scene for the image target. A greater amount of diversity would be appreciated to better gauge the qualitative effects of decoupled flow on semantic grounding.

W3: Given the lack of important introductory sections, significant parts of the manuscript are dedicated to toy evaluations in support of the main thesis that would better belong in the supplemental material. The authors' definition of "good noise rate" is confusing and insufficiently supported by citations or theoretical explanations.

Questions for authors

Q1: Could the authors repeat the experiment in Fig. 1a, reporting standard deviations and statistical significance tests for p<0.01? Could the performance of FlowSeed be compared alongside the baselines in Fig. 1b?

Q2: I understand that significant restructuring of the manuscript would be required in order to introduce an Introduction and Related Work section, which is potentially not within the scope of this conference's rebuttal. Could the authors discuss the reason for the absence of a Related Work section detailing diffusion models, and, if possible, suggest a correction that could improve the paper's presentation (by separating the introductory section from the methodology, making the manuscript's contribution clearer)?

Q3: Could the authors revisit the toy verification section, so that the experiment's necessity is clearer and the reported results are more structured? Could it be perhaps better shown in a plot?

Reviewer_qqucOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents a method to learn text-side flow independently of the image generator, using aligned representations to obtain condition-dependent source initializations. It combines with global-token FlowEdit-style sampling.

This method is evaluated on text-to-image generation task, across a few different representation settings and demonstrates uniformly superior results on GenEval, DPG-Bench, FID, and CLIP-score.

Critical strengths

Consistent reported prompt-following improvements. The positive numerical results are the most standing-out factor when reading this paper. The reported metrics are sufficient and in many perspectives.

Critical weaknesses

Lack of strong qualitative results. Despite showing some numerical results as comparion, this is far from enough to qualify as a good research paper. The only cases that is shown in the paper, including appendix, fig.4 and fig. 8-10. Among them, some cases are even reused in both figures, which cannot support a claim that the proposed method should be significant.

Low quality figures. As a good academic practice, it is recommended to use high resolution figures in paper, rather than a probably python-generated figure being directly used in paper. Fig 4 and 8-10 are all lacking sufficient resolution, and in some way reduced the credibility and soundness of this paper. Also, the paper is written in a form that is fairly unconventional, without a clear structure for sec. 1-5, and missing literature review or related work study.

Questions for authors
  1. Is there more results that I missed, except for fig. 4, 8-10? Or is it that the proposed method has limited applicability and usage scope?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.