acceptodds
Under review as a conference paper at ICLR 2027

Flow Matching with Arbitrary Auxiliary Paths

Abstract

We introduce **Flow Matching with Arbitrary Auxiliary Paths (AuxPath-FM)**, a generative modeling framework that makes the auxiliary distribution an explicit choice in probability-path design. AuxPath-FM constructs trajectories , allowing to be continuous, discrete, or coupled to the endpoints without requiring Gaussianity or independence. Under explicit moment and coefficient regularity conditions, we establish endpoint preservation, a weak continuity equation, and a tractable regression objective equivalent to marginal flow matching for each fixed path. This formulation accommodates Gaussian, Uniform, Laplace, Rademacher, and learned semantic auxiliaries within a common training framework. Semantic auxiliaries encode labels in the path and enable trajectory-level guidance with one backbone evaluation per sampling step. Experiments reveal how auxiliary choice affects generation across datasets and how semantic guidance improves label alignment. On ImageNet-1k, auxiliary guidance reduces FID from 23.25 to 18.01 at approximately the one-pass CFM cost and roughly half the computation of two-pass classifier-free guidance (CFG) at the same step count. AuxPath-FM connects flexible path design with a practical quality–cost tradeoff in conditional generation.

Primary area:
generative models
Keywords:
flow matching, stochastic interpolants, auxiliary distributions, conditional generation, classifier-free guidance

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_ca3TOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes AuxPath-FM, which incorporates auxiliary variables with continuous & discrete distributions, inclouding endpoint-dependent auxiliary variables, into the path of flow matching, and presents the theoretical justification for the continuity equation and regression objective. By applying these auxiliary variables to conditional generation, the method presents the learned semantic auxiliary as an explicit drift component. This enables one-pass guidance, allowing the influence of the conditioning information to be adjusted using only a single backbone evaluation per sampling step. Experiments on datasets such as MNIST, CIFAR-10, and ImageNet demonstrate improvements in both conditional alignment and generation quality.

Critical strengths
  • This paper explores various auxiliary variables in flow matching and provides a theoretical foundation for them. It offers valuable insight by systematically categorizing the options available for designing paths.

  • One-pass guidance utilizing semantic auxiliary variables provides a useful control mechanism for conditional generation. This approach is practically significant, as it demonstrates the potential to improve generation quality without substantially increasing computational overhead.

Critical weaknesses
  • While this study offers valuable insights, the reasons for the performance gain are not sufficiently clear. Although some experiments show improvement, there is limited explanation clarifying the specific role of the proposed training method in these gains.

  • There is a lack of comparison based on similar computational costs (ex., Table 4). Since the proposed method and existing guidance approaches require different computational costs, it is difficult to determine whether it offers an advantage when using the same computational resources.

Questions for authors
  • Regarding weakness 1, could the author clarify the reasons for the observed performance gains, particularly the contribution of the proposed training method?

  • Are there any comparison results between the proposed method and existing guidance methods under comparable computational budgets?

4 comments

Thank you for your time and suggestions on performance gains and matched-cost evaluation. We added controls separating auxiliary-path training from sampling-time guidance.

1. Contribution and sources of the performance gains.

AuxPath-FM makes the auxiliary distribution a design variable of an explicit, endpoint-preserving flow-matching path. The auxiliary may be discrete, learned, or coupled to the endpoints; our weak-transport and fixed-law regression guarantees do not require an auxiliary density. For conditional generation, this construction lets semantic information shape the transport path itself and exposes its effect as an explicit velocity component.

This path design supports both model adaptation and controllable generation. Training adapts the backbone to transport along the semantic path, while auxiliary CFG adjusts the explicit semantic component with one backbone prediction per integration step. It can also be combined with velocity CFG, giving the same model two independently adjustable controls. We assess training at unit auxiliary strength and sampling-time guidance by varying the coefficient with the checkpoint fixed.

Training and sampling controls. We compare ordinary CFM, frozen CFM with auxiliary guidance, and adapted AuxPath-FM. The learned-FF rows reuse the same auxiliary network. All settings use 50 Euler steps, no velocity CFG, and 50,000 ImageNet-256 samples, evaluated with the original FID protocol.

BackboneAuxiliaryAux. scaleFID ↓
Frozen CFMNone—23.3294
Frozen CFMClass mean123.0011
Frozen CFMClass mean216.8829
Frozen CFMLearned F123.0431
Frozen CFMLearned F217.2810
Adapted AuxPath-FMLearned F121.6411
Adapted AuxPath-FMLearned F217.4678

Frozen CFM uses translated-input sampling. Adapted AuxPath-FM uses its native residual field trained on the auxiliary path.

At auxiliary scale 1, adapted AuxPath-FM achieves FID 21.6411, compared with 23.3294 for ordinary CFM and 23.0431 for frozen CFM with the same learned FF. The benefit is therefore already present with the prescribed auxiliary term, before increasing its guidance coefficient. The paired DiT-B/2 experiment from random initialization also improves FID from 85.6556 to 83.6281 at scale 1. These comparisons support the contribution of backbone adaptation on the auxiliary path.

Keeping the adapted checkpoint fixed and increasing the auxiliary scale from 1 to 2 further improves FID from 21.6411 to 17.4678. Frozen CFM with the same learned FF reaches 17.2810 at scale 2. These results identify a substantial sampling-time contribution alongside the unit-strength improvement of the adapted model. We thank the reviewer for encouraging these controls, which also motivated auxiliary guidance with no additional backbone training.

Why subtract the auxiliary displacement from the input. Let Fy=F(y)F_y=F(y) and c(0)=c(1)=0c(0)=c(1)=0. Writing the auxiliary-path state as a translated CFM state gives

xt=zt+c(t)Fy,zt=xt−c(t)Fy.x_t=z_t+c(t)F_y,\qquad z_t=x_t-c(t)F_y.

The chain rule yields

x˙t=vCFM(xt−c(t)Fy,t,y)+c′(t)Fy.\dot x_t=v_{\mathrm{CFM}}(x_t-c(t)F_y,t,y)+c'(t)F_y.

Subtracting c(t)Fyc(t)F_y therefore returns the input to the coordinates in which the frozen CFM was trained. Auxiliary CFG adjusts the explicit term:

\dot x_t=v_{\mathrm{CFM}}(x_t-c(t)F_y,t,y)+c'(t)F_w,$$ where $F_u$ is the null-condition auxiliary. The input translation uses $F_y$, so the guided dynamics in the original coordinates become $$\dot z_t=v_{\mathrm{CFM}}(z_t,t,y)+(w-1)c'(t)(F_y-F_u).$$ At $w=1$, the extra term vanishes. Since $c(0)=c(1)=0$, exact integration preserves the original CFM endpoint map. At $w=2$, the remaining term $c'(t)(F_y-F_u)$ changes the trajectory and can change the endpoint distribution without updating the backbone. A fixed empirical class mean is a valid deterministic auxiliary and avoids training an auxiliary network. Its FID of 16.8829 at scale 2 supports this simple choice for class-conditioned generation. Even when classes share a mean, the label-conditioned residual field can represent their different structures. Learned $F$ serves a broader conditioning interface: a shared map from text embeddings computes auxiliaries for new prompts on demand. This extends auxiliary design to text-to-image sampling, where storing an empirical mean for every possible description is impractical. **Planned revision.** Section 4.3 will explain how semantic path design supports backbone adaptation and controllable guidance. Section 5.4 will include these controls and distinguish training from guidance gains, with the coordinate derivation in Appendix B.

2. Generation quality at matched inference budgets.

At NFE=10, AuxPath-FM at auxiliary scale 2 improves FID, precision, and recall over the tested CFG settings. Each setting below uses 50,000 generated ImageNet-256 images and the original FID protocol. NFE counts conditional and unconditional backbone evaluations separately, including batched branches. Single-branch Euler sampling uses one evaluation per step; velocity CFG uses two:

MethodCFG scaleAux. scaleSampling stepsNFEFID ↓Precision ↑Recall ↑
CFM1—101038.76570.41880.6106
CFG1.5—51036.70150.38530.4002
CFG2—51028.33240.44450.2959
AuxPath-FM11101033.91820.44560.5823
AuxPath-FM12101027.39340.47060.5345

At guidance scale 2, AuxPath-FM achieves FID 27.3934 versus 28.3324 for CFG, with precision 0.4706 versus 0.4445 and recall 0.5345 versus 0.2959. Thus, the improvement includes both distributional fidelity and coverage at the same number of backbone evaluations. AuxPath-FM can use 10 integration steps within this budget, whereas two-branch CFG uses 5.

Performance across sampling budgets. AuxPath-FM improves FID over the matched CFM model at all four tested budgets. We evaluated the same checkpoints at 10, 20, 30, and 50 Euler steps with velocity CFG disabled, so each step uses one backbone evaluation:

Sampling stepsNFECFM: FID ↓AuxPath w=1: FID ↓AuxPath w=2: FID ↓
101038.765733.918227.3934
202027.728324.971820.0395
303024.762922.955218.4473
505023.329421.641117.4678

At auxiliary scale 2, the 20-step result, FID 20.0395, is better than the 50-step CFM result of 23.3294; 30 steps further improve FID to 18.4473. Scale 1 also improves over CFM at every tested budget. These results complement the equal-NFE CFG comparison and show the quality–computation trade-off across the measured range.

The NFE=50 comparison against CFG requires 25 CFG integration steps. That matched-base evaluation is pending; the table above reports the completed CFM and AuxPath-FM results. We will add the 25-step CFG result when it is available.

Joint auxiliary guidance and CFG. Jointly tuning auxiliary guidance and velocity CFG improves generation quality: the best setting in our ImageNet-256 sweep reaches FID 4.1076. Both coefficients are adjusted on the same trained model. The following sweep uses 50 Euler steps and 50,000 generated images per setting under the original FID protocol:

CFG scaleAux. scaleSampling stepsNFEFID ↓
10.8505023.3339
10.9505022.4540
11505021.6411
12505017.4678
1.51501004.9719
1.51.5501004.8539
1.52501005.2349
20.8501004.1076
20.9501004.2334
21501004.3679
22501005.6816

At CFG=2, auxiliary scales 0.8, 0.9, and 1 give FIDs 4.1076, 4.2334, and 4.3679. At CFG=1.5, scale 1.5 gives 4.8539, while scale 2 gives 5.2349. The best tested auxiliary coefficient therefore depends on the velocity-guidance coefficient, and values below 1 are useful in the joint sampler. The 50-step configurations cost NFE=50 at CFG=1 and NFE=100 with two-branch CFG; the table makes this budget difference explicit.

Planned revision. Section 5.4 will include the equal-NFE comparison alongside Table 4, with separate columns for integration steps and NFE. Appendix D will include the sampling-budget curve and joint-guidance sweep. The accompanying discussion will connect generation quality to the corresponding computation budget.

I appreciate the author's rebuttal and the additional experiments.

I acknowledge that the proposed training method shows benefits at auxiliary strength 1. However, at strength 2, a frozen CFM using the same learned auxiliary performs comparably to or even better than the trained AuxPath-FM. Consequently, the substantive contribution of the additional training to the main performance gains remains unclear.

Could you please clarify how these results should be interpretred? Specifically, what additional advantages does the proposed training offer over simply applying sampling time guidance to a frozen backbone? Based on the evidence currently provided, I will maintain my original score.

Dear Reviewer ca3T,

Thank you for clarifying the training question. We assess its additional contribution using an equal-budget continuation experiment with a shared auxiliary network.

1. Matched training and its relation to the earlier results.

The earlier comparison used the original CFM without further backbone training and an existing adapted AuxPath-FM checkpoint. Its learned-FF evaluations shared the auxiliary network from that existing checkpoint.

For the matched control, both branches start from the original CFM and receive 40K additional steps with the remaining training settings matched. One learns on ordinary CFM paths; the other learns residual transport on auxiliary paths. Thus, the CFM control also receives 40K backbone updates. At inference, it uses translated-input auxiliary guidance, while the AuxPath branch uses its trained residual field. Both fixed backbones use the same FF from the new AuxPath branch's 40K checkpoint.

These FIDs correspond to new backbone and auxiliary weights, explaining their differences from the earlier results. The present table measures the training-path contribution at equal continuation budget. Both branches use 50-step Euler, CFG=1, NFE=50, and 50K samples.

Auxiliary wwCFM training + sampling-only guidance: FID ↓AuxPath training: FID ↓FID reduction
124.1423.230.91
1.520.3820.160.22
320.5120.440.08

Path training provides its largest additional benefit at the lower guidance strength. The gap narrows as guidance increases; reductions use unrounded FIDs.

Endpoint behavior at unit strength. At w=1w=1, translated-input sampling preserves the original CFM endpoint under exact integration. Using the original frozen backbone, 1,000 paired noise–label samples, CFG=1, and FP32, we compare it with ordinary CFM at each Euler step count. We average the per-sample endpoint L2 difference divided by the corresponding ordinary-CFM endpoint norm:

Auxiliary10 Euler steps25 steps50 steps
Learned FF8.69%3.61%1.88%
Class mean9.29%3.86%2.02%

The decreasing discrepancy is consistent with the endpoint-preserving coordinate transformation at unit strength.

2. Adaptation with larger auxiliary displacement.

We increased c(t)c(t) from t(1−t)t(1-t) to 4t(1−t)4t(1-t), raising its peak from 0.25 to 1. We jointly trained the backbone and FF for 300K steps from pretrained CFM-B/2. The baseline freezes the original backbone and uses class means. Both use 20-step Euler, CFG=1, NFE=20, and 50K images.

Auxiliary wwFrozen CFM + class means: FID ↓Jointly adapted backbone + learned FF: FID ↓
125.4123.29
1.222.8818.63
1.428.1920.04
251.2042.17

The complete adaptation scheme lowers FID by 4.25 and 8.15 at w=1.2w=1.2 and 1.41.4. This comparison includes learning FF and adapting the backbone; the 40K control above isolates the training-path contribution.

3. Text-conditioned path learning.

A shared F(y)F(y) computes auxiliaries for new prompts without storing a mean for every text. This generalization serves both frozen and adapted variants; adaptation learns residual transport on the text-conditioned paths. Appendix F evaluates AuxPath-LoRA on SANA using BLIP-VQA, 30 held-out prompts, and 129 atomic questions. At w=2w=2, atomic accuracy reaches 85.27%, versus 80.62% for Standard LoRA; all-correct rates are 53.33% versus 40.00%. These results evaluate the complete learned approach.

4. Summary of contributions.

Finally, we summarize our three contributions.

(i) Theory: Wei–Ma Section 3.1 allows general path laws, while Appendix J.1 uses density-form integrals and Gaussian regularization with assumed limit exchanges. Stochastic interpolants' density analysis uses Gaussian auxiliary noise. Our test-function proof covers discrete, singular, and endpoint-dependent auxiliaries without density assumptions, establishing endpoint preservation and weak continuity for the proposed path family under finite first moments and absolutely continuous coefficients satisfying the endpoint constraints.

(ii) Auxiliary distributions: Wei–Ma's experiments use Gaussian-process streams; the cited stochastic-interpolants paper uses Gaussian auxiliary noise. They do not compare other auxiliary laws. Table 3 tests Gaussian, uniform, Laplace, Rademacher, and label-guided auxiliaries, including discrete −1/+1-1/+1 noise and continuous non-Gaussian laws.

(iii) Learned semantic paths: Joint learning adapts auxiliary representations and residual transport for conditional generation, exposing an explicit velocity component for guidance.

We will add the new amplitude results to the appendix. We hope this framework supports further exploration of auxiliary distributions for flow learning, a direction we will continue pursuing. Thank you again for the constructive discussion.

The authors

Reviewer_23KdOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes AuxPath-FM, a framework for flow matching with auxiliary variables that can follow arbitrary distributions or depend on the data. It establishes theoretical properties of the resulting paths and introduces a label-dependent auxiliary variable for conditional generation. Experiments show improvements in conditional generation and propose a guidance mechanism requiring one model evaluation per sampling step.

Critical strengths
  • The theoretical analysis is generally rigorous. I found no clear mathematical errors in the main results, and the assumptions and limitations are appropriately stated.

  • The semantic guidance results are promising. On ImageNet, FID improves from 23.25 to 18.01 with almost no additional inference cost compared with unguided CFM.

Critical weaknesses
  1. The generalization to arbitrary auxiliary distributions appears closely related to existing work on stochastic interpolants and general flow-matching paths, particularly Stream-level Flow Matching with GPs (Wei and Ma, ICML 2025). The main theoretical results only extend established arguments. The paper does not sufficiently explain what fundamentally new capability it introduces.

  2. The benefit of the proposed training method is not evaluated in isolation. Proposition 3 shows that deterministic semantic auxiliaries correspond to translations of ordinary conditional paths, with unchanged optimal regression error. But it's unclear whether the improvements arise from the proposed training method or just from the additional guidance applied during sampling. It would be good to see a comparison applying equivalent guidance to a standard CFM model for example.

  3. The main efficiency claim compares auxiliary guidance and CFG using the same number of sampling steps, although CFG requires twice as many model evaluations. A comparison at equal computational cost should be included. Moreover, all random auxiliary distributions worsen ImageNet FID, and improvements on smaller datasets are inconsistent. Also ,results are reported from single runs, without error bars.

  4. It seems like the released code implements the Uniform and Rademacher auxiliaries identically, despite reporting separate results? Additionally, Table 2 reports a CIFAR-10 FID of 3.68 for learned auxiliaries, whereas Table 6 reports 4.14 at guidance strength 1, which should recover ordinary auxiliary sampling. The differences in experimental settings are not explained.

Questions for authors
  • How does the theoretical contribution differ substantially from existing general flow-matching formulations, particularly Wei and Ma (2025)?

  • Can the authors compare against standard CFM with the same auxiliary guidance applied only during sampling, without auxiliary-path retraining?

  • What FID does standard CFG achieve with 25 sampling steps, matching approximately the inference cost of 50-step auxiliary guidance?

  • Why do Tables 2 and 6 report different results for apparently equivalent CIFAR-10 configurations?

  • Was the identical implementation of Uniform and Rademacher auxiliaries used to obtain Table 3?

7 comments

Thank you for your detailed assessment of the theory and experimental settings. We clarify the contribution and explain the implementation and table discrepancies below.

1. Theoretical contribution and relation to prior work.

AuxPath-FM extends additive path design to general auxiliary joint laws and turns the resulting semantic velocity term into a controllable guidance mechanism. Its explicit endpoint-preserving construction is

Xt=a(t)X1+b(t)X0+c(t)η,(X0,X1,η)∼Q,X_t=a(t)X_1+b(t)X_0+c(t)\eta,\qquad (X_0,X_1,\eta)\sim Q,

where a(0)=b(1)=0a(0)=b(1)=0, a(1)=b(0)=1a(1)=b(0)=1, and c(0)=c(1)=0c(0)=c(1)=0. The auxiliary can be discrete, deterministic, learned, or coupled to the endpoints. With absolutely continuous coefficients and finite first moments, the sample derivative gives

=\mathbb E[\nabla\varphi(X_t)\cdot\dot X_t], \qquad u_t(x)=\mathbb E[\dot X_t\mid X_t=x].$$ This weak transport identity is formulated through expectations under $Q$ and requires no auxiliary density. With space-time square integrability, regression on $\dot X_t$ and on $u_t$ has the same population minimizers for each fixed path law. For learned auxiliaries, this applies at each fixed parameter value; joint optimization also changes the path-dependent variance. A unique deterministic ODE realization additionally requires velocity-field regularity. **Stochastic interpolants.** Their standard additive construction uses an independent Gaussian latent. Remark 2 allows other noise distributions, but the paper develops this auxiliary-noise construction using Gaussian noise. Its density-regularity proof uses Gaussian smoothing to establish a smooth, strictly positive interpolant density, on which the score derivation relies (Definition 1, Theorems 6 and 8, Appendix B.1). Our extension makes the full joint law $Q(X_0,X_1,\eta)$ available for path design, including density-free, endpoint-dependent auxiliaries. For example, $\eta=X_0$ with non-degenerate $X_0$ introduces explicit endpoint dependence. The new flexibility lies in this joint-law class, together with its weak-transport and regression guarantees. **Stream-level Flow Matching with GPs.** AuxPath-FM expands auxiliary-law design beyond GP-CFM's conditional Gaussian construction, admitting discrete and singular laws without an auxiliary density. For example, fix $X_0=0$, $X_1=1$, and let $\eta=\pm1$ with equal probability. Then $$X_t=t+t(1-t)\eta,\qquad \dot X_t=1+(1-2t)\eta.$$ The endpoints are preserved, while $X_{1/2}$ equals $1/4$ or $3/4$ with equal probability. This two-atom law has no Lebesgue density, yet the path has an exact derivative for regression. A single GP conditioned on the same endpoints has Gaussian, possibly degenerate, marginals and cannot reproduce this two-atom law. This gives a concrete extension of the auxiliary distribution choices beyond that GP construction. For $\eta=F(y)$, the explicit semantic term $c'(t)F(y)$ also enables auxiliary CFG independently of velocity CFG. **Auxiliary CFG.** With $\eta=F(y)$, the semantic velocity term is $\dot c(t)F(y)$. For residual field $v_\theta$, we define $$v_{\mathrm{aux}}(x,t,y)=v_\theta(x,t,y)+\dot c(t) \left[F(\emptyset)+w_{\mathrm{aux}}\big(F(y)-F(\emptyset)\big)\right].$$ At $w_{\mathrm{aux}}=1$, this recovers the prescribed term. Varying the coefficient controls semantic drift with one backbone prediction per integration step. Velocity CFG can independently act on the residual predictions, allowing both guidance mechanisms to be combined. This connects the theoretical path family to conditional generation. **2. Uniform/Rademacher and CIFAR-10 settings.** Table 3 used two distinct distributions: continuous Uniform noise on $[-1,1]$ and equiprobable Rademacher values $-1$ and $+1$. During anonymized-repository assembly, the Rademacher implementation was copied into the Uniform branch. We apologize for this release error. We will restore the intended Uniform sampler, document both formulas, and add a support check so the released code matches the reported experiments. The CIFAR-10 discrepancy comes from different backbones. Our experiment reports confirm that Table 2 uses U-Net and gives learned-auxiliary FID 3.68; Table 6 uses DiT and gives FID 4.14 at auxiliary strength 1. **Planned revision.** Related Work and the path section will state the construction-level contribution explicitly. Appendices A–B will distinguish weak transport, fixed-law regression, and ODE assumptions. The experiment captions and reproducibility appendix will specify both distributions and backbones, with consistent values wherever a configuration is repeated.

3. Sampling-only guidance on a frozen CFM backbone.

Your suggestion inspired a sampling-only extension requiring no additional backbone training. We fix the CFM backbone and reuse learned FF from an AuxPath checkpoint, or use class means. All controls use 50 Euler steps and no velocity CFG:

BackboneAuxiliaryAux. scaleFID ↓
Frozen CFMNone—23.3294
Frozen CFMClass mean123.0011
Frozen CFMClass mean216.8829
Frozen CFMLearned F123.0431
Frozen CFMLearned F217.2810
Adapted AuxPath-FMLearned F121.6411
Adapted AuxPath-FMLearned F217.4678

Frozen CFM uses translated-input sampling. Adapted AuxPath uses its native residual field trained on the auxiliary path; both apply the same auxiliary CFG operation.

Why subtract the auxiliary displacement from the input. Let Fy=F(y)F_y=F(y) and c(0)=c(1)=0c(0)=c(1)=0. Writing the auxiliary-path state as a translated CFM state gives

xt=zt+c(t)Fy,zt=xt−c(t)Fy.x_t=z_t+c(t)F_y,\qquad z_t=x_t-c(t)F_y.

The chain rule then yields

x˙t=vCFM(xt−c(t)Fy,t,y)+c′(t)Fy.\dot x_t=v_{\mathrm{CFM}}(x_t-c(t)F_y,t,y)+c'(t)F_y.

Thus, subtracting c(t)Fyc(t)F_y returns the state to the coordinates in which the unchanged CFM velocity is evaluated. Auxiliary CFG replaces the explicit semantic term by

\dot x_t=v_{\mathrm{CFM}}(x_t-c(t)F_y,t,y)+c'(t)F_w,$$ where $F_u$ is the null-condition auxiliary. The input translation uses $F_y$, while the guidance coefficient acts on $F_w$. At $w=1$, this is an exact coordinate change with the same endpoint map under exact integration. The finite-step results, 23.0011/23.0431, are close to the CFM reference of 23.3294. At $w=2$, the residual control in the $z_t$ coordinates is $c'(t)(F_y-F_u)$ and can change the endpoint distribution. With the same learned $F$, adaptation improves scale-1 FID from 23.0431 to 21.6411. At scale 2, frozen CFM reaches 17.2810 versus 17.4678 for adapted AuxPath: sampling-time control already explains a substantial gain. Class means reach 16.8829 at scale 2; learned $F$ provides a shared map for new text prompts. **4. Matched inference budgets and repeated sampling.** At NFE=10, AuxPath at scale 2 improves FID, precision, and recall over CFG at scale 2. NFE counts conditional and unconditional backbone evaluations separately, including batched branches: | Method | CFG scale | Aux. scale | Sampling steps | NFE | FID ↓ | Precision ↑ | Recall ↑ | | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | | CFM | 1 | — | 10 | 10 | 38.7657 | 0.4188 | 0.6106 | | CFG | 1.5 | — | 5 | 10 | 36.7015 | 0.3853 | 0.4002 | | CFG | 2 | — | 5 | 10 | 28.3324 | 0.4445 | 0.2959 | | AuxPath-FM | 1 | 1 | 10 | 10 | 33.9182 | 0.4456 | 0.5823 | | AuxPath-FM | 1 | 2 | 10 | 10 | 27.3934 | 0.4706 | 0.5345 | **Repeated sampling.** We report the paper results and two additional evaluations, each covering CFM and AuxPath at $w=1$ and $w=2$. Both additional evaluations use fixed checkpoints, 50-step Euler integration, and 50,000 samples per setting. | Method | Original paper | Seed 2027 | Seed 20261114 | Mean ± sample SD | | --- | ---: | ---: | ---: | ---: | | CFM | 23.25 | 23.33 | 23.26 | 23.28 ± 0.04 | | AuxPath-FM, $w=1$ | 21.84 | 21.64 | 21.53 | 21.67 ± 0.16 | | AuxPath-FM, $w=2$ | 18.01 | 17.47 | 17.47 | 17.65 ± 0.31 | AuxPath achieves lower FID than CFM in all three evaluations, with further improvement at $w=2$. **Planned revision.** Section 5 and Appendix D will present the translated-input control, matched-NFE comparisons, and the repeated-sampling results. Appendix B will explain the coordinate transformation.
Reviewer_23KdOpenReview ↗

Response to Authors rebuttal

Thank you for the detailed response and the additional experiments.

The explanations regarding the Uniform/Rademacher implementation and the CIFAR-10 results address my concerns about those discrepancies. The repeated evaluations and matched-compute comparison are also useful additions.

My main concern now is the contribution of auxiliary-path training itself. The new sampling-only experiments are interesting, but they actually seem to suggest that most of the improvement comes from the guidance mechanism rather than the proposed training procedure. In particular, frozen CFM with class means achieves an FID of 16.88, compared to 17.47 for adapted AuxPath-FM. I think this deserves more discussion, as it raises the question of whether auxiliary-path training is necessary in the first place.

I also remain somewhat unconvinced by the novelty of the theoretical framework. Section 3.1 of Wei and Ma (2025) already considers general distributions over paths, not just the Gaussian-process construction introduced later. I would appreciate a more precise explanation of what is new relative to that formulation.

Finally, while the 10-NFE comparison is helpful, I would still like to see the 25-step CFG versus 50-step AuxPath comparison, with both methods properly tuned. This would be more directly relevant to the main experimental results.

Since the PDF can be updated during rebuttal, I would encourage the authors to incorporate these results and clarifications into the paper, particularly the sampling-only experiments, which seem important for understanding where the gains actually come from.

For now, I am keeping my rating unchanged.

Reviewer_oGt5OpenReview ↗
Rating
2(−1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose AuxPath-FM, a new framework for learning new transport paths between source and target distributions via flow matching. AuxPath-FM does this by constructing trajectories that are not only dependent on source and target, but also on an additional component sampled from a specified distribution, whilst ensuring that the resultant trajectory satisfies the flow matching endpoint constraints. The authors show how to adapt the flow matching objective, training and generation formulations to their framework, and illustrate the generality of their framework by constructing auxiliary paths with different distributions (e.g., Gaussian, Uniform) and learned-label-guided formulations. They validate their approach on MNIST, CIFAR-10 and ImageNet-1K 256x256, finding that one-pass (non-CFG) generation substantially improves with AuxPath-FM, though not under two pass (CFG).

Critical strengths
  • The authors introduce a framework with general auxiliary laws for additive paths, supporting several different types of additive paths including discrete or continuous distributions, or from learned functions. They further theoretically support their methodology, and validate it on an established benchmark (ImageNet) and with a used architecture (DiT-B), outperforming standard flow matching in non-CFG generation settings. The authors further analyze label-guided AuxPath-FM methods, finding it possible to further increase the FID gap to almost 5 FID50K compared to conditional flow matching.
Critical weaknesses
  • CFG results are quite poor on ImageNet (Table 4). Is there a better adaptation for the method with CFG? The best shown CFG result is over 13 FID50K lower than the best single-pass AuxPath-FM result shown.
  • To my understanding, only results at NFE=50 are shown for ImageNet. How does AuxPath-FM compare to one-pass CFM at varying NFE? Do the two methods converge at higher denoising steps? Similarly, does AuxPath-FM achieve better generations with fewer denoising steps (e.g., 20,30)? A figure tracking FID with denoising steps would be very helpful in understanding the path stability of the method.
  • It is mentioned in Section 5.3, that AuxPath-FM is employed on a partially trained checkpoint originally trained with flow matching. Is this necessary? How does AuxPath-FM fare against CFM when trained from scratch (random init)? I wonder whether AuxPath-FM is a less stable objective early on.
  • Small, but how do larger architectures fair on ImageNet? E.g., DiT-XL/2.
Questions for authors
  • I am unconvinced that a learned label-guided auxiliary is necessary. The method achieves very similar results to the Uniform approach on ImageNet and CIFAR-10. How does AuxPath-FM act when Fϕ(Y)F_\phi(Y) is exactly the class mean in latent space? Instead of a learned function? Identical class means can indeed co-exists as the authors state in lines 251-252, though it is not clear whether this is an empirical / performance concern. Non-learned conditiona-auxiliaries also would reduce the training complexity of the approach.
  • Very small, just a flag: II is overloaded. In line 138 it is a map, while immediately afterwards in line 141 it is the Identity matrix.
4 comments

Thank you for your time and suggestions on guidance, sampling efficiency, initialization, and auxiliary representations. We have added the requested comparisons.

1. Joint auxiliary guidance and CFG.

Jointly tuning auxiliary guidance and velocity CFG improves generation quality: the best setting in our ImageNet-256 sweep reaches FID 4.1076. Both coefficients are adjusted on the same trained model. Auxiliary guidance controls the explicit semantic drift, while velocity CFG combines conditional and unconditional backbone predictions. Each setting below uses 50,000 generated images and the original FID protocol.

CFG scaleAux. scaleSampling stepsNFEFID ↓
10.8505023.3339
10.9505022.4540
11505021.6411
12505017.4678
1.51501004.9719
1.51.5501004.8539
1.52501005.2349
20.8501004.1076
20.9501004.2334
21501004.3679
22501005.6816

At CFG=2, auxiliary scales 0.8, 0.9, and 1 give FIDs 4.1076, 4.2334, and 4.3679. At CFG=1.5, scale 1.5 gives 4.8539, while scale 2 gives 5.2349. The best tested auxiliary coefficient therefore depends on the velocity-guidance coefficient, and values below 1 are useful in the joint sampler.

Sampling steps denote Euler integration steps. NFE (number of function evaluations) counts backbone evaluations per generated image, with conditional and unconditional branches counted separately even when batched together. Thus, the 50-step configurations cost NFE=50 at CFG=1 and NFE=100 with two-branch CFG. The table makes this budget difference explicit.

2. Performance across sampling budgets.

AuxPath-FM improves FID over the matched CFM model at all four tested budgets. We evaluated the same checkpoints at 10, 20, 30, and 50 Euler steps with velocity CFG disabled, so each step uses one backbone evaluation:

Sampling stepsNFECFM: FID ↓AuxPath w=1: FID ↓AuxPath w=2: FID ↓
101038.765733.918227.3934
202027.728324.971820.0395
303024.762922.955218.4473
505023.329421.641117.4678

At auxiliary scale 2, the 20-step result, FID 20.0395, is better than the 50-step CFM result of 23.3294; 30 steps further improves FID to 18.4473. Scale 1 also improves over CFM at every tested budget. The benefit is therefore already present with the prescribed auxiliary term, before increasing its guidance coefficient.

All three curves improve as the integration budget increases. The gaps narrow but remain visible at NFE=50, where CFM and AuxPath at scales 1/2 give 23.3294, 21.6411, and 17.4678. These measurements characterize the observed convergence trend over 10–50 steps; its continuation at larger budgets remains to be measured.

Planned revision. Section 5 and Appendix D will include the joint-guidance sweep and FID–NFE curves, specify the Euler solver, and list sampling steps and NFE separately. The accompanying discussion will explain coefficient interaction and the quality–computation trade-off.

3. Random initialization and DiT-XL/2.

AuxPath-FM trains from random initialization and improves over CFM in our paired DiT-B/2 experiment. We originally initialized from an existing FM checkpoint to reduce training cost. We have now trained CFM and AuxPath-FM from scratch with DiT-B/2, and additionally trained AuxPath-FM with DiT-XL/2. The following results use 50 Euler steps, no velocity CFG, and 50,000 generated images:

BackboneMethodAux. scaleFID ↓IS ↑Top-1 (%) ↑
DiT-B/2CFM—85.655613.74076.17
DiT-B/2AuxPath-FM183.628113.94006.38
DiT-B/2AuxPath-FM273.744815.86548.91
DiT-XL/2AuxPath-FM159.480719.651914.23
DiT-XL/2AuxPath-FM251.314522.648217.81

On B/2, AuxPath improves FID from 85.6556 to 83.6281 at auxiliary scale 1 and to 73.7448 at scale 2. With XL/2, AuxPath reaches 59.4807 and 51.3145, respectively, with higher IS and classification accuracy. The B/2 rows provide the paired CFM comparison; the XL/2 rows demonstrate operation with a larger backbone.

Under the same 50-step evaluation, the existing-checkpoint B/2 experiments give FID 23.3294 for CFM and 21.6411/17.4678 for AuxPath at scales 1/2. Random initialization therefore works, while initialization from existing weights achieves better quality at the evaluated checkpoints. The latter benefits from its earlier training history, which we will document alongside the added training schedules. These measurements assess final-checkpoint quality; early-stage stability requires training-trajectory comparisons.

4. Fixed class means and learned auxiliaries.

A fixed empirical class mean is a valid deterministic auxiliary and avoids training an auxiliary network. We evaluated fixed means and a pretrained FF on the frozen CFM backbone using translated-input sampling at scales 1/2. Every row uses 50 steps without velocity CFG:

BackboneAuxiliaryAux. scaleFID ↓
Frozen CFMNone—23.3294
Frozen CFMClass mean123.0011
Frozen CFMClass mean216.8829
Frozen CFMLearned F123.0431
Frozen CFMLearned F217.2810
Adapted AuxPath-FMLearned F121.6411
Adapted AuxPath-FMLearned F217.4678

Why subtract the auxiliary displacement from the input. Let Fy=F(y)F_y=F(y) and c(0)=c(1)=0c(0)=c(1)=0. Translate the CFM state as

xt=zt+c(t)Fy,zt=xt−c(t)Fy.x_t=z_t+c(t)F_y,\qquad z_t=x_t-c(t)F_y.

The chain rule gives

x˙t=vCFM(xt−c(t)Fy,t,y)+c′(t)Fy.\dot x_t=v_{\mathrm{CFM}}(x_t-c(t)F_y,t,y)+c'(t)F_y.

Subtracting c(t)Fyc(t)F_y returns the input to the original CFM coordinates. Auxiliary CFG then replaces the explicit auxiliary term:

\dot x_t=v_{\mathrm{CFM}}(x_t-c(t)F_y,t,y)+c'(t)F_w,$$ where $F_u$ is the null-condition auxiliary. The input shift uses $F_y$, not $F_w$. At $w=1$, exact integration preserves the original endpoint map; at $w>1$, the residual guidance can change the endpoint distribution. *Frozen rows share one CFM checkpoint. Learned $F$ is reused from trained AuxPath, which uses its native residual field trained on the auxiliary path.* Fixed class means with input translation achieve FID 16.8829, compared with 23.3294 for ordinary CFM and 17.4678 for adapted AuxPath at scale 2. These ImageNet results support the simpler fixed-mean choice and identify a substantial sampling-time contribution. When two classes share a mean, the residual velocity still receives their distinct labels and can represent class structure beyond that shared prototype. Learned $F$ serves a broader conditioning interface: a shared map from text embeddings can compute auxiliaries for new prompts on demand. This makes the representation scalable to text-to-image sampling, where storing an empirical mean for every possible description is impractical. Fixed means and learned mappings are two supported parameterizations with different practical uses. **5. Notation.** We will use $\mathcal I_t$ for the interpolation map and $I_d$ for the identity matrix throughout the text and proofs. **Planned revision.** Section 5 will include the initialization and backbone comparisons, with training histories in the experimental appendix. The semantic-auxiliary section will present fixed means alongside learned $F$ and explain their respective roles.
Reviewer_oGt5OpenReview ↗

Response to authors

I thank the authors for their detailed response and efforts.

The CFG results are quite good, as is the experiment varying NFE.

However,

  • in response to (3), the authors do not include comparisons against DiT-XL/2 with CFM, making it impossible to understand how it compares to CFM at larger scales. Additionally, although AuxPath-FM improves upon CFM with DiT-B/2 in early training from scratch (83.63 vs 85.66 FID), both models are far from converged. It is unclear how this trend continues through training (e.g., whether AuxPath-FM continues to improve, converges to CFM, or is ultimately beaten by CFM).
  • Perhaps more importantly, fixed class means on a frozen CFM backbone match learned F at scale 1 on the same backbone, and outperform it (and the adapted AuxPath-FM variant) at scale = 2. Reading the authors' discussion with reviewer 23Kd, auxiliary-path training does appear to yield some benefit over CFM training with sampling only guidance, but it is relatively minor (0.91 FID at scale = 1, 0.22 at scale = 1.5 and 0.08 at scale = 3) Thus, it does not appear that learned F and the auxiliary-path training play a critical role in performance.

As such, I lower my rating to 2.

Dear Reviewer oGt5,

Thank you for the time you have devoted to reviewing our paper and the additional experiments. We respect your assessment and appreciate your positive comments on the CFG and NFE evaluations. We would like to clarify how these findings relate to the broader contribution of the paper.

The current XL/2 results evaluate AuxPath-FM without a matched CFM-XL/2 baseline. The B/2 results from random initialization likewise establish performance at the tested training budget, but do not resolve the longer-term convergence trend. With the discussion period now nearing its end, there is insufficient time to complete the matched XL/2 training and extended convergence experiments needed to answer these questions.

Auxiliary CFG is one application of our framework. The central contribution is an endpoint-preserving path construction that accommodates general auxiliary laws, together with a weak-transport guarantee that does not require probability densities under the stated moment and coefficient assumptions. Table 3 tests Gaussian, Uniform, Laplace, Rademacher, and semantic auxiliaries. In particular, the Rademacher experiment demonstrates the use of a discrete auxiliary law within the same construction. These experiments establish concrete applications of the distributional flexibility supported by the theory; they are not intended to establish that every auxiliary law improves generation or that auxiliary guidance outperforms standard CFG in every setting.

The class-mean results establish a useful sampling-only alternative, while path training supports learning transport on semantically modified intermediate states. We agree that the additional training gains in the matched shared-F experiment are modest, especially at stronger guidance. Fixed class means are effective for class-conditional generation. A learned auxiliary map serves a further purpose in text-to-image generation: it computes an auxiliary representation for each prompt without storing a separate mean for every text. Backbone adaptation then learns residual transport on these text-conditioned paths.

Table 2 and Appendix F provide evidence for this semantic use of the framework. In Table 2, combining semantic auxiliary paths with network conditioning improves label agreement; on CIFAR-10, the learned auxiliary also improves FID, KID, and IS. Appendix F applies AuxPath-LoRA to SANA and evaluates 30 held-out compositional prompts. The adapted model improves atomic and all-correct prompt-following scores over Standard LoRA, and changing auxiliary strength on the same trained checkpoint provides further control. These results support the usefulness of semantic auxiliary paths beyond the frozen-backbone class-mean setting.

Our contribution therefore combines a density-free transport formulation, experiments with continuous and discrete auxiliary laws, and a semantic path-conditioning mechanism. We hope this work provides a basis for exploring auxiliary distributions that improve flow-matching learning and conditional generation. Thank you again for your careful evaluation and for helping us clarify both the contribution and the scope of the evidence.

Comments on the paper

1 comment

Dear Reviewers and Area Chair,

Thank you for your careful reviews and continued discussion. Your questions helped us distinguish the roles of path construction, training, and guidance, and strengthen the evidence for each.

1. General path distributions and stochastic interpolants (23Kd)

The revised proof makes explicit how our auxiliary-path family extends the distributional coverage of earlier results. Wei and Ma's general-path formulation and stochastic interpolants allow broad modeling choices, while their density-based arguments do not explicitly establish transport for arbitrary discrete or singular auxiliary laws. Our weak formulation covers these laws without requiring auxiliary or state-marginal densities.

2. What auxiliary-path training adds (23Kd, ca3T)

The matched study starts both branches from the same CFM checkpoint, gives them equal additional training budgets, and uses the same learned auxiliary at inference. Auxiliary-path training improves on ordinary-path training followed by sampling-only guidance, with the larger benefit at lower guidance strength.

At larger auxiliary displacement, joint backbone and auxiliary adaptation also gives lower FID than frozen CFM with class means. The shared-auxiliary comparison measures the training-path contribution; the larger-displacement comparison evaluates the complete adaptation scheme.

3. Guidance at matched inference budgets (oGt5, 23Kd, ca3T)

Step-count curves and matched-budget evaluations show where auxiliary guidance is useful: it improves FID in the tested low-budget comparison, while standard CFG achieves lower FID at the higher budget. The joint-guidance experiments requested by oGt5 show that auxiliary control can further improve generation on top of fixed velocity-field CFG.

4. Fixed and learned auxiliaries (oGt5, ca3T)

We compared class means with learned auxiliaries, derived translated-input sampling for frozen CFM, and used paired endpoint diagnostics to examine endpoint preservation at unit strength. A shared F(y)F(y) computes auxiliary representations for new text prompts without storing a separate mean for each prompt. The text-to-image appendix reports improved prompt following with AuxPath-LoRA under BLIP-VQA.

5. Initialization, model size, and notation (oGt5)

Matched random-initialization experiments show that AuxPath-FM can train from scratch; a larger-backbone evaluation extends the results to a larger model. Pretrained initialization reduces training cost. We also separated the notation for the interpolation map and identity matrix.

6. Implementation and evaluation consistency (23Kd)

We explained the Uniform/Rademacher copy error in the anonymous release and specified the distributions used in the experiments. The CIFAR discrepancy is explained by the different backbones. Repeated sampling evaluations report individual results and mean ± sample standard deviation.

Our main contributions

  • Extending transport theory to general auxiliary laws within an explicit path family. AuxPath-FM constructs endpoint-preserving paths that allow a general joint law of endpoints and auxiliaries. Under finite first moments and absolutely continuous coefficients satisfying the endpoint constraints, we establish the weak continuity equation without density assumptions. This extends the relevant density-based results to discrete, singular, and endpoint-dependent auxiliary distributions.
  • Testing non-Gaussian and discrete auxiliaries in flow matching. We evaluate continuous non-Gaussian, discrete Rademacher, and semantic auxiliaries within the same framework. These experiments demonstrate the use of auxiliary distributions beyond Gaussian noise and compare how the auxiliary law and temporal path affect learning and sampling.
  • Using semantic auxiliary paths for both learning and sampling control. The backbone learns residual transport on auxiliary-shifted states, and the explicit auxiliary velocity provides guidance distinct from velocity-field CFG. Matched-training experiments quantify the benefit of path adaptation, joint-guidance experiments test composition, and AuxPath-LoRA extends the learned construction to text-to-image generation.

Thank you again for helping us strengthen the paper and explain these contributions more clearly.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.