acceptodds
Under review as a conference paper at ICLR 2027

Score Addition Is Not Composition: Closed-Form Path Corrections for Training-Free Composition of Pretrained Flows

Abstract

Pretrained flow-matching models are becoming a dominant class of generative models, and there is a growing need to compose two of them without retraining, as a product, a geometric mean, or a negation. Current practice relies on heuristics, such as adding velocities or annealing along summed scores, or on particle-based correctors. The heuristics do not target the composition: adding velocities overshoots the product mean of two unit-variance Gaussians by a factor of , and summed-score annealing follows the wrong intermediate curve, potentially misallocating mass across modes even with longer equilibration. We start from an exact identity. For flows trained with independent couplings from a common source, the intermediate marginals of any tilted composition factorize into powered component marginals and a kernel-corrected overlap integral of the endpoint posteriors, and the gradient of the log-overlap term is the exact correction to summed-score annealing. Gaussian moment matching gives the term a closed form that can be evaluated from black-box flows using only velocities and velocity Jacobians, and a well-posedness certificate identifies where the correction applies. Under frozen covariances the correction is a precision-weighted force that pulls samples toward agreement between the two models' endpoint predictions. The resulting sampler, CompFM, corrects the annealing path per sample and needs no particle population. At matched score-call budgets on class-intersection tasks with independently trained audit classifiers, CompFM attains 100.0% and 96.6% audited target-class purity on MNIST and FashionMNIST, respectively, versus 79.8% and 79.3% for summed-score annealing. On CIFAR-10, it concentrates 75.3% of audited mass on vehicle groups, compared with 1.1% for summed scores at the matched score-call budget. Composing two conditional flows on colored MNIST gives 96–99% joint accuracy, against 78–87% for summed scores. Classifier-free guidance (CFG) is the equal-covariance special case of the fusion rule, which also yields a covariance-aware correction to CFG.

Primary area:
generative models
Keywords:
flow matching, generative model composition, training-free sampling, posterior fusion, path correction

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_2N8aOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CompFM, a training-free method for composing pretrained flow models by introducing a closed-form score correction based on Gaussian approximations of endpoint posteriors. Experiments on synthetic and image datasets demonstrate improved compositional generation compared to score addition and velocity combination baselines.

Critical strengths

The paper provides a theoretically grounded approach to sampling from a specified compositional distribution, supported by theoretical analysis and experimental validation. The proposed method also achieves promising results without additional training.

Critical weaknesses
  1. Limited justification of the target distribution. It remains unclear whether accurately sampling from (p_A^\alpha p_B^\beta) necessarily improves generation quality, particularly for CFG and high-dimensional image generation. The product-based distribution may not always be the optimal target for semantic composition or perceptual quality.
  2. Computational overhead and scalability. The method requires additional velocity-Jacobian-related computations, introducing substantial inference overhead. Its efficiency and scalability to higher-dimensional, high-resolution image generation remain insufficiently demonstrated.
Questions for authors
  1. Could the authors further evaluate CompFM on more challenging, higher-dimensional generation tasks, such as ImageNet-64 in pixel space or ImageNet-256 in latent space? Given the reliance on covariance approximations and additional Jacobian-related computations, such experiments would help assess the scalability, computational efficiency, and practical effectiveness of the proposed method.
  2. Practical benefits for CFG: The covariance-aware CFG formulation is theoretically interesting, but its evaluation is mainly limited to Gaussian settings. Could the authors clarify whether more accurate distribution composition translates into better generation quality or conditional alignment compared with standard CFG on realistic image generation tasks?

I would be willing to reconsider my rating if the authors can provide convincing evidence or clarification addressing these concerns, particularly regarding the practical benefits of the proposed composition target and the scalability of the method to more challenging generation tasks.

5 comments

Thank you for your thoughtful review and for stating clearly what evidence would change your assessment. We ran all of the higher-dimensional evaluations you suggested, including both pixel-space ImageNet-64 and latent-space ImageNet-256. Part 1 addresses the practical benefits of the composition target and covariance-aware CFG (Weakness 1 & Question 2). Part 2 covers ImageNet-64 and ImageNet-256 scalability, computational efficiency, and clarity revisions (Weakness 2 & Question 1).

1. Practical value of the target: when the product is the desired target, accurate path sampling improves image quality and reconstruction (Weakness 1) Selecting the composition operator (product, geometric mean, or negation in §3, or KK-factor Bayesian conjunction in Rem. 3) is an upstream modeling choice, and the raw product is not intended as a universal semantic target. Once a target is chosen, however, summed scores and CompFM share that exact target distribution at t=1t=1. Summed scores follow biased intermediate marginals that omit ∇log⁡Rt\nabla\log R_t (Thm. 2), while CompFM estimates ∇log⁡Rt\nabla\log R_t in closed form to correct the path without training. Comparing CompFM with summed scores under the identical sampler therefore isolates what accurate sampling buys.

Benefit A: Lower KID than summed scores in class intersection. On AFHQ-64 and pixel-space ImageNet-64 (ADM, Part 2), each flow models a subset of classes, so their product targets the shared classes. Using the identical 100-step predictor-corrector sampler, our path correction lowers KID to real target-class images (×10−3\times10^{-3}, lower is better, 5 seeds) across all six targets, each by more than two standard errors:

DatasetShared targetSummed scoresCompFM
AFHQ-64Dog42.442.437.3\mathbf{37.3}
Wildlife16.616.613.4\mathbf{13.4}
Cat19.919.912.8\mathbf{12.8}
ImageNet-64 (ADM)Church85.185.174.3\mathbf{74.3}
Golf ball107.7107.799.8\mathbf{99.8}
Three-class set31.531.527.0\mathbf{27.0}

Benefit B: Bayesian restoration where Bayes' rule fixes the weighted product. When independent sensor measurements y1y_1 and y2y_2 observe an image xx without paired training data, Bayes' rule fixes the posterior p(x∣y1,y2)∝p(x∣y1)p(x∣y2)/p(x)p(x|y_1,y_2)\propto p(x|y_1) p(x|y_2)/p(x), our (1,1,−1)(1,1,-1) member. At three scales, we separately trained a 50%50\% pixel-inpainting flow, a noisy block super-resolution flow, and an unconditional prior, then composed them. Reconstruction PSNR on 256 test images (dB, 3–5 seeds):

Dataset & scaleSummed scoresVelocity avg.CompFMJointly trained flow
CIFAR-10, 32×3232\times3229.329.328.628.630.9\mathbf{30.9}31.731.7
AFHQ, 64×6464\times6430.730.730.330.332.8\mathbf{32.8}32.832.8
ImageNet-256, latent (SiT-XL/2, 10 classes)20.820.820.720.721.3\mathbf{21.3}21.221.2

Without ever seeing paired observations during training, CompFM beats both baselines at every scale and matches the flow trained on both observations at 64×6464\times64 and ImageNet-256.

2. Practical benefits for CFG: on ImageNet-256, covariance-aware CFG matches standard CFG at a smaller guidance scale, as our fusion rule predicts (Question 2) CFG is a case where users do not want faithful samples from pc1+γp∅−γp_c^{1+\gamma}p_\varnothing^{-\gamma}. Instead, CFG sharpens samples toward high-confidence class modes, as shown by Bradley & Nakkiran (2024) and Chidambaram et al. (2024). Our fusion rule (Eq. 3, Cor. 1) predicts what covariance-aware fusion does here: along class-informative directions where the unconditional posterior is broader than the conditional one, precision weighting pulls less strongly than standard CFG.

We tested this on 1,000-class ImageNet-256 with pretrained SiT-XL/2 (5,000 images per point). Full-latent scalar variances of the two branches are nearly equal (ratio 0.960.96–1.001.00), so scalar fusion is almost identical to CFG. Testing the variant that differs most from CFG (a separate covariance on the coarsest frequency band) gives the following generation quality (FID) and conditional alignment (ResNet-50 top-1 accuracy):

CFG scale s=1+γs=1+\gamma2.02.03.03.04.04.0
Standard CFG: FID / top-1 (%)13.813.8 / 94.294.219.319.3 / 96.996.922.222.2 / 97.597.5
Covariance-aware CFG: FID / top-1 (%)13.113.1 / 93.493.418.118.1 / 96.596.520.420.4 / 97.297.2

At each scale ss, covariance-aware CFG lowers FID and raises recall while slightly lowering top-1 accuracy. This traces the same trade-off as standard CFG at a smaller scale (within noise at equal FID). We therefore do not claim an improvement over CFG, and the abstract now calls this a covariance-aware form of CFG rather than a correction.

We then performed the pixel-space ImageNet-64 and latent-space ImageNet-256 experiments, analyzed computational efficiency, and summarized our clarity revisions (Weakness 2 & Question 1).

1. Scalability: CompFM raises target purity across both pixel-space ImageNet-64 and latent-space ImageNet-256 (Question 1) We ran both benchmarks you proposed, keeping our scalar covariance estimator completely unchanged and pairing it with a 100-step predictor-corrector sampler.

Composition combines separately trained models with overlapping support, whereas public checkpoints are single conditional networks. We therefore fine-tuned public ADM checkpoints (296M parameters) for pixel-space ImageNet-64 (12,28812{,}288 dimensions) and public SiT-XL/2 checkpoints (675M parameters) for latent-space ImageNet-256 into three unconditional flows each on a 10-class ImageNet subset. Each flow covers 5 classes, and each pair shares either one class (church or golf ball) or three classes (tench, English springer, and cassette player). Target purity is the percentage of generated images assigned to the shared target by a 10-class ResNet-50 classifier (5 seeds, 256 samples per seed):

Benchmark & modelShared targetVelocity avg.Summed scoresCompFM
ImageNet-64, pixel (ADM)Church31.431.432.732.750.1\mathbf{50.1}
Golf ball22.022.030.330.337.5\mathbf{37.5}
Three-class set58.158.161.661.674.9\mathbf{74.9}
ImageNet-256, latent (SiT-XL/2)Church22.522.530.230.242.0\mathbf{42.0}
Golf ball19.219.225.025.041.0\mathbf{41.0}
Three-class set67.367.368.768.771.971.9

CompFM raises target purity over both baselines on all six benchmarks, exceeding two standard deviations on five of the six. Together with our ImageNet-256 Bayesian restoration result in Part 1 (21.321.3 dB for CompFM vs. 20.820.8 dB for summed scores and 21.221.2 dB for the flow trained on both observations, on the same 10 classes), this confirms both the scalability and practical effectiveness of our scalar approximation on class intersection (ImageNet-64 and ImageNet-256) and restoration (ImageNet-256).

2. Computational efficiency: no full Jacobians, and a flat runtime ratio from 9M to 675M parameters (Weakness 2) Our theoretical formulas write posterior covariances using velocity Jacobians, so it is natural to ask about their cost. In practice, CompFM never forms a Jacobian matrix (§4 now states this upfront). Each model's scalar variance uses a single Hutchinson Jacobian-vector product (JVP), and the score correction uses one scalar backward pass.

Because one JVP and one backward pass each cost a fixed small multiple of a forward pass, the runtime ratio does not grow with model size. Under our 100-step predictor-corrector sampler (re-estimating covariances at every step), CompFM takes about 8×8\times the time of summed scores on a 9M-parameter UNet (AFHQ-64), 8.7×8.7\times on the 296M-parameter ADM (ImageNet-64), and 7.2×7.2\times on the 675M-parameter SiT-XL/2 (ImageNet-256) (and 2.28×2.28\times end-to-end under our CIFAR-10 annealed sampler). Refreshing the covariance every 25 steps instead of every step cuts the AFHQ-64 runtime ratio to 1.7×1.7\times (3434 s vs. 2020 s on an idle GPU) without losing purity. The main cost is peak GPU memory (44–9×9\times summed scores at ImageNet scales: 42.942.9 vs. 4.74.7 GB on ADM, and 56.056.0 vs. 13.913.9 GB on SiT-XL/2) from saving activations for the backward pass, which can be lowered with smaller batches.

3. Manuscript clarity improvements (Presentation) To make the paper easier to read, we added a scope paragraph in §1 separating target choice from path correction, a step-by-step compute summary in §4 clarifying JVP estimation, and a notation table alongside explicit KK-factor fusion formulas in App. A.

Dear reviewer, as the discussion deadline approaches, we wanted to share quick pointers to where your questions are addressed and check if anything remains unclear:

  • Practical applications where composition beats single conditionals (§1, §6.3, §6.5, Tables 3–4): Without paired training data, dual-sensor Bayesian restoration (32232^2, 64264^2, and ImageNet-256 latents) beats single-sensor flows and both baselines at every scale—matching jointly trained flows at 64264^2 (32.8 dB32.8\text{ dB}) and 2562256^2 (21.3 dB21.3\text{ dB})—and 30-pair attribute composition averages 90.2%90.2\% vs. 30.8%30.8\% for a single flow.
  • Inference cost breakdown (§4, §6.4, App. B.1 Table 11): CompFM never forms full Jacobians; at 2 steps/slice on an H100, JVP divergence probes take 4848–50%50\% of time, the log⁡R^t\log\hat{R}_t backward pass ∼30%\sim30\%, and forward passes 20%20\% (2.28×2.28\times CIFAR end-to-end).
  • Scaling with model size (§6.4, Table 11): Because JVPs and backward passes cost a fixed multiple of a forward pass, the 100-step PC runtime ratio stays flat from 9M to 675M parameters (∼8×\sim8\times UNet, 8.7×8.7\times ADM, 7.2×7.2\times SiT-XL/2), and 25-step covariance refresh lowers AFHQ-64 to 1.7×1.7\times.

Please let us know if any additional details would be helpful—thank you again for your review!

Reviewer_tAuYOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies training-free composition of pretrained flow-matching models, aiming to sample from a target distribution defined by the product of independently learned distributions. The authors derive an exact correction term for the intermediate score and propose a practical approximation based on Gaussian posterior fusion. The resulting method, CompFM, incorporates this correction into the sampling process without retraining the original models. Experiments on synthetic distributions and image datasets demonstrate improvements over naive score and velocity composition.

Critical strengths
  1. The paper rigorously explains why direct velocity or score composition fails to recover the target distribution. The derivation of the correction term is mathematically well motivated and provides useful insights into compositional generation.

  2. The proposed method uses pretrained flow-matching models and requires no additional training. The Gaussian posterior approximation provides a practical way to estimate the correction term from the models' predicted velocities.

  3. The experiments demonstrate substantial improvements over naive composition methods on several synthetic and image-generation benchmarks, supporting the effectiveness of the proposed correction.

Critical weaknesses
  1. The experiments primarily use synthetic distributions and relatively simple image datasets. Most experiments focus on combining distributions associated with different classes or attributes. It remains unclear how the proposed method applies to more practical generative modeling problems, where composing multiple pretrained models would clearly outperform existing conditional generation approaches.

  2. The proposed method requires additional computations, including divergence estimation and correction gradients. The paper reports approximately 4.7–4.8 wall-clock overhead compared with summed-score annealing.

Questions for authors
  1. Could the authors provide a more concrete discussion and experimental demonstration of practical applications where composing independently pretrained flow models is particularly beneficial, e.g., image editing?

  2. Could the authors provide a more detailed breakdown of the additional inference cost introduced by the correction term? In particular, how much time is spent on divergence estimation and correction-gradient computation, and how does the overhead scale with model size?

3 comments

Thank you for your positive assessment of our theory and precise questions on applications and inference cost.

1. When composing pretrained flows beats conditional generation: settings without paired data (Weakness 1 & Question 1) When paired multi-condition data is plentiful, training one joint conditional model is ideal. Composing independent flows outperforms single-condition models when a joint conditional model cannot be trained. This arises in three cases: (1) separate sensors have plenty of individual data but few paired measurements, (2) separate datasets each label only one attribute, and (3) new constraints are combined after training without retraining for every pair.

We test the first case on dual-sensor Bayesian restoration across three scales. We trained three separate models: a 50%50\% pixel-inpainting flow p(x∣y1)p(x|y_1), a noisy block super-resolution flow p(x∣y2)p(x|y_2), and an unconditional prior p(x)p(x). None sees paired (y1,y2)(y_1,y_2) data during training. For conditionally independent sensors, Bayes' rule fixes p(x∣y1,y2)∝p(x∣y1)p(x∣y2)/p(x)p(x|y_1,y_2)\propto p(x|y_1) p(x|y_2)/p(x), matching our (1,1,−1)(1,1,-1) fusion (Rem. 3). For evaluation only, we also trained a reference flow directly on paired observations. PSNR on 256 test images (dB, 3–5 seeds):

Dataset & scaleSummed scoresVelocity avg.CompFMJointly trained flow
CIFAR-10, 32×3232\times3229.329.328.628.630.9\mathbf{30.9}31.731.7
AFHQ, 64×6464\times6430.730.730.330.332.8\mathbf{32.8}32.832.8
ImageNet-256, latent (SiT-XL/2, 10 classes)20.820.820.720.721.3\mathbf{21.3}21.221.2

The stronger single-sensor flow gets only 28.228.2, 30.530.5, and 20.220.2 dB at the three scales. Without paired training data, CompFM beats single-condition models and both composition baselines at all scales, matching the jointly trained flow at 64×6464\times64 and ImageNet-256.

In the second case (separately labeled attributes), across all 30 digit-color pairs on Colored MNIST where neither flow sees the other label, the digit flow alone averages 30.8%30.8\% joint accuracy, summed scores get 69.4%69.4\%, and CompFM gets 90.2%90.2\%. CompFM wins on 27 pairs and recovers 96%96\% of a jointly supervised reference (94.0%94.0\%).

2. Cost breakdown: divergence takes ∼50%\sim50\% of corrected time and backprop takes ∼30%\sim30\% (Weakness 2 & Question 2a) At two Langevin steps per slice, divergence estimation takes 4848–50%50\% of the corrected time, the correction backward pass takes about 30%30\%, and the shared forward passes take 20%20\%. The 4.74.7–4.8×4.8\times figure in the paper is the full annealing wall-clock ratio at this setting.

CompFM never forms a full Jacobian. Per step, summed scores run two forward passes. CompFM runs the same two forward passes with the graph retained and backpropagates the scalar log⁡R^t\log\hat R_t once through both networks. Once per slice, it adds one Hutchinson Jacobian-vector product (JVP) per model for the divergence. Timings on one H100 GPU at batch 512:

Component (ms)32ch (28228^2)64ch (32232^2)128ch (32232^2)
Plain step (two forward passes)36.636.672.372.3145.3145.3
Corrected step (forward with graph + backward)94.894.8155.7155.7316.3316.3
Divergence per slice (two JVPs)174.4174.4322.9322.9644.6644.6
Per-slice ratio (2 / 5 / 10 steps)4.974.97 / 3.543.54 / 3.073.074.394.39 / 3.053.05 / 2.602.604.394.39 / 3.063.06 / 2.622.62
Peak memory, plain / corrected (GB)2.92.9 / 18.718.71.81.8 / 28.428.43.53.5 / 56.856.8

Using five steps per slice shares the JVP across more steps and drops the ratio to 3.13.1–3.5×3.5\times.

3. Scaling with model size: the runtime ratio stays flat from 9M to 675M parameters (Question 2b) One JVP and one backward pass each cost a fixed multiple of a forward pass, so the time ratio does not grow with model size. From 32 to 128 channels, the per-slice ratio stays at 4.44.4–5.0×5.0\times. At 64×6464\times64 and 256×256256\times256, our 100-step predictor-corrector sampler re-estimates covariances every step without slice amortization. Across a 75×75\times range in model size, the ratio remains flat: about 8×8\times on a 9M UNet (AFHQ-64), 8.7×8.7\times on ADM (296M, ImageNet-64), and 7.2×7.2\times on SiT-XL/2 (675M, ImageNet-256). Refreshing every 25 steps cuts the AFHQ-64 ratio to 1.7×1.7\times (3434 s vs. 2020 s on an idle GPU) without losing purity. Peak memory is 44–9×9\times the plain path at ImageNet scales and drops with smaller batches.

This compute pays off at equal or smaller budgets. On MNIST at equal wall-clock time, CompFM gets 100.0%100.0\% purity vs. 94.0%94.0\%. On CIFAR-10 class intersection, end-to-end cost is 2.28×2.28\times (both share the final ODE), and summed scores at 20×20\times the steps get only 7.0%7.0\% vehicle purity vs. 60.7%60.7\% for CompFM at 1×1\times. On restoration, 20 CompFM steps (2222 s, 31.031.0 dB) beat summed scores up to 2,000 steps (183183 s, 30.830.8 dB).

I thank the authors for their response. The response addresses some of my concerns. However, two concerns remain: (1) the experiments still use image datasets rather than real-world multi-sensor data, and (2) the computational cost is higher than that of the baselines. I will maintain my original ratings.

Thank you so much for your follow-up and kind response.

1. Real-world multi-sensor data: KITTI camera and LiDAR

We ran CompFM on KITTI depth completion (Uhrig et al., 2017), a real driving dataset in which each scene is recorded by an RGB camera (y1y_1) and a Velodyne LiDAR (y2y_2). The task is to predict metric depth xx (0 to 85 m).

We trained three separate flows so that no model sees both sensors: p(x∣y1)p(x|y_1), p(x∣y2)p(x|y_2), and an unconditional depth prior p(x)p(x). We composed them via Bayes' rule, p(x∣y1,y2)∝p(x∣y1)p(x∣y2)/p(x)p(x|y_1,y_2) \propto p(x|y_1) p(x|y_2) / p(x), using the same predictor-corrector sampler as before. As a reference for when paired training data is available, we also trained one flow directly on (y1,y2)(y_1,y_2). All flows are small UNets (about 9M parameters) at 56×15256 \times 152 resolution (a 4×4 \times downsampled 224×608224 \times 608 crop). Test error on 256 frames from 13 unseen drives (meters, lower is better, 3 seeds, seed s.d. at most 0.03 m):

MetricCamera onlyLiDAR onlyVelocity avg.Summed scoresCompFMJointly trained flow (paired data)
RMSE (m)5.114.194.113.953.733.49
MAE (m)2.651.942.091.941.791.66

Among all methods that use no paired data, CompFM has the lowest RMSE and MAE. In RMSE, it beats summed scores, velocity averaging, and LiDAR alone on all 13 drives, and it closes 65% of the gap between LiDAR alone and the jointly trained flow. We will add this experiment to the final manuscript if you find this helpful.

2. Computational cost

We agree. CompFM needs more computational cost than the baselines, and we will state this clearly in the Limitations section of the final manuscript.

Thank you again for your constructive feedback. It led us to make the study much stonger.

Reviewer_NtpqOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies training-free composition of pretrained flow-matching models. It derives a correction term that accounts for the mismatch between summed scores and the score of the noised target distribution. The proposed method, CompFM, approximates this term through Gaussian posterior fusion, using moments obtained from model velocities and their Jacobians. Experiments on synthetic distributions, MNIST, FashionMNIST, and CIFAR-10 compare it with velocity composition and summed-score annealing, showing improved mode allocation and target-class or group purity.

Critical strengths

The paper is clearly presented and takes an interesting approach to flow composition. It derives the missing correction to summed scores and gives a closed-form approximation through Gaussian posterior fusion. The detailed analysis helps explain both why naive composition fails and when the proposed fusion is well posed.

The synthetic experiments provide convincing evidence that the correction improves sampling from the target product distribution over naive score and velocity combinations. Using analytically tractable targets and oracle moments makes these experiments useful for isolating composition error from model error.

Critical weaknesses

My background is primarily in practical applications of diffusion models, such as generation and editing, rather than theoretical analysis.

  1. I find the problem of sampling from the product of two pretrained models’ distributions interesting, and I appreciate the theoretical analysis. However, the practical motivation remains unclear to me. For example, training one model on digits 8 and 9 and another on 3 and 8, then composing them to generate 8, is a useful controlled experiment, but when would this setting arise in practice? A concrete application where this composition is needed would substantially strengthen the paper’s significance.

  2. The experiments provide limited evidence of scalability. The image experiments only reach CIFAR-10, and even there the composition involves three classes. This makes it difficult to assess whether the approximations remain effective on more complex distributions or whether the additional computational cost is justified. My concern is not the absence of a particular benchmark, but whether the method extends beyond these restricted settings.

  3. The last two experiments in Table 2 lack a CFG baseline. One could apply digit CFG (e.g., 8 versus no condition) and color CFG (e.g., red versus no condition) to the respective models, then combine their scores. Although this changes the target distribution, it is a relevant baseline for the reported joint-condition accuracy. Stronger guidance may already improve this metric substantially, so it is unclear how much additional benefit the correction provides. Given the widespread use of CFG, this comparison is important for assessing the practical gains.

Questions for authors
  1. Can the authors identify a practical use case where composing two pretrained models is needed? This would help clarify the significance of the proposed method.
  2. Could the authors compare against summed scores with CFG applied to each conditional model on colored MNIST? Does the correction still improve joint-condition accuracy over this baseline?
2 comments

Thank you for your constructive review and practical perspective.

1. Practical use cases: composition is needed when paired training data does not exist (Weakness 1 & Question 1) Overlapping digit sets like {3,8}×{8,9}\{3,8\}\times\{8,9\} serve as controlled tests with known ground truth. In practice, composition is needed when independent models must be combined because no dataset pairs their conditions.

Use case A: Multi-sensor Bayesian restoration from separately trained models. We trained three separate models: a 50%50\% pixel-inpainting flow p(x∣y1)p(x|y_1), a noisy block super-resolution flow p(x∣y2)p(x|y_2), and an unconditional prior p(x)p(x). None sees paired (y1,y2)(y_1,y_2) data during training. For conditionally independent measurements given xx, Bayes' rule gives p(x∣y1,y2)∝p(x∣y1)p(x∣y2)/p(x)p(x|y_1,y_2)\propto p(x|y_1) p(x|y_2)/p(x), our (1,1,−1)(1,1,-1) weighted product (Rem. 3). PSNR on 256 test images (dB, 3–5 seeds):

Dataset & scaleSummed scoresVelocity avg.CompFMJointly trained flow
CIFAR-10, 32×3232\times3229.329.328.628.630.9\mathbf{30.9}31.731.7
AFHQ, 64×6464\times6430.730.730.330.332.8\mathbf{32.8}32.832.8
ImageNet-256, latent (SiT-XL/2, 10 classes)20.820.820.720.721.3\mathbf{21.3}21.221.2

CompFM beats both baselines at every resolution and matches the flow trained on both observations at 64×6464\times64 and 256×256256\times256. Even the stronger single-sensor flow gets only 28.228.2, 30.530.5, and 20.220.2 dB across the three scales, and on CIFAR-10 CompFM fits observed pixels as well as the inpainting flow (MSE 0.00030.0003 vs. 0.00170.0017 for summed scores).

Use case B: Attributes labeled in separate datasets. On Colored MNIST, the digit flow never sees color labels and the color flow never sees digit labels. Across all 30 pairs, CompFM averages 90.2%90.2\% joint accuracy vs. 69.4%69.4\% for summed scores and wins on 27 pairs. A jointly trained flow gets 94.0%94.0\%, so CompFM recovers 96%96\% of joint training without paired labels.

2. Scalability: CompFM extends to AFHQ-64, ImageNet-64, and ImageNet-256 (Weakness 2) The restoration flows above cover all 10 CIFAR-10 classes, AFHQ-64, and ImageNet-256 latents. Class intersection scales too. Using a 100-step predictor-corrector sampler with our unchanged estimator, CompFM beats summed scores and velocity averaging on all nine targets across AFHQ-64, pixel-space ImageNet-64 (ADM, 296M), and latent-space ImageNet-256 (SiT-XL/2, 675M, 5 of 10 classes per flow). For instance, church purity rises from 32.7%32.7\% to 50.1%50.1\% on ImageNet-64 and from 30.2%30.2\% to 42.0%42.0\% on ImageNet-256 (table in our reply to 2N8a).

Is the extra cost worth it? Yes, because giving summed scores more steps does not close the gap. In CIFAR-10 restoration, 20 CompFM steps (2222 s, 31.031.0 dB) beat summed scores at every step count up to 2,000 steps (183183 s, 30.830.8 dB). At equal wall-clock time on MNIST, CompFM gets 100.0%100.0\% purity vs. 94.0%94.0\%. Refreshing covariances every 25 steps on AFHQ-64 cuts the runtime ratio to 1.7×1.7\times (3434 s vs. 2020 s on an idle GPU) without losing purity (breakdown in our reply to tAuY).

3. Per-model CFG baseline: CFG reaches higher joint accuracy because it sharpens the target (Weakness 3 & Question 2) We ran this baseline, and the results confirm your intuition. Per-model CFG (γ≥1\gamma\ge1) reaches 9898–100%100\% joint accuracy, above CompFM. We retrained both flows with 10%10\% label dropout and reran every row on the same checkpoints (shifting γ=0\gamma=0 from Tables 2 and 10 in the revised version, e.g., 62.0%62.0\% vs. 87.1%87.1\% for summed scores on red eights, with the same ranking). Joint accuracy (%, 5 seeds, n=512n=512):

MethodRed eightsGreen threes
Jointly trained flow (reference)86.286.291.391.3
Summed scores (γ=0\gamma=0)62.062.063.963.9
Summed scores + per-model CFG (γ=1\gamma=1 / 22)98.298.2 / 100.0100.098.398.3 / 99.999.9
Composable Diffusion (γ′=1\gamma'=1 / 22)82.382.3 / 97.597.587.687.6 / 99.299.2
CompFM (no guidance)96.196.188.788.7

Even a flow trained on ground-truth joint labels gets 86.2%86.2\% and 91.3%91.3\%. CFG exceeds that reference because increasing γ\gamma tilts the target to p(x∣c)p(c∣x)γp(x|c) p(c|x)^\gamma, pushing samples toward high-confidence examples. CompFM keeps the target fixed and corrects the sampling path toward it with nothing to tune. Composable Diffusion at γ′=1\gamma'=1 targets the exact joint conditional and lands near the reference, as expected.

This gives a clear rule, which §6.3 now states. When both models have unconditional branches and the goal is maximum joint accuracy, use CFG. When the problem fixes the target (as in restoration) or models lack unconditional branches (as in our class-intersection tasks), use CompFM.

Thank the authors for providing the details rebuttal. It addresses all my concerns, therefore I increase my rating to Clear acceptance.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.