acceptodds
Under review as a conference paper at ICLR 2027

When is Compositional Generation Feasible? Distributional Estimation Error and Inference-time Approximation Error in Diffusion Models

Abstract

The task of compositional generation involves using a conditional generative model, trained only on a subset of the possible conditions, to produce samples from compositionally-defined target distributions such as a geometric combination of the source distributions. In this work, we explore what makes this challenging by separating out several sources of error and analyzing their interplay. To begin, we show that even when source distributions are estimated with arbitrarily small error, their composition may be arbitrarily far from the true composition. This leads us to derive sufficient conditions to prevent error blowup, and we propose a metric based on the resulting error-sensitivity bounds which can separate error-sensitive compositions from stable ones without access to ground truth distributions. Then, we carefully design experiments with diffusion models for both synthetic and realistic data to analyze the interaction between distributional estimation error and inference-time approximation error. In particular, while recent methods can reduce inference-time approximation error in principle, our results show that distributional estimation error has a more catastrophic effect on performance when the target distribution is out-of-distribution with respect to the sources.

Primary area:
generative models
Keywords:
compositional generation, diffusion models, error sensitivity

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_rKRLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies geometric compositional generation in diffusion models. The authors decompose the total generation error into two sources: inference-time approximation error and distributional estimation error. From this perspective, they analyze the sensitivity of geometric compositions to source-distribution estimation errors and show that even small estimation errors can be significantly amplified when the target distribution is poorly covered by the source distributions. They further demonstrate that improving inference-time approximation through correction methods does not necessarily improve compositional generation and may even worsen performance when distributional estimation errors are present.

Critical strengths

The paper provides an interesting theoretical analysis showing that even small errors in estimating source distributions can lead to large errors in their geometric composition. Another interesting empirical evidence that despite reducing inference-time approximation error in principle, can worsen compositional generation when source distributions are imperfectly estimated.

Critical weaknesses

Concerns with the paper

Interpretation of the two error sources: Equation (3) only provides an upper bound through the triangle inequality. The two error terms do not form an exact additive decomposition of the total error. Furthermore, the underlying deviations may counteract one another, meaning that reducing either error term does not necessarily reduce the total generation error. The authors should clarify the practical significance of separately analyzing these two errors and how their interaction affects compositional generation.

Consider three one-dimensional Gaussian distributions with identical variance:

P⋆w=N(0,1)P_\star^w=\mathcal N(0,1)

$ \hat P^w=\mathcal N(2,1) $

$ \tilde P=\mathcal N(0,1). $

Suppose we use the Wasserstein-2 distance. For Gaussians with identical variance, this is simply the distance between the means.

undefined
Questions for authors

Addressing all the weakness will answer my questions

7 comments

We thank the reviewer for their thorough feedback. Below, we address the weaknesses raised and what we believe may be some conceptual misunderstandings about the content of the paper.

Weaknesses

W1: Interpretation of the two error sources

We agree that the underlying deviations can cancel one another out. However, this worst-case upper bound is very standard in statistical analysis, see for instance [1, 2, 3, 4]. We reiterate that, without further assumptions, the triangle inequality in Equation (3) can be tight, and the bound in Theorem 1 is tight (distributions exist whose composition error matches the upper bound up to constant factors), though we agree that exactly characterizing what further assumptions are both reasonable and useful for analysis of the interactions is a promising (but very challenging) line of future work.

A large portion of our analysis is dedicated to how the two error sources interact in practice, and we characterize four separate regimes, the quadrants of Figure 2, Figure 3, and Table 1, where these interactions (or lack thereof) play out differently. We additionally provide several theoretical examples highlighting a form of 'bias-variance' tradeoff that can occur between naive composition and exact composition under estimation errors (Section 3.2, and Corollaries 1.1, 1.2 in Appendix G.4). Consequently, we feel this separation is already clearly justified. If the reviewer has specific concerns or questions about these points, we would be happy to provide further clarification.

W2: Limited scope of compositional generation

We acknowledge that the form of composition we study is not all-encompassing, and there are some notions of composition which do not fit under this umbrella. We focus on weighted composition because of its rigorous motivation through identifiability theory (Section 2) and causal representation learning (Appendix B), recent methodological advances like FKC that reduce inference-time approximation error, and because it covers a wide range of practical settings (Section 1), including applications in biology, classifier-free guidance, Bayes composition, and popular forms of score-level arithmetic, for instance.

Indeed, CoInD by Gaudi et al. (2025) explicitly uses the notion of geometric composition expressed at the score level (their Section 2, Eqs 1-3), which falls under our Definition 1. In fact, the CoInD training objective explicitly penalizes differences between the joint score and the score corresponding to the compositions of the marginal scores, though as discussed in our Section 3.2 the score-addition identity need not hold at intermediate timesteps. In fact, the CoInD objective can be viewed as a regularization encouraging agreement between joint conditioning and naive score composition, and thus it may exhibit a kind of bias-variance tradeoff as discussed for naive composition in our work; our analysis may be directly useful for understanding these contributions.

Okawa et al. (2023) define classes as attribute tuples, and define a concept graph describing relationships between classes, but do not provide a formal definition of the target distribution in terms of the source distributions. Such settings fall within our framework if the underlying data-generating process admits the causal representation and modular structure described in Appendix B, which yields our weighted composition formula. For instance, their (shape, size, color) experiment could be represented this way. We would be happy to provide further details if the reviewer would find it helpful, but for now we omit formal notation for the sake of brevity.

We will add these as cited examples to the paper and discuss the connection between weighted composition and logical score operations, as in CoInD, and the Okawa et al. concept graph framework.

W3: Insufficient explanation of distributional estimation error

Nearly all statistical learning techniques produce estimation errors. The fact that estimation errors arise is unavoidable and taken as a given. For further details on properties of estimation error in diffusion models, we recommend [5, 6], but this is not our focus.

We disagree that the paper "does not adequately explain why models can accurately generate individual conditions while failing at their composition". This is exactly the point of the paper: estimation error on individual conditions can be low, so that models can accurately generate individual conditions, but this does not imply that they will accurately generate from the composition. This is directly established in Proposition 1 and Theorem 1, and further discussed in Section 3.2 where inference-time approximation error adds another potential failure mode.

AuthorsOpenReview ↗

Response (part 2)

W4: Conditional independence and inference-time approximation

It is true that conditional independence need not be preserved during noising. Note that the Factorized Conditionals assumption is slightly stronger than conditional independence, and is preserved during noising (Lemma 2 in G.5). We can attribute increased failures in non-Factorized settings (compared to their Factorized counterpart) to inference-time approximation error precisely because of this. Of course, this can be muddied by introducing estimation error, which is also what our experiments are designed to analyze.

W5: Insufficient experimental separation of errors

We disagree. Our synthetic experiments explicitly include the analytic score functions, where there is no distributional estimation error, and the Factorized Conditional distributions are designed to incur no inference-time approximation error. This allows us to directly measure the impacts of each source of error and their interplay.

The two error terms are not directly measured, limiting the conclusions about their relative importance.

For our non-synthetic settings, the terms cannot be directly measured, hence our setup. We carefully design experiments so that they vary whether the target is ID (low estimation error) or OOD (high estimation error), and vary inference-time approximation error by varying whether they are approximately factorized (low approximation error) or non-factorized (high approximation error). The experimental results follow the same patterns as those in the Gaussian settings, as well as the intuition from Section 3.2, giving us high confidence that the experimental settings are effective at both disentangling the two errors and studying their combination (depending on quadrant).

W6: Limited practical contribution

We agree that the framework does not directly establish how to improve compositional generation. Our goal is complementary: we seek to provide a deeper understanding of the problem of compositional generation, identifying two highly relevant sources of error and analyzing their interplay. We believe scientific understanding of a challenging problem is by itself valuable, whether or not it immediately produces a solution.

However, it can also be demonstrated how a straightforward application of our analysis can immediately yield better results by selecting the inference procedure. As a simple instantiation of this, the rESS sensitivity metric can be combined with the factorization heuristic in Appendix E.2 to determine whether the target composition is likely factorized or not and if it is likely OOD or not. This can then be used to select between FKC (K=16) and naive composition, which here would yield up to +20.1%/+10.4% success rate (rooms/faces) or 16x cheaper inference over using only naive composition or using only FKC, depending on the setting.

Additionally, the sensitivity metric may be used by practitioners to assess when compositions may be unreliable, without access to ground-truth data from the composition. Lastly, as a straightforward proof-of-concept, we demonstrate with an additional experiment that the sensitivity metric can be used to decide how to allocate data to each source when such data is expensive to collect (see our response to Reviewer Kkz4, Q4 for further details).

So, while the goal of our work is not to propose methods to improve compositional generation, the analysis it provides can be used to do so, and we are hopeful that the increased scientific understanding of the problem will be used to build further progress. We will expand our current discussion of these implications in the Discussion section.

References

[1] Block et al. (2022), Generative Modeling with Denoising Auto-Encoders and Langevin Sampling

[2] Chen et al. (2023), Sampling is as Easy as Learning the Score

[3] Chen et al. (2023), Improved Analysis of Score-based Generative Modeling: User-Friendly Bounds under Minimal Smoothness Assumptions

[4] Benton et al. (2024), Nearly d-Linear Convergence Bounds for Diffusion Models via Stochastic Localization

[5] Chen et al. (2023), Score Approximation, Estimation and Distribution Recovery of Diffusion Models on Low-Dimensional Data

[6] Oko et al. (2023), Diffusion Models are Minimax Optimal Distribution Estimators

Reviewer_rKRLOpenReview ↗

W1 Interpretation of the two error sources

I understand that Equation (3) provides a mathematically valid decomposition and that the two terms correspond to distinct error mechanisms. However, I am still unclear about the extent to which this separation is informative for understanding the total compositional error.

The triangle inequality permits introducing any intermediate distribution, so its validity and worst-case tightness do not themselves justify the explanatory value of this particular separation.

More importantly, both terms in Equation (3) depend on the estimated source distributions. Consequently, the inference-time approximation error is not independent of distributional estimation error, and the two errors may reinforce or compensate for one another.

While the authors provide specific examples demonstrating such behavior, I do not see a general mathematical justification for analyzing these errors separately when drawing conclusions about overall compositional performance.

Could the authors clarify whether their decomposition is intended primarily as an operational distinction between two error mechanisms, or whether they claim that the separate error terms provide a meaningful characterization of the total error beyond a worst-case upper bound? If the latter, what theoretical result supports this interpretation?

Reviewer_Kkz4OpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper analyzes distributional estimation error and inference-time approximation error in diffusion-based compositional generation. It establishes sensitivity bounds for error amplification, proposes an rESS-based sensitivity estimator, and studies the interaction between these errors through controlled experiments.

Critical strengths

Theoretical characterization: The paper rigorously shows that small source estimation errors can lead to catastrophic compositional errors and provides a theoretically justified sensitivity bound.

Error interplay: The comparison between analytical and learned scores demonstrates that more accurate inference-time composition can worsen generation performance when source estimation errors dominate.

Critical weaknesses

Limited practical relevance: The experiments primarily use artificially constructed source distributions, where ID/OOD performance differences are largely expected from target coverage. It remains unclear whether the proposed analysis explains failures in realistic compositional generation tasks.

Unclear practical value of sensitivity estimation: The rESS estimator is sensitive to density-ratio estimation and timestep selection. Its robustness, advantages over existing methods, and usefulness for improving compositional generation are insufficiently demonstrated.

Questions for authors
  • What practical compositional generation problem requires the proposed OOD geometric composition? The paper assumes that the target distribution �0��0� can be identified from the source distributions through geometric composition, but this may not hold for interacting conditions. Could the authors clarify which realistic tasks satisfy this assumption and why the constructed ID/OOD image experiments are representative of those tasks?

  • The degradation under Feynman-Kac correction is demonstrated primarily using controlled conditional diffusion models. Does the same phenomenon occur in existing pretrained compositional diffusion models? I would suggest evaluating naive score composition and FKC on a pretrained text-to-image diffusion model using the same condition pairs and random seeds, while varying correction strength and source-model accuracy.

  • What additional information does the proposed rESS-based sensitivity metric provide over existing diffusion-based likelihood-ratio measures, such as CompLift[1]?

  • If compositional failure is primarily attributed to distributional estimation error, why is collecting additional source data not sufficient? More importantly, how can the proposed sensitivity characterization improve this process?

  • [Minor formatting concern]: The main title and subtitle appear to use different font sizes.

[1]Yu, Chenning, and Sicun Gao. “Improving Compositional Generation with Diffusion Models Using Lift Scores.” Proceedings of the 42nd International Conference on Machine Learning (ICML), 2025.

6 comments

W1: Limited practical relevance

See responses to questions below.

W2: Unclear practical value of sensitivity estimation

We stress that the sensitivity estimation is not intended to improve generations, and is better suited for understanding where compositions may be unreliable and require more data. However, as we discuss in further detail in our response to Reviewer F9pQ W1, the understanding provided by our analysis has several immediate practical consequences. In particular, to improve compositional generation, we could (as a simple instantiation) combine the sensitivity metric with the Factorization heuristic in Appendix E.2 to select between naive composition or FKC depending on the setting, which achieves up to +20.1%/+10.4% success rate (rooms/faces) or 16x cheaper inference over naive composition alone or FKC alone.

As another use case, we demonstrate with an additional experiment that the sensitivity metric can be used to decide how to allocate data to each source when such data is expensive to collect (see the response to Q4 below for details). We clarify connections between the rESS-based sensitivity metric and other diffusion-based likelihood ratio methods in Q3.

Lastly, improved performance under timestep selection is not unique to our method. In fact, this technique is inspired by Feynman-Kac Correctors, where a similar advantage to restricting to only an 'active region' is reported.

Questions

Q1: Practicality of geometric composition

What practical compositional generation problem requires the proposed OOD geometric composition?

Could the authors clarify which realistic tasks satisfy this assumption and why the constructed ID/OOD image experiments are representative of those tasks?

This form of geometric composition is common to many settings, and arises naturally from causal representation learning theory, as discussed in Appendix B. A motivating application is predicting the distribution of cellular responses to a combination of perturbations from control and single-perturbation data, including responses measured through microscopy images. This motivation is discussed in the Introduction and Section 2, Example 1. For example, Xu et al. (2024, Theorem 3.6) [1], cited in the Introduction, use the same density-ratio identity as in our definition of base-composition, which they derive under separability assumptions and use to study interactions in real cell images.

Note that the assumptions which lead to these geometric compositions allow causal dependencies between variables, they do not require independent features to be observed. As acknowledged in page 1's footnote, we do not expect the assumptions to hold for every perturbation pair. Our question is whether accurate compositional generation is achievable even when such identification is valid.

The ID/OOD image experiments test a difficulty relevant to this application, as combined outcomes may be well represented or poorly represented among the source data. By controlling this coverage and varying approximate factorization, they test how estimation and approximation errors interact in complex image data.

Beyond intervention prediction, weighted geometric compositions are also practical generation objectives in their own right. This includes the multi-objective molecular generation settings studied by Skreta et al. (2025) for Feynman-Kac Correctors [2], as well as classifier-free guidance and Bayes composition as discussed in Sections 2 and 3.2. Our analysis can also be viewed as providing context for other previously observed behaviors in image generation, such as the failure to combine certain concepts but not others [8], and which is related to the notion of projective composition discussed in [3].

We also note that, if the target distribution cannot be identified from the source distributions through geometric composition, then there are two possible scenarios. The first is that the interacting conditions are not identifiable from the sources, in which case recovering the true target distribution from the source distributions alone cannot be guaranteed in principle. In the second case, the composition is somehow identifiable, but not through a geometric composition. We expect that a similar kind of sensitivity analysis to the one presented in this paper could be extended to cases like these as well, but we do not claim to represent every possible formulation of composition. Our experiments are also broad enough to cover some notions of projective composition from Mechanisms of Projective Composition (Bradley et al., 2025) [3].

AuthorsOpenReview ↗

Response (part 2)

Q2: Extension to pretrained models

The degradation under Feynman-Kac correction is demonstrated primarily using controlled conditional diffusion models. Does the same phenomenon occur in existing pretrained compositional diffusion models?

Yes. As an additional experiment, we evaluate FKC on two CelebA latent diffusion models released by Gaudi et al., (2025) [4], one trained with their independence regularizer (CoInD) and the matched model trained without it. The task is to generate smiling men by composing the Male and Smiling conditions, when no smiling men were included in training/finetuning. FKC with 16 particles reduced the conformity score from 25.5% to 8.1% for the unregularized model, and from 32.7% to 28.2% for the CoInD model using the authors' strong smiling guidance setting of (3, 24). For their vanilla finetuned text-to-image model, performance is lowered a small amount from 9.4% to 7.2%, and was stagnant for their CoInD-finetuned text-to-image model. We tested across 1000 generation seeds. This aligns with what our analysis predicts for OOD, partially factorized compositions. Should the reviewer find it helpful, we will provide further experimental details, and we will add these results to the appendix.

We further conduct an experiment on Stable Diffusion XL text-to-image compositional generation, and find that the factorization heuristic described in Appendix E.2, calculated with the model's CLIP text encoder, correlates with composition performance. Correlation reaches ρ=−0.49\rho=-0.49 (p < 0.001), indicating naive composition success is correlated with higher prompt factorization. Within groups of similar conditions, e.g. pictures of an apple or banana paired with various other conditions, the heuristic also achieves correlation up to ρ=+0.64\rho =+0.64 (p = 0.002) with how much FKC improves the performance, aligning with predictions that FKC helps most for less factorized conditions. However, it does not predict FKC improvement globally. These trends persisted across different composition weight strengths. We are happy to provide further experimental details should the reviewer find it helpful.

However, without access to the training data, it is not possible for us to assess whether the compositions are ID or OOD, which severely limits the utility of studying large-scale pretrained models. Furthermore, for text-to-image generation from a single model, sampling from a composition is typically done with a prompt containing both conditions, rather than score addition from two prompts as done here. Most importantly, text-to-image generation represents a different setting than conditional compositional generation as used in e.g. perturbational cell modeling. Regardless of this difference, as discussed in Section 1, conditional compositional generation is still of wide interest and importance.

It is also important to stress that we do not intend to critique Feynman-Kac correction. Our sensitivity analysis extends to any weighted geometric composition, regardless of the presence of inference-time approximation error. We employ FKC because it is a powerful and theoretically principled method of correcting inference-time approximation error for diffusion models, but find that, in certain settings of interest, it too has limitations due to the interplay between estimation error and approximation error.

Q3: rESS comparison

What additional information does the proposed rESS-based sensitivity metric provide over existing diffusion-based likelihood-ratio measures, such as CompLift?

CompLift seeks to measure a different property (whether a sample aligns with a condition), and use this to improve generation, whereas rESS seeks to measure potential compositional error sensitivity. The metrics are complementary, not directly comparable, though there is a connection between the two. In particular, let Lj(x)=log⁡Pj(x)P0(x)L_j(x)=\log\frac{P^j(x)}{P^0(x)} denote the log-lift, as used in CompLift, when P0P^0 is unconditional and PjP^j is a conditional. For the base composition Pw∝P1P2/P0P^w \propto P^1P^2/P^0 we have that rESS1=(EP1[eL2(X)])2EP1[e2L2(X)],\mathrm{rESS}_1 = \frac{\left(\mathbb{E}_{P^1}[e^{L_2(X)}]\right)^2}{\mathbb{E}_{P^1}[e^{2L_2(X)}]}, and similar for the other rESS values. That is, rESS can be interpreted as a population-level moment ratio of exponentiated log lifts, generalized to generic weighted compositions. Consequently, our theory provides a principled connection between CompLift and compositional error-sensitivity. The same connection extends to Condition Alignment Score (Hong et al., 2024) [5], which uses the same underlying log density ratio as CompLift to similarly rank or reject samples, but estimates it through a different method.

AuthorsOpenReview ↗

Response (part 3)

A related line of work uses diffusion-based proxies for conditional log likelihoods to perform zero-shot classification, where differences between class scores similarly approximate log density ratios [6, 7]. As described in Section 4.3, we adapt the technique from [7] of using paired single-step denoising inferences to estimate these ratios. To our knowledge, prior work has not used diffusion-based likelihood-ratio estimates to measure the error-sensitivity of geometric composition.

We will add a discussion of the relationship of rESS to these existing works.

Q4: Data collection

If compositional failure is primarily attributed to distributional estimation error, why is collecting additional source data not sufficient? More importantly, how can the proposed sensitivity characterization improve this process?

We agree that collecting more data is typically sufficient to improve distributional estimation error. This is largely true of any statistical property stemming from estimation error. However, for trained neural models, while additional source data helps ID settings, this is muddier for OOD settings, and particularly OOD NFC settings. This is because the composed sampling procedure requires evaluating at points from the composition, which were not within the training data, so these points may still yield estimation error.

More importantly, in practice, much more data may be necessary. Indeed, Proposition 1 demonstrates that even negligible estimation error can blow up to be arbitrarily large in the composition. Furthermore, data collection can be very expensive in scientific applications. The sensitivity characterization can help inform how much data is necessary, and it can be used to help assess when to trust that compositionally generated samples may be accurate or whether the condition pair requires data to be collected.

To address this, we additionally performed an experiment to test whether a simple interpretation of the sensitivity metric produces meaningful results for data allocation. If we assume that source-wise estimation error decreases proportional to 1/ni1/\sqrt{n_i} (where nin_i is the number of training samples for source ii), then the bound from Theorem 1 suggests allocating a total sample budget NN by minimizing ∑iκi1ni,subject to ∑ini=N.\sum_i \kappa_i \frac{1}{\sqrt{n_i}}, \qquad \text{subject to } \sum_i n_i = N. The optimum of this is at ni=Nκi2/3∑jκj2/3∝rESSi−1/3.n_i = N \frac{\kappa_i^{2/3}}{\sum_j \kappa_j^{2/3}} \propto \mathrm{rESS}_i^{-1/3}. Accordingly, we fix a dataset size of N=4000N=4000 room training images, and compare models trained with images allocated uniformly across conditions versus models with the rESS-given allocations, using the rESS values from the paper. Interestingly, we find that while naive composition success rate is effectively unchanged between the two, the success rate of FKC for the models trained with rESS-based data allocation increases by +10.5% for FC OOD and +5% for NFC ID on average. The fact that it increases performance most for FC OOD aligns with the intuition that estimation error drove degradation in this case.

We would be happy to provide further details on this experiment and its results, though we note that this is intended only as a simplest-possible extension, and it is probable that more sophisticated methodologies could make better use of the sensitivity analysis.

References

[1] Xu et al. (2024), Automated Discovery of Pairwise Interactions From Unstructured Data

[2] Skreta et al. (2025), Feynman-Kac Correctors in Diffusion: Annealing, Guidance, and Product of Experts

[3] Bradley et al. (2025), Mechanisms of Projective Composition of Diffusion Models

[4] Gaudi et al. (2025), CoInD: Enabling Logical Compositions in Diffusion Models

[5] Hong et al., (2024), CAS: A Probability-Based Approach for Universal Condition Alignment Score

[6] Clark & Jaini (2023), Text-to-Image Diffusion Models are Zero-Shot Classifiers

[7] Li et al. (2023), Your Diffusion Model Is Secretly a Zero-Shot Classifier

[8] Liu et al. (2022), Compositional Visual Generation With Composable Diffusion Models

[9] Yu, Chenning, and Sicun Gao, (2025), Improving Compositional Generation with Diffusion Models Using Lift Scores

Reviewer_F9pQOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the feasibility of compositional generation by distribution estimation error from inference-time approximation error. It shows that small errors in source distributions can produce arbitrarily large composition errors. Through controlled Gaussian experiments and simple image-generation setups, the paper demonstrates that FKC can worsen generation quality when applied to learned scores, particularly in OOD settings. It also investigates an ESS-based diagnostic for composition sensitivity.

Critical strengths
  • Useful analysis of correction methods: The paper explains an important caveat of FKC-style methods (which is widely believed to be promising): improving sampling fidelity need not improve agreement with the true target. This is a valuable perspective for interpreting inference-time correction.

  • Clear motivation: The Gaussian constructions make the instability, while the formal local sensitivity bounds provide a principled explanation. The examples show that naive sampling (using the score functions) can be more robust.

  • Supporting experiments: The experiments systematically vary factorization/source coverage/training-set size/particle count. Comparisons between analytic and learned scores provide useful controls.

Critical weaknesses
  • Limited actionable algorithm: Although the paper identifies when correction can be harmful, it does not develop an alternative that improves this tradeoff. The image-level ESS diagnostic relies on approximate density ratios, but evaluation on a large-scale setup and a broader range of concept combinations would strengthen the practical relevance of the conclusions.

  • Accessibility and presentation. (I feel like) the presentation is mathematically dense. A more intuitive explanation of the failure mechanism, together with a clearer distinction between local sensitivity guarantees and practical reliability claims, would help readers understand the results.

Questions for authors

See the weakness section.

1 comment

We thank the reviewer for their feedback and positive assessment. Below, we address the weaknesses which were raised.

Weaknesses

W1: Limited actionable algorithm

We agree that we do not provide an explicit actionable algorithm, and note that the goal of the paper is not to develop an alternative approach to compositional generation. Rather, the goal is to develop a deeper understanding of the general task and existing approaches, similar to prior works that study compositional generation [1,2,3]. It is our belief that works furthering scientific understanding of a challenging problem are valuable whether or not they propose a method to solve it.

Nevertheless, our results do provide a few different actionable insights. Firstly, the sensitivity metric may be used by practitioners to assess when compositions may be unreliable, without access to ground-truth data from the composition. This can be important in applications where it may not be obvious whether compositions are correct, as is the case in many settings requiring extrapolation to unseen combinations (e.g., in biology).

Secondly, our analysis can be used to help understand when FKC (or other approximation-error correction) is likely to be helpful, inert, or harmful, and act accordingly. As a simple instantiation of employing this in practice, we can combine the sensitivity metric with the factorization heuristic in Appendix E.2 to determine whether the target composition is likely factorized or not and if it is likely OOD or not. This can then be used to select between FKC (K=16) and naive composition, which here would yield up to +20.1%/+10.4% success rate (rooms/faces) or 16x cheaper inference over using only naive composition or using only FKC, depending on the setting.

Lastly, as a straightforward proof-of-concept, we demonstrate with an additional experiment that the sensitivity metric can be used to decide how to allocate data to each source when such data is expensive to collect (see our response to Reviewer Kkz4, Q4 for further details).

We will expand our current discussion of the first two implications in the Discussion section, and add a discussion of this third one.

evaluation on a large-scale setup and a broader range of concept combinations would strengthen the practical relevance

We have performed additional ablations for our experiments over a wider array of object and feature combinations (for example, Rug + Mirror for rooms and Eyeglasses + Smiling for CelebA). We observe similar trends for these as in the paper. We would be happy to share the full tables during the discussion period should the reviewer find it helpful, and we will add these ablations to the appendix. We additionally describe results for pretrained models in response to Reviewer Kkz4 Q2.

We hope these clarifications and additional results have helped to alleviate the reviewer's concerns and increased their confidence in the practical relevance of the conclusions.

W2: Accessibility and presentation

We apologize that the presentation felt mathematically dense, and we will add more intuition for the failure mechanism. Intuitively, composition can concentrate probability in regions that are rare under the individual source distributions, amplifying estimation errors of those sources. Our sensitivity bound provides a worst-case quantification of this amplification. Additionally, diffusion introduces a second source of error: because composition and noising do not commute, naive score composition can produce sampling errors even with exact source scores. Corrections such as FKC aim to reduce this inference-time approximation error, but must necessarily target the composition of the learned sources, which have estimation error. Sampling that composition more faithfully can therefore reduce sample quality rather than improve it. In our experiments, this plays out in four different regimes, where correction helps when the composition remains in-distribution and the conditions to compose are not fully factorized, but can hurt when the compositions are out of distribution, particularly when the conditions are factorized.

References

[1] Bradley et al. (2025), Mechanisms of Projective Composition of Diffusion Models

[2] Bradley and Nakkiran (2024), Classifier-Free Guidance is a Predictor-Corrector

[3] Okawa et al. (2023), Compositional Abilities Emerge Multiplicatively

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of responses

We thank the reviewers for their feedback. Our responses clarify the generality of the paper's scope and practical implications, and provide additional experiments testing the proposed mechanisms across a broader range of compositions and pretrained models.

Practical implications

While the primary goal of this work is to further understanding of compositional generation, and we believe the main contribution is the theoretical and empirical analysis of distributional estimation error, inference-time approximation error, and their interaction during composition, we have also shown how this understanding can be used to:

  • Select between composition procedures, improving compositional accuracy or reducing inference cost depending on the setting (existing results, clarified in e.g. response to Reviewer F9pQ, W1).
  • Improve data allocation across source distributions when data collection is costly (response to Reviewer Kkz4, Q4).
  • Improve predictions of compositional performance without access to samples from the ground-truth composition (second response to Reviewer Kkz4).

Scope

We have clarified the breadth and practical motivation of weighted geometric composition (response to Reviewer Kkz4, Q2), including how it covers prior works on compositional generation and may help explain the behavior of methods such as CoInD (response to Reviewer kKRL, W2).

Additional evidence

Finally, we perform ablations across object and attribute compositions, together with experiments on pretrained models (response to Reviewer kKRL, Q2). These exhibit the trends in naive composition and FKC performance predicted by our analysis, providing further evidence for the proposed mechanisms and the practical relevance of the framework.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.