acceptodds
Under review as a conference paper at ICLR 2027

When Conditional Generators Cannot Extrapolate: An Exactly Measurable Decomposition of Out-of-Distribution Error

Abstract

Conditional generators answer requests they never encountered in training, but when an answer fails we cannot tell whether the model is inadequate or the target was unreachable from the available data. A recent theory separates this failure into target geometry, representation distortion, and training fit. Its sharpest criterion, however, relies on a proof step that controls a different, alignment-dependent quantity, while its experiments estimate every term through proxies. The theory therefore neither certifies the claimed boundary nor reveals which cause actually binds. We repair it by separating an unconditional certificate from a sharper criterion that holds under coherent alignment, formalizing every statement in a proof assistant, and turning distance from the training hull into a test of what any inference-time reweighting can reach. We then construct two benchmarks-capital-letter point clouds and single-cell perturbation populations-where the distances, barycenters, error terms, and sampling floor are computed exactly under held-out evaluation. In both domains, geometry predicts the unseen-condition error while representation distortion and training fit do not bind. Reweighting, condition mixing, and representation regularization leave the geometric gap in place; one genuine target example collapses it. The implication is a practical triage rule before training: determine whether a request is already reachable, requires new examples, or lies beyond what tuning alone can recover.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Out of Distribution, Few Shot, Deep Learning Theory

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_oxhYOpenReview β†—
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies why conditional generators fail on unseen conditions. It builds on prior work that divides out-of-distribution (OOD) error into three parts: target geometry, embedding distortion, and training fit.

The authors revisit the theoretical analysis and clarify the conditions under which some bounds hold. They also introduce two benchmarks based on letter point clouds and single-cell perturbation data. These benchmarks allow more direct measurement of the error terms using Wasserstein distances.

Experiments show that the geometric term strongly correlates with OOD error in both domains. Reweighting and representation regularization have limited effects, while adding real target examples greatly reduces the error. The paper aims to identify when generation errors can be improved by model tuning and when additional data may be needed.

Critical strengths

1. Well-defined and Measurable Benchmarks

The paper introduces two well-designed benchmarks that allow direct measurement of Wasserstein-based error terms, rather than relying on proxies. This provides a clear and controlled setting to study the sources of OOD generation error. I think this is a useful contribution, as it makes the analysis more concrete and easier to validate.

2. Comprehensive Experimental Validation

The experiments are thorough and well organized. The authors evaluate two different domains and include held-out evaluation, fidelity checks, ablation studies, and robustness tests across different settings. They also compare different interventions and show how adding target examples affects the error. These experiments provide good support for the main empirical findings.

Critical weaknesses

1. Limited Generality and Novelty of the Main Finding

The main finding is somewhat expected under the barycentric generation setting. If a model can only combine existing training distributions, changing their weights cannot reach targets outside the reachable set. While the paper provides a careful validation of this result, it is unclear how much new insight this offers beyond the existing theory. The experiments also use a small per-point MLP with barycentric conditioning. It remains unclear whether the same geometric limitations apply to more general conditional generators.

2. Limited Practical Applicability

The paper aims to use geometric information to identify OOD generation limits before training. However, computing the proposed diagnostic requires information about the target distribution. The experiments use Wasserstein distances involving held-out target samples, which may not be available in real OOD scenarios. Although the authors acknowledge this limitation, it remains unclear how the diagnostic can be applied when the target distribution is unknown. This limits the practical impact of the proposed approach.

Questions for authors

1. Generality to Large-scale Generative Models

The experiments mainly use a small per-point MLP and low-dimensional point clouds. How do the authors envision applying this framework to mainstream large-scale generative models, such as diffusion or flow-based models? Would the proposed geometric diagnostic and the conclusions about irreducible OOD error still apply to these models?

2. Practical Use of the Diagnostic

The paper proposes using Term I to identify OOD generation limits before training. However, the experiments use information from held-out target samples to compute Wasserstein distances. How could Term I be estimated when no target samples are available? How practical is the proposed diagnostic in a true zero-shot setting?

Reviewer_E5xSOpenReview β†—
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper claims that the OOD error of conditional flow-matching generators is governed by the geometric distance of the target from the training conditions, and that this gap can be closed only by new data, not by tuning. It supports this with a Lean-checked revision of a prior three- term bound (Pham, 2026) and with leave-one-condition-out experiments on letter point clouds and Perturb-seq data, where Wasserstein distances and barycenters are computed exactly.

Critical strengths

S1. Exact measurement. Computing π‘Š2, barycenters, 𝛿 and the sampling floor exactly, instead of through VAE/Jacobian proxies, is the right way to test this kind of bound. The robustness checks (leverage, jackknife, temperature, seeds, 8D) are careful. This is what makes the empirical results worth discussing at all. S2. The coherent-alignment gap. The paper shows that the barycentric argument bounds 𝛽 by 𝐼coh β‰₯ TERM I, not by TERM I. It also measures the residual directly and reports the 5/80 failures openly (Fig. 4). This is the clearest new content of the paper.

Critical weaknesses

W1. The main conclusion is built into the protocol. An unseen condition is fed 𝑒 = βˆ‘π‘– 𝑀𝑖𝑒𝑖 (L177–180), and the fidelity check (Sec. 4.2) confirms the output is the barycenter 𝐡𝑀. It follows by design that the error tracks the distance from the training conditions and that reweighting cannot reduce it. A model-free oracle alone gives π‘Ÿ = 0.87 (App. I). No conditioning mechanism that could extrapolate is tested, such as gene-level side information or compositional perturbations, the standard use of Norman-2019. So the title and abstract (β€œa property of the data, not of the tuning”) go well beyond what is shown. The proposed β€œtriage before training” also needs about 256 target samples to compute the weights (L273, App. G). W2. What is measured is not what is proved. (a) The floor in Prop. 4 is TERM I⋆ = inf𝑀 π‘Š2(𝐡𝑀, πœ‡π‘β‹† ), which is never reported. The experiments measure TERM I and call it the floor (L363, Fig. 5). TERM I is not a distance to the hull, and L154 is false: for πœ‡1 = π›Ώβˆ’1, πœ‡2 = 𝛿+1, πœ‡π‘β‹† = 𝛿0, the target is the barycenter (𝛽 = 0) but TERM I = 1. At 𝜏 = 0.06, TERM I is essentially the nearest-neighbor π‘Š2. (b) The proof of Thm. 1 (App. C.1, L770) uses π‘Š2(βˆ‘π‘– π‘€π‘–πœ‡π‘–, πœ‡π‘β‹† ) ≀ βˆ‘π‘– π‘€π‘–π‘Š2(πœ‡π‘–, πœ‡π‘β‹† ). This is false: π‘Š2( 1/2 𝛿0 + 1/2 𝛿1, 𝛿0) = 1/√2 > 1/2, and the ratio grows as 1/βˆšπ‘€, so no constant 𝐾 absorbs it. The paper maps this step to the Lean lemma weighted_dist_triangle (L777–778, Table 2). That lemma is a weighted triangle inequality in an abstract pseudometric space, which cannot express mixtures of measures. So the false step is not what Lean checks, and β€œevery statement in this section is formalized” (L158) does not hold for Thm. 1. Together with an abstract that never names flow matching and captions that assert conclusions rather than describe the plots (Figs. 1b, 3, 5, 13), this makes the paper’s claims hard to verify.

Questions for authors

Clarifications

  1. TERM I vs. TERM I⋆. Can you report TERM I⋆ per condition and redo Figs. 3 and 5 against it? Does the zero-shot gap actually match TERM I⋆, as Prop. 4 would require?
  2. Thm. 1. How is the step π‘Š2(βˆ‘π‘– π‘€π‘–πœ‡π‘–, πœ‡π‘β‹† ) ≀ βˆ‘π‘– π‘€π‘–π‘Š2(πœ‡π‘–, πœ‡π‘β‹† ) (App. C.1) justified, given π‘Š2( 1/2 𝛿0 + 1/2 𝛿1, 𝛿0) = 1/√2 > 1/2? Does the decomposition still hold with 𝐼2 in place of TERM I, or with the barycenter in place of the mixture?
  3. Extrapolating conditioning. Do the conclusions hold when the unseen condition’s embedding is not a convex combination of training embeddings? Two examples are an encoder from gene-level features and held-out double perturbations in Norman-2019. If not, would you restrict the claims in the title and abstract to this protocol?
  4. Few-shot protocol. Is β€œone example” a single point or cell, or a full 256-point cloud? Is it disjoint from the evaluation sample? Several gaps fall below the floor (e.g. L: βˆ’0.027, L364), which suggests fitting the evaluation draw.
  5. Sharp-bound violations. Is there an analytic counterexample to 𝛽 ≀ TERM I for exact barycenters (e.g. Gaussians in 𝑑 = 2)? Could the 5/80 violations, all below the floor, come from the free-support iteration reaching a local fixed point with 𝑁 -point support?
  6. Single-cell preprocessing. Why is each perturbation’s cloud centered and rescaled (App. F)? This removes the mean shift, which is the main perturbation effect. Were HVG selection, scaling and PCA fitted without the held-out perturbations?

Suggestions (future work, would not change my score) β€’ Test real classifier or classifier-free guidance. Replacing the weights with 𝑝(𝑐 ∣ π‘₯) is not classifier guidance, yet Prop. 4 claims to cover it. β€’ Justify the |Ξ”| < 0.05 β€œno meaningful change” band relative to typical gap sizes (0.1–0.2). β€’ Fix the numerical inconsistencies: – 𝛿 β‰ˆ 0.004 (L297) vs. β€œfrom 0.001” (L310); – in-distribution gap +0.011 (Sec. 4.1) vs. +0.048 (App. H); – the k = 0 gaps for C and L (0.197, 0.206; L364) vs. Table 3, where total minus floor gives 0.211 and 0.145; – I and J near 0 in Fig. 3a but about 0.1–0.15 at π‘˜ = 0 in Fig. 5c. β€’ Rewrite the abstract to name flow matching and state concrete results. Make the captions describe what is plotted.

1 comment

Dear Reviewer E5xS,

Because of the discussion periods will close soon. Dont know if my comments have solved all your concerns ?

Thank you again for your time and thoughtful feedback.

Reviewer_u1qKOpenReview β†—
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates out-of-distribution (OOD) generation errors in conditional flow-matching models. It proposes an upper bound on this error, decomposed into three components: interpolation sparsity, embedding distortion, and in-distribution fit. The authors refine existing derivations, provide formal verification, and establish a benchmark where each component can be directly measured. Experiments involving capital-letter point clouds and single-cell perturbation data reveal that, in the settings examined, the geometric term is the primary driver of the error. Furthermore, the inference-time reweighting strategies tested prove ineffective at closing this gap, whereas the inclusion of even a small number of real target samples significantly reduces the error.

Critical strengths

The authors propose an upper bound on the out-of-distribution (OOD) generation error, formulated as the sum of three computable terms: interpolation sparsity, embedding distortion, and in-distribution fit. They refine existing derivations and provide a formal verification using Lean 4 and Mathlib. By explicitly calculating these three error terms on a constructed benchmark, the authors bridge rigorous theoretical analysis with actionable experimental measurement. This enhances the understanding of OOD errors in conditional flow-matching models and provides a basis for diagnosing primary sources of error and identifying targeted avenues for improvement.

Critical weaknesses
  • The text is cluttered with an excessive use of formatting for emphasis and colloquial language. For instance, in the "Contributions" section, a span of only about ten lines incorporates a chaotic mix of various emphasis stylesβ€”such as \textbf, \textit, and \textscβ€”alongside mathematical symbols. This approach distracts readers from the core arguments, creating unnecessary obstacles to reading and comprehension.
  • The validation experiments are limited to 2D capital-letter point clouds and 2D or 8D PCA embeddings of a single-cell perturbation dataset, all evaluated using a single per-point MLP model. This setup lacks validation on higher-dimensional data, alternative model architectures, and a broader range of inference-time interventions. Furthermore, the work fails to sufficiently demonstrate how the insights gained from this decomposition can effectively guide improvements in out-of-distribution generation performance.
Questions for authors

See the "weakness" part.

Comments on the paper

1 comment

Dear Reviewers,

We have submitted our rebuttal and revised paper, addressing your comments and suggestions.

We hope the revisions help clarify our contributions and address your concerns.

Thank you for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.