acceptodds
Under review as a conference paper at ICLR 2027

Latent Causal Propagation for Out-of-Distribution Single-Cell Perturbation Prediction

Abstract

Predicting single-cell responses to unseen genetic or chemical perturbations requires generalization beyond the perturbations observed during training. Yet the conditions under which latent representations support such generalization remain insufficiently understood. We study this problem through latent causal propagation: perturbations intervene on a basal cell state, propagate through a shared latent structural causal model (SCM), and produce transcriptional responses through a domain-invariant conditional generation kernel. Under explicit assumptions, we establish identifiability results for perturbation-induced latent distributions and, under additional restrictions, the latent structural matrix. We further characterize the cross-domain optimality of an oracle Bayes rule conditioned on the causal latent state and bound target-domain generation error in terms of latent representation and conditional generator approximation errors. Guided by this framework, we introduce scCausalDiff, which combines a learned latent SCM with conditional diffusion generation. Experiments on genetic and chemical perturbation benchmarks under strict OOD splits demonstrate competitive predictive performance across mean-response and distributional metrics.

Primary area:
causal reasoning
Keywords:
single-cell perturbation prediction, out-of-distribution generalization, latent causal models, conditional diffusion, computational biology

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_jmh5OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates when learned representations for single-cell perturbation prediction support OOD generalization and proposes scCausalDiff, a three-stage framework: (1) projecting control-cell expression into a basal latent space via a variational encoder and predicting an additive shift with a perturbation encoder; (2) performing causal propagation on the combined representations using a learned linear SCM; and (3) feeding the propagated representation into a conditional diffusion generator. In particular, the SCM is trained under sparsity and acyclicity regularization.

The paper also provides a theoretical analysis establishing latent-distribution identifiability under a fixed conditional kernel and an injectivity assumption. It further discusses structural identifiability under additional constraints, proves oracle Bayes optimality across domains, and derives a generation-error bound controlled by representation error and generator approximation error.

The empirical evaluation spans held-out genetic perturbations on Replogle, unseen gene combinations on Norman, and held-out compound-dose conditions on sci-Plex3. Based on the reported figures, scCausalDiff is competitive in terms of mean-response accuracy, while its most substantial advantage lies in distributional metrics on sci-Plex3 and Norman.

Critical strengths

The strengths lie primarily in the empirical results, demonstrating practical value for distributional prediction. scCausalDiff achieves superior performance in terms of MMD and energy distance on sci-Plex3, and Appendix D.1 reports consistent improvements across all five folds on Norman. The evaluation of both mean responses and cell-population distributions is appreciated.

Critical weaknesses
  1. The theoretical analysis does not establish identifiability for the jointly learned model or OOD generalization from source-domain training.

Theorem 4.4 treats the conditional kernel KK as fixed and assumes that the observation operator is injective, whereas the proposed method jointly learns the latent SCM and the conditional operator. For example, jointly scaling the latent representation and reparameterizing the conditional kernel can leave the observed distribution unchanged. This does not contradict Theorem 4.4, but shows that its fixed-kernel guarantee leaves the joint-learning ambiguity unresolved. Also, Theorem 4.10 relies on Assumption B.1(c–d) to bound target-domain representation and generator errors, but it does not show how source-domain training actually controls these terms. Neither result explains why the proposed causal propagation improves OOD generalization.

  1. The existing ablation study also can not support the benefits of the learned DAG.

Figure 5 replaces the propagation map only at inference time, while keeping all other weights fixed. The drop in performance shows that the trained model depends on this map, but it does not show that training with a DAG is better than training with other architectures. For example, a comparison would be models trained from scratch with A=0A=0 and with an unconstrained learned linear map, using the same embeddings, denoiser, and training budget. Also, sci-Plex3 is split by compound-dose conditions, but it does not establish compound-disjoint evaluation. These results therefore do not establish generalization to completely unseen compounds.

Questions for authors
  1. Theorem 4.4 fixes KK, while Assumption B.1(c–d) already bounds target-domain errors. What do these results guarantee for the actual model, which learns AA and KK jointly from source-domain data?

  2. Does the DAG outperform models trained from scratch with A=0A=0 or an unconstrained learned linear map, using the same embeddings, denoiser, and training budget?

  3. How do you position your novelty compared to DiscrepancyVAE (Zhang et al., 2023), SAMS-VAE (Bereket and Karaletsos, 2023), and PerturbedVAE (Jiang et al., 2026) in its modeling assumptions, identifiability guarantees, and prediction of unseen perturbations? Where the prediction settings overlap, are direct empirical comparisons with these methods available?

References:

Zhang J, Greenewald K, Squires C, et al. Identifiability guarantees for causal disentanglement from soft interventions. NeurIPS 2023.

Bereket M, Karaletsos T. Modelling cellular perturbations with the sparse additive mechanism shift variational autoencoder. NeurIPS 2023.

Jiang W, Liu Y, Cai Y, et al. What Makes a Representation Good for Single-Cell Perturbation Prediction? ICML 2026.

3 comments

We thank the reviewer for the careful reading and the precise questions. We address them in turn below.


On what the theory guarantees for the jointly learned model (Weakness 1, Question 1)

The fixed-kernel condition in Theorem 4.4 concerns population-level identifiability, rather than a restriction on joint optimization. Given a shared conditional kernel satisfying injectivity, the theorem establishes identifiability of perturbation-induced latent distributions and their relative shifts. While jointly learned latent representations may admit equivalent parameterizations, our objective is accurate perturbation-response prediction rather than unique recovery of the structural matrix AA. Such structural identifiability is not required for our OOD results.

Crucially, Theorem 4.10 provides a generation-error guarantee for the jointly learned model. It explicitly considers both the learned representation U^p\hat U^p and the learned generator QgenQ_{\mathrm{gen}}, without requiring separate optimization, independent estimation errors, or a uniquely identifiable AA. Under Assumption B.1, the target-domain generation error is bounded by ω(δU)+δQ\omega(\delta_U)+\delta_Q. Conditions B.1(c–d) specify the required target-domain approximation accuracy; the theorem establishes how these two errors jointly control the observable prediction error, rather than assuming accurate predictions themselves. Thus, the guarantee applies to jointly learned models satisfying these conditions, without requiring the generation kernel to remain fixed during training.


On whether the learned DAG improves over alternative architectures (Weakness 2, Question 2).

We trained the learned DAG and A=0A=0 variants from scratch on the same five folds of Replogle and sci-Plex3, using the same perturbation embeddings and denoiser architecture within each dataset. The table reports mean differences in the original metric units. Positive values favor the DAG and negative values favor A=0A=0. We subtract A=0A=0 from the DAG for Δ-P, Δ-S, and PDS, and reverse the subtraction for MMD and E-dist.

DatasetΔ-P ↑Δ-S ↑PDS-cos ↑PDS-L1 ↑MMD ↓E-dist ↓
Replogle+0.0353+0.0316+0.0135+0.0014+0.0047+0.1300
sci-Plex3+0.0234+0.0301≈0.0000−0.0053+0.0178+0.4283

The DAG improved Δ-P, Δ-S, MMD, and E-dist in all five folds on both datasets, supporting the use of a learned propagation operator.

In a separate five-fold comparison, the learned DAG and unrestricted propagation achieved similar overall predictive performance. The triangular DAG used approximately half as many free structural coefficients (d(d−1)/2d(d-1)/2 versus d2d^2).


On compound-disjoint evaluation for sci-Plex3 (Weakness 2)

Our description in Appendix C.1 ("formed over conditions") was imprecise. The sci-Plex3 folds are compound-disjoint: the 187 compounds are partitioned across the five folds, and all four doses of a held-out compound go to the test set as a block. We re-verified on the released split files that the train–test compound overlap is exactly zero on every fold, with no cell of a test compound in training. Table 1 therefore measures generalization to completely unseen compounds.


On DiscrepancyVAE, SAMS-VAE, and PerturbedVAE (Question 3)

Modeling. DiscrepancyVAE uses latent soft interventions, SAMS-VAE sparse offsets, and PerturbedVAE label-dependent SCM edges. Ours predicts sp=f(rp)s_p=f(r_p) from perturbation representations, uses shared B=(I−A)−1B=(I-A)^{-1}, and adopts conditional diffusion for unseen perturbations.

Identifiability. DiscrepancyVAE identifies DAGs/targets up to CD-equivalence under intervention coverage; PerturbedVAE identifies responsive factors under invertibility and sufficient environments; SAMS-VAE has no formal identifiability theorem. Our Theorem 4.4 instead identifies latent distributions under Assumption 4.3, without requiring identifiable AA or per-node intervention coverage.

Experiments. We retrained all three baselines on our five-fold Norman and Replogle splits.

Norman

MethodΔ-P ↑Δ-S ↑PDS-cos ↑PDS-L1 ↑MMD ↓E-dist ↓
Ours0.84530.52710.92480.93180.02010.3654
PerturbedVAE0.75710.41920.93930.83030.03530.6298
DiscrepancyVAE0.69530.32410.86010.79020.03410.5680
SAMS-VAE0.68240.45270.88160.67210.06351.0418

Replogle

MethodΔ-P ↑Δ-S ↑PDS-cos ↑PDS-L1 ↑MMD ↓E-dist ↓
Ours0.68930.68140.78340.76890.02320.7606
PerturbedVAE0.47780.49430.51410.51100.04381.4546
SAMS-VAE0.42070.40110.50850.50870.13913.9503
DiscrepancyVAE0.35170.34370.50570.51030.04591.4115

Ours leads on all six Replogle metrics and five of six Norman metrics.


We thank the reviewer for the constructive feedback and hope our responses address their concerns.

Thank you for the detailed response and additional experiments. The retrained identity ablation, new baseline comparisons, and clarification of the compound-disjoint splits address important aspects of my empirical concerns. Three points remain:

  • Connection between the theory and the jointly learned model. I agree that Theorem 4.10 can apply to jointly trained representations and generators. However, Assumption B.1(c–d) already requires control of their errors in the target domains. My concern is how source-domain training provides this control; the response does not establish that connection. Similarly, Theorem 4.4 identifies latent distributions under a fixed kernel. The observation that unique recovery of AA is unnecessary for prediction does not resolve the ambiguity in latent distributions when the kernel is learned jointly.

  • The specific benefit of the DAG constraint. The retrained identity comparison supports the value of learned propagation. However, the reported similarity between DAG and unrestricted propagation does not establish that acyclicity improves prediction. Comparable performance with fewer parameters could be useful, but please provide the full comparison and clarify the parameterization. The response describes a ''triangular DAG'', whereas Section 3 specifies NOTEARS without a prescribed topological order. Were the new comparisons conducted using the model described in the manuscript?

  • Incorporation of the rebuttal evidence. I could not find these additions in the supplied manuscript. Appendix C.1 still describes sci-Plex3 splits as formed over conditions, Figure 5 still reports inference-time replacements, and the new baseline comparisons are absent. Please incorporate the corrected split description, retrained ablations, and baseline results, with sufficient implementation detail to assess their comparability.

I appreciate the stronger empirical evidence. My remaining concerns focus on what the theory establishes for the trained model and what the experiments show specifically about the DAG constraint.

We thank the reviewer for acknowledging the additional evidence and clarifications. To maintain a consistent manuscript version for all reviewers during the ongoing discussion, we have presented the additional results and clarifications in our responses. We plan to incorporate these updates together in the final manuscript, should the paper be accepted.

We provide further clarification on the two remaining points below.

On joint learning and transfer to unseen perturbations

To connect source-domain fitting with OOD prediction under joint learning, we consider the observable cell-population expression distributions. Let PpP_p and P^p\hat P_p denote the true and predicted distributions for perturbation pp, with finite first moments, and define Dp=W1(Pp,P^p)D_p=W_1(P_p,\hat P_p).

Following our structural formulation, write the true and learned propagated states as V+c(rp)V+c(r_p) and V^+c^(rp)\hat V+\hat c(r_p), respectively. Within each model, the basal-state distribution is shared across perturbations. Suppose cc and c^\hat c are Lipschitz in the perturbation embedding, with constants Lc,Lc^L_c,L_{\hat c}, and the corresponding generation kernels K,QK,Q are W1W_1-Lipschitz in their latent inputs, with constants LK,LQL_K,L_Q. Let M=LKLcM=L_KL_c and M^=LQLc^\hat M=L_QL_{\hat c}.

By coupling the same basal state across perturbations within each model, we have

\qquad W_1(\hat P_p,\hat P_q)\leq \hat M\|r_p-r_q\|.$$ Applying the triangle inequality through any source perturbation $q$ gives $$\boxed{ D_p\leq \min_{q\in\mathcal P_{\mathrm{src}}} \left[ D_q+(M+\hat M)\|r_p-r_q\| \right]. }$$ Hence, source-domain prediction accuracy can extend to unseen perturbations when the response mappings vary smoothly and the target perturbations are sufficiently covered in the embedding space. The bound applies to the jointly fitted predictor and depends on observable source-domain errors rather than requiring alignment between the true and learned latent coordinates. The same reasoning applies to two independently fitted models, even if their latent representations and generators differ. If their observable response mappings have Lipschitz constants $M_1,M_2$, then $$\boxed{ \begin{aligned} &W_1(\hat P_p^{(1)},\hat P_p^{(2)})\\ &\leq \min_{q\in\mathcal P_{\mathrm{src}}} \left[ D_q^{(1)}+D_q^{(2)} +(M_1+M_2)\|r_p-r_q\| \right], \end{aligned} }$$ where $D_q^{(i)}=W_1(P_q,\hat P_q^{(i)})$. Thus, different latent parameterizations need not lead to substantially different OOD predictions: their observable disagreement is controlled by source-domain fitting accuracy and response variation for nearby unseen perturbations. This analysis complements Theorem 4.4, which concerns latent identifiability under a fixed shared kernel, and Theorem 4.10, which bounds generation error through representation and generator approximation errors. It further relates the jointly learned model's source-domain predictive accuracy to its behavior on unseen perturbations under the stated regularity and coverage conditions. ___ ### On the DAG constraint and unrestricted propagation The "triangular DAG" in our earlier response refers to the structural property that a DAG admits a topological ordering under which its adjacency matrix is strictly triangular, with at most $d(d-1)/2$ possible edges. In our implementation, this property is enforced through the NOTEARS acyclicity constraint. When the constraint is satisfied exactly, the learned graph likewise has at most $d(d-1)/2$ edges. This describes the DAG structure, not the number of optimized parameters. The fully unrestricted variant starts from a randomly initialized dense matrix and jointly optimizes its entries without acyclicity or L1 constraints, allowing cyclic dependencies and self-loops. **DAG vs. unrestricted propagation.** Signed mean differences over five folds are computed as DAG minus free for ↑ metrics and free minus DAG for ↓ metrics. Positive values favor the DAG. | Dataset | Δ-P ↑ | Δ-S ↑ | PDS-cos ↑ | PDS-L1 ↑ | MMD ↓ | E-dist ↓ | |---|---:|---:|---:|---:|---:|---:| | Replogle | −0.0055 | −0.0047 | +0.0055 | +0.0038 | −0.0005 | −0.0148 | | sci-Plex3 | −0.0048 | −0.0070 | +0.0011 | +0.0006 | −0.0022 | −0.0531 | The results show broadly comparable predictive performance. The value of the DAG constraint lies in its structural inductive bias: perturbation effects propagate along directed paths without returning through feedback cycles, consistent with our latent SCM formulation, rather than through arbitrary cyclic feature mixing. Together with the end-to-end retrained $A=0$ ablation, these findings support two complementary observations: learning propagation improves prediction over identity propagation, and these improvements can largely be retained under DAG-based structural constraints.
Reviewer_PhH9OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors propose a new model, scCausalDiff, based on three key component: An encoding mechanisms to encode unseen genetic or chemical perturbations, a (linear) SCM that propagates the effect of perturbations over a learnt DAG, and finally a Diffusion model that generates the predicted perturbed transcriptional profile given the learnt latent factor. Importantly, the authors propose a theoretical framework for OOD perturbation prediction, which is of great value. The validation of the model is against well known models, and performs very well. The biological interpretation of the learnt causal mechanisms also adds value.

Critical strengths

The main strength of the paper is the theoretical framework for OOD perturbation prediction. It represents a step forward form recent previous works in terms of the identifiability guarantees for OOD prediction, which is an important step forward for causal representation learning in perturbation prediction.

The proposed model is also an important addition to the current state of the art, as it performs very well across different datasets and modalities (chemical and genetic perturbations).

Critical weaknesses

In my opinion the critical weakness is the biological evaluation of the inferred latent factors. It looks like a cherry-picked example to show that the latent factors recapitulate known biology. In perturbation prediction models, and more importantly, in those that claim to uncover causal relationships, it is very important to show that the underlying causal structure makes sense from the biological point of view. Biology is a very complex field in which spurious correlations/causal mechanisms can be learnt from the high dimensional data.

The second weakness is the lack of assessment of different components using the Norman dataset (both for single and double perturbations). Even more, given that the results for two-gene combinatorial perturbation prediction (Table 2), shows the proposed model underperforming with respect to previous models.

Questions for authors
  • Why constraining the biological evaluation to the Replogle dataset? It would be good to see how it works when predicting double perturbations, or chemical perturbations. Do the latent programs also replicate known biology?

  • Why ablation studies skipped the Norman dataset and the double perturbation prediction? This would be very important since double perturbation prediction is a critical task to be solved in perturbation biology.

  • Why did the authors no used recent datasets such as the HepG2-Essential Perturb-seq dataset? Expanding over the (overused) datasets is important to make sure that the models are not somehow overfitting to the characteristics of the widely used Norman and Replogle datasets.

1 comment

We thank the reviewer for the careful reading, constructive feedback, and thoughtful questions. We address the concerns point by point below.


On the biological evaluation of the latent factors (Weakness 1, Question 1)

Norman double perturbations. Following the variance-based criterion in the original paper, we retained six PCs explaining at least 90% of the training-set latent variance. Both PCA and expression-program regression were fitted using only training double perturbations. The associated gene programs were enriched for cell-cycle pathways, while the sixth PC additionally captured associations with respiratory electron transport and oxidative phosphorylation. These associations remained significant after additional correction across all 12 PC-direction gene lists.

Chemical perturbations. Drug representations required 23–25 PCs to explain 90% of the variance across folds. We therefore complemented the PCA analysis with a full-space assessment of mechanism organization, covering all 24 annotated mechanism classes containing at least two compounds. Eight classes showed significant same-mechanism neighborhood agreement after correction across all 72 class-by-representation tests, including histone deacetylation, Aurora kinase activity, nuclear receptor activity, and JAK kinase activity.

Together, these analyses extend the biological assessment beyond Replogle to both combinatorial and chemical perturbations, providing systematic evidence beyond a single selected example.


On component assessment on Norman and combinatorial prediction (Weakness 2, Question 2)

We have run the component ablations on Norman two-gene combinations, five folds, with the same six metrics and the same readout as in the paper.

Latent propagation.

armΔ-P ↑Δ-S ↑PDS-cos ↑PDS-L1 ↑MMD ↓E-dist ↓
learned0.84420.52650.92470.93120.02000.3675
A = 00.82060.52830.92470.82840.05110.7623
random A0.79310.49640.91820.76830.05160.8278
orthogonal0.21700.27950.49970.51850.10211.6195

The learned structure is best or tied-best on five of the six metrics, and ahead of all three alternatives on every one of the five folds for Δ-P, MMD and E-distance. On Δ-S the A = 0 arm is ahead by 0.0018. The ordering reproduces what the paper reports on Replogle and sci-Plex3.

Conditional generation.

armΔ-P ↑Δ-S ↑PDS-cos ↑PDS-L1 ↑MMD ↓E-dist ↓
diffusion0.79710.52660.85760.87560.02470.4748
VAE0.70680.48160.72060.68100.09543.4369

Diffusion is ahead on all six metrics on all five folds, with E-distance differing by a factor of seven.


On the Table 2 comparison (Weakness 2)

On Norman, no single method dominates across the mean-shift metrics: TxPert leads on Δ-P, while scLAMBDA leads on Δ-S, and our method performs comparably to both. The relative rankings also vary across folds, with no consistent winner on Δ-P. In contrast, the distributional metrics show a clear and consistent advantage for our method. scCausalDiff achieves the lowest MMD (0.0201 vs. 0.0319 for scLAMBDA) and E-distance (0.3654 vs. 0.6110 for Doloris), outperforming every baseline on both metrics across all five folds. Thus, our method maintains competitive mean-response accuracy while substantially improving the predicted cell-population distributions for unseen gene combinations.


Evaluation beyond Replogle and Norman (Question 3).

To further assess generalization beyond the commonly used benchmarks, we extended our evaluation to the HepG2-Essential Perturb-seq dataset. Under five-fold OOD evaluation, scCausalDiff achieves the best performance on five of the six metrics, including both mean-response correlations and both distributional metrics (MMD and E-distance). These additional results further demonstrate the effectiveness of our approach beyond the original benchmarks.

MethodΔ-P ↑Δ-S ↑PDS-cos ↑PDS-L1 ↑MMD ↓E-dist ↓
Ours0.48160.44800.58670.60470.041051.9835
scLAMBDA0.47610.44420.58660.60020.041852.0311
TxPert0.47070.43640.65250.57780.080185.7046
Doloris0.32220.28120.50520.50450.062552.7643
GEARS0.27540.24020.56230.54890.048782.3006

We hope these clarifications and additional results adequately address the reviewer's concerns. We thank the reviewer again for the constructive feedback and valuable suggestions.

Reviewer_WzQ6OpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

In biology, methods for observing and measuring perturbation responses are crucial for understanding the roles of various factors, such as genes and drugs. Even in the single-cell data addressed in this paper, it is a mainstream biological experiment to measure how cell fate responds to perturbations—such as those induced by knockdown, knockout, or overexpression of specific genes, as in PerturbSeq. This paper focuses on the general challenge of predicting perturbation responses, specifically highlighting the importance of generalization for unseen perturbations, and proposes a framework in which perturbations act on latent shifts, which in turn function as domain-shared mechanisms. The authors have successfully separated, in theory, the identifiability of hidden causal graphs from the challenge of out-of-distribution (OOD) prediction. They clearly demonstrate that their proposed method outperforms state-of-the-art methods in two practical experimental applications: gene perturbation and drug perturbation.

Critical strengths
  • This paper addresses a conceptually novel topic—generalization to unseen perturbations—within computational biology, one of the most prominent fields in AI for science in recent years.

  • The model representation that decomposes a specific type of artificial operation—perturbation—into two distinct mechanisms—latent shift and domain-shared mechanism—represents a highly contemporary and robust approach.

  • The experiments in this paper, including ablation studies, clearly demonstrate application cases in which the proposed method functions effectively.

Critical weaknesses

This paper devotes three pages to analyzing the theoretical soundness of the proposed framework. While these arguments appear mathematically very clear, they give the impression of being somewhat removed from the interests of readers who would utilize the paper’s results (such as computational biologists). More specifically, it seems that strong, artificial assumptions have been introduced to derive the mathematical claims, and these assumptions appear to be quite detached from the original problem of “unseen perturbations.” Furthermore, I sense a risk that the essence of the theory—despite the authors’ sincere efforts to clearly state the limitations and assumptions in these discussions—remains largely obscured from the reader’s perspective within this information-rich theoretical framework. I have specifically outlined these concerns in the “Questions” section.

Questions for authors

The discussion regarding the identifiability of the structural matrix A appears to be a straightforward fact derived from the covariance factorization of a very general linear Gaussian SEM. Furthermore, as the authors explicitly mention, the identifiability of the structural matrix A requires the assumption that the ordering is known. I remain somewhat concerned as to whether this line of argument can be improved upon any further, even by utilizing insights from recent research such as the following:

Zhou, Q., & Chang, H. (2023). Complexity analysis of Bayesian learning of high-dimensional DAG models and their equivalence classes. The Annals of Statistics, 51(3), 1058–1085.

Similarly, very strong assumptions—which seem almost tautological—are placed on the causal latent representation UpU^{p}. In Assumption 4.3, the authors assume the injectivity of the observation operator \Psi on the Gaussian measure. As a result, they derive the identifiability of the input distribution from the output distribution. This is mathematically very straightforward. However, this fact feels as though it simply replaces the identifiability of U^{p} entirely with the richness of Y^{p}.

The logic by which these two forms of identifiability contribute to OOD prediction does not seem clear. The difficulty in OOD prediction discussed in this paper appears to lie in estimating the shift sps_{p} under unseen perturbations. I do not agree with the way this paper is structured, as it isolates the essence of this difficulty within Assumption B.1 in the appendix. This goes beyond assumptions about error in the source domain and directly assumes the quality of the representation and the generator approximation in the target domain. If the paper aims to make a theoretical contribution to OOD prediction, these assumptions—which seem to be very strong—should be handled with greater caution (and at the very least, not hidden away in an appendix).

1 comment

We thank the reviewer for the careful reading of our theory and the constructive comments. We address the three questions below.


On the identifiability of the structural matrix AA (Question 1)

For a known ordering, Theorem 4.5 is the standard triangular (Cholesky) factorization of ΣU\Sigma_U in a linear Gaussian SEM; none of our OOD results (Theorems 4.9–4.10) rely on it.

Without an ordering. In linear Gaussian SEMs, the DAG is identifiable from the distribution only up to its Markov equivalence class under faithfulness (Peters & Bühlmann, 2014). Thus ΣU\Sigma_U (identified by Theorem 4.4) determines the equivalence class of AA and nothing finer. As the reviewer suggests, Zhou & Chang (2023) show that this class can be learned consistently, with polynomial complexity, for sparse high-dimensional DAGs, so the equivalence class is the attainable structural target.

Beyond the equivalence class. Further refinement requires additional structure, such as equal noise variances (Remark 4.7; Peters & Bühlmann, 2014), non-Gaussian noise (as in LiNGAM), or interventions on known latent nodes. Our model imposes none of these: we do not assume equal latent noise variances, non-Gaussian noise would break the Gaussian location-family structure of Theorem 4.4, and our perturbations enter as learned shifts sps_p in a latent space whose nodes are not tied to specific genes. Our prediction guarantees are therefore stated without requiring identifiability of AA; they rely only on the identifiable latent distributions of Theorem 4.4.

On Assumption 4.3 and Theorem 4.4 (Weakness, Question 2)

Assumption 4.3 concerns the information preserved by the observation mechanism, and admits concrete sufficient conditions. For example, consider a conditional generator with E[Y∣U=u]=Wu+b\mathbb E[Y\mid U=u]=Wu+b, where WW has full column rank, and a conditional covariance CC independent of uu. The observed mean and covariance then uniquely determine the mean and covariance of UU. Since UU is Gaussian under Assumption 4.2, these moments determine its entire distribution, establishing the injectivity required by Theorem 4.4. Linearity is not essential: the same conclusion holds for additive-noise generators Y=g(U)+εY=g(U)+\varepsilon with injective gg and independent noise having a nonvanishing characteristic function (e.g., Gaussian noise).

These examples show that Assumption 4.3 can be satisfied by concrete generation mechanisms. Appendix D.5 further provides complementary empirical diagnostics: the learned basal representation exhibits approximate Gaussianity, and the denoiser has full-rank restricted Jacobians along the tested latent directions, indicating non-degenerate local sensitivity to latent variations.


On the connection to OOD prediction and Assumption B.1 (Weakness, Question 3)

The key challenge in OOD prediction is estimating the shift sps_p for perturbations not seen in training. Our framework addresses this challenge through a shared latent mechanism.

In our model, each perturbation acts through a latent shift sp=f(rp)s_p = f(r_p), produced by a perturbation encoder ff from the perturbation representation rpr_p, and propagated as Up=B(Z+sp)U^p = B(Z + s_p). The distribution of ZZ, the propagation BB, the generator KK, and the encoder ff are all shared across perturbations. An unseen perturbation therefore differs from the training ones only through its representation rpr_p.

The two forms of identifiability play different roles. Identifiability of AA concerns the interpretation of the learned propagation structure and, as discussed in our answer to Question 1, is not required for OOD prediction. Identifiability of the latent distributions (Theorem 4.4) is more directly relevant: on each source perturbation qq, the propagated shift B(sq−sctrl)=mq−mctrlB(s_q-s_{\mathrm{ctrl}})=m_q-m_{\mathrm{ctrl}} is uniquely determined by the observed data under the stated conditions. Thus, the source response distributions uniquely constrain the propagated shifts that the shared encoder is trained to predict, rather than leaving them ambiguous within the fixed latent coordinate system. Theorem 4.9 then establishes that the same oracle Bayes decision rule remains optimal across unseen perturbation domains.

What remains is how well the shared encoder extrapolates from training representations rqr_q to an unseen rpr_p, and how accurate the generator is at the resulting latent states. Conditions (c)–(d) of Assumption B.1 (Appendix B.4) measure the target-domain representation and generator approximation errors, and Theorem 4.10 bounds the error of the generated distribution by ω(δU)+δQ\omega(\delta_U)+\delta_Q. How small these errors are on unseen perturbations depends in part on how informative the representation rpr_p is, and this is what our strict OOD splits evaluate empirically, with entire perturbation conditions held out from training.


We appreciate the reviewer's constructive feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.