acceptodds
Under review as a conference paper at ICLR 2027

Learning Causal Normalizing Flows from Incomplete Data via Observed-Data Likelihood

Abstract

Causal Normalizing Flows (CNFs) enable causal inference from observational data given the causal structure, but they assume fully observed training data. We introduce MissCNF, which trains CNFs directly on incomplete data by maximizing the marginal likelihood of each partially observed sample, without discarding rows or constructing a completed dataset. Thanks to the causal structure encoded in the autoregressive factorization of CNFs, only missing variables in the ancestral closure of the observed set are integrated out, while the others are dropped without computation. We further establish the conditions under which MissCNF recovers the true joint distribution, and introduce causal-family positivity, where identification is possible even when no record in the dataset is ever complete. We compare MissCNF with two common strategies for handling missing data: listwise deletion and impute-then-fit pipelines. Across eight synthetic causal benchmarks, three missingness mechanisms, and missing rates up to 90%, MissCNF achieves the lowest KL divergence in 23 of 24 nonlinear MCAR and MAR settings and in all nonlinear MNAR settings, as well as the lowest counterfactual error in 20 of 24 settings. On linear SCMs, where linear imputation performs best, MissCNF ranks in the top two in 22 of 24 settings.

Primary area:
causal reasoning
Keywords:
causal inference, missing data, causal normalizing flows, structural causal models, causal effect estimation, imputation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_VnhAOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how to train causal normalizing flows directly on incomplete data. Given a known causal graph and no unobserved confounding, MissCNF maximizes the likelihood of the observed entries and uses Monte Carlo integration only for missing ancestral variables. It also introduces causal-family positivity, under which the full joint distribution is identifiable under MAR even without complete records. Experiments cover eight synthetic causal models and three missingness mechanisms. MissCNF achieves the lowest KL divergence in 23 of 24 nonlinear MCAR/MAR settings and the lowest counterfactual error in 20. MICE generally performs best on linear models.

Critical strengths
  1. Causal-family positivity establishes when the joint distribution can be recovered without complete records, under a weaker condition than full-pattern positivity. This is the main theoretical contribution: it clarifies when incomplete data suffice for causal modeling.
  2. The observed-data objective and ancestral integration provide a training method without a separate imputation stage. Density and counterfactual results on nonlinear models support its practical value.
Critical weaknesses
  1. The imputation comparison is incomplete. MICE is included, but only one completed dataset is used for downstream training. A comparison with several completions, fitting a CNF to each and combining predictions, is missing. Does MissCNF still have a clear advantage under that setup? Reporting the extra cost would also help judge the value of direct likelihood training.
  2. The larger-graph experiment reports time and memory usage for up to 64 variables, but not recovery accuracy. The Monte Carlo sample-count study is also limited to small graphs. The results therefore leave open how much sampling is needed as the number of missing ancestors grows. Larger-graph experiments should report density and causal-query accuracy together with runtime.
  3. The gains are smaller for ATE than for density fitting and counterfactual prediction. The paper reports this difference but does not explain it in much depth. Since causal inference motivates the method, the authors should discuss which queries benefit from the better density estimate and where the improvement remains limited.
Questions for authors
  1. Does the advantage over MICE persist when several imputations are retained, a CNF is fitted to each, and the predictions are combined? Please include the extra training cost in the comparison.
  2. For the larger graphs used in the runtime study, how do KL, ATE, and counterfactual errors change as the Monte Carlo budget increases? At what computational cost does accuracy stabilize?
  3. What explains the smaller gains in ATE estimation? Are there settings where a clear KL improvement brings little or no ATE improvement, and what distinguishes them from settings with larger causal gains?
4 comments
AuthorsOpenReview ↗

Response to Reviewer VnhA

We thank the reviewer for the positive assessment and the constructive suggestions. Following them, we added a comparison with multiple imputation. The experiments on larger graphs and the Monte Carlo budget are still running, and we will post the results as soon as they are ready. Our answers to each point are below.


Weakness 1 and Question 1: Multiple imputation comparison, with cost.

Thank you for this suggestion. We agree this comparison matters. Appendix A.4 explains why our main experiments use m=1m = 1. The downstream object is a full generative model, not a finite-dimensional estimand. As far as we know, no standard Rubin-style rule exists for combining mm learned densities. A natural alternative is the equally weighted mixture p~(x)=1m∑jpθ(j)(x)\tilde{p}(x) = \frac{1}{m} \sum_j p_{\theta^{(j)}}(x).

We use this mixture for KL: to generate nn samples, we draw n/mn/m samples from each of the mm models. For RMSE-ATE and RMSE-CF: we average the mm models' ATE estimates and counterfactual predictions, respectively.

We ran this ensemble (m∈{1,5,10}m \in \{1, 5, 10\} on Triangle-LIN and Triangle-NLIN, at MAR missingness of rates 30% and 60%, 5 seeds. Table 1 reports three metrics and total wall-clock cost (imputation plus all mm CNF fits).

Table 1. MissCNF vs. an mm-completion MICE ensemble. Mean ± std over 5 seeds; cost is imputation + all mm CNF-fit wall-clock time, in seconds. Best KL, RMSE-CF and RMSE-ATE per setting in bold.

TaskMethodKL (↓)RMSE-CF (↓)RMSE-ATE (↓)Cost (s)
Triangle-LIN, p=30%p=30\%MissCNF0.014 ± 0.0070.327 ± 0.0830.367 ± 0.102369 ± 5
MICE, m=1m=10.007 ± 0.0040.279 ± 0.0670.300 ± 0.10214 ± 4
MICE, m=5m=50.002 ± 0.0030.263 ± 0.0420.348 ± 0.08254 ± 7
MICE, m=10m=100.001 ± 0.0030.266 ± 0.0190.351 ± 0.04680 ± 5
Triangle-LIN, p=60%p=60\%MissCNF0.034 ± 0.0160.465 ± 0.2840.492 ± 0.345422 ± 11
MICE, m=1m=10.009 ± 0.0060.254 ± 0.0600.297 ± 0.06214 ± 5
MICE, m=5m=50.003 ± 0.0020.216 ± 0.0380.281 ± 0.06742 ± 4
MICE, m=10m=100.002 ± 0.0020.229 ± 0.0420.304 ± 0.07182 ± 5
Triangle-NLIN, p=30%p=30\%MissCNF0.028 ± 0.0100.240 ± 0.0620.153 ± 0.029373 ± 10
MICE, m=1m=110.42 ± 1.732.338 ± 0.0672.298 ± 0.07215 ± 5
MICE, m=5m=510.70 ± 1.592.335 ± 0.0412.325 ± 0.06448 ± 6
MICE, m=10m=1011.18 ± 1.612.339 ± 0.0362.303 ± 0.05380 ± 5
Triangle-NLIN, p=60%p=60\%MissCNF0.052 ± 0.0320.325 ± 0.0550.190 ± 0.043426 ± 13
MICE, m=1m=119.93 ± 2.473.532 ± 0.0833.429 ± 0.12112 ± 3
MICE, m=5m=519.87 ± 2.933.485 ± 0.0563.383 ± 0.12155 ± 19
MICE, m=10m=1019.68 ± 2.373.488 ± 0.0523.372 ± 0.12778 ± 2

The results match Experiment 1, where MICE does best on linear tasks and worst on nonlinear ones. On Triangle-LIN, where the MICE model is correctly specified, more imputations lower the ensemble's KL (0.007→0.002→0.0010.007 \to 0.002 \to 0.001 at p=0.3p=0.3). On Triangle-NLIN, increasing mm from 1 to 10 brings no improvement. Averaging over mm imputations reduces the variance from any single imputation, but it does not fix the bias that comes from MICE's misspecified conditional models. MissCNF is slower (370–430 s vs. approximately 80 s for m=10m=10), but on nonlinear tasks the extra cost buys a large gain in accuracy. We will add this experiment to Appendix A.4.


Weakness 2 and Question 2: Accuracy on larger graphs and Monte Carlo budget.

We agree this is important, we're currently rerunning these experiments and will report the results as soon as possible.


Weakness 3 and Question 3: Why are ATE gains smaller than density and counterfactual gains?

We thank the reviewer for this question. ATE depends only on the means of two interventional distributions, while KL and counterfactuals depend on the full shape of the distribution. A baseline model can recover the means well while distorting variances, making the benefit of our method stronger in KL and counterfactual error, but much less in ATE.

However, lower KL still corresponds to lower causal error. By Javaloy et al. (2023, Thm. 1), recovering the joint identifies interventional and counterfactual queries, and this holds empirically across all 540 runs in Experiments 1–4. The Spearman correlation between KL and RMSE-ATE is ρ=0.63\rho = 0.63, and between KL and RMSE-CF it is ρ=0.73\rho = 0.73 (both p<10−59p < 10^{-59}). KL tracks counterfactual error more closely than ATE error in every experiment, which matches the explanation above. We will add this discussion and the correlation analysis to the paper.

Weakness 2 and Question 2: Accuracy on larger graphs and Monte Carlo budget.

We ran MissCNF on Chain-8, Chain-16, and Erdős–Rényi graphs ER(16,16) and ER(16,64) following DECI [1], with a training size of 1000, MAR with a missing rate of 60%, and KK from 32 up to 16384 Monte Carlo samples. We report results on Chain-8 in Table 21 on the larger graphs in Table 2, and the imputation baselines for reference in Table 3.

On Chain-8 (Table 1), accuracy improves with KK. At K=16384K = 16384, MissCNF matches the complete-data model in KL (1.61 vs. 1.62). Training time is almost constant up to K=2048K = 2048, and grows roughly linearly after that. On the larger graphs, we could only run K≤512K \le 512 due to GPU memory limits. In this range, the changes are small, as they are on Chain-8, except for a small increase in KL on ER(16,64). We therefore expect larger graphs, which have more missing ancestors per row, to also need KK in the thousands. We leave this for future work. Even with a moderate budget (Table 3), MissCNF outperforms imputation baselines on Chain-8, Chain-16 and ER(16,16). On ER(16,64), it has the lowest RMSE-ATE and is on par with the best baseline in KL and RMSE-CF.

Table 1. Effect of the Monte Carlo budget KK on Chain-8 (2 seeds).

KKKLRMSE-CFRMSE-ATETrain time
323.55 ± 0.302.46 ± 0.050.72 ± 0.070.32h
643.74 ± 0.242.46 ± 0.100.73 ± 0.030.32h
1283.48 ± 0.662.44 ± 0.040.68 ± 0.060.32h
2563.18 ± 0.822.40 ± 0.040.64 ± 0.010.33h
5123.38 ± 0.372.39 ± 0.130.67 ± 0.050.34h
10242.88 ± 0.182.33 ± 0.050.60 ± 0.050.35h
20482.58 ± 0.682.29 ± 0.010.56 ± 0.120.35h
40961.77 ± 0.202.20 ± 0.110.36 ± 0.061.17h
81922.09 ± 0.492.16 ± 0.030.34 ± 0.061.78h
163841.61 ± 0.872.09 ± 0.060.29 ± 0.022.66h

Table 2. Effect of the Monte Carlo budget K≤512K \le 512 on larger graphs.

GraphKKKLRMSE-CFRMSE-ATETrain time
Chain-16 (2 seeds)3212.69 ± 2.333.81 ± 0.031.17 ± 0.008.55h
6412.34 ± 0.713.80 ± 0.011.14 ± 0.048.49h
12812.35 ± 2.513.81 ± 0.031.17 ± 0.028.55h
25612.13 ± 0.343.80 ± 0.011.11 ± 0.108.79h
51212.23 ± 0.633.80 ± 0.011.14 ± 0.058.59h
ER(16,16) (3 seeds)328.97 ± 0.302.16 ± 0.090.72 ± 0.131.94h
649.55 ± 1.102.20 ± 0.010.72 ± 0.081.95h
1288.39 ± 0.202.13 ± 0.020.70 ± 0.101.94h
2569.46 ± 1.212.19 ± 0.060.66 ± 0.181.98h
5128.57 ± 0.082.14 ± 0.040.71 ± 0.111.98h
ER(16,64) (3 seeds)3218.26 ± 1.043.74 ± 0.051.65 ± 0.196.78h
6418.61 ± 0.673.78 ± 0.041.59 ± 0.126.82h
12819.72 ± 1.073.82 ± 0.041.63 ± 0.226.87h
25620.19 ± 0.743.81 ± 0.051.66 ± 0.127.07h
51220.56 ± 0.663.84 ± 0.021.68 ± 0.236.82h

Table 3. Comparison with imputation baselines on larger graphs (MissCNF with K=256K = 256). Best non-oracle result in bold.

GraphMethodKLRMSE-CFRMSE-ATE
Chain-8Complete data (oracle)1.62 ± 0.112.19 ± 0.050.61 ± 0.27
MissCNF3.18 ± 0.822.40 ± 0.040.64 ± 0.01
MIRACLE7.35 ± 0.142.84 ± 0.011.12 ± 0.04
KPI8.82 ± 0.193.03 ± 0.021.40 ± 0.04
MICE11.70 ± 0.263.09 ± 0.001.52 ± 0.03
Chain-16Complete data (oracle)12.30 ± 0.073.79 ± 0.011.15 ± 0.10
MissCNF12.13 ± 0.343.80 ± 0.011.11 ± 0.10
MIRACLE16.32 ± 1.153.94 ± 0.031.33 ± 0.08
KPI17.78 ± 0.944.00 ± 0.061.47 ± 0.07
MICE20.53 ± 1.073.99 ± 0.031.31 ± 0.00
ER(16,16)Complete data (oracle)11.56 ± 4.962.00 ± 0.060.62 ± 0.05
MissCNF9.46 ± 1.212.19 ± 0.060.66 ± 0.18
MIRACLE13.79 ± 1.192.61 ± 0.101.20 ± 0.11
KPI13.40 ± 0.602.77 ± 0.071.38 ± 0.11
MICE18.76 ± 1.082.98 ± 0.061.55 ± 0.05
ER(16,64)Complete data (oracle)12.16 ± 0.273.46 ± 0.071.42 ± 0.08
MissCNF20.19 ± 0.743.81 ± 0.051.66 ± 0.12
MIRACLE23.29 ± 0.393.80 ± 0.031.88 ± 0.03
KPI20.38 ± 0.193.82 ± 0.022.07 ± 0.01
MICE24.70 ± 0.123.83 ± 0.022.21 ± 0.04

[1] T. Geffner, J. Antorán, A. Foster, W. Gong, C. Ma, E. Kıcıman, A. Sharma, A. Lamba, M. Allamanis, N. Pawlowski, et al. (2024). Deep end-to-end causal inference. Transactions on Machine Learning Research (TMLR).

Dear reviewer VnhA,

Since the discussion deadline is approaching, could you kindly confirm if our response has addressed your concerns? If not, we would be happy to answer any follow-up questions and concerns.

Thank you again for your constructive review!

Thank you for your responses. I will keep my original score.

Reviewer_JV93OpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a method using normalizing flows to estimate the data generating mechanism of multivariate data under missingness, assuming a known DAG of an underlying SCM and no hidden variables. The method combines some useful computational observations that enable efficient estimations. The paper provides a theoretical sufficient condition for recovering the causal mechanism under missingness. The proposed method is evaluated on 4 different graphs.

Critical strengths

None

Critical weaknesses
  • The method requires that the DAG is known.
  • The methodological and theoretical contributions (identifiability, etc.) seem incremental. Graphical marginalization seems standard when the DAG is known. The sufficient identifiability condition works because the DAG gives the true conditional independences, so for example for the graph X_1 -> X_2 -> X_3 even if (X_1,X_2,X_3) is not observed together, if one observes (X_1,X_2) and (X_2,X_3), one can reconstruct the full distribution.
  • The evaluation is very limited: only on 4 graphs, each has very small number of nodes (3 or 4).
Questions for authors
  • How sensitive the method is to mis-specification/mis-estimation of the DAG?
  • The method should be evaluated in much larger graphs, with multiple simultaneously missing ancestors and multiple different mechanisms.
4 comments
AuthorsOpenReview ↗

Response to Reviewer JV93

We thank the reviewer for the review and the clear questions. Following the suggestions, we added experiments on larger graphs and on DAG misspecification. Our answers are below.


Weakness 1: The method requires a known DAG.

We agree that a known DAG is a strong assumption. However, it is a standard requirement in this line of work, including causal normalizing flows [1], DeCaFlow [2], DoFlow [3], Causally Consistent NF [4], and Time-Series Causal Normalizing Flow (TSCNF) [5]. These models target causal queries such as interventions and counterfactuals, which are not identifiable without structural knowledge. In applied health research, DAGs are commonly drawn from background knowledge and assumptions rather than learned from data [6, 7]. The Sachs network [8] is one example, as its graph is a consensus from the biology literature. To measure how much the method relies on a correct graph, we added a DAG misspecification experiment (see our answer to Question 1). We will state this assumption more prominently and discuss it in the limitations.

[1] Javaloy, A., Sánchez-Martín, P., & Valera, I. (2023). Causal Normalizing Flows: From Theory to Practice. Advances in Neural Information Processing Systems (NeurIPS).

[2] Almodóvar, A., Javaloy, A., Parras, J., Zazo, S., & Valera, I. (2025). DeCaFlow: A deconfounding causal generative model. Advances in Neural Information Processing Systems (NeurIPS).

[3] Wu, D., Qiu, F., & Xie, Y. (2026). DoFlow: Flow-based generative models for interventional and counterfactual forecasting on time series. International Conference on Learning Representations (ICLR).

[4] Zhou, Q., Lu, K., & Xu, M. (2025). Causally consistent normalizing flow. AAAI Conference on Artificial Intelligence (AAAI).

[5] Yeom, Y., Kim, J., Yun, T., Lyu, J., Kim, J. H., Lee, S., ... & Lee, S. (2026). Estimating interventional outcomes over time with causal normalizing flow. Uncertainty in Artificial Intelligence (UAI).

[6] Shrier, I., & Platt, R. W. (2008). Reducing bias through directed acyclic graphs. BMC Medical Research Methodology.

[7] Tennant, P. W., Murray, E. J., Arnold, K. F., Berrie, L., Fox, M. P., Gadd, S. C., ... & Ellison, G. T. (2021). Use of directed acyclic graphs (DAGs) to identify confounders in applied health research: review and recommendations. International Journal of Epidemiology.

[8] Sachs, K., Perez, O., Pe'er, D., Lauffenburger, D. A., & Nolan, G. P. (2005). Causal Protein-Signaling Networks Derived from Multiparameter Single-Cell Data. Science.


Weakness 2: Contributions seem incremental; identifiability follows from the DAG's conditional independences.

We agree with the reviewer's example: recovering the full joint from partially observed subsets relies on the conditional independences given by the DAG. Our contribution is to make this recovery work for a flexible, nonparametric model. Ordering the flow by the DAG makes these marginals tractable: non-ancestral missing variables drop out exactly, and the integral over missing ancestors can be computed by reparameterizing through the noise variables. To our knowledge, no prior work trains a causal normalizing flow on incomplete data.

Weakness 3 and Question 2: Evaluation limited to 4 small graphs.

Thank you for this suggestion. We added new experiments on larger graphs, which also address the same concern raised by reviewer Akxh. Following DECI [1], we added Erdős–Rényi graphs ER(dd, ee) with dd nodes and ee edges: ER(16,16), ER(16,64) and ER(64,64). We use a training size of 1,000, with validation and test sizes of 2,500 each, K=256K=256 MC samples, and 3 seeds. ER(64,64) is tested under MCAR, MAR and MNAR at 60% missingness, and the other two under MAR at 60%. We drop listwise deletion due to no or too few training rows. We will add this experiment to our experiment section.

MissCNF has the lowest mean error of all missing-data methods on ER(64,64) and ER(16,16) (Tables 1 and 2), with RMSE-ATE close to the oracle. On the densest graph, ER(16,64), it has the lowest KL and RMSE-ATE, while counterfactual errors are similar across methods.

Table 1. ER(64,64), N=1000N=1000, K=256K=256, 60% missingness. Mean ± std over 3 seeds; lower is better. No row is fully observed, so listwise deletion is not reported. Best missing-data method in bold.

MethodMCAR KLMCAR RMSE-ATEMCAR RMSE-CFMAR KLMAR RMSE-ATEMAR RMSE-CFMNAR KLMNAR RMSE-ATEMNAR RMSE-CF
Complete data46.48 ± 1.851.30 ± 0.241.58 ± 0.1746.48 ± 1.851.30 ± 0.241.58 ± 0.1746.48 ± 1.851.30 ± 0.241.58 ± 0.17
MissCNF45.21 ± 2.281.25 ± 0.251.55 ± 0.1748.61 ± 1.791.24 ± 0.311.54 ± 0.2152.66 ± 3.301.21 ± 0.271.52 ± 0.19
Mean55.17 ± 1.151.62 ± 0.091.75 ± 0.0763.91 ± 1.811.65 ± 0.071.77 ± 0.0574.91 ± 2.921.66 ± 0.071.78 ± 0.07
MICE53.33 ± 2.151.33 ± 0.171.59 ± 0.1055.97 ± 2.841.43 ± 0.161.63 ± 0.1360.33 ± 3.641.40 ± 0.231.65 ± 0.14
MissForest50.57 ± 2.661.38 ± 0.201.66 ± 0.0956.71 ± 1.291.42 ± 0.191.69 ± 0.1065.90 ± 1.901.44 ± 0.071.67 ± 0.04
KPI49.85 ± 1.681.49 ± 0.151.67 ± 0.1154.13 ± 0.781.54 ± 0.111.70 ± 0.0964.87 ± 1.651.50 ± 0.061.69 ± 0.05
O-MIRACLE47.11 ± 1.171.35 ± 0.251.62 ± 0.1749.79 ± 3.431.34 ± 0.271.62 ± 0.1956.01 ± 3.651.33 ± 0.191.61 ± 0.15
DiffPuter55.99 ± 0.831.64 ± 0.081.74 ± 0.0863.67 ± 1.021.70 ± 0.091.79 ± 0.0775.17 ± 3.021.70 ± 0.041.81 ± 0.04

Table 2. ER(16,16) and ER(16,64), MAR, 60% missingness, N=1000N=1000, K=256K=256. Mean ± std over 3 seeds; lower is better. Listwise deletion is only possible on ER(16,16). Best missing-data method in bold.

MethodER(16,16) KLER(16,16) RMSE-ATEER(16,16) RMSE-CFER(16,64) KLER(16,64) RMSE-ATEER(16,64) RMSE-CF
Complete data (oracle)11.56 ± 4.960.62 ± 0.052.00 ± 0.0612.16 ± 0.271.42 ± 0.083.46 ± 0.07
MissCNF9.46 ± 1.210.66 ± 0.182.19 ± 0.0620.19 ± 0.741.66 ± 0.123.81 ± 0.05
Mean16.78 ± 3.111.54 ± 0.122.98 ± 0.1725.86 ± 0.252.23 ± 0.043.86 ± 0.02
MICE18.76 ± 1.081.55 ± 0.052.98 ± 0.0624.70 ± 0.122.21 ± 0.043.83 ± 0.02
MissForest15.65 ± 1.941.34 ± 0.052.76 ± 0.0427.63 ± 0.522.01 ± 0.113.88 ± 0.03
KPI13.40 ± 0.601.38 ± 0.112.77 ± 0.0720.38 ± 0.192.07 ± 0.013.82 ± 0.02
O-MIRACLE13.79 ± 1.191.20 ± 0.112.61 ± 0.1023.29 ± 0.391.88 ± 0.033.80 ± 0.03
DiffPuter16.58 ± 3.831.54 ± 0.112.93 ± 0.1426.83 ± 1.132.13 ± 0.033.87 ± 0.03

[1] T. Geffner, J. Antorán, A. Foster, W. Gong, C. Ma, E. Kıcıman, A. Sharma, A. Lamba, M. Allamanis, N. Pawlowski, et al. (2024). Deep end-to-end causal inference. Transactions on Machine Learning Research (TMLR).

Question 1: Sensitivity to DAG misspecification.

We thank the reviewer for this question. We added an experiment where the model gets a misspecified DAG while the data are still generated from the true graph. We use an ER(16,16) graph, similar to the previous experiment. We add, delete, or reverse 10%, 20%, or 30% of the true edges, under MAR with 60% missingness on every non-root node (training size N=1000N=1000, MC samples K=256K=256, 3 seeds). All baselines use the same perturbed graph.

MissCNF is the best method in every setting on all three metrics (Table 3). Adding spurious edges hurts least, since the true parents are still included. Deleting edges degrades performance gradually, and reversing edges hurts most. At 30% reversal, RMSE-ATE rises from 0.66 to 1.37, which is still below O-MIRACLE (1.59) and MICE (1.75) given the same graph. We will add this experiment to our appendix section.

Table 3. DAG misspecification on ER(16,16), training size N=1000N=1000, MC samples K=256K=256, MAR, 60% missingness. The model and all graph-aware baselines receive the perturbed DAG; data are generated from the true DAG. Mean ± std over 3 seeds; lower is better. MissCNF achieves the lowest value in every row.

PerturbationLevelMissCNFMICEO-MIRACLE
KL
None0%9.46 ± 1.2118.76 ± 1.0813.79 ± 1.19
Addition10%9.85 ± 1.9318.48 ± 0.6214.29 ± 0.79
Addition20%8.50 ± 1.8919.07 ± 0.4013.90 ± 0.47
Addition30%9.37 ± 0.8018.51 ± 0.2014.05 ± 1.11
Deletion10%10.41 ± 2.0319.06 ± 0.9614.92 ± 2.17
Deletion20%10.86 ± 0.9219.03 ± 0.4714.84 ± 1.05
Deletion30%12.89 ± 0.8720.07 ± 0.7515.26 ± 0.81
Reversal10%13.65 ± 8.0718.73 ± 1.0514.70 ± 2.10
Reversal20%11.69 ± 1.5219.29 ± 0.3315.09 ± 1.51
Reversal30%13.56 ± 1.6219.87 ± 1.0616.18 ± 1.42
RMSE-ATE
None0%0.66 ± 0.181.55 ± 0.051.20 ± 0.11
Addition10%0.73 ± 0.131.56 ± 0.101.23 ± 0.05
Addition20%0.66 ± 0.141.58 ± 0.081.25 ± 0.03
Addition30%0.67 ± 0.061.54 ± 0.061.20 ± 0.08
Deletion10%0.95 ± 0.191.57 ± 0.041.40 ± 0.21
Deletion20%1.07 ± 0.021.62 ± 0.081.35 ± 0.06
Deletion30%1.18 ± 0.151.68 ± 0.131.49 ± 0.15
Reversal10%0.95 ± 0.441.61 ± 0.081.33 ± 0.23
Reversal20%1.04 ± 0.101.64 ± 0.041.38 ± 0.08
Reversal30%1.37 ± 0.231.75 ± 0.211.59 ± 0.13
RMSE-CF
None0%2.19 ± 0.062.98 ± 0.062.61 ± 0.10
Addition10%2.14 ± 0.102.97 ± 0.072.68 ± 0.04
Addition20%2.10 ± 0.112.99 ± 0.062.67 ± 0.03
Addition30%2.15 ± 0.052.97 ± 0.012.66 ± 0.09
Deletion10%2.28 ± 0.183.01 ± 0.052.74 ± 0.18
Deletion20%2.42 ± 0.053.02 ± 0.012.80 ± 0.07
Deletion30%2.57 ± 0.213.09 ± 0.082.91 ± 0.15
Reversal10%2.37 ± 0.343.01 ± 0.072.78 ± 0.28
Reversal20%2.54 ± 0.093.05 ± 0.022.87 ± 0.10
Reversal30%2.83 ± 0.203.14 ± 0.153.05 ± 0.16

Dear reviewer JV93,

Since the discussion deadline is approaching, could you kindly confirm if our response has addressed your concerns? If not, we would be happy to answer any follow-up questions and concerns.

Thank you again for your constructive review!

Reviewer_AkxhOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

In this paper, the authors propose an approach for performing causal inference from observational data with missing values in observations. They represent the structural causal model as a causal normalizing flow and estimate the marginal likelihood with monte Carlo estimator where they sample ancestor missing variables and maximizing the likelihood of whatever each row shows. Also under MAR, a causal-family positivity condition is establish that can recover the full joint even when no row is fully observed. Finally, empirical performance is demonstrated on synthetic datasets.

Critical strengths

The paper addresses a very important problem, i.e. how to approximate the structural causal model with CNF from missing data. The approach seemed novel to me compared to existing work. I appreciate how the authors provided intuitions after making complex statements. The figures also help understand the approach. I also appreciate the extensive empirical experiments performed in this paper.

Critical weaknesses

Below I provide my concerns.

  1. To my understanding, causal normalizing flow is trained to learn a proxy structural causal model from observational data by matching the joint distribution. However, the main goal of learning a proxy SCM is not just matching the joint distribution, rather predicting correct (identifiable )interventional and counterfactual queries. However, the main paper have very limited such discussion. Only in the experiment section, a few evaluation is performed. More theoretical and empirical analysis is required.
  2. Many important related work in the Deep Structural Causal Models literature is missing. For example, [1] estimates causal effects by parameterizing an SCM with neural networks; [2] CausalGAN represents a DAG with GAN networks mimicking an SCM and generates high-dimensional interventions; [3] Modular-DCM learns an SCM containing high-dimensional variables in a modular fashion in the presence of unobserved confounders; [4] extends NCM for continuous data. The authors should carefully compare with them qualitatively and as baselines in their experiment section to evaluate for interventional and counterfactual queries. They can be equipped with any imputation methods.
  3. The approach should be evaluated on more complicated and larger causal graphs. How does the performance scale with larger graphs?
  4. The authors did not show performance on any real-world dataset which is required to demonstrate the proposed approach performance in complex and critical setup. I would highly recommend to perform a few extensive on such datasets.
  5. The authors should provide algorithmic complexity for their approach.

I would request the authors to address the above concerns in a concise manner in their rebuttal. I am willing to increase score if properly addressed.

Questions for authors

Below I ask some questions.

  1. In Figure 1b, are tau and tau^-1 used properly?
  2. Line 216, whats the dimension of R=1_{0}?
  3. Why does MICE has the lowest KL for ChainLIN and ForkLIN in Figure 4?
  4. Why does not the same approach of Monte Carlo sampling work for unobserved confounders? What are the challenges there compared to missing mediators (both parent & child) or missing root nodes (only parents)?
8 comments
AuthorsOpenReview ↗

Response to Reviewer Akxh

We thank the reviewer for the careful review, the detailed questions, and for being open to revising the score. Below we answer each point. We ran several new experiments in response (Deep SCM baselines, larger graphs, real-world data).


Weakness 1: Causal queries are the main goal of a CNF; more theoretical and empirical analysis needed.

We agree that the ultimate goal of learning an SCM with a Causal NF is to answer counterfactual and interventional queries. Javaloy et al. (2023, Thm. 1) [1] show that under a diffeomorphic data-generating process, an acyclic causal graph, and causal sufficiency, any model in the CNF family that matches p∗(x)p^\ast(x) differs from the true SCM only by a component-wise invertible transformation of the exogenous variables. This transformation leaves interventional and counterfactual queries unchanged. Therefore, our theory in Theorem 1 focuses only on recovering p∗(x)p^\ast(x).

Empirically, all our experiments already report RMSE-ATE and RMSE-CF alongside KL (full results in Appendix C). Our results also show that matching the joint distribution leads to better predictions for interventional and counterfactual queries. Across all 540 runs in Experiments 1–4, the Spearman correlation between KL and RMSE-ATE is ρ=0.63\rho = 0.63, and between KL and RMSE-CF it is ρ=0.73\rho = 0.73 (both p<10−59p < 10^{-59}). The correlation is positive and significant in every experiment, and it is weakest under MNAR (Experiment 3: ρ=0.38\rho = 0.38 and 0.530.53), where the joint is not identified in general. We will add this analysis to our discussion.

[1] Javaloy, A., Sánchez-Martín, P., & Valera, I. (2023). Causal Normalizing Flows: From Theory to Practice. Advances in Neural Information Processing Systems, 36, 58833–58864.

Weakness 2: Missing Deep SCM literature and baselines.

We thank the reviewer for the suggestion. In this work, we focus on learning CNFs directly from incomplete data. The advantage of CNFs is that they use a single neural network to learn the full causal-generating process, unlike other deep SCM approaches that employ separate neural networks for each variable, which might suffer from error propagation [1].

However, we agree that CausalGAN [2], Modular-DCM [3], and Neural Causal Models [4] are relevant and will expand our discussion accordingly. CausalGAN supports interventions but lacks exact abduction, while Modular-DCM and NCM additionally accommodate latent confounding. To answer counterfactual queries, they also need generators that can recover the noise. On a DAG without hidden confounders, as in our setting, using an invertible generator for each variable turns both into a Deep SCM [5]. We therefore now added a comparison against the invertible, explicit Deep SCM, combined with each imputer, on four graphs under MAR at 60% (Tables 1,2).

MissCNF is best or second best on every graph and metric. On the nonlinear graphs, it has the lowest KL and counterfactual error, while MICE + Deep SCM fails on Fork-NLIN. On the linear graphs, MICE + Deep SCM is best, consistent with our experiment in Section 4.2, where MICE's linear model is correctly specified. We will add this to the appendix.

Table 1. Comparison with Deep SCM combined with imputation, linear graphs, MAR at 60%. Mean ± std over 5 runs. Lower is better. Best in bold, second best in italics (within each table).

MethodChain-LIN KLChain-LIN RMSE-CFChain-LIN RMSE-ATEFork-LIN KLFork-LIN RMSE-CFFork-LIN RMSE-ATE
Listwise deletion + Deep SCM0.850 ± 0.2030.248 ± 0.0380.167 ± 0.06411.9 ± 22.60.112 ± 0.0310.065 ± 0.034
MICE + Deep SCM0.023 ± 0.0120.081 ± 0.0080.073 ± 0.0250.018 ± 0.0060.026 ± 0.0030.034 ± 0.006
KPI + Deep SCM0.824 ± 0.5790.203 ± 0.0370.123 ± 0.0421.213 ± 0.3250.092 ± 0.0190.062 ± 0.011
O-MIRACLE + Deep SCM3.624 ± 4.2070.342 ± 0.0850.105 ± 0.0671.420 ± 0.8560.086 ± 0.0330.037 ± 0.004
MissCNF (ours)0.029 ± 0.0170.112 ± 0.0090.093 ± 0.0340.033 ± 0.0130.033 ± 0.0060.037 ± 0.007

Table 2. Same comparison, nonlinear graphs, MAR at 60%. Mean ± std over 5 runs. Lower is better. Best in bold, second best in italics (within each table).

MethodChain-NLIN KLChain-NLIN RMSE-CFChain-NLIN RMSE-ATEFork-NLIN KLFork-NLIN RMSE-CFFork-NLIN RMSE-ATE
Listwise deletion + Deep SCM1.417 ± 1.1570.152 ± 0.0320.075 ± 0.0302.020 ± 1.0840.389 ± 0.0760.603 ± 0.264
MICE + Deep SCM0.941 ± 0.5100.218 ± 0.0180.083 ± 0.02321.3 ± 1.82.664 ± 0.0602.249 ± 0.180
KPI + Deep SCM0.728 ± 0.6130.181 ± 0.0130.118 ± 0.0231.908 ± 0.6190.647 ± 0.0300.202 ± 0.026
O-MIRACLE + Deep SCM1.982 ± 1.4890.209 ± 0.0460.032 ± 0.0111.570 ± 0.7660.477 ± 0.0850.195 ± 0.120
MissCNF (ours)0.025 ± 0.0220.062 ± 0.0120.045 ± 0.0100.032 ± 0.0110.147 ± 0.0040.083 ± 0.024

[1] Javaloy, A., Sánchez-Martín, P., & Valera, I. (2023). Causal Normalizing Flows: From Theory to Practice. Advances in Neural Information Processing Systems (NeurIPS).

[2] Kocaoglu, M., Snyder, C., Dimakis, A. G., & Vishwanath, S. (2018). CausalGAN: Learning Causal Implicit Generative Models with Adversarial Training. International Conference on Learning Representations (ICLR).

[3] Rahman, M. M., & Kocaoglu, M. (2024). Modular Learning of Deep Causal Generative Models for High-dimensional Causal Inference. International Conference on Machine Learning (ICML).

[4] Xia, K., Lee, K.-Z., Bengio, Y., & Bareinboim, E. (2021). The Causal-Neural Connection: Expressiveness, Learnability, and Inference. Advances in Neural Information Processing Systems (NeurIPS).

[5] Pawlowski, N., Castro, D. C., & Glocker, B. (2020). Deep Structural Causal Models for Tractable Counterfactual Inference. Advances in Neural Information Processing Systems (NeurIPS).

Weakness 3: Larger, more complicated graphs.

Thank you for this suggestion. We added new experiments on larger graphs, which also address the same concern raised by reviewer JV93. Following DECI [1], we added Erdős–Rényi graphs ER(dd, ee) with dd nodes and ee edges: ER(16,16), ER(16,64) and ER(64,64). We use a training size of 1,000, with validation and test sizes of 2,500 each, K=256K=256 MC samples, and 3 seeds. ER(64,64) is tested under MCAR, MAR and MNAR at 60% missingness, and the other two under MAR at 60%. We drop listwise deletion due to no or too few training rows. We will add this experiment to our experiment section.

MissCNF has the lowest mean error of all missing-data methods on ER(64,64) and ER(16,16) (Tables 1 and 2), with RMSE-ATE close to the oracle. On the densest graph, ER(16,64), it has the lowest KL and RMSE-ATE, while counterfactual errors are similar across methods.

Table 1. ER(64,64), N=1000N=1000, K=256K=256, 60% missingness. Mean ± std over 3 seeds; lower is better. No row is fully observed, so listwise deletion is not reported. Best missing-data method in bold.

MethodMCAR KLMCAR RMSE-ATEMCAR RMSE-CFMAR KLMAR RMSE-ATEMAR RMSE-CFMNAR KLMNAR RMSE-ATEMNAR RMSE-CF
Complete data46.48 ± 1.851.30 ± 0.241.58 ± 0.1746.48 ± 1.851.30 ± 0.241.58 ± 0.1746.48 ± 1.851.30 ± 0.241.58 ± 0.17
MissCNF45.21 ± 2.281.25 ± 0.251.55 ± 0.1748.61 ± 1.791.24 ± 0.311.54 ± 0.2152.66 ± 3.301.21 ± 0.271.52 ± 0.19
Mean55.17 ± 1.151.62 ± 0.091.75 ± 0.0763.91 ± 1.811.65 ± 0.071.77 ± 0.0574.91 ± 2.921.66 ± 0.071.78 ± 0.07
MICE53.33 ± 2.151.33 ± 0.171.59 ± 0.1055.97 ± 2.841.43 ± 0.161.63 ± 0.1360.33 ± 3.641.40 ± 0.231.65 ± 0.14
MissForest50.57 ± 2.661.38 ± 0.201.66 ± 0.0956.71 ± 1.291.42 ± 0.191.69 ± 0.1065.90 ± 1.901.44 ± 0.071.67 ± 0.04
KPI49.85 ± 1.681.49 ± 0.151.67 ± 0.1154.13 ± 0.781.54 ± 0.111.70 ± 0.0964.87 ± 1.651.50 ± 0.061.69 ± 0.05
O-MIRACLE47.11 ± 1.171.35 ± 0.251.62 ± 0.1749.79 ± 3.431.34 ± 0.271.62 ± 0.1956.01 ± 3.651.33 ± 0.191.61 ± 0.15
DiffPuter55.99 ± 0.831.64 ± 0.081.74 ± 0.0863.67 ± 1.021.70 ± 0.091.79 ± 0.0775.17 ± 3.021.70 ± 0.041.81 ± 0.04

Table 2. ER(16,16) and ER(16,64), MAR, 60% missingness, N=1000N=1000, K=256K=256. Mean ± std over 3 seeds; lower is better. Listwise deletion is only possible on ER(16,16). Best missing-data method in bold.

MethodER(16,16) KLER(16,16) RMSE-ATEER(16,16) RMSE-CFER(16,64) KLER(16,64) RMSE-ATEER(16,64) RMSE-CF
Complete data (oracle)11.56 ± 4.960.62 ± 0.052.00 ± 0.0612.16 ± 0.271.42 ± 0.083.46 ± 0.07
MissCNF9.46 ± 1.210.66 ± 0.182.19 ± 0.0620.19 ± 0.741.66 ± 0.123.81 ± 0.05
Mean16.78 ± 3.111.54 ± 0.122.98 ± 0.1725.86 ± 0.252.23 ± 0.043.86 ± 0.02
MICE18.76 ± 1.081.55 ± 0.052.98 ± 0.0624.70 ± 0.122.21 ± 0.043.83 ± 0.02
MissForest15.65 ± 1.941.34 ± 0.052.76 ± 0.0427.63 ± 0.522.01 ± 0.113.88 ± 0.03
KPI13.40 ± 0.601.38 ± 0.112.77 ± 0.0720.38 ± 0.192.07 ± 0.013.82 ± 0.02
O-MIRACLE13.79 ± 1.191.20 ± 0.112.61 ± 0.1023.29 ± 0.391.88 ± 0.033.80 ± 0.03
DiffPuter16.58 ± 3.831.54 ± 0.112.93 ± 0.1426.83 ± 1.132.13 ± 0.033.87 ± 0.03

[1] T. Geffner, J. Antorán, A. Foster, W. Gong, C. Ma, E. Kıcıman, A. Sharma, A. Lamba, M. Allamanis, N. Pawlowski, et al. (2024). Deep end-to-end causal inference. Transactions on Machine Learning Research (TMLR).

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of revisions

We thank all reviewers for their time and constructive feedback, which has substantially improved our manuscript. We have addressed all comments from the reviews and uploaded a revised version of the paper, with all changes marked in blue. Below we summarize the main changes and the comments they address.

New experiments

  1. Larger graphs (Akxh W3, JV93 W3/Q2, VnhA W2). We added Erdős–Rényi graphs ER(16,16), ER(16,64), and ER(64,64) under MAR at 60% to the main text (Sec. 4.4). MissCNF has the lowest KL and lowest RMSE-ATE on all three graphs. Full results, including ER(64,64) under MCAR and MNAR at 60%, are in Appendix C.3.
  2. Real-world data (Akxh W4). We added Light Tunnel (from the Causal Chamber testbed) and Sachs to the main text (Sec. 4.5), with MCAR and MAR at 30% and 60%. Since the ground-truth SCM is unknown, we report held-out NLL and MMD. MissCNF performs best on Light Tunnel. On Sachs, it is second to MICE in every setting. Full results are in Appendix C.4.
  3. Deep SCM baselines (Akxh W2). We added a comparison with Deep SCM trained on complete cases or on data imputed by MICE, KPI, and O-MIRACLE (Appendix D.3). MissCNF is best or second best on every task and metric, and best in KL and RMSE-CF on the nonlinear tasks.
  4. Multiple-imputation ensemble (VnhA W1/Q1). We added a comparison with ensembles of m ∈\in {1, 5, 10} CNFs, each trained on a separate MICE imputation, including training cost (Appendix D.4). The discussion of multiple imputation in Appendix A.4 now points to it.
  5. DAG misspecification (JV93 Q1). We added an experiment where 10–30% of the edges are added, deleted, or reversed (Appendix D.5). MissCNF has the lowest error in every setting.

Text changes

  1. Causal queries and KL (Akxh W1, VnhA W3/Q3). We now highlight in the background (Sec. 2) and the experimental setup (Sec. 4.1) that matching the joint distribution is sufficient under the CNF assumptions. We also report the Spearman correlation of KL with RMSE-ATE (0.63) and with RMSE-CF (0.73) over all runs on the small SCMs (Appendix D.1).
  2. Related work (Akxh W2). We added a paragraph on causal generative models (Sec. 2) that clarifies how CNFs differ from other deep structural causal models.
  3. Limitations (JV93 W1, Akxh Q4, VnhA W2). The limitations now discuss the known-DAG assumption, unobserved confounders, and the Monte Carlo cost on dense graphs.
  4. Other fixes (Akxh Q1–Q3). We corrected τ−1\tau^{-1} in Fig. 1b, stated R=1O∈0,1dR = \mathbf{1}_O \in \\{0,1\\}^d, and added a sentence on why MICE performs best on the linear SCMs.

We also updated the abstract and the list of contributions to reflect these changes.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.