acceptodds
Under review as a conference paper at ICLR 2027

Causal Posterior Estimation

Abstract

We present Causal Posterior Estimation (CPE), a novel method for Bayesian inference in simulator models, where evaluating the likelihood function is intractable or computationally expensive, but generating outputs given parameter values is straightforward. CPE approximates the posterior distribution using flow matching while directly incorporating the conditional dependence structure induced by the model's graphical representation into the neural network architecture. Across extensive experiments, we demonstrate that hard-coding these conditional dependencies into the network, rather than requiring them to be learned from data, enables CPE to achieve highly accurate posterior inference that matches or outperforms state-of-the-art baselines.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
simulation-based inference, Bayesian inference, flow matching, causality

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_ejAgOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Causal Posterior Estimation (CPE), which uses the posterior program to constrain the neural network used for flow matching. The method combines causal block-autoregressive layers with the prior as the base distribution and learnable skip connections. The authors evaluate CPE on several SBI benchmarks and report competitive posterior accuracy with fewer trainable parameters than the compared baselines.

Critical strengths
  1. The paper addresses a relevant question: how to use known model structure when designing a posterior estimator. It provides a concrete architecture for incorporating this information.

  2. The experiments cover several SBI benchmarks and include a comparison with BNPE to examine the effect of causal masks. The reported results provide evidence that masking can help on several tested problems.

Critical weaknesses
  1. The posterior conditional independencies is not directly implied by the constraints on the velocity field. Lemma 5 establishes a block lower-triangular Jacobian, but does not establish that the resulting posterior satisfies the claimed conditional independencies. It is necessary to have a further proof that the velocity constraints implies actual conditional independence structures in the resulting posterior distribution. The authors also need to explain whether these constraints are compatible with the linear interpolation used for flow matching, or whether they introduce posterior approximation bias.

  2. The novelty over existing structured inference architectures is unclear. Using graphical model structure to constrain continuous normalizing flows has already been explored, e.g. Weilbach et al. (2020). The authors should clarify what their proposed architecture adds. Although the BNPE comparison evaluates the structural masks, the separate benefits of using the prior as the base distribution and introducing learnable skip connections remain unclear.

  3. In the SIR experiment, the information loss due to non-sufficient summary statistics is not separated from posterior estimation error. All methods are trained on these summaries, but the reference posterior is computed from the full time series. The information loss of non-sufficient summary statistics, which is not measured here, might contribute to posterior learning error. This issue has already been discussed in the literature. Joyce and Marjoram (2008) discusses how conditioning on non-sufficient summary statistics can yield a posterior different from that based on the full data.

  • Joyce and Marjoram (2008). Approximately Sufficient Statistics and Bayesian Computation

A counterexample illustrating Weakness 1. Fix the observation xx and consider the posterior program θ3→θ2→θ1\theta_3\to\theta_2\to\theta_1, which requires

θ1⊥θ3∣θ2,x.\theta_1\perp\theta_3\mid\theta_2,x.

Let the base distribution be θ(0)=U∼N(0,I3)\theta(0)=U\sim N(0,I_3), and consider

\dot\theta_2(t)=\theta_3(t),\qquad \dot\theta_1(t)=\theta_2(t).$$ Each velocity component depends only on its direct parent. Its Jacobian is therefore block lower triangular in the ordering $(\theta_3,\theta_2,\theta_1)$, satisfying an even stronger restriction than the ancestor dependence allowed in Lemma 5. The exact solution at $t=1$ is $$\theta_3=U_3,\qquad \theta_2=U_2+U_3,\qquad \theta_1=U_1+U_2+\tfrac12U_3.$$ However, the resulting Gaussian distribution has $$\text{Cov}(\theta_1,\theta_3\mid\theta_2,x) = \text{Cov}(\theta_1,\theta_3) - \frac{\text{Cov}(\theta_1,\theta_2)\text{Cov}(\theta_2,\theta_3)}{\text{Cov}(\theta_2)}.$$ Substituting the covariances gives $$\text{Cov}(\theta_1,\theta_3\mid\theta_2,x) = \frac12-\frac{(3/2)\cdot1}{2} = -\frac14\neq0.$$ Thus, the required conditional independence does not hold, even though the velocity field respects the direct-parent dependencies. The issue is that $\theta_1$ depends on the entire trajectory of $\theta_2$, which carries information about $\theta_3$. Conditioning on the terminal value of $\theta_2$ does not generally remove this dependence. This example uses an exact ODE solution, so numerical integration error is not responsible. This does not show that a trained CPE model necessarily violates the desired conditional independencies. It shows that the velocity constraints alone do not guarantee them. The authors should clarify what additional conditions connect the restrictions established in Lemma 5 to the posterior factorization described in Section 3.2.
Questions for authors
  1. Could the authors report training and posterior sampling times for CPE and the baselines on the same hardware, alongside posterior accuracy? Sampling time should include the cost of discarding and replacing samples outside the prior support.

  2. On a Gaussian model with an analytically known correlated posterior, does the covariance error decrease toward zero as the simulation budget increases, both with and without the structural masks?

Reviewer_rqWHOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper introduces "causal poterior estimation" to find a posterior distribution of parameter given a black-box simulation. It introduces a masking mechanism over the flow matching process to the flow-based inference algorithm.

Critical strengths

I don't see any.....

Critical weaknesses

The posterior of Figure 1 does not necessarily have to be the dependence of \theta_2 from \theta_3. The relation can be in the reverse direction. The current diagram is just a selection out of many possible scenario, so it needs to be mentioned in the caption.

Regarding Figure 5 and Figure 6, authors claim "CPE performs or matches all baseline on all eight benchmark models...". The wording "matches" is very ambiguous. I would mention that BNPE could work better in some cases.

This method may not use the parameter as much as the compared models as authors argue "CPE reaches this performance with generally fewer...". However, the author does not mention how much computation is being utilized. The most gain would come from the block augmentation and masking. As more fine masking is given, the K causal block layers increases. That will make the Figure 2 computation chain longer.

Having said that, the K causal block layer needs to be ablated to get to know its effect. Authors are experimenting this method with simple benchmark distributions and simulation, so they do not test the variation of K, but if we test more realistic distribution/simulation, you would have some versions of K-causal blocks that need to be ablation studied.

All these experiments are very hypothetical and simple models regarding simulaiton or evaluation distribution. There is no significant real world benchmark evaluations. I know that the underlying true posterior distribution may not be observable in the real world, but your posterior distribution should be accessible and has some meaning to the dataset.

In general, this paper assumes a certain posterior inference structure, that does not necessarily to be causality model. The title and the usage of "causality" is essentially not a correct choice in the given method. "Causality" will require "do" operation to hypothesize a counterfactual case that the dataset does not reveal. The current version is just strengthening a particular sampling relations in the posterior model. This could be "structured", but that research is well researched in the proabilistic modeling community.

Questions for authors

please answer to my weakness section.

Reviewer_WnYXOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes a method that encodes the known conditional dependence structure that is defined in the data model (and therefore simulation) – a hierarchical or graph model typically is used to define this structure, which the authors call "causal". This inductive bias, knowing the structure of the inverted graph (the posterior program), is implemented architecturally with the aim of improving simulation-based inference.

It is implemented architecturally, as flow matching, in two different ways: using the prior as the flow's base distribution (with skip connections), and an explicit block structure in the autoregressive layers following the graph structure. The block structure is via a combination of posterior factorisation and a masking strategy, plus a "Block-Matrix Augmentation" taken from block neural autoregressive flows.

This is tested on standard benchmarks against alternatives measured against MCMC references with H-min and C2ST. There is also a comparison of the different solver choices in the proposed model (for the integrator of the flow-type velocity), and a claim of fewer trainable parameters.

Critical strengths

This paper describes a well-motivated, principled approach. The foundations are described well and match the stated problem. The implementation appears sound.

The motivation to solution link is very clear. The particular inductive bias, which may appear for a particular simulation-based inference or density estimation task, is implemented directly.

The paper includes a fair assessment of the state of the art and alternatives in the related work (clearly describing which aspects of encoding structure is not new).

The paper includes experiments with standard benchmarks, and perform fair comparison (same training protocol for all methods), and the results, according to the metrics, are good.

The paper acknowledges the limitations, in particular scaling with the size of the parameter space, and the naive graph inversion. These are important limitations, but the authors are clear about them.

Critical weaknesses

The results, though promising, are not overwhelming or a clear winner on all fronts. All-in-One Posterior Estimation (AIO) outperforms the proposed method on SIR in some cases. Similarly, the alternatives CPE on the hierarchical model at the smallest "budget", which is in fact the task chosen to demonstrate the novel architectural/causal structure.

Also, the ablation is limited, which makes it hard to distinguish different effects, and hard for a reader to clearly see which (if any) of the different contributions are directly the improvements. In reality, only the mask is really ablated. No ablation of prior- base distribution or of other aspects (e.g. the convex combination?).

Questions for authors

Given the method implements the causal structure, why does it fail compared to the alternatives on the hierarchical model?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.