- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes Causal Posterior Estimation (CPE), which uses the posterior program to constrain the neural network used for flow matching. The method combines causal block-autoregressive layers with the prior as the base distribution and learnable skip connections. The authors evaluate CPE on several SBI benchmarks and report competitive posterior accuracy with fewer trainable parameters than the compared baselines.
Critical strengths
-
The paper addresses a relevant question: how to use known model structure when designing a posterior estimator. It provides a concrete architecture for incorporating this information.
-
The experiments cover several SBI benchmarks and include a comparison with BNPE to examine the effect of causal masks. The reported results provide evidence that masking can help on several tested problems.
Critical weaknesses
-
The posterior conditional independencies is not directly implied by the constraints on the velocity field. Lemma 5 establishes a block lower-triangular Jacobian, but does not establish that the resulting posterior satisfies the claimed conditional independencies. It is necessary to have a further proof that the velocity constraints implies actual conditional independence structures in the resulting posterior distribution. The authors also need to explain whether these constraints are compatible with the linear interpolation used for flow matching, or whether they introduce posterior approximation bias.
-
The novelty over existing structured inference architectures is unclear. Using graphical model structure to constrain continuous normalizing flows has already been explored, e.g. Weilbach et al. (2020). The authors should clarify what their proposed architecture adds. Although the BNPE comparison evaluates the structural masks, the separate benefits of using the prior as the base distribution and introducing learnable skip connections remain unclear.
-
In the SIR experiment, the information loss due to non-sufficient summary statistics is not separated from posterior estimation error. All methods are trained on these summaries, but the reference posterior is computed from the full time series. The information loss of non-sufficient summary statistics, which is not measured here, might contribute to posterior learning error. This issue has already been discussed in the literature. Joyce and Marjoram (2008) discusses how conditioning on non-sufficient summary statistics can yield a posterior different from that based on the full data.
- Joyce and Marjoram (2008). Approximately Sufficient Statistics and Bayesian Computation
A counterexample illustrating Weakness 1. Fix the observation and consider the posterior program , which requires
Let the base distribution be , and consider
\dot\theta_2(t)=\theta_3(t),\qquad \dot\theta_1(t)=\theta_2(t).$$ Each velocity component depends only on its direct parent. Its Jacobian is therefore block lower triangular in the ordering $(\theta_3,\theta_2,\theta_1)$, satisfying an even stronger restriction than the ancestor dependence allowed in Lemma 5. The exact solution at $t=1$ is $$\theta_3=U_3,\qquad \theta_2=U_2+U_3,\qquad \theta_1=U_1+U_2+\tfrac12U_3.$$ However, the resulting Gaussian distribution has $$\text{Cov}(\theta_1,\theta_3\mid\theta_2,x) = \text{Cov}(\theta_1,\theta_3) - \frac{\text{Cov}(\theta_1,\theta_2)\text{Cov}(\theta_2,\theta_3)}{\text{Cov}(\theta_2)}.$$ Substituting the covariances gives $$\text{Cov}(\theta_1,\theta_3\mid\theta_2,x) = \frac12-\frac{(3/2)\cdot1}{2} = -\frac14\neq0.$$ Thus, the required conditional independence does not hold, even though the velocity field respects the direct-parent dependencies. The issue is that $\theta_1$ depends on the entire trajectory of $\theta_2$, which carries information about $\theta_3$. Conditioning on the terminal value of $\theta_2$ does not generally remove this dependence. This example uses an exact ODE solution, so numerical integration error is not responsible. This does not show that a trained CPE model necessarily violates the desired conditional independencies. It shows that the velocity constraints alone do not guarantee them. The authors should clarify what additional conditions connect the restrictions established in Lemma 5 to the posterior factorization described in Section 3.2.Questions for authors
-
Could the authors report training and posterior sampling times for CPE and the baselines on the same hardware, alongside posterior accuracy? Sampling time should include the cost of discarding and replacing samples outside the prior support.
-
On a Gaussian model with an analytically known correlated posterior, does the covariance error decrease toward zero as the simulation budget increases, both with and without the structural masks?