acceptodds
Under review as a conference paper at ICLR 2027

Scalable Diffusion SBI for Compositional Inference under Simulator Misspecification

Abstract

Simulation-based inference (SBI) is challenging when many heterogeneous observations must be composed, hierarchical latent structure must be preserved, and the simulator is misspecified relative to observed data. We develop sampling and fine-tuning methods for diffusion-based SBI in design-conditional settings, where the same simulator is queried across different experimental conditions . We extend compositional score-based inference with a continuous-time diffusion coefficient that accounts for the number of observations, avoiding Jacobian and auxiliary-covariance corrections. We introduce Hierarchical Blockwise Diffusion Sampling (HBDS), which infers shared parameters and group-specific latent states using a single pretrained (PT) model, with the hierarchy specified only at sampling time. Together, these methods support variable observation sets and groupings without retraining. To address misspecification, we introduce path-regularized fine-tuning that adapts the learned likelihood to observations and transfers corrections to posterior inference. Using Girsanov's theorem, we quantify path divergence between PT and fine-tuned (FT) models across experimental designs and interpret it alongside predictive errors to distinguish candidate misspecification correction from unnecessary adaptation. We evaluate compositional sampling on exact-score Gaussian and SLCP benchmarks, HBDS with analytic and learned scores on a controlled hierarchical model, and fine-tuning and localization on a separate analytic model with known design-dependent discrepancy. Finally, we apply the framework to 940 measurements across four cell lines in a mechanistic Bone Morphogenetic Protein (BMP) signaling model, where fine-tuning improves both posterior-predictive accuracy and posterior parameter inference relative to the PT model.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
simulation-based inference, amortized Bayesian inference, diffusion models, compositional inference, simulator misspecification, hierarchical inference, stochastic differential equations, scientific machine learning, biophysical modeling, systems biology

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_fC6fOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a diffusion-based SBI framework to address three challenges: combining multiple observations, handling hierarchical latent variables, and simulator misspecification. The authors extend existing compositional sampling methods with an observation-count-dependent diffusion coefficient (F-NPSE-SDE). They also introduce HBDS to infer shared and group-specific parameters using a single pretrained model. To address simulator misspecification, they develop a fine-tuning method and use path-space divergence to measure model adaptation. The methods are evaluated on synthetic benchmarks and a biological model with 940 observations across four cell lines.

Critical strengths
  1. This paper extends existing compositional sampling methods with a new diffusion coefficient and introduces HBDS for hierarchical inference. These methods allow a single pretrained model to handle different observation sets and hierarchical structures, while avoiding some of the computational costs of existing sampling methods.

  2. The paper evaluates the proposed framework on a biological signaling model with 940 observations across four cell lines. The results show improvements in predictive performance in several experimental settings and demonstrate the potential of the framework for real-world scientific inference.

Critical weaknesses
  1. Limited improvements and comparisons with existing methods. Although F-NPSE-SDE improves upon F-NPSE on SLCP, the improvement is relatively modest, and it remains less accurate than GAUSS and JAC (Table 1). Moreover, HBDS is mainly compared with pooled sampling, rather than existing hierarchical SBI methods. These results make it difficult to assess whether the proposed methods offer substantial improvements over existing approaches.

  2. Limited theoretical guarantees. Although the paper derives a new diffusion coefficient and provides a path-space analysis using Girsanov's theorem, it lacks theoretical guarantees for the accuracy of the proposed inference algorithms. In particular, it is unclear whether F-NPSE-SDE and HBDS converge to the correct target posterior, even with exact scores and sufficiently small sampling steps. The absence of convergence or approximation error guarantees limits the theoretical support for the proposed methods.

Questions for authors
  1. Can the authors provide stronger comparisons with existing hierarchical SBI methods for HBDS? Also, given the limited accuracy improvement of F-NPSE-SDE on SLCP, could the authors clarify its advantages in terms of posterior accuracy versus computational cost?
  2. Can the authors provide theoretical justification for the correctness of F-NPSE-SDE and HBDS? In particular, under exact scores and vanishing discretization steps, do these methods converge to the target posterior? If not, can the authors characterize the approximation error?
3 comments

We thank the reviewer for their time and thoughtful review, and are glad they found the paper clear and easy to follow. We believe the revised manuscript addresses the main concerns and substantially strengthens both the experimental and theoretical contributions.

Response to weaknesses:

  • As discussed in our main rebuttal and response to reviewer RjX1, we corrected a sampling implementation bug and substantially improved F-NPSE-SDE's performance. It now achieves accuracy comparable to GAUSS and JAC on the Gaussian and SLCP benchmarks, with improvements on some sW and MMD metrics. We also added comparisons with CSM-Damped from Arruda et al. (2026) and a computational complexity analysis in section A.8, demonstrating the practical advantages of avoiding Jacobian and covariance calculations.
  • We expanded the theoretical analysis in section A.4 to explain when Langevin corrections reduce sampling error. Proposition A.2 bounds the Wasserstein distance to the reference intermediate posterior marginal, separating composed-score error, incoming predictor error, and Euler discretization error. The derived coefficient gn(t)g_n(t) controls how strongly score and discretization errors affect each correction, but also influences the contraction rate. Corollaries A.3 and A.4 extend this analysis to LL corrections and establish sufficient conditions for error reduction. We also added Proposition B.1, establishing exact recovery of the hierarchical posterior for idealized HBDS under exact conditional probability flows and mixed-time conditioning.

Response to questions:

  1. We implemented the hierarchical AR(1) benchmark of Arruda et al. (2026), finding that HBDS achieves lower global parameter NRMSE than the published Hierarchical-CSM result, although local parameter inference remains less accurate. As discussed in the main rebuttal, HBDS's main advantage is introducing hierarchical structure at sampling time using a single pretrained joint model. F-NPSE-SDE now achieves competitive posterior accuracy without the Jacobian or covariance estimation costs of GAUSS and JAC, as demonstrated by the complexity analysis and BMP runtimes in section A.8.

  2. The new theoretical results in sections A.4 and B.1 clarify the guarantees and remaining approximations. Proposition B.1 shows that idealized HBDS recovers the correct hierarchical posterior under exact conditional flows, whereas practical HBDS uses learned scores and finite sampling steps. For F-NPSE-SDE, our Wasserstein bounds characterize how composed-score approximation and numerical error affect the intermediate marginals, but do not establish general convergence to the exact posterior under vanishing step size alone. Extending these bounds to the alternating global and local updates of practical HBDS is an important direction for future work.

Thank you for the detailed responses and additional experiments. I appreciate the new comparisons and theoretical analysis, which address most of my concerns. While the theoretical guarantees are still limited, I think the revised experimental results provide stronger support for the proposed methods. I will raise my rating from 2 to 3.

AuthorsOpenReview ↗

Additional Analysis of Score and Conditioning Errors in HBDS

We thank the reviewer for their time and appreciate their updated assessment of the revised experiments and theoretical analysis. Following their question about approximation error, we were also curious about this relationship between error in the score and conditioning values. Thus, we investigated this and added Corollary B.2 after Proposition B.1 to examine how Tweedie conditioning affects a single-observation HBDS block update.

The corollary separates score estimation error, error in the supplied denoising estimate, and the remaining gap from replacing uncertainty about the clean global state with a single conditioning value via a telescoping decomposition. The ideal HBDS update in Proposition B.1 uses the exact mixed-time score, which averages clean-conditioned scores over plausible clean global states given the noisy global state, the current local state, and the observation. Corollary B.2 bounds the discrepancy between the learned Tweedie-conditioned score and this exact score, making the score-error term in Proposition A.2’s one-step Wasserstein bound explicit for a local HBDS correction.

This helps distinguish several cases where different improvements may be useful:

  • Concentrated conditional posterior and accurate denoising. When the Tweedie estimate is close to the conditional mean and the remaining spread is small, the conditioning approximation can be accurate. Improving the learned score then addresses the remaining score estimation error.
  • Inaccurate denoising. Even an accurate clean-conditioned score model can receive a poor conditioning value. Moving the estimate closer to the relevant conditional mean tightens the denoising contribution to the bound, with the size of the improvement controlled by the score’s sensitivity to that value.
  • Broad or multimodal conditional posterior. Even the exact mean can lie between modes. If the clean score varies nonlinearly across plausible global states, evaluating it at the mean can differ from averaging those scores. This motivates exploring multiple conditioning values and their conditional weights during sampling.

These cases suggest concrete directions for improving HBDS. Flow maps could make this exploration more practical by reducing the cost of generating multiple candidate clean conditioning states within each HBDS update. We leave this for future work.

Finally, we are also revising the notation throughout the HBDS derivation and algorithm to represent group-specific latent variables generically as rkr_k, reserving receptor-specific notation for the BMP application. This clarification emphasizes that HBDS is applicable to hierarchical SBI problems beyond receptor inference, with the BMP model serving as one scientific application.

Reviewer_RjX1OpenReview ↗
Rating
4(+2)
Confidence
2
Informativeness
2
Presentation
2
Soundness
3(+1)
Read review
Paper summary

This paper contributes solutions to three problems in simulation-based inference: First, the problem of simulator misspecification, for which the authors propose a fine-tuning method based on pathspace-KL divergence; second, the problem of compositional sampling across many different observations, for which they propose an SDE-based extension of a prior compositional sampling algorithm; and third, the problem of hierarchical structure in the latent variables to be estimated, for which they propose a block-based sampler, similar to Gibbs sampling, that alternates between updating group-specific and globally shared latent variables. The authors validate these methods on several toy models, a simple likelihood/complex posterior example, and a real-world biological model.

Critical strengths
  1. The use of path-space KL to regularize the fine tuning the simulator provides a mathematically motivated method for reducing simulator misspecification. The paper demonstrates that the path-space divergence can identify areas of design space where the surrogate undergoes substantial change, and thus could provide a useful diagnostic for simulator misspecification.

  2. The results on the BMP example provide evidence that the proposed framework is applicable to realistic scientific inference problems, and the results in general show improvement relative to the authors' selected baselines.

Critical weaknesses

Overall, I am unconvinced by the experimental results of the utility of the proposed methods:

  • The paper proposes two new sampling algorithms, F-NSPE-SDE and HBDS. I am not convinced that the experimental results adequately demonstrate their advantages over existing methods. Section 4.1 shows that F-NPSE-SDE performs similarly to the original F-NPSE, and shows only modest improvement in the SLPC example, and relatively large reported standard deviations make it unclear if that improvement is statistically significant. Comparable performance would be reasonable if F-NPSE-SDE offered other advantages over F-NPSE, but it is unclear from the presentation what those advantages are.

  • The authors also compare to JAC and GAUSS, and these methods generally achieve better sampling accuracy, especially on SLCP. F-NPSE-SDE does not incur costs from estimating the Jacobian or covariance, which could be an important advantage, but beyond some runtime observations for the BMP application in the appendix, a rigorous comparison of computational cost is missing from the paper.

  • The paper compares HBDS to pooled compositional sampling. This comparison shows that HBDS performs better in a toy setting in which different groups have different latent values. Since pooled compositional sampling assumes all observations share the same latents, it is naturally ill-suited to this setting. This shows that HBDS can handle a setting that pooled compositional sampling can't, but missing is a comparison to existing hierarchical SBI methods can handle this situation: [1], for example, already propose a diffusion-based method for hierarchical SBI. A comparison with this method or others would help isolate the contribution of the HBDS sampler.

[1] Jonas Arruda, Vikas Pandey, Catherine Sherry, Margarida Barroso, Xavier Intes, Jan Hasenauer, and Stefan T. Radev. Compositional amortized inference for large-scale hierarchical bayesian models. In International Conference on Learning Representations, 2026.

Questions for authors

Below are the outstanding questions I have after reading the paper, along with related questions that could help me understand the answer:

  1. In what way is F-NPSE-SDE a better sampling algorithm than F-NPSE-SDE? Given that F-NPSE-SDE doesn't seem to significantly outperform F-NPSE in the demonstrated examples, could the authors clarify what the principal advantage of F-NPSE-SDE is over F-NPSE? It remains unclear to me if the main benefit is intended to be improved sample accuracy, computational efficiency, or something else.

  2. Is F-NPSE-SDE a better sampling algorithm than existing SDE-based compositional samplers? Could the authors provide comparisons to existing SDE-based compositional samplers, such as [1]?

  3. Is F-NPSE-SDE more computationally efficient for a given level of accuracy? Can the authors provide a more direct comparison of the computational efficiency of F-NPSE-SDE to that of JAC, GAUSS, or other relevant methods? As it stands, JAC and GAUSS seem to out-perform F-NPSE-SDE in accuracy, so it is unclear to me why F-NPSE-SDE should be preferred.

  4. Is HBDS a better hierarchical inference algorithm than existing methods? Could the authors provide comparisons to existing hierarchical SBI methods such as that of [1]? A comparison with an existing sampler that can handle the setting of 4.2 would help to isolate the contribution of HBDS.

2 comments

We thank the reviewer for their warranted skepticism and we believe the updated results in the main paper will alleviate this skepticism. We address their 3 critical weaknesses below:

  • We have included a comparison to the compositional sampling (CSM-Damped) of Arruda et al. 2026 to F-NPSE-SDE in the main paper Table 1, showing how it performs. As mentioned in the main rebuttal, we spotted a bug in our implementation and improved our results even further (F-NPSE also improved since it shared the same prior). We now see consistent improvement over F-NPSE in both Gaussian and SLCP tasks, as well as competitiveness with JAC/GAUSS on both and CSM-Damped - without the required Bayesian optimization, as discussed in the response to reviewer ZH1S. We believe this answers this concern.
  • We added a more detailed comparison of the time and memory complexity of F-NPSE-SDE against JAC/GAUSS in Appendix A.8, using the motivating BMP model as an example. For BMP, each HBDS round requires approximately 61,100 reverse-mode differentiation passes per particle and diffusion step with JAC, compared with 11,280 ordinary score evaluations for F-NPSE-SDE (L=5L=5). Their approximate computational-work ratio is (2Cs+65Cr)/(12Cs)(2C_s+65C_r)/(12C_s), excluding JAC's additional matrix operations, explaining our previous observation that JAC took 10 hours to sample rather than an hour using HBDS. GAUSS faces a different bottleneck, that is, it generates MM preliminary posterior samples per observation through SS-step DDIM rollouts to estimate covariance corrections before compositional sampling. In HBDS, alternating global and local conditioning can require repeating these rollouts as the conditioning variables change. Under fully batched, particle-specific covariance estimation for the global block, auxiliary memory can exceed HBDS particle-state storage by a factor of 705M+42,300705M+42{,}300. These estimates help explain the observed computational challenges we previously observed for JAC and GAUSS on the BMP model.
  • As discussed in the main rebuttal, we implemented the hierarchical AR(1) benchmark of Arruda et al. (2026) to address the reviewer's concern. HBDS achieves lower global parameter NRMSE than the published Hierarchical-CSM results, although it performs worse on local parameter inference. These results use independently generated test sets, so we interpret the comparison cautiously and investigate the local performance gap in Appendix B.3. Importantly, the contribution of HBDS is not hierarchical inference itself, but the ability to introduce hierarchical structure at sampling time using a single pretrained joint model, rather than training separate global and local score networks as in Hierarchical-CSM. This also supports our fine-tuning framework, where likelihood adaptation transfers to posterior inference through shared representations. Extending Hierarchical-CSM to support this transfer may be possible, but would require additional development. Thus, HBDS offers a complementary approach that prioritizes flexibility and reuse of the pretrained model across inference tasks.

Answering the reviewer's questions:

  1. Following correction of the prior-score implementation bug, F-NPSE-SDE now substantially outperforms F-NPSE and Langevin on SLCP, reducing sW from 0.82 to 0.52 and MMD from 0.11 to 0.04 relative to F-NPSE. On Gaussian, it achieves comparable accuracy to F-NPSE, GAUSS, and JAC. Its principal advantage is accurate, stable compositional sampling without expensive Jacobian or covariance calculations. Our new theoretical analysis (section A.4) further establishes conditions under which the derived schedule and Langevin corrections reduce sampling error.
  2. As requested, we implemented CSM-Damped from Arruda et al. (2026), including 20 Bayesian optimization trials. F-NPSE-SDE achieves better max-sW on Gaussian (0.24 vs. 0.75). On SLCP, CSM-Damped has slightly better sW (0.50 vs. 0.52), while F-NPSE-SDE has better MMD (0.04 vs. 0.06). Unlike CSM-Damped's tuned log-SNR shift, our adjustment is determined analytically by diffusion time and observation count (section A.9).
  3. We expanded section A.8 with time and memory complexity comparisons, including the additional cost of JAC's score differentiation and GAUSS's preliminary DDIM rollouts for covariance estimation. F-NPSE-SDE achieves accuracy comparable to JAC/GAUSS on our benchmarks without these operations. On BMP, F-NPSE-SDE completed HBDS in approximately one hour, JAC required over ten hours, and GAUSS exhausted GPU memory.
  4. We implemented the hierarchical AR(1) benchmark of Arruda et al. (2026). HBDS achieves lower global NRMSE than the published Hierarchical-CSM result (0.071 vs. 0.080), but higher local NRMSE (0.228 vs. 0.100), using independently generated test sets. HBDS's distinctive contribution is introducing hierarchical structure at sampling time using a single pretrained joint model, which also supports our likelihood-to-posterior fine-tuning framework.

Thanks to the authors for their work to address my concerns. I think the added results substantially strengthen the paper, and as such I will raise my score to a 4.

Reviewer_ZH1SOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper tackles Simulation-based inference and introduces Hierarchical Blockwise Diffusion Sampling (HBDS), which infers shared parameters and group-specific latent states using a single pre-trained model. The paper derives an observation-count-dependent diffusion coefficient that enables efficient compositional SBI without expensive Jacobian or covariance calculations. The experiments show practical advantage over existing methods.

Critical strengths

This paper has strengths overall and instead of a single strength, the "package" is good: The problem is well motivated and proposed methodology appears sound overall. The presentation is good and the paper reads well. Topic-wise this is a good fit for the conference and it should be of interest to at least parts of the ICLR audience.

Critical weaknesses

These are not "critical" weaknesses but rather points to raise for the discussion phase:

  1. The individual components/contributions in the model build on existing diffusion/SBI methods, compositional inference, and fine-tuning. This is clearly also said/implied in the paper, but worth pointing out here.

  2. There is only limited means for theoretical analysis of the approximations in the method, and therefore including some more quantitative analysis in the experiments would give more insights.

Questions for authors

If my understanding is correct, the method relies on the construction of approximate intermediate distributions. How does the effect of this show in the experiments you conducted?

2 comments

We thank the reviewer for their positive feedback. Indeed, our suite of methods arose from the challenges of fitting the BMP model using existing SBI approaches, and we appreciate the recognition of this combined contribution. We promoted Table 3 to the main paper, showing that fine-tuning with our FiLM architecture and sampling with HBDS achieves the lowest predictive errors among the ablations on this nontrivial BMP model.

We'd like to further address the reviewer's concern about sampling and diffusion fine-tuning novelty in SBI.

Diffusion fine-tuning novelty. We build on the Simformer of Gloeckler et al. 2024, whose joint representation supports the conditional queries required by HBDS. However, adapting this representation to real observations under simulator misspecification remains an open challenge.

Existing SBI methods approach misspecification differently. Ward et al. 2022 introduce a spike-and-slab discrepancy model over summary statistics, requiring an explicit observation-error model that would need further adaptation to our 940 heterogeneous design–response pairs. Schmitt et al. 2024 detect misspecification through learned summary representations, while Huang et al. 2023 learn robust statistics; neither directly adapts a shared likelihood/posterior model. The optimal-transport approach of Wehenkel et al. 2025 elegantly aligns learned representations of real and simulated observations, but extending this alignment to design-conditioned SBI, where hundreds of heterogeneous observations must be composed into a shared posterior, is not straightforward.

Our approach instead fine-tunes the surrogate likelihood using observed responses at a simulator-aligned reference, transferring adaptation to posterior inference through shared FiLM embeddings. Table 3 demonstrates the importance of this architectural modification, with substantially worse predictive performance in the separately trained no-FiLM ablation. Moreover, comparing design-dependent path divergence with predictive error provides a new diagnostic for localizing potential simulator misspecification without explicitly specifying a discrepancy model.

Sampling novelty. As discussed in the main rebuttal, we have updated the theory and experiments behind F-NPSE-SDE and can provide further background to its origin here. While Geffner et al. 2023 provided the first F-NPSE implementation, when we read the paper and followed their derivation of F-NPSE we were left at a loss for how the authors intuited the reverse transition kernel and its variance in Algorithm 2 as there is no algebra that connects equation 27 and equation 28. We sought to improve our understanding by deriving the reverse transition kernel they proposed, which we do in section A.2, filling a gap in the field.

We then made a similar observation to Arruda et al. 2026 that this compositional sampling kernel was made in a discrete sampling formulation, whereas most diffusion models are sampled using continuous-time dynamics. We diverge from Arruda et al. 2026 in that we used the insights gained from our derivation to help answer the question "what would the ideal diffusion coefficient look like if we knew, and noised, the true posterior?" Since sampling and noising share the same coefficient, we sought to propose one that could also be used in ULA corrector steps during sampling. Deriving a "true" forward noising kernel is quite difficult, and we leave for future work, so we instead proposed a heuristic based on the variance of the noising process from the "true" posterior that leveraged the compositional precision factor κ(n)(t)\kappa^{(n)}(t) that we had previously derived for discrete sampling, but now in continuous time. We then use that variance approximation to derive the schedule gn(t)g_n(t) and, thanks to all reviewers' request, discuss the theoretical benefit of it in section A.4. Our subsequent toy experiments in section A.5 show the benefit of this new schedule in the toy Gaussian model and our updated main-paper experiments show how it compares to the SDE sampler proposed by Arruda et al. 2026 (CSM-Damped).

In summary, both our approach and CSM-Damped of Arruda et al. 2026 seek to improve the stability of compositional diffusion sampling, but arrive at different solutions. Building on the discrete kernel of Geffner et al. 2023, we derive gn(t)g_n(t) from an observation-count-dependent variance prescription, while Arruda et al. explore score damping and a tunable log-SNR shift. As we show in section A.9, these approaches are closely related through the nominal SNR, but our adjustment varies analytically with diffusion time and observation count, without requiring Bayesian hyperparameter optimization to determine the shift. We discuss the benefits and limitations of our approach in section A.4.

We again thank the reviewer for their positive review and probing questions. We hope this background satisfies their curiosity as to the motivation and novelty of our methods.

Thank you for your rebuttal and edits to the manuscript. My views remain on the positive side.

Comments on the paper

1 comment

Dear Reviewers, The authors have now submitted their responses to the reviews. There is only a limited amount of time left for you to review their replies and provide any further comments.

Your feedback is crucial to ensure that the review process proceeds smoothly. We would greatly appreciate it if you could share your reactions before the discussion period closes.

Thank you very much for your work. Best regards.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.