acceptodds
Under review as a conference paper at ICLR 2027

PACE: Parameter-Efficient Calibration for Robust Posterior Inference under Simulator Misspecification

Abstract

Simulator based inference provides a framework for inferring parameters of complex stochastic simulators in the absence of explicit likelihood. However, misspecification in the simulator often creates systemic discrepancies, which leads to poor posterior estimates for real data samples. Recent works have attempted to address this by explicitly handling the discrepancy using methods such as transport-based correction, or robust summary learning. However, this involves additional objectives or multi-stage procedures. We show that, surprisingly, such elaborate adaptation is often not required in many cases of practical interest. We introduce PACE, a simple and deliberately minimal calibration strategy for simulator-pretrained neural posterior inference.Rather than defining explicit correction mechanisms, PACE leverages the representation and posterior structure learnt using abundant simulated data and calibrates the posterior using light weight feature wise transformations learnt using a small set of labeled real observations. These light-weight transformations are parameterized using a small set of calibration parameters and optimized directly for posterior inference. Despite its compact calibration space and simple strategy, PACE demonstrates effective outcome across diverse data modalities ranging from time-series to images. On multiple evaluation metrics, PACE yields comparable or better results than SOTA approaches while requiring adaptation of a very small number of parameters. Our results suggest that when simulation-trained representations retain the information required for posterior inference, expensive re-learning or explicit alignment to the target distribution is unnecessary and a lightweight calibration of existing inference pathway suffices.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Simulation-Based Inference, Simulator Misspecification, Robust Neural Posterior Estimation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_Cxs2OpenReview ↗
Rating
1(−1)
Confidence
3
Informativeness
1(−1)
Presentation
2
Soundness
1(−1)
Read review
Paper summary

PACE is a parameter-efficient calibration framework developed to address simulator misspecification in neural posterior estimation. The method keeps a pre-trained network frozen and adds lightweight, feature-wise transformations across representations, using a small labeled calibration set. This allows fine-tuning with a limited number of parameters: rather than changing 100% of the parameters (fine-tuning), PACE adapts only the small number of added calibration parameters.

Critical strengths
  • The idea in itself is interesting, since networks tend to overfit in low-data regimes, and the idea of needing fewer parameters to fine-tune might be helpful.
  • Enough benchmarking tasks and two real-world tasks have been used to verify the contribution of the method.
  • Many metrics have been used to evaluate the method.
Critical weaknesses

Figures

  • Figure 1: Please change the figure layout; some text falls outside one of the boxes. The colors, color spots, resolution, and letter spacing strongly suggest the figure was generated with AI. However, the AI use statement did not disclose that generative AI was used to create figures directly; please be more transparent about this if this is the case, and make the figures yourself. In addition, the naming is inconsistent: in the figure, one uses xc(i)x_c^{(i)}, while the text uses xc,ix_{c, i}.

The results do not support the claims in the contributions section.

  • Please tone down the main contributions in the introduction section. For instance, "Line 101: Strong performance across diverse data modalities and benchmarks." This line seems overstated. The method does not always achieve "performance comparable to or better than state-of-the-art approaches" (see Figure 2, LPP evaluation, and discussion in the results section).

Sloppy language

  • Line 093: “A different hypothesis creates figures directly about misspecified SBI”: This title is very broad and unspecific. What, specifically, is different about the hypothesis? 
  • Line 201: “The simulator-trained NSE parameters ψ and posterior estimator parameters ϕ remain completely frozen. Instead, we introduce only a small set of trainable parameters…”. The structure of this sentence is unclear: the first part describes how the method works, and the second part introduces a contradiction with the word “instead”.
  • Line 341: “The Base NPE is trained only on simulations”: I assume you mean “misspecified simulations”?
  • Line 334: J-NPE: I assume it means that you perform joint (J) training? Please clarify the sentence describing the method.
  • Line 370, Fig. 2:  “Results on Synthetic and Physical tasks.”The caption is too broad. It would be beneficial to mention what the error bars represent in the figure caption for clarity (i.e., the sample std across 3 seeds)
  • Table 7 reports 0.42–3.19%, while the main text rounds it up to 0.5–4%.
  • Line 394: “On Wind Tunnel, RoPE is strongest at the smallest budgets, but PACE catches up in LPP and reaches the leading group by n = 500”: strange/unprofessional wording.
  • There are a lot of typos. A few of them (but not all):
    • Line 13: Simulator based inference -> simulation-based inference
    • Line 22: levareges -> leverages
    • Line 53: In practice settings -> in practical settings
    • Line 215: “Eq. equation 2” -> Eq. and equation are redundant
    • Line 216: “As Illustrated in Figure 1, We …”: -> “Illustrated” and “We” should not be capitalized.
    • Line 456: sift parameters -> shift
    • Line 658: "sshow the results" ->show
    • ...

Related work

Reproducibility issues

  • No code has been shared; results cannot be verified.
  • The LPP for RoPE/SBI in this paper is very different from the results in the RoPE paper (Wehenkel et al., 2025) across almost all benchmark tasks.
  • Figure 2: Please add more independent repetitions to better estimate differences between the methods. 3 seeds seem too few for the small dataset ranges of the experiments.
Questions for authors
  • Line 333: “For a fair comparison, all methods use the same task-specific NPE architecture and calibration split.” What is the total number of simulations used for each method? For example, how many unique samples does J-NPE use compared with, for instance, full fine-tuning?

  • The Log Posterior Probability (LPP) results for RoPE in this paper are substantially different from the results in the RoPE paper (Wehenkel et al., 2025) across almost all tasks. Could you elaborate on where this difference is coming from?

Reviewer_ZHMnOpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes PACE, which addresses simulator misspecification by adapting a pretrained neural posterior estimator (on simulated data) using a small set of real datasets. It freezes the pretrained weights and learns feature-wise scale-and-shift parameters in the encoder, conditioning interface, and flow conditioner using the standard posterior negative log-likelihood. Experiments on five synthetic and physical benchmarks show substantial improvements over the unadapted estimator while training only substantially less parameters than the full model.

Critical strengths
  1. A simple-yet-effective adaptation strategy. PACE preserves ordinary amortized posterior inference after calibration and requires relatively few trainable parameters. A reliable demonstration that this restricted adaptation is sufficient would be valuable for practical SBI.

  2. Meaningful empirical results. PACE approaches the learned oracle reference on CS and Wind Tunnel and obtains strong results on Light Tunnel at larger calibration budgets. The Light Tunnel improvements are supported by LPP, posterior-mean error, and energy score.

  3. The evaluation is well executed. The paper reports multiple calibration budgets, shared calibration splits, a disjoint test set, and means and standard deviations over three calibration seeds. The appendix provides full metric tables and detailed NPE/PACE architecture and optimization settings.

Critical weaknesses
  1. The methodological contributation is limited. The experiments do not sufficiently establish the contribution beyond applying known lightweight adaptation to NPE. The scale-and-shift parameterization is from SSF [1], and the calibration objective is standard supervised posterior NLL. The main contribution therefore rests on the empirical finding that restricted adaptation suffices under simulator misspecification.

  2. The empirical study lacks a very important baseline. A comparably parameter-efficient baseline such as LoRA would be particularly relevant here. LoRA can adapt the encoder and compatible flow-conditioner layers using the same labeled calibration data and posterior NLL while retaining amortized inference. It is unclear how PACE's tradeoff between accuracy and the amount of real datasets compares with standard PEFT.

[1] Lian, D., Zhou, D., Feng, J., & Wang, X. Scaling & shifting your features: A new baseline for efficient model tuning. NeurIPS 2022.

Questions for authors
  1. Could the authors provide further experimental results including standard PEFT methods such as LoRA? It is a simple modification and I think it should be relatively quick to run if the whole experimental framework is there.
Reviewer_WE9vOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The method freezes a pretrained neural posterior estimator and trains only identity-initialized feature-wise scale-and-shift (SSF) adapters, using a small set of labeled real data for calibration. Method is compared with full fine-tuning, J-NPE, RoPE and FRISBI on five tasks.

Critical strengths

The method is simple, amortized and cheap, adding 0.4 to 3.2% parameters. It is a useful baseline for misspecified simulation-based inference. The metrics are broad and include calibration, and the method is strongest on the high-dimensional image task (Light Tunnel).

Critical weaknesses
  • Limited novelty: This is SSF applied unchanged to NPE. No mechanism is specific to SBI. Therefore the paper feels a bit short on its surprise. The authors somewhat consider that argument in their intro, "The simplicity of PACE is intentional", but I find this a bit limited for an ICLR paper.
  • Missing PEFT baselines: The paper's own question is "how little adaptation is enough", yet the only baseline in the same family is FT. Any of these would likely be needed to be more convincing: LoRA, BitFit, head/flow-only or encoder-only fine-tuning, and FT with L2-SP regularization. FMCPE is cited (Sec. 4) but not compared, nor are RNPE or Huang et al.
  • FT is essentially tied: On CS, SIR, Pendulum and Wind Tunnel, FT matches or beats PACE at n ≥ 500 (e.g., SIR LPP 6.60 vs 6.09; Pendulum 1.06 vs 0.49), and J-NPE wins SIR. So the claim of "comparable or better than SOTA" mostly holds against RoPE/FRISBI.
  • Baseline tuning is suspect: This is perhaps the most damaging in my opinion. RoPE and FRISBI LPP barely moves from n=50 to 1000 (Tables 2, 3, 5). This is surprising. On SIR their jC2ST is about 0.81–0.84, close to the Prior's 0.885. This suggests under-tuned or misconfigured baselines. Only 3 seeds are used.
  • The ablation is over-claimed: Sec. 6.1 says full PACE "achieves the best results across all metrics". Table 1 contradicts this: Flow+context has much better coverage MAE (0.014 vs 0.078) and ACAUC (-0.004 vs 0.063). The ablation also covers only one task at one n.
  • The theory is thin overall... Prop. 1 follows from Bayes' rule once the induced conditionals are assumed equal and q is assumed exact. It is never checked empirically. The O(·) estimation bound in D.3 is schematic and not proven. Sec. 7 breaks the shared-prior assumption and compares against no adaptation baselines. Presentation also needs work: an empty "Impact Statement", frequent typos, a "context adapter" (Sec. 6) that is never defined, and an ambiguous Light Tunnel implementation note (App. C).
Questions for authors

How do LoRA, BitFit and regularized fine-tuning compare at matched parameter budgets? How were RoPE and FRISBI tuned? Under what level of misspecification does PACE fail and full fine-tuning win?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.