- Rating
- 1(−1)
- Confidence
- 3
- Informativeness
- 1(−1)
- Presentation
- 2
- Soundness
- 1(−1)
Read review
Paper summary
PACE is a parameter-efficient calibration framework developed to address simulator misspecification in neural posterior estimation. The method keeps a pre-trained network frozen and adds lightweight, feature-wise transformations across representations, using a small labeled calibration set. This allows fine-tuning with a limited number of parameters: rather than changing 100% of the parameters (fine-tuning), PACE adapts only the small number of added calibration parameters.
Critical strengths
- The idea in itself is interesting, since networks tend to overfit in low-data regimes, and the idea of needing fewer parameters to fine-tune might be helpful.
- Enough benchmarking tasks and two real-world tasks have been used to verify the contribution of the method.
- Many metrics have been used to evaluate the method.
Critical weaknesses
Figures
- Figure 1: Please change the figure layout; some text falls outside one of the boxes. The colors, color spots, resolution, and letter spacing strongly suggest the figure was generated with AI. However, the AI use statement did not disclose that generative AI was used to create figures directly; please be more transparent about this if this is the case, and make the figures yourself. In addition, the naming is inconsistent: in the figure, one uses , while the text uses .
The results do not support the claims in the contributions section.
- Please tone down the main contributions in the introduction section. For instance, "Line 101: Strong performance across diverse data modalities and benchmarks." This line seems overstated. The method does not always achieve "performance comparable to or better than state-of-the-art approaches" (see Figure 2, LPP evaluation, and discussion in the results section).
Sloppy language
- Line 093: “A different hypothesis creates figures directly about misspecified SBI”: This title is very broad and unspecific. What, specifically, is different about the hypothesis?
- Line 201: “The simulator-trained NSE parameters ψ and posterior estimator parameters ϕ remain completely frozen. Instead, we introduce only a small set of trainable parameters…”. The structure of this sentence is unclear: the first part describes how the method works, and the second part introduces a contradiction with the word “instead”.
- Line 341: “The Base NPE is trained only on simulations”: I assume you mean “misspecified simulations”?
- Line 334: J-NPE: I assume it means that you perform joint (J) training? Please clarify the sentence describing the method.
- Line 370, Fig. 2: “Results on Synthetic and Physical tasks.”The caption is too broad. It would be beneficial to mention what the error bars represent in the figure caption for clarity (i.e., the sample std across 3 seeds)
- Table 7 reports 0.42–3.19%, while the main text rounds it up to 0.5–4%.
- Line 394: “On Wind Tunnel, RoPE is strongest at the smallest budgets, but PACE catches up in LPP and reaches the leading group by n = 500”: strange/unprofessional wording.
- There are a lot of typos. A few of them (but not all):
- Line 13: Simulator based inference -> simulation-based inference
- Line 22: levareges -> leverages
- Line 53: In practice settings -> in practical settings
- Line 215: “Eq. equation 2” -> Eq. and equation are redundant
- Line 216: “As Illustrated in Figure 1, We …”: -> “Illustrated” and “We” should not be capitalized.
- Line 456: sift parameters -> shift
- Line 658: "sshow the results" ->show
- ...
Related work
-
It would be beneficial if you related this work to multifidelity SBI. In particular, these papers, because they use pre-training and fine-tuning for SBI:
The idea of fine-tuning a pre-trained network on a small number of high-fidelity simulations and evaluating it on true observations that come from a high-fidelity simulator (to rule out the problem of misspecification) is very similar to using (partial) fine-tuning on a calibration set. The range of calibration samples (50-5000 simulations) is also similar to the amount used in multifidelity SBI.
-
Line 693: cite the paper on neural spline flows (https://proceedings.neurips.cc/paper_files/paper/2019/file/7ac71d433f282034e088473244df8c02-Paper.pdf).
Reproducibility issues
- No code has been shared; results cannot be verified.
- The LPP for RoPE/SBI in this paper is very different from the results in the RoPE paper (Wehenkel et al., 2025) across almost all benchmark tasks.
- Figure 2: Please add more independent repetitions to better estimate differences between the methods. 3 seeds seem too few for the small dataset ranges of the experiments.
Questions for authors
-
Line 333: “For a fair comparison, all methods use the same task-specific NPE architecture and calibration split.” What is the total number of simulations used for each method? For example, how many unique samples does J-NPE use compared with, for instance, full fine-tuning?
-
The Log Posterior Probability (LPP) results for RoPE in this paper are substantially different from the results in the RoPE paper (Wehenkel et al., 2025) across almost all tasks. Could you elaborate on where this difference is coming from?