- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Kalman Posterior Sampling (KPS) is a method for diffusion posterior sampling that addresses a more general form of inverse problems in which the measurement likelihood is intractable. KPS decomposes posterior sampling into transitions . It approximates using Tweedie identities and uses statistical linearization to form a Gaussian approximation to ; this yields an algorithm that iteratively denoises samples and contains an inner Gibbs loop to apply Kalman updates and renoising at each noise level. The paper presents a gradient-based variant and a more general simulator gradient-free variant, demonstrating strong results across several scientific inverse problems.
Critical strengths
1. The problem is well-motivated and timely. Derivative-free posterior sampling for inverse problems with inaccessible likelihoods is practically useful for real world scientific tasks. The focus on evaluation of posterior coverage, rather than simply on deterministic reconstruction metrics, is valuable.
2. The proposed algorithm is conceptually simple and well motivated: it combines diffusion-based local Gaussian approximations with statistical linearization and a Kalman update, while the re-noising/Gibbs loop provides a principled way to maintain consistency across diffusion levels.
Critical weaknesses
1. I'm concerned about the novelty and contribution of KPS relative to existing work like Blade [1], which also focuses on derivative-free posterior sampling and uses ensemble statistical linearization in a Gibbs-style sampler. The paper briefly mentions Blade in the related work but does not include comparison against it in the experiments, despite Blade, to my understanding, being the closest prior work to KPS. This is especially important for Navier–Stokes, where Blade evaluates on the same InverseBench problem (as far as I can tell) and, based on the published results, substantially outperforms KPS-H at the same ensemble size.
2. I don't think the experiments convincingly validate the generality of KPS to problems with intractable measurement likelihoods. Several of the evaluated tasks use the standard additive Gaussian measurement noise; while the black hole imaging task does not, the results are not fairly comparable with the baselines due to the difference in noise model as reported by the paper. So the experiment most relevant to the paper’s central motivation lacks a controlled comparison to prior methods under the same observation model.
Questions for authors
- Major: can the authors clarify the novelty/contribution of KPS w.r.t. Blade? And can they please provide experimental comparison with Blade, such as on Navier Stokes and the other tasks?
- Moderate: Black hole imaging appears to be the main experiment that directly tests the non-additive/non-Gaussian, simulation-based likelihood setting. But there isn't a fair comparison to the baselines due to the difference in noise model. Could the authors adapt some representative baselines to the same noise model setting to enable fair comparison? It doesn't have to be all the reported baselines, just a few could be useful. This would help emphasize the strength of KPS's general likelihood-free formulation.
- Clarification: The paper provides per-update complexity and simulator-call counts, but no end-to-end runtime or matched-compute comparison against baselines. Could you report wall-clock time and/or total simulator/denoiser evaluations for KPS-H and KPS-G relative to the closest baselines, especially Blade on Navier–Stokes?
[1] Zheng et al., TMLR “Blade: A Derivative-free Bayesian Inversion Method using Diffusion Priors.”
Most concerns have been satisfied. Updating score from 2 to 3.
5 comments
Thanks for the response. The clarification of the methodological differences between KPS and BLADE is appreciated. However, the Navier–Stokes results show that BLADE substantially outperforms KPS-H in both reconstruction accuracy and calibration at the same ensemble size. This leaves the practical benefit of KPS unclear.
The additional black hole experiments under the baselines’ observation model are useful and an improvement.
My question about end-to-end computational cost remains unresolved. The reported operation counts alone do not establish the computational tradeoff relative to the closest baselines.
I appreciate the improvements, but the remaining uncertainty about the practical contribution and computational cost leads me to maintain my weak reject score.
BLADE on latent problem and further results
We thank the reviewer for the follow-up and for acknowledging the black-hole experiments. We now have first results with BLADE on the latent problem, which bear directly on both remaining concerns.
Navier–Stokes. Beyond the reasons discussed in the revised paper, BLADE's results on this task rely on careful per-task tuning. Its hyperparameters are set per task and per noise level (BLADE, Table 4), with the effective observation noise ranging from 0.03 to 0.3. The BLADE authors identify as the most important hyperparameter ("Impact of different hyperparameters", Section 4.3). They set it below the true noise level and tune it so that the spread-skill ratio is close to 1 ("Effective observation noise", Appendix B.1). Its calibration on Navier–Stokes is therefore partly obtained by tuning toward the calibration target. KPS has no step size or observation noise to tune. It estimates from the simulated pairs, and uses a single configuration across the InverseBench tasks (Table 5).
BLADE on the latent problem. We adapted the official implementation (blade.py) to our setting. We used its main (most calibrated) version with 2048 particles, as for all ensemble methods except KPS-G, and 60 Langevin steps. Following Appendix B.1 of the BLADE paper, we set below the true observation noise . We kept the ratio of their Navier–Stokes configuration at (), which gives , without further tuning. The results are in the updated Fig. 3:
| Method | Skill ↓ | SSR → 1 | CRPS ↓ |
|---|---|---|---|
| BLADE | 0.32 ± 0.13 | 0.38 ± 0.09 | 425 ± 214 |
| KPS-G | 0.23 ± 0.03 | 1.17 ± 0.12 | 266 ± 39 |
| KPS-H | 0.23 ± 0.03 | 1.04 ± 0.10 | 266 ± 36 |
On this task, BLADE is under-dispersed and varies strongly across instances. The galleries show why. On some instances, BLADE is qualitatively on par with the best methods (Figs. 19 and 20). On others, it does not diverge but misses the target entirely (Figs. 21 and 22). This is consistent with the BLADE paper's finding that must be tuned per problem: a setting that works well on one task does not transfer directly to another.
Black-hole imaging. We also realized that BLADE reports black-hole imaging results (BLADE, Appendix Table 3), for point accuracy only, and we have added them to Table 8. Under the same eht noise model, BLADE (main) leads in PSNR (31.3 vs. 28.9 for KPS-G) and in closure-phase fit, while KPS-G fits closure amplitudes better ( 1.57 vs. 1.95). BLADE reports no calibration metrics on this task, whereas KPS-G remains close to calibrated under both noise models (Figs. 4 and 9), with the same configuration in both.
Computational cost. With particles, noise levels (or split-Gibbs iterations), Langevin steps and Gibbs sweeps, the number of forward-model calls scales as follows:
| Method | Forward-model calls | Simulator gradient |
|---|---|---|
| DPS, MMPS, TDS | yes | |
| EnKG | no | |
| DAPS, PnP-DM | yes | |
| BLADE | no | |
| KPS-H | no | |
| KPS-G | yes |
Concretely, BLADE's Navier–Stokes configuration runs 25 split-Gibbs iterations of 50 Langevin steps each (BLADE, Appendix Table 2). This amounts to 1,250 forward-model calls per particle, against 64 for KPS (, ). At 512 particles, BLADE makes 640,000 calls against 32,768 for KPS-H, about 20 times more. This matters when the simulator is expensive. We have added this table to Appendix A.8.
Our preliminary observations on wall-clock time are the following:
- In these benchmarks, the simulator is cheap, so calling it less often does not yet show in wall-clock time.
- The main costs of KPS are the linear solve of the Kalman update and the vector-Jacobian products through the denoiser for the Tweedie covariance. We keep both because they make the update fully defined, with no step size or noise level to tune.
- KPS-G is particle-efficient. Its slope has full rank whatever the ensemble size, so few particles suffice: 16 on InverseBench, and 64 on the latent problem against 2048 for the other ensemble methods.
- Neither variant has been optimized for speed, and standard engineering could reduce their cost.
We ran into implementation issues while setting up the timing, memory and Laplace noise experiments, and are working to resolve them. We will post these results as soon as they are available, and aim to do so before the end of the discussion period. We hope the results above already address the reviewer's main concerns, and would be grateful if they could reconsider their score in their light, or let us know what would help them do so.
Direct answer
We thank the reviewer for insisting on BLADE and for the concrete suggestions.
W1 & Q1: Novelty with respect to BLADE, and experimental comparison.
We agree that BLADE is the closest prior work and that the submission did not acknowledge it enough. Both methods build on the ensemble statistical linearization of EnKG. They differ in how they use it and in what they assume about the likelihood:
These choices have a cost. A single closed-form step inherits the error of the linear surrogate when the forward model is strongly non-linear, where BLADE's small steps are likely more robust. We see the two methods as complementary. For example, BLADE could query the simulator and estimate Σy as KPS does, and KPS could relinearize within a noise level. We now describe BLADE in detail in Section 3 and discuss the comparison in a new paragraph, "Relation to BLADE" (Section 6).
On experiments, we added BLADE's reported Navier–Stokes results (N=512, most calibrated configuration) to Table 2. The reviewer is right: BLADE outperforms KPS-H there in both relative L2 (0.110 vs. 0.313) and calibration. We attribute this to the strong non-linearity of the PDE solver, where a single Kalman step on a linear surrogate is less robust than many small steps. We are also adding BLADE to the latent problem, where all methods are run by us under identical settings, and will share these results during the discussion.
W2 & Q2: Controlled comparison on black-hole imaging.
Rather than adapting the baselines to our noise model, we adapted KPS to theirs. Both variants now also run under the
ehtnoise model used for the published baselines (Table 2, Table 8, Appendix C.3.1). Under this model, KPS queries the simulator directly and both variants still exceed the best published blur PSNR (34.8 and 34.7 vs. 32.9). Theehtmodel applies non-differentiable station gains, so KPS-G must take its Jacobian from the differentiablevis_thermalmodel. This makes it a misspecified-gradient setting, in which KPS-G remains close to calibrated (MIRA lower by only 0.028±0.006, Fig. 9). Figure 8 quantifies how the two noise models differ.On the broader point, the other tasks have tractable likelihoods by design: the baselines need an explicit likelihood to be run at all. KPS never uses it and treats every task as if the likelihood were intractable. We will also add a non-Gaussian (Laplace) noise experiment on the latent problem.
Q3: End-to-end cost.
We will report wall-clock time and peak memory for all methods, including BLADE, on the latent problem, normalized per posterior sample since KPS produces the whole ensemble in parallel. No method, including ours, was tuned for efficiency: we used the configurations of the original code.
While waiting for additional results on BLADE on the latent problem, we do believe that our responses have addressed your main concerns. We would be happy to provide further clarification if any issues remain. Otherwise, we would be grateful if you could reconsider your score in light of our revisions.