acceptodds
Under review as a conference paper at ICLR 2027

Getting Your Guidance Weights Right in diffusion and flow-matching posterior sampling

Abstract

Training-free posterior sampling methods, also known as Plug-and-Play methods, leverage pretrained unconditional diffusion or flow-matching models to solve inverse problems. Most existing approaches rely on guidance weights to balance, at each time step, prior information from the unconditional score or velocity network with measurement consistency, yet the tuning of these weights is often not discussed and is largely left to heuristics. We introduce a simple and principled offline strategy for automatically tuning these guidance weights. Our key observation is that, at each time step, the conditional denoising score-matching objective for diffusion models, or the conditional flow-matching objective for flow-matching models, is a least-squares objective. Therefore, when the conditional prediction is expressed as a weighted sum of the unconditional network output and a measurement-guidance term, optimizing over these weights reduces to a two-dimensional linear least-squares problem. The resulting time-dependent guidance weights can be optimized offline for a given measurement operator, noise level and sampler at the cost of a single minibatch of sampling trajectories, without retraining or fine-tuning the pretrained generative model. Instantiated with the standard Tweedie-based measurement-consistency term, our approach improves posterior sampling and achieves state-of-the-art reconstruction performance across diffusion- and flow-matching-based methods. Moreover, the optimized guidance weights enable diffusion samplers to reduce the number of sampling steps from 1000 to 50 with no significant degradation in reconstruction quality. Code will be made available.

Primary area:
generative models
Keywords:
diffusion models, flow matching, posterior sampling, guidance, inverse problems, conditional generation, Bayesian inference, sampling efficiency

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_TM7jOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
3(+1)
Read review
Paper summary

This paper addresses the problem of setting time-dependent guidance weights in training-free posterior sampling (Plug-and-Play) algorithms for linear inverse problems using pretrained diffusion or flow-matching models. Rather than relying on heuristic schedules or expensive iterative optimization, the authors make the observation that the conditional denoising score-matching (diffusion) and conditional flow-matching objectives can be formulated as least-squares regression problems. Consequently, optimizing the balance between the unconditional prior field and the measurement-guidance term reduces at each step to a two-dimensional linear least-squares problem.

To account for exposure bias and the distribution shift between forward diffusion marginals and actual sampling trajectories, the authors propose SIMCA (SIMulation-based CAlibration), a greedy, simulation-based offline calibration scheme that sequentially optimizes guidance weights using a single minibatch of synthetic trajectories. Experiments across diffusion models (FFHQ, ImageNet) and flow-matching models (CelebA, AFHQ-Cat) demonstrate that SIMCA matches or outperforms competitive training-free baselines, collapses performance differences among various guidance formulations (e.g., DPS, Π\PiGDM, TMPD), and enables up to a 20×20\times reduction in sampling steps (1000 down to 50) without severe perceptual loss.

Critical strengths
  1. Conceptual Simplicity and Elegance: Formulating the guidance weight tuning problem through the lens of conditional score matching and conditional flow matching targets (Eqs. 4, 10, 12) is mathematically sound, elegant, and provides a principled ground for what was previously mostly treated as ad-hoc empirical tuning.
  2. Computational Efficiency: Because the objective is quadratic in wt=[wu,t,wf,t]⊤w_t = [w_{u,t}, w_{f,t}]^\top, each step requires solving an elementary 2×22 \times 2 linear system. Calibration takes only seconds to a few minutes offline (Table 5) and avoids backpropagation through ODE/SDE trajectories (unlike diffusion optimal control methods).
  3. Unified Perspective Across Model Families: The paper provides a unified notation and formulation encompassing both diffusion models and Gaussian flow-matching backbones. The fact that SIMCA operates out-of-the-box on flow velocity fields without score conversion highlights the generality of the principle.
  4. Acceleration via Discretization-Aware Calibration: Calibrating weights specifically for reduced step budgets (K=500,50K=500, 50) allows DDPM to retain high reconstruction quality and even achieve higher PSNR/SSIM, providing a practical path toward fast posterior sampling.
Critical weaknesses
  1. Statistical Reliability of Low-Sample Distribution Metrics (FID on N=100N=100):

    • While I acknowledge that evaluating on 100 images follows the benchmark protocol of DAPS (Zhang et al., 2025) to allow direct comparison with published baselines, computing Fréchet Inception Distance (FID) on only 100 samples is statistically ill-conditioned. Estimating a 2048-dimensional covariance matrix from 100 samples produces a rank-deficient matrix (rank≤99\text{rank} \le 99) with extreme sample variance and severe positive bias.
    • Small differences in FID (such as those in Tables 1 and 6) under this regime are within the noise margin and cannot reliably support claims of superior distribution coverage.
    • The authors should consider reporting sample-efficient distribution metrics designed for smaller sample sizes, such as CMMD (CLIP Maximum Mean Discrepancy, Jayasumana et al., 2024), to confirm that distribution-level conclusions hold.
    • Clarification on text: In line 911, the text states "and FID [...] averaged over individual images". Please clarify this wording, as FID is a distribution-level metric and cannot be evaluated or averaged per-instance.
  2. Structural Ceiling on Ill-Conditioned Operators:

    • The paper acknowledges in lines 374–377 that SIMCA underperforms on deblurring and denoising. Specifically, on ImageNet Gaussian deblurring, SIMCA trails DDNM by almost 2.8 dB (25.28 dB vs. 28.06 dB) and has a substantially worse FID (121.85 vs. 81.43).
    • This demonstrates that scalar-tuning of Tweedie guidance cannot overcome the fundamental limitation of isotropic score approximations on ill-conditioned / low-pass operators, where subspace/null-space decomposition (e.g., DDNM) remains strictly superior.
  3. Lack of Variance Analysis for the "Single-Image" (N=1N=1) Calibration Claim:

    • In Section 5.3 and Table 5, the authors assert that a single image (N=1N=1) is sufficient for calibration. However, this is shown for only one specific sample.
    • If N=1N=1 is drawn as a low-texture or low-contrast image (e.g., flat sky or background), the conditioning of the 2×22 \times 2 Gram matrix HH in Algorithm 1 will differ drastically from an image with high-frequency edges. To substantiate the claim that N=1N=1 is universally sufficient, the authors should report the mean and standard deviation over 10–20 independent single-image draws.
Questions for authors
  1. Behavior Under Varying Noise Levels (σm\sigma_m):

    • In the paper, the measurement noise level is kept fixed at σm=0.05\sigma_m = 0.05. How do the optimal trajectory weights (wu,t,wf,t)(w_{u,t}, w_{f,t}) shift when evaluating across different noise regimes, such as σm∈{0.01,0.05,0.1,0.2}\sigma_m \in \{0.01, 0.05, 0.1, 0.2\}?
    • Does wf,tw_{f,t} scale inversely with noise variance (e.g., proportional to 1/σm21/\sigma_m^2 as classical Bayesian likelihood weighting would suggest), and does wu,tw_{u,t} stay closer to 1.01.0 as noise increases to regularize the reconstruction?
  2. Robustness to Operator Misspecification (Semi-Blind Settings):

    • In real-world applications, acquisition parameters are rarely known with perfect accuracy. How sensitive is SIMCA if there is a mismatch between the calibration operator and the test operator?
    • For example, if guidance weights are calibrated using a Gaussian blur kernel with standard deviation σ=3.0\sigma = 3.0, how significantly does reconstruction quality degrade if the test image was blurred with σ=3.5\sigma = 3.5, or if the true noise level is σm=0.08\sigma_m = 0.08 instead of 0.050.05?
  3. Transferability Across Datasets and Backbones:

    • Do the calibrated schedules {wt}\{w_t\} primarily capture the geometry of the measurement operator AA, or are they sensitive to the specific dataset prior and network architecture?
    • For a fixed operator (e.g., 4×4\times super-resolution), could the authors:
      • Compare the schedules: Overlay the calibrated curves (wu,t,wf,t)(w_{u,t}, w_{f,t}) obtained across different datasets (e.g., FFHQ vs. ImageNet) and backbones on the same plot (analogous to Figure 1) to visually show whether the trajectories align or diverge.
      • Evaluate the numbers: Report quantitative reconstruction metrics (PSNR, SSIM, LPIPS) when transferring weights directly (e.g., applying FFHQ-calibrated weights to ImageNet, or across different checkpoints) without recalibrating.
    • This would clarify whether practitioners can reuse pre-computed schedules for a given operator, or if calibration must strictly be repeated for every new dataset and model.
  4. Comparison with Baseline Guidance Schedules (in Appendix):

    • In Figure 1, the calibrated schedules (wu,t,wf,t)(w_{u,t}, w_{f,t}) are shown in isolation. Could the authors overlay on the same axes (e.g., in an appendix figure) the effective schedules used by key baselines? Specifically, plotting the empirical average of the residual-normalized weight of DPS (wf,t=ζ/∥y−Ax^0(xt)∥2w_{f,t} = \zeta / \|y - A\hat{x}_0(x_t)\|_2 with wu,t≡1.0w_{u,t} \equiv 1.0) and the analytical weights of TMPD, Π\Pi-GDM, or PnP-Flow.
    • Visually comparing these curves would help explain where existing heuristics fail (e.g., do they over-guide or under-guide as t→0t \to 0?) and provide intuitive support for the claim in Table 4 that baselines mostly suffer from poor scaling.
  5. Applicability to Non-Linear Inverse Problems:

    • The method is strictly formulated for linear measurement models (y=Ax+ny = Ax + n). How does the least-squares formulation in Equations 10–12 behave when the forward operator is non-linear (e.g., phase retrieval y=∣F(x)∣y = |\mathcal{F}(x)|, non-linear camera tone curves, or JPEG quantization)?
    • Does non-linearity in ∇xt∥y−A(x^0)∥2\nabla_{x_t} \|y - \mathcal{A}(\hat{x}_0)\|^2 cause instability in the 2D linear regression step, or can the framework be adapted to non-linear inverse problems without modifications?
8 comments
AuthorsOpenReview ↗

Response to Reviewer TM7j (1/5)

We thank the reviewer for the detailed feedback. Please find our answers below.

"The authors should consider reporting sample-efficient distribution metrics designed for smaller sample sizes, such as CMMD (CLIP Maximum Mean Discrepancy, Jayasumana et al., 2024), to confirm that distribution-level conclusions hold."

We computed CMMD for Table 4 of the paper (see below): it preserves the ordering given by FID, with SIMCA improving on the simulation-free variant for all three measurement terms. We will add CMMD to Tables 4 and 6 of the paper in the revised version.

PSNRSSIMLPIPSFIDCMMD
DPS25.460.8230.20369.200.1729
+ sim.-free31.690.8970.10333.710.1061
+ SIMCA32.910.9160.06919.160.0480
Π\PiGDM25.100.7190.24867.930.1601
+ sim.-free32.120.9030.09630.680.0941
+ SIMCA32.970.9160.06918.350.0464
TMPD27.890.8350.15656.430.1594
+ sim.-free32.020.8790.11530.720.0942
+ SIMCA32.840.9150.07020.210.0485

"Clarification on text: In line 911, the text states "and FID [...] averaged over individual images". Please clarify this wording, as FID is a distribution-level metric and cannot be evaluated or averaged per-instance."

The wording at line 911 is incorrect: PSNR, SSIM and LPIPS are averaged over images, while FID is computed on the set of 100 reconstructions. We will correct this sentence.

"To substantiate the claim that N=1N=1 is universally sufficient, the authors should report the mean and standard deviation over 10–20 independent single-image draws."

We ran this experiment on FFHQ (DDPM, 5050 steps, σm=0.05\sigma_m = 0.05) for Gaussian deblurring, box inpainting and random inpainting. We calibrate 2020 schedules, each on a different single image, and reconstruct the same 100100 test images with each of them. The observations and the sampling noise are shared across runs, so the calibration image is the only source of difference. Table A reports the mean and standard deviation of the test metrics over these runs.

Table A. Sensitivity of the test metrics to the calibration image on FFHQ (DDPM, 50 steps, σm=0.05\sigma_m = 0.05, 100 test images): mean ±\pm std over 20 runs, each calibrated on a different single image.

TaskPSNR ↑\uparrowSSIM ↑\uparrowLPIPS ↓\downarrow
Gaussian deblurring28.33±0.0628.33 \pm 0.060.819±0.0020.819 \pm 0.0020.232±0.0020.232 \pm 0.002
Box inpainting23.43±0.0923.43 \pm 0.090.851±0.0020.851 \pm 0.0020.144±0.0030.144 \pm 0.003
Random inpainting33.77±0.0433.77 \pm 0.040.935±0.0010.935 \pm 0.0010.117±0.0020.117 \pm 0.002

Across the single-image calibrations, the standard deviation is below 0.10.1 dB in PSNR, at most 0.0020.002 in SSIM and at most 0.0030.003 in LPIPS. For Gaussian deblurring and box inpainting, the mean PSNR is also within 0.020.02 dB of the value obtained with the full calibration set of 100100 images.

In these settings, a single calibration image is sufficient, and the choice of that image has little influence on reconstruction quality. We will add this table to the revised paper.

AuthorsOpenReview ↗

Response to Reviewer TM7j (2/5)

"In the paper, the measurement noise level is kept fixed at σm=0.05\sigma_m=0.05 . How do the optimal trajectory weights (wu,t,wf,t)(w_{u,t},w_{f,t}) shift when evaluating across different noise regimes, such as σm∈0.01,0.05,0.1,0.2\sigma_m\in 0.01, 0.05, 0.1, 0.2? Does wf,tw_{f,t} scale inversely with noise variance (e.g., proportional to 1σm2\frac{1}{\sigma_m^2} as classical Bayesian likelihood weighting would suggest), and does wu,tw_{u,t} stay closer to 1.0 as noise increases to regularize the reconstruction?"

We calibrated the weights on FFHQ (DDPM, 5050 steps, 1010 calibration images) for noise levels from σm=0.01\sigma_m = 0.01 to 0.50.5, on box inpainting, random inpainting and 4×4\times super-resolution. The likelihood term is taken at the true noise level, f=−12σm2∇xt∥y−Ax^0(xt)∥2f = -\tfrac{1}{2\sigma_m^2}\nabla_{x_t}\Vert y - A\hat{x}_0(x_t)\Vert^2, so that classical Bayesian weighting corresponds to wf,t=1w_{f,t} = 1. Table B reports wf,tw_{f,t} at selected steps.

Likelihood weight. In these experiments, the classical 1/σm21/\sigma_m^2 scaling is observed only toward the end of the trajectory. Over roughly the first 2020 steps, wf,tw_{f,t} is approximately proportional to σm2\sigma_m^2 (the ratio wf,t/σm2w_{f,t}/\sigma_m^2 varies by less than 3% for σm≤0.2\sigma_m \le 0.2), so the effective guidance is nearly independent of the noise level. In the last steps, wf,tw_{f,t} becomes of order one, closer to the classical weighting, and this happens earlier when the noise is larger.

Prior weight. wu,tw_{u,t} appears largely insensitive to the noise level: ∣wu,t−1∣≤0.017|w_{u,t} - 1| \le 0.017 for all tasks and noise levels except in the last three steps, with a deviation that increases slightly with σm\sigma_m. We therefore do not observe wu,tw_{u,t} moving closer to 11 as the noise increases.

Table B. Calibrated likelihood weight wf,tw_{f,t} on FFHQ (DDPM, 50 steps, 10 calibration images). Step 4949 is the first, noisiest step.

Taskσm\sigma_mt=49t=49t=30t=30t=10t=10t=5t=5t=1t=1
Box inpainting0.010.018.75e−98.75\mathrm{e}{-9}2.88e−62.88\mathrm{e}{-6}2.64e−42.64\mathrm{e}{-4}1.21e−31.21\mathrm{e}{-3}0.02210.0221
0.050.052.19e−72.19\mathrm{e}{-7}7.17e−57.17\mathrm{e}{-5}6.57e−36.57\mathrm{e}{-3}0.03000.03000.5110.511
0.10.18.76e−78.76\mathrm{e}{-7}2.88e−42.88\mathrm{e}{-4}0.02620.02620.1180.1181.611.61
0.20.23.50e−63.50\mathrm{e}{-6}1.14e−31.14\mathrm{e}{-3}0.1030.1030.4370.4372.942.94
0.50.52.19e−52.19\mathrm{e}{-5}7.25e−37.25\mathrm{e}{-3}0.5680.5681.701.703.073.07
Random inpainting0.010.012.38e−82.38\mathrm{e}{-8}9.84e−69.84\mathrm{e}{-6}7.93e−47.93\mathrm{e}{-4}3.65e−33.65\mathrm{e}{-3}0.04720.0472
0.050.055.95e−75.95\mathrm{e}{-7}2.46e−42.46\mathrm{e}{-4}0.01970.01970.08970.08971.011.01
0.10.12.38e−62.38\mathrm{e}{-6}9.84e−49.84\mathrm{e}{-4}0.07800.07800.3360.3362.402.40
0.20.29.51e−69.51\mathrm{e}{-6}3.94e−33.94\mathrm{e}{-3}0.2960.2961.051.052.842.84
0.50.55.94e−55.94\mathrm{e}{-5}0.02460.02461.291.292.262.263.563.56
SR (×4\times 4)0.010.011.04e−71.04\mathrm{e}{-7}4.55e−54.55\mathrm{e}{-5}4.54e−34.54\mathrm{e}{-3}0.02320.02320.4680.468
0.050.052.59e−62.59\mathrm{e}{-6}1.14e−31.14\mathrm{e}{-3}0.1110.1110.5180.5183.943.94
0.10.11.04e−51.04\mathrm{e}{-5}4.58e−34.58\mathrm{e}{-3}0.4170.4171.511.514.344.34
0.20.24.15e−54.15\mathrm{e}{-5}0.01840.01841.291.292.802.805.335.33
0.50.52.59e−42.59\mathrm{e}{-4}0.1140.1142.842.843.893.899.019.01

These results suggest that the calibrated schedule depends on the noise level in a fairly regular way: the prior weight changes little, and the likelihood weight mainly changes in the last steps. We will add the corresponding figures, with the full schedules for all noise levels, to the revised paper.

AuthorsOpenReview ↗

Response to Reviewer TM7j (3/5)

"In real-world applications, acquisition parameters are rarely known with perfect accuracy. How sensitive is SIMCA if there is a mismatch between the calibration operator and the test operator? For example, if guidance weights are calibrated using a Gaussian blur kernel with standard deviation σ=3.0\sigma=3.0 , how significantly does reconstruction quality degrade if the test image was blurred with σ=3.5\sigma=3.5, or if the true noise level is σm=0.08\sigma_m=0.08 instead of 0.05?"

We ran the two cases suggested by the reviewer on FFHQ (DDPM, 10001000 steps, 100100 test images). In both, the sampler is left untouched: it uses the weights calibrated for the nominal model and evaluates its likelihood term with the nominal operator and noise level. Only the observations change, with the sampling noise shared across settings.

Misspecified noise level. The sampler assumes σm=0.05\sigma_m = 0.05, while the true noise level ranges from 0.020.02 to 0.10.1 (Table C). For deviations of ±20%, the metrics change little (at most 0.70.7 dB and 0.0150.015 LPIPS). The sensitivity is asymmetric: a lower true noise level does not degrade the reconstruction, whereas a higher one does. For the reviewer's example (0.080.08 instead of 0.050.05), PSNR decreases by 0.10.1 dB on box inpainting and 2.02.0 dB on random inpainting, and LPIPS increases by 0.030.03 and 0.050.05.

Misspecified blur kernel. The sampler assumes a Gaussian kernel of standard deviation 3.03.0, while the observations are blurred with 3.53.5. PSNR decreases from 28.2328.23 to 27.4927.49 dB, SSIM from 0.7940.794 to 0.7750.775, and LPIPS increases from 0.1630.163 to 0.1820.182.

Table C. Misspecified noise level on FFHQ. The sampler always assumes σm=0.05\sigma_m = 0.05.

TaskTrue σm\sigma_mPSNR ↑\uparrowSSIM ↑\uparrowLPIPS ↓\downarrowCMMD ↓\downarrow
Box inpainting0.020.0222.950.8550.1030.195
0.040.0422.930.8510.1030.151
0.050.05 (matched)22.900.8460.1060.151
0.060.0622.870.8370.1120.273
0.080.0822.760.8070.1401.149
0.10.122.570.7460.1791.537
Random inpainting0.020.0233.890.9370.0570.086
0.040.0433.260.9240.0620.044
0.050.05 (matched)32.910.9160.0690.048
0.060.0632.190.8960.0840.071
0.080.0830.900.8540.1200.247
0.10.129.590.8060.1590.890

In these experiments, SIMCA degrades gradually under moderate misspecification and appears more sensitive to an underestimated noise level than to an overestimated one. We will add these results to the revised paper.

Reviewer_vVwDOpenReview ↗
Rating
4
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

Summary

  • This paper propose a least square approach towards auto-tuning the weight of DPS. It treat the combination of the unconditional score and the guidance term as a least square estimation of ground truth vector field. The key observation is the ground truth vector field is invariant to conditioning.
  • This paper propose two variant of method based on the least square auto-tuning, one is simulation free, the other one is simulation based. The simulation free variant improves the baseline by large margin, and the simulation based is also quite competitive among all methods.
Critical strengths

Strength

  • The utilization of ground truth vector field does not change with conditioning to construct a least square is very clever. The proposed approach is neat, clean and theoretically grounded.
  • The proposed approach seems to work well. The similation free variant seems to improve the baseline by large margin, at the cost of batch running.
Critical weaknesses

Weakness

  • One clear weakness is that the proposed approach is only adaptative per operator. Using minibatch for least square lose the potential to make the step size dependent on sample.
  • The simulation based approach clearly add non trivial complexity overhead to existing methods. Does this overhead makes the simulation based methods slower in Table 1? Or are there any other efforts to make the comparsion fair?
  • Another group of paper propose to view DPS as maximizing a posterior, and they claim that the error in score estimation is a normal phenomena, and even benefical to final performance. It would be interesting if their opinion, which is kind of opposite to this paper, could be discussed [ICLR 25 Rethinking Diffusion Posterior Sampling: From Conditional Score Estimator to Maximizing a Posterior] [ICML 26 Local MAP Sampling for Diffusion Models].
Questions for authors

Questions

  • See weakness

Justification

  • Overall I find the least square solution quite clever and it works very well. I recommend a clear acceptance.
1 comment
AuthorsOpenReview ↗

Response to Reviewer vVwD

We thank the reviewer for the feedback. Please find our answers below.

"One clear weakness is that the proposed approach is only adaptative per operator. Using minibatch for least square lose the potential to make the step size dependent on sample."

We agree that losing the potential to make the step size dependent on sample, i.e. being self-adaptive, is a limitation of SIMCA that should be explicitly acknowledged in the paper. Nevertheless, we would like to highlight an interesting empirical finding: SIMCA's performance is remarkably insensitive to the calibration minibatch size (see Table 5 of the paper), with even a single image (i.e., a minibatch of size 1) yielding very strong results. This implies that different images lead to very similar step size. We believe this observation provides an indication that self-adaptive calibration may offer only limited additional improvements over SIMCA.

"The simulation based approach clearly add non trivial complexity overhead to existing methods. Does this overhead makes the simulation based methods slower in Table 1? Or are there any other efforts to make the comparison fair?"

We believe there may be a misunderstanding regarding the "computational overhead" introduced by SIMCA. Importantly, SIMCA's calibration is performed entirely offline and therefore introduces no additional computational overhead at inference time. In particular, for an equal number of sampling steps KK, SIMCA has essentially the same inference runtime as DPS in Table 1 of the paper.

Regarding the fairness of the comparison, we would like to emphasize that most methods in Table 1 of the paper tuned their hyperparameters for each problem. For instance, the DPS hyperparameters were tuned separately for each inverse problem in Table 1 of the paper to maximize PSNR, a process that requires evaluating multiple hyperparameter configurations through full sampling trajectories and is therefore time-consuming. In contrast, SIMCA replaces this tuning process with a computationally efficient, automatic calibration procedure, requiring only approximately 2 minutes for K=1000K=1000 sampling steps and a few seconds for K=50K=50. We therefore believe that SIMCA offers a favorable trade-off between calibration cost and performance, while maintaining the same inference-time computational complexity as DPS.

"Another group of paper propose to view DPS as maximizing a posterior, and they claim that the error in score estimation is a normal phenomena, and even beneficial to final performance. It would be interesting if their opinion, which is kind of opposite to this paper, could be discussed [ICLR 25 Rethinking Diffusion Posterior Sampling: From Conditional Score Estimator to Maximizing a Posterior] [ICML 26 Local MAP Sampling for Diffusion Models]."

We thank the reviewer for highlighting these relevant references, which we will discuss in Section 3. Xu et al. (2025) show that the approximate conditional score employed by DPS deviates substantially from that of a trained conditional diffusion model, while producing high-quality reconstructions with significantly reduced diversity. This suggests that DPS behaves more like a MAP estimator than a faithful posterior sampler. We find this analysis particularly interesting, although a similar investigation would be necessary to determine whether these conclusions also hold for SIMCA, whose calibrated guidance can substantially alter the sampling trajectories compared to DPS.

Regarding LMAPS (Zhang et al., 2026), the method explicitly solves a local MAP optimization problem at each sampling step, achieving substantial improvements over DPS in reconstruction metrics such as PSNR, SSIM, and LPIPS. However, we view LMAPS as complementary rather than opposed to SIMCA. Indeed, LMAPS introduces guidance parameters k1k_1 and k2k_2 that are tuned separately for each inverse problem (see Table 4 in their appendix). While LMAPS focuses on improving the reconstruction procedure through local MAP optimization, SIMCA addresses the complementary question of how to automatically calibrate guidance weights. We therefore believe that extending our calibration strategy to LMAPS represents a promising direction for future research.

Reviewer_QPTuOpenReview ↗
Rating
3(+2)
Confidence
3
Informativeness
2(+1)
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes SIMCA, an offline calibration method for automatically tuning time-dependent guidance weights in training-free posterior sampling using diffusion/flow priors. By formulating conditional score- and flow-matching objectives as least-squares problems, it reduces weight estimation to small linear systems and reports improvements across multiple inverse problems and generative backbones.

Critical strengths
  1. The method is simple, computationally efficient, and compatible with both diffusion and flow-matching models.
  2. The experiments cover multiple datasets, inverse problems, and samplers, with useful ablations.
  3. This is a very relevant direction to pursue in inverse problems using generative priors.
Critical weaknesses
  1. The core least-squares observation is intuitive, but the paper should better distinguish its methodological novelty from existing guidance-weighting and calibration approaches.
  2. Is SIMCA’s greedy, trajectory-dependent optimization guaranteed to find globally optimal weights, and can it ensure convergence to the posterior or improve posterior fidelity? Since the evaluation is predominantly empirical, in my opinion the paper does not sufficiently establish when calibration can be expected to work or how robustly it generalizes across tasks.
  3. The reported gains in the results section are also uneven across tasks, with notable regressions on some deblurring and denoising settings.

Overall: Although SIMCA is computationally attractive and shows some promise, the demonstrated conceptual novelty is limited and lacks stronger justification for generalization. Further, I feel inconsistencies in empirical improvements make its overall contribution less convincing for acceptance. A stronger positioning against prior methods and more controlled, reproducible evaluations with some theoretical guarantees in posterior sampling would be needed to substantiate the paper’s claims.

Questions for authors
  1. How sensitive are the calibrated weights to the calibration distribution and minibatch size?
  2. Can the authors provide stronger evidence that improvements generalize beyond the chosen calibration and test settings?
  3. What would it take to extend this to nonlinear inverse problems? What exactly is preventing this method from extending to nonlinear settings?
5 comments
AuthorsOpenReview ↗

Response to Reviewer QPTu (1/3)

We thank the reviewer for the feedback and appreciate the opportunity to clarify several aspects of our work. Please find our answers below.

"The paper should better distinguish its methodological novelty from existing guidance-weighting and calibration approaches."

Guidance weighting and calibration in training-free posterior sampling methods have previously been treated largely as ad hoc empirical tuning problems, with relatively few works explicitly investigating this question. Perhaps the closest work to ours is Li & Pereira (2024), which we discuss in the extended related work (Appendix A). However, the computational complexity of their approach makes it impractical for the setting we target. To the best of our knowledge, our work is the first to provide a principled formulation of guidance calibration based on conditional score matching and conditional flow matching targets. We propose to revise Section 3 accordingly and integrate the extended related work into the main Related Work section.

"Is SIMCA [...] guaranteed to find globally optimal weights, and can it ensure convergence to the posterior or improve posterior fidelity?"

SIMCA is a greedy calibration algorithm whose validation is primarily empirical, with no guarantee of finding globally optimal weights for the objective in Eq. (12). This is largely a consequence of our focus on practical constraints, particularly the requirement that calibration introduce negligible computational overhead. Indeed, we believe that simultaneously designing a method that (i) is computationally highly efficient, (ii) demonstrates strong empirical performance across diverse inverse problems, and (iii) comes with strong theoretical guarantees is particularly challenging. SIMCA successfully achieves the first two objectives, which we believe already constitutes a significant practical contribution. While establishing stronger theoretical guarantees would certainly be valuable, we view this as an important direction for future work rather than a prerequisite for the practical relevance of our approach.

"The reported gains in the results section are also uneven across tasks, with notable regressions on some deblurring and denoising settings."

We acknowledge that SIMCA does not achieve state-of-the-art performance on every metric and every task. However, we emphasize that our evaluation covers four datasets and multiple inverse problems, using both diffusion- and flow-matching-based methods, for a total of 20 experimental settings. Across these settings, SIMCA ranks either first or second in at least 17 out of 20 cases. No competing method demonstrates a comparable level of consistency across the evaluated settings. This demonstrates SIMCA's strong and consistent performance across a diverse range of problems, alongside the important practical advantage that SIMCA can be readily applied to new linear inverse problems with minimal effort.

Regarding the "regression" on deblurring and denoising, we understand the reviewer to be referring to the fact that SIMCA achieves the best performance for deblurring and denoising on CelebA (Table 2 of the paper), but not on AFHQ-Cat (Table 3 of the paper). We would like to emphasize that, even on AFHQ-Cat, SIMCA remains among the top two methods on 4 out of 6 metrics. Moreover, no competing method consistently outperforms SIMCA in these settings: PnP-GS ranks first on 2 metrics but falls outside the top two on the remaining 4, while Flower-OT ranks among the top two on 3 metrics but not on the other 3. The most challenging setting for SIMCA is Gaussian deblurring on ImageNet (Table 1 of the paper), where it nevertheless ranks second in LPIPS and among the top three in PSNR and SSIM. Overall, these results demonstrate that SIMCA provides a favorable combination of consistent empirical performance, computational efficiency, and ease of applicability across diverse inverse problems.

AuthorsOpenReview ↗

Response to Reviewer QPTu (2/3)

"How sensitive are the calibrated weights to the calibration distribution and minibatch size?"

To answer this question, we ran additional experiments on both aspects, and found the calibrated weights to be only mildly sensitive to either. We discuss the minibatch size first, then the calibration distribution.

Minibatch size. For each N∈1,2,5,10N \in \\{1, 2, 5, 10\\}, we draw 2020 disjoint sets of NN calibration images and calibrate independently on each (DDPM, 5050 steps, σm=0.05\sigma_m = 0.05, FFHQ and ImageNet; box inpainting, random inpainting and Gaussian deblurring), with the sampling noise shared across sets so that the variation comes from the calibration images alone. Table R1 reports the likelihood weight wf,tw_{f,t} for Gaussian deblurring. With a single image, the standard deviation of wf,tw_{f,t} across sets is typically 11–14% of its mean on FFHQ and 16–21% on ImageNet, while the schedule itself spans about eight orders of magnitude. It falls to 4–5% and 7–8% for N=10N = 10. The mean stays close to the full-set weight, except at the first step, where small sets slightly overestimate it. The prior weight wu,tw_{u,t} is far more stable, with a relative spread of about 0.05% (median over tt). The inpainting tasks behave in the same way.

Table R1. Gaussian deblurring (DDPM, 50 steps): wf,tw_{f,t} as mean ± std across 20 disjoint calibration sets of NN images, against the full calibration set (100 images on FFHQ, 50 on ImageNet).

DatasetStepFull setN=1N=1N=10N=10
FFHQ49 (noise)1.5×10−71.5\times10^{-7}(1.8±0.2)×10−7(1.8 \pm 0.2)\times10^{-7}(1.6±0.1)×10−7(1.6 \pm 0.1)\times10^{-7}
396.2×10−66.2\times10^{-6}(6.3±0.9)×10−6(6.3 \pm 0.9)\times10^{-6}(6.3±0.3)×10−6(6.3 \pm 0.3)\times10^{-6}
291.07×10−41.07\times10^{-4}(1.06±0.14)×10−4(1.06 \pm 0.14)\times10^{-4}(1.11±0.04)×10−4(1.11 \pm 0.04)\times10^{-4}
191.26×10−31.26\times10^{-3}(1.23±0.15)×10−3(1.23 \pm 0.15)\times10^{-3}(1.26±0.04)×10−3(1.26 \pm 0.04)\times10^{-3}
90.01610.01610.0161±0.00240.0161 \pm 0.00240.0162±0.00070.0162 \pm 0.0007
0 (last)42.142.140.8±1.540.8 \pm 1.541.6±0.841.6 \pm 0.8
ImageNet49 (noise)2.3×10−72.3\times10^{-7}(3.0±1.1)×10−7(3.0 \pm 1.1)\times10^{-7}(2.5±0.2)×10−7(2.5 \pm 0.2)\times10^{-7}
396.8×10−66.8\times10^{-6}(7.9±1.5)×10−6(7.9 \pm 1.5)\times10^{-6}(6.4±0.8)×10−6(6.4 \pm 0.8)\times10^{-6}
290.92×10−40.92\times10^{-4}(1.04±0.17)×10−4(1.04 \pm 0.17)\times10^{-4}(1.03±0.06)×10−4(1.03 \pm 0.06)\times10^{-4}
191.18×10−31.18\times10^{-3}(1.19±0.11)×10−3(1.19 \pm 0.11)\times10^{-3}(1.16±0.07)×10−3(1.16 \pm 0.07)\times10^{-3}
90.01530.01530.0158±0.00250.0158 \pm 0.00250.0157±0.00120.0157 \pm 0.0012
0 (last)45.145.142.5±2.642.5 \pm 2.644.1±1.044.1 \pm 1.0

Calibration distribution. Changing the calibration dataset altogether has a similarly small effect: weights calibrated on ImageNet and applied to FFHQ, or the reverse, lose at most 0.30.3 dB in PSNR compared with weights calibrated on the target dataset (see our answer to the next question). What matters more is the distribution of the states used in the fit: calibrating on the sampler's own states rather than on forward-process states improves LPIPS from 0.1030.103 to 0.0690.069 for DPS (Table 4 of the paper).

In these experiments, the calibrated weights are only mildly sensitive to the calibration images: a single image recovers the full-set schedule up to a spread of 10–20% in wf,tw_{f,t}, which is small compared with the range of the schedule, and a few images reduce it further. We will add these results to the revised paper.

AuthorsOpenReview ↗

Response to Reviewer QPTu (3/3)

"Can the authors provide stronger evidence that improvements generalize beyond the chosen calibration and test settings?"

We ran two additional experiments that change the calibration setting and the test setting respectively.

Beyond the calibration setting: transfer across datasets and backbones. We reconstruct the test images of FFHQ and ImageNet with each dataset's own model but the weights calibrated on the other dataset, without recalibration (DDPM, 5050 steps, σm=0.05\sigma_m = 0.05). Table R2 shows that PSNR drops by at most 0.30.3 dB and LPIPS changes by at most 0.0040.004. The two calibrated schedules are themselves close: the likelihood weights wf,tw_{f,t} agree within 8–9% (median over timesteps), while spanning about seven orders of magnitude along the trajectory.

Table R2. Native weights (calibrated on the target dataset) vs. transferred weights (calibrated on the other dataset), 100 test images.

TaskWeightsFFHQ PSNRFFHQ SSIMFFHQ LPIPSImageNet PSNRImageNet SSIMImageNet LPIPS
Box inpaintingNative23.420.8550.11019.860.7780.210
Transferred23.120.8490.11219.620.7770.212
Random inpaintingNative33.760.9300.07229.460.8520.132
Transferred33.550.9260.07229.480.8520.136
SR (×4\times 4)Native30.000.8530.14426.450.7260.244
Transferred29.920.8500.14526.390.7250.246

Beyond the test setting: misspecified observation model. We keep the sampler fixed (weights and likelihood term for the nominal model) and change only the observations, on FFHQ. When the true noise level deviates by ±20% from the assumed σm=0.05\sigma_m = 0.05, the metrics change by at most 0.70.7 dB and 0.0150.015 LPIPS. For a larger deviation (0.080.08 instead of 0.050.05), PSNR decreases by 0.10.1 dB on box inpainting and 2.02.0 dB on random inpainting, part of which is inherent to the noisier observations. For Gaussian deblurring with a kernel of standard deviation 3.53.5 instead of the assumed 3.03.0, PSNR decreases from 28.2328.23 to 27.4927.49 dB. The full tables are given in our response to Reviewer TM7j.

In these experiments, the calibrated weights transfer between FFHQ and ImageNet, in both directions, with little loss, and the sampler degrades gradually when the test observations depart from the calibration model. This suggests that the weights depend mainly on the measurement setup and only weakly on the dataset and the network. We will add these results to the revised paper.

"What would it take to extend this to nonlinear inverse problems? What exactly is preventing this method from extending to nonlinear settings?"

While our paper focuses on linear measurement operators A\mathcal{A} (Eq. 7), this operator only enters the SIMCA calibration procedure through Eq. 9. Consequently, the formulation of SIMCA does not inherently require linearity and could, in principle, be extended to nonlinear measurement operators, provided they are differentiable and the resulting computations remain numerically stable. However, we believe that the main practical limitation may arise from the Tweedie-based measurement-consistency term borrowed from DPS. Indeed, as reported in DAPS (Table 1), DPS performs poorly on nonlinear inverse problems such as phase retrieval. In preliminary experiments, we attempted to apply SIMCA to phase retrieval but encountered numerical instabilities that would require further investigation. In particular, these instabilities may originate from backpropagation through the nonlinear measurement operator, although we have not yet established their precise cause. We therefore leave a thorough investigation of nonlinear inverse problems to future work and will clarify this limitation in the revised manuscript.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Revised Manuscript Uploaded

Dear Reviewers and AC,

We thank all reviewers for their detailed and constructive feedback. We have uploaded a revised manuscript in which all changes are highlighted in blue. This new version includes all the additional results obtained during this discussion, together with the clarifications promised in our individual responses.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.