acceptodds
Under review as a conference paper at ICLR 2027

Backward-Consistent Diffusion Sampling for Sparsely Observed PDE Inverse Problems

Abstract

Recovering Partial Differential Equation (PDE) coefficient fields from extremely sparse observations is a severely ill-posed inverse problem for which generative machine learning methods (e.g., diffusion models) have become a leading way to encode the prior. Recent state-of-the-art diffusion solvers lift these priors to function spaces, finding a physics-consistent reconstruction in the output space of the diffusion denoiser. We prove that, in a discontinuous PDE setting, output space methods can result in failure to appropriately minimize the unobserved error with the correct coefficient field. Consequently, we propose Function space Backward-Consistent Sampling (FunBCS), an input space optimization approach for solving PDE problems which aims to find the best input such that the denoiser reconstruction is physics-consistent. We then prove that FunBCS appropriately minimizes the unobserved error, unlike output space optimization methods. Per our theoretical analysis, we also provide insights on how to dynamically allocate the number of input space optimization steps used throughout the sampling process. Our evaluations, across four PDE inverse problems (including the discontinuous Darcy flow), demonstrate that FunBCS reduces the reconstruction error by -% while running - faster when compared to the current state-of-the-art. Code is available at https://anonymous.4open.science/r/FunBCS-7C24/.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Inverse Problems, Partial Differential Equations, Diffusion Models, Neural Operators, Function Space, AI for Science

Reviews

Among:
Shape:
1234this paper, Top 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.50 over 2 reviews, above 99% of the others.
Reviewer_EXwUOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes FunBCS for PDE inverse problems with sparse solution observations. The method enforces measurement consistency by optimizing the input of a pretrained denoiser, combines this update with re-noising, and allocates optimization steps across noise levels. An analysis under a two-point prior explains how the method can correct measurement-nullspace errors and escape an incorrect mode, while motivating the allocation of computational effort. Experiments on Poisson, Helmholtz, Navier–Stokes, and Darcy problems demonstrate substantial improvements in reconstruction accuracy and favorable runtime relative to the evaluated baselines.

Critical strengths
  1. The paper addresses a relevant difficulty in sparsely observed PDE inversion, particularly the recovery of discontinuous coefficient fields. Reconstruction improvements are consistent across four tasks and especially substantial on Darcy flow. Sharing the pretrained prior and forward surrogate with DDIS strengthens this comparison.
  2. The theoretical analysis provides an interpretable mechanism for the proposed approach. Figure 4 demonstrates recovery from incorrect modes, while Appendix I.3 shows reduced nullspace errors in a discontinuous Poisson problem using a learned diffusion prior and a neural-operator surrogate.
  3. The empirical study includes runtime–accuracy comparisons, extensive scheduling ablations, and an application of the scheduler to DDIS. The independent calibration samples and the discussion of the spectral-metric discrepancy are also valuable.
  4. The detailed theoretical analysis contributes to understanding how denoiser-input optimization couples measurement consistency with nullspace correction, and how noise levels affect escape from incorrect modes. The paper is clearly written, with a well-organized presentation of the method, theoretical results, and supporting experiments.
Critical weaknesses
  1. The removal of the input-proximity constraint is insufficiently validated. FunBCS omits SITCOM’s [a] regularization term that penalizes deviations from the initial input, but provides no direct ablation of this choice. Under a matched computational budget, comparing variants with λ∥v−v0∥22\lambda\|v-v_0\|_2^2 across different values of λ\lambda would clarify the benefits and trade-offs in reconstruction accuracy, escape from incorrect modes, and optimization stability.
  2. A comparison with prior-covariance-based preconditioning is missing. Under small-step gradient descent, the proposed input optimization can be approximately interpreted as an output-space update preconditioned by ∇vD∗(v,tk)∇vD∗(v,tk)⊤\nabla_v D^{*}(v, t_k)\nabla_v D^{*}(v, t_k)^\top, whereas FAPS uses prior covariance to propagate measurement corrections to correlated unobserved components. I recommend adding a comparison with FAPS [b] and equipping DDIS [c] with covariance preconditioning under the same prior, forward model, computational budget, and optimization-step schedule to clarify the contributions of input parameterization and preconditioner choice. A separate scheduling ablation would help isolate the gains from optimization-step scheduling.

[a] SITCOM: Step-wise triple-consistent diffusion sampling for inverse problems. ICML 2025.

[b] Flow Annealing Posterior Sampling for Function-Space Regression and Inverse Problems. arXiv preprint arXiv:2606.22346 (2026).

[c] Decoupled diffusion sampling for inverse problems on function spaces. arXiv preprint arXiv:2601.23280 (2026).

Questions for authors

The connection between the linear nullspace analysis and the nonlinear experiments needs clarification. The nullspace guarantees rely on a linear observation map A\mathcal{A}, while the experiments also include nonlinear Navier–Stokes and Darcy inverse problems. Although the simplified analysis provides useful insight, the paper should clarify which aspects of the proposed mechanism remain applicable in these settings and which conclusions depend on linearity.

3 comments
AuthorsOpenReview ↗

Response to Reviewer EXwU (1/2)

We thank the reviewer EXwU for their useful feedback. We appreciate their positive remarks on our method's strong reconstruction accuracy (including on the challenging Darcy flow), the interpretability of our studied theoretical mechanism and its accompanying empirical analysis, and the presentation of our paper. We address the reviewer's questions below.

[W1] Regularization (proximity) term.

We thank the reviewer for this question. We did initially consider the regularization (proximity) term that comes with the SITCOM [A] sampler, but found that in most of the cases, its impact is negligible, so we decided not to include it in our original submission.

However, with the additional fifth PDE that we ran in the rebuttal (Wave-Layer, see our response to reviewer 2wEK Q1), we found that its impact is not marginal.

In the following table, we examine the regularization term (λ∥v−v0∥2\lambda\|\boldsymbol{v}-\boldsymbol{v}_0\|^2) at the same budget of M=800M = 800 optimization steps as FunBCS-AC, on 50 test samples of each PDE including the Wave-Layer PDE, on a test set of 50 samples.

λ\lambdaPoissonHelmholtzNavier–StokesDarcyWave-Layer from Poseidon [B]
09.559.275.702.3819.52
0.019.569.315.602.3819.72
0.19.829.795.592.4919.37
116.0014.896.524.0818.12

As observed, for all PDEs, other than the Wave-Layer, when λ>0\lambda>0, the improvement is negligible. For Wave-Layer, the error improves as λ\lambda increases.

This experiment is added to App. I.7 of the revised paper. We clarify the inclusion of the regularization term in the Setup section of Sec. 5.

Additionally, we study how the regularization term affects escaping the incorrect mode on Darcy flow, in a similar setting to our experiment in Sec. 5. We find that for λ\lambda zero or small, we achieve a strong escape rate in line with our theory. However, for λ\lambda large, the escape rate drops significantly. We hypothesize that this is because a large regularization makes the optimization variable close to the initialization. Consequently, the input to the denoiser is not substantially updated at a fixed tkt_k; per our theory in Sec. 3/4, the input to the denoiser should be updated to escape to the true mode.

MethodEscape rate
FunBCS, λ\lambda = 094%
FunBCS, λ\lambda = 0.0194%
FunBCS, λ\lambda = 0.196%
FunBCS, λ\lambda = 122%

This experiment is added to App. I.7 of the revised paper.

[W2] Comparison with FLAPS and preconditioned DDIS.

As requested by the reviewer, we compare FunBCS to FLAPS [C] and a preconditioned version of DDIS [D] across all four of our PDE. We use the publicly available FLAPS Hugging Face checkpoints and code, and are able to reproduce the numbers reported in the FLAPS paper in their setting.

We provide results for FLAPS and preconditioned DDIS on our setting of 3% noiseless observations. We provide these results for both the methods with their original hyperparameters as well as matching our FunBCS (uniform) method. To be precise:

  • FLAPS (native) uses the original FLAPS prior (flow), forward surrogate, and inference-time compute budget.
  • Preconditioned DDIS (native) uses the same prior (diffusion) and forward surrogate as FunBCS, with its original inference-time compute budget. The preconditioner used here is the sample covariance matrix of the prior.
  • FLAPS (matched) uses the same prior (diffusion) forward surrogate, and inference-time compute budget (±5\pm5% s/sample) as FunBCS (uniform).
  • Preconditioned DDIS (matched) uses the same prior (diffusion) forward surrogate, and inference-time compute budget (±5\pm5% s/sample) as FunBCS (uniform).

We provide the reconstruction error of these baselines compared to FunBCS (uniform) below, with full results (e.g., spectral error) in App. I.6 of the revised manuscript:

PDE (units)FLAPS (native)Preconditioned DDIS (native)FLAPS (matched)Preconditioned DDIS (matched)FunBCS (uniform)
Poisson (rel. ℓ2\ell_2 %)39.60 ± 13.11564.65 ± 193.6235.21 ± 11.92556.73 ± 192.8910.51 ± 3.38
Helmholtz (rel. ℓ2\ell_2 %)38.84 ± 11.74373.39 ± 116.9932.91 ± 9.16354.01 ± 114.509.90 ± 2.71
Navier–Stokes (rel. ℓ2\ell_2 %)34.26 ± 6.9367.11 ± 14.7633.85 ± 7.2865.31 ± 13.706.23 ± 1.13
Darcy (pixel error %)15.18 ± 6.9920.75 ± 4.8612.50 ± 5.8120.36 ± 4.872.82 ± 1.56

We find that both baselines do significantly worse than FunBCS. Additionally, we find that preconditioned DDIS diverges on Poisson and Helmholtz. However, a comprehensive analysis of this is outside the scope of our work, since our proposed method does not use the sample covariance matrix as a preconditioner.

AuthorsOpenReview ↗

Response to Reviewer EXwU (2/2)

[Q1] Linearity of the forward operator in the theoretical analysis.

We thank the reviewer for raising this point. We agree that while our experiments on Darcy flow, which has a nonlinear solution operator, provide empirical evidence for the theoretical mechanism we discuss, the formal guarantees are only for linear solution operators (such as those of Poisson and Hemholtz).

We note that Prop. 1 continues to hold for the linearized forward operator A:=DA\mathbf{A} := D\mathcal{A}, and FunBCS continues to precondition the measurement gradient per the chain rule (Prop. 7). However, nonlinear analogues of Cor. 1 and Thm. 1 are more involved, and we leave this to future work. We have updated the manuscript in App. D and Remark 11 in App. K to highlight this.

To provide further evidence of our theoretical results extending to nonlinear PDE, in App. I.5 of the revised manuscript, we examine if Prop. 3 continues to hold for nonlinear Darcy flow after linearizing the forward operator by taking the Jacobian. Specifically, we find that the unobserved error to the true discontinuity location is significantly lower across the reverse process for FunBCS than DDIS.

We provide a summary of these results in the table below, evaluating the unobserved pixel error (%) across the reverse process, with a figure in App. I.5 (Fig. 18a) of the revised manuscript:

Reverse timestep tkt_kFunBCS unobserved pixel error (%)DDIS unobserved pixel error (%)
10210^218.6444.83
10010^09.8913.38
10−210^{-2}8.5211.91
10−410^{-4}8.5211.91

Note that we use pixel error for Darcy, in line with our experiments and the DiffusionPDE paper, but we find that the trend described above holds similarly for the ℓ2\ell_2 error used in our theory: FunBCS has lower unobserved error at each tkt_k.


We again thank the reviewer and are happy to clarify further if any questions remain.


References

[A] I. Alkhouri et al. SITCOM: Step-wise Triple-Consistent Diffusion Sampling for Inverse Problems. ICML, 2025.

[B] M. Herde et al. Poseidon: Efficient Foundation Models for PDEs. NeurIPS, 2024.

[C] Y. Shi et al. Flow Annealing Posterior Sampling for Function-Space Regression and Inverse Problems. NeurIPS, 2026.

[D] T. Lin et al. Decoupled diffusion sampling for inverse problems on function spaces. arXiv preprint arXiv:2601.23280 (2026).

I thank the authors for thoroughly addressing most of my concerns, revising the manuscript, and providing additional experimental results. At this stage, I am inclined to maintain my current score.

Reviewer_2wEKOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The authors of this work propose Function-Space Backward-Consistent Sampling (FunBCS) to reconstruct the coefficient map from PDEs in which the solution is only sparsely observed. The main contribution of this work is a theoretically motivated change to the diffusion sampler. In contrast to previous baselines such as DDIS, the proposed alternative optimizes a previously trained diffusion model around the input of this sampler to improve measurement consistency, rather than keeping it fixed. The change is theoretically motivated, and the analysis is thoroughly supported by the proposed experiments, yielding large improvements over DDIS while retaining DDIS's pretrained denoiser and forward surrogate. The theoretical results identify the mechanism where, in some cases, input optimization can correct components that the DDIS correction preserves. The authors also explore adding a schedule that controls how the optimization is carried out and how much time each noise level is optimized for, showing that tuning this parameter is important, as it can improve all the methods compared.

Critical strengths
  1. The method showcases solid improvement over the relevant baselines across all the tested benchmarks. The gains shown are not the result of increased computational capability, as the method not only performed better but was also faster than its respective baselines, underscoring the importance of the theoretical gap between DDIS and the proposed FunBCS.
  2. The paper finds a concrete limitation of DDIS and then shows how optimizing the denoiser's input can address this limitation. Although, the analysis is restricted to one case, it provides the required intuition and justification of the algorithm.
  3. The contrast with DDIS is particularly strong in the evaluation. I believe that the comparison, which reuses the exact same components as DDIS and yields much better results, substantially supports the main claim, providing evidence that this method addresses incorrect structural predictions and can correct them when DDIS fails.
Critical weaknesses
  1. Robustness to noise in measurements is not evaluated. In real application scenarios the measurements have sources of noise such as sensor noise. It is not realistic to test only in the ideal scenario where the solution to the equation is exact, leaving unclear whether the reported gains persist when the samples used have noise.
  2. Robustness to surrogate accuracy. FunBCS and its assumptions assume that the surrogate model that solves the PDE provides an accurate solution to the system. Even though the authors acknowledge that this should be future work (and I agree), it is important to measure the method's sensitivity to different surrogates and degrees of accuracy.
  3. Robustness to Denoiser accuracy. This is also related to the previous point. It is important to isolate the contribution of each component in the accuracy of the solution. The wrong mode experiment addresses this partially, as it can show that FunBCS can recover from an initially poor initialization, but it does not address what happens if the denoiser itself is less accurate. I would ask the authors to evaluate if their gains persist in the presence of a worse denoiser.
  4. The proposed method is stochastic in nature as it learns the posterior of the coefficients given the observed solution and produces samples of this posterior. Table 1 reports the mean and SD across different test instances but it does not tell anything about the variability of the reconstruction given the same observation.
  5. Clarity of notation and writing. The method itself proposes a fairly compact modification of the inference algorithm and I believe it should be simplified as it is difficult to follow. Please revise your notation, simplify it and address some inconsistencies found, for instance γ\gamma means different things across the document.
Questions for authors
  1. Could you compare your method against the baselines using the acoustic scattering maze from The Well [1], using the existing architectures and standard training procedures? Please include a small robustness check using checkpoints from different training stages and with reduced training data availability, while analyzing the denoiser component and the surrogate model separately (keep one fixed and vary the other).
  2. Could you report the forward prediction errors for all your experiments and also the measurement residuals (the full data consistency loss) for each PDE? Please use a normalized metric such as NMSE.
  3. Could you perform a sensitivity analysis in one of your experiments to measurement noise and compare FunBCS with the baselines at several noise levels?
  4. Could you run the stochastic methods repeatedly and report the mean reconstruction, its error against the ground truth, and the standard-deviation map?

[1] Ohana, Ruben, Michael McCabe, Lucas Meyer, et al. “The Well: A Large-Scale Collection of Diverse Physics Simulations for Machine Learning.” arXiv.org, February 21, 2025. https://arxiv.org/abs/2412.00568.

4 comments
AuthorsOpenReview ↗

Response to Reviewer 2wEK (1/2)

We thank Reviewer 2wEK for their comprehensive feedback. We are glad that the reviewer acknowledges our method's improvement across all tested benchmarks in both reconstruction accuracy and computational efficiency (and the reproducibility of our results), as well as the theoretical analysis of the limitation of DDIS and how FunBCS solves it. We address the reviewer's questions below.

[W1+Q3] Evaluation on additive noisy measurements.

We thank the reviewer for this suggestion. We initially did not consider noise following previous diffusion-based methods (DDIS [A], DiffusionPDE [B], and FunDPS [C]). However, we agree that including noise is more practical.

Therefore, here, we consider additive i.i.d. Gaussian noise to the 500 observed solution values on Darcy flow, with variances 10−310^{-3} and 10−410^{-4}, as in FLAPS [D].

For FunBCS, we use hyperparameter δ\delta as the stopping criterion (tuned), akin to the SITCOM [E] sampler, which we use inside the optimization in Alg. 1.

For DDIS, following the DAPS [F] sampler paper (which tunes on the noisy settings), we tuned the weight of its measurement consistency term (γ\gamma in Eq. 5 of our paper) at each noise level and report its best value.

For FunDPS and DiffusionPDE, both utilize the DPS [G] sampler. DPS tunes the guidance weight on noisy tasks, so we tuned FunDPS and DiffusionPDE's DPS guidance weight as well. However, we found that running both methods' default settings as published obtains better results than tuned guidance weights, so we kept the guidance weights as published and report the error rate on noisy Darcy.

Results are given in the following table, where all methods are evaluated on the same 50 test samples. As observed, FunBCS-AC remains the most accurate method at both noise levels. We again thank the reviewer for this suggestion. We include this experiment in App. I.4 of the revised paper.

MethodDarcy error rate (%) with additive noise with variance 10−410^{-4}Darcy error rate (%) with additive noise with variance 10−310^{-3}
DiffusionPDE8.2910.45
FunDPS5.406.53
DDIS7.718.16
FunBCS-AC3.555.72

[W2+W3+Q1] Evaluation on weakened surrogate and prior.

Following the reviewer's suggestion, we vary one component (the prior or the surrogate) at a time and keep the other at its final checkpoint.

More specifically, to deliberately test the robustness of FunBCS to weak surrogates and priors, we consider the following experiment on discontinuous Darcy flow. For the prior, we use two checkpoints, taken after 5% and 10% of the training epochs. For the surrogate, we consider a checkpoint at 100 (out of 500) epochs, in addition to retraining the surrogate on 10% and 1% of the training pairs.

Results are given in the following table, averaged over 50 test samples where the first row corresponds to the error using the fully trained prior and surrogate. The prior test error is the relative ℓ2\ell_2 error of a single denoising step at noise level σ=1\sigma = 1, and the surrogate test error is the relative ℓ2\ell_2 error of the predicted solution, both on the first 100 test samples.

As observed, the error of FunBCS grows gradually on weakened priors and surrogates and remains below that of DDIS with the final prior and surrogate (7.95% on the same 50 test samples) in every setting. For example, when the prior test error increases from 13.0% to 20.2%, FunBCS overall error increases only from 2.39% to 3.28%.

These results are included in App. I.9 of the updated paper.

Prior settingPrior error of test dataSurrogate settingSurrogate error of test dataFunBCS error
Final prior13.0%Final surrogate0.22%2.39%
5% of training epochs20.2%Final surrogate0.22%3.28%
10% of training epochs15.6%Final surrogate0.22%2.56%
Final prior13.0%At 100 of 500 epochs0.30%2.48%
Final prior13.0%Trained on 10% of the data1.04%3.73%
Final prior13.0%Trained on 1% of the data2.89%6.32%

[W4+Q4] Understanding variability of individual reconstructions.

We thank the reviewer for this question. In the following table, for five Darcy test samples, we draw eight reconstructions from each observation (test sample) using 8 different seeds, and report the average error rate ±\pm std per sample.

As observed, our method provides not only better reconstructions, but also less variability than DDIS. In addition to including the table, we include more samples and a figure of three visualizations in the updated paper (App. I.8).

SampleFunBCS mean error rate ±\pm stdDDIS mean error rate ±\pm std
Test Observation 13.78 ±\pm 0.527.28 ±\pm 0.88
Test Observation 23.60 ±\pm 0.308.51 ±\pm 0.68
Test Observation 31.75 ±\pm 0.304.72 ±\pm 0.78
Test Observation 41.28 ±\pm 0.413.81 ±\pm 0.59
Test Observation 52.10 ±\pm 0.165.97 ±\pm 0.82
AuthorsOpenReview ↗

Response to Reviewer 2wEK (2/2)

[Q1] Comparison on a fifth PDE with discontinuities.

We thank the reviewer for suggesting the acoustic scattering maze PDE from the Well. We considered this PDE, but found that it provides only 2,000 trajectories (1,600 for training under The Well's split). However, 50,000 training fields were needed for each of our (and DDIS's) four PDEs. Furthermore, we have not found any other diffusion-based pretrained prior for this PDE.

That said, we will try our best to conduct experiments for this specific PDE. If 1,600 training instances is sufficient for a working prior and neural operator surrogate, we will report the results either during the rebuttal or for the camera ready (if the paper is accepted).

To fully address the reviewer, here, we include another PDE, which is the Wave-Layer from Poseidon [H]. This PDE models acoustic waves from seismic sources in a layered subsurface medium and has 10,512 trajectories at our resolution of 1282128^2. The task is to recover the piecewise-constant wave speed, with 3 to 6 layers separated by curved interfaces and a random speed in each layer, from 500 observations of the wavefield at t=0.7t = 0.7 given the initial displacement. This PDE, similar to Darcy, is discontinuous in the coefficient field and comes with its own challenges in several respects. The forward map is a wave equation, which is oscillatory and nonlinear in the wave speed, the coefficient field value is not binary like Darcy, and regions that no wave reaches before t=0.7t = 0.7 are not constrained by the observations at all.

We train the diffusion prior and FNO surrogate with the architectures of the other four tasks on its 10,016 training trajectories. The surrogate's relative test error at the true wave speed is 4.9%, which is 20 to 40 times that of the other four tasks, so this task also tests FunBCS and DDIS with a relatively inaccurate surrogate.

We compare with DDIS, which uses the same prior and surrogate. Results are in the following table. As observed, FunBCS achieves better results in terms of both the reconstruction error and the spectral energy error. This experiment (along with reconstruction visualizations) is added to App. J of the revised paper.

100 test samplesRel. ℓ2\ell_2 (%)Spectral error EsE_s
DDIS26.02 ±\pm 8.830.124 ±\pm 0.172
FunBCS-AC20.13 ±\pm 6.610.105 ±\pm 0.105

[Q2] Forward prediction error and measurement residual.

We would like to thank the reviewer for this suggestion.

As the reviewer suggested, at the 500 observed points, for the runs of Table 1, we report

  • The forward prediction NMSE ∣Gh,ϕ(a^)−u∣22/∣u∣22\\|\mathcal{G}_{h,\phi}(\hat{\mathbf{a}})-\mathbf{u}\\|_2^2/\\|\mathbf{u}\\|_2^2 over the full grid, which depends on the surrogate, and

  • The measurement residual NMSE ∣Oh∘Gh,ϕ(a^)−uobs∣22/∣uobs∣22\\|\mathcal{O}_h\circ\mathcal{G}_{h,\phi}(\hat{\mathbf{a}})-\mathbf{u}_{\mathrm{obs}}\\|_2^2/\\|\mathbf{u}_{\mathrm{obs}}\\|_2^2, which represents 3% of the solution space.

Please refer to Figure 20 of App. I.10 for the full box plot results for the NMSE comparison for reconstruction error, forward prediction error, and measurement residual between our method and DDIS.

[W5] Notation.

We thank the reviewer for pointing this out. γ\gamma denotes the likelihood scale in the main text but was reused for the Adam learning rate in the experimental details of the Appendix. We now denote the learning rate by α\alpha, as in Algorithm 1, reserve γ\gamma for the likelihood scale. We ask the reviewer to please let us know if there are any other inconsistencies in our notation or ways we can simplify, and we would be happy to address.


We again thank the reviewer and are happy to address any remaining concerns.


References

[A] T. Lin et al. Decoupled diffusion sampling for inverse problems on function spaces. arXiv preprint arXiv:2601.23280 (2026).

[B] J. Huang et al. DiffusionPDE: Generative PDE-Solving Under Partial Observation. NeurIPS, 2024.

[C] J. Yao et al. Guided Diffusion Sampling on Function Spaces with Applications to PDEs. NeurIPS, 2025.

[D] Y. Shi et al. Flow Annealing Posterior Sampling for Function-Space Regression and Inverse Problems. NeurIPS, 2026.

[E] I. Alkhouri et al. SITCOM: Step-wise Triple-Consistent Diffusion Sampling for Inverse Problems. ICML, 2025.

[F] B. Zhang et al. Improving Diffusion Inverse Problem Solving with Decoupled Noise Annealing. CVPR, 2025.

[G] H. Chung et al. Diffusion Posterior Sampling for General Noisy Inverse Problems. ICLR, 2023.

[H] M. Herde et al. Poseidon: Efficient Foundation Models for PDEs. NeurIPS, 2024.

I think most of my concerns have been addressed I'm updating the score accordingly.

We are glad that our responses have addressed most of the reviewer’s concerns. We would like to thank them for updating their score to clear acceptance.

Comments on the paper

3 comments
AuthorsOpenReview ↗

Response to Reviewer HAwq (1/2)

We thank Reviewer HAwq for their thoughtful feedback. We appreciate their recognition of FunBCS's improvements in reconstruction accuracy and compute efficiency over DDIS [A] as well as the incorrect-mode experiments supporting our proposed theoretical mechanism. We address the reviewer's questions below.

[W1+Q1] DiffusionPDE and FunDPS failure modes and categorization as output space optimization methods.

We agree with the reviewer that these methods do not suffer from the exact same failure mechanism as DDIS highlighted in our theory. That is to say, output space optimization methods can have this failure mechanism, but it is not necessary that they will. We have edited the manuscript in the abstract (lines 21-22), Sec. 2 (lines 181-183), Sec. 3 (lines 262-264), and App. E (lines 1122-1125) to explicitly acknowledge that while the current SOTA DDIS has this failure mechanism, not all output space optimization methods do.

To clarify our categorization of DiffusionPDE [B] and FunDPS [C] as output space optimization methods, we define output space optimization methods to be algorithms such that the denoising (D) and measurement consistency enforcement (M) are separate so that the input to the denoiser is not updated during (M) at a fixed tkt_k.

Although DiffusionPDE and FunDPS differentiate through the denoiser, the input to the denoiser is not updated at a fixed timestep tkt_k per Eq. 22-23. Thus, DiffusionPDE and FunDPS are output space optimization methods. We have updated our manuscript systematically, especially in Sec. 2 (lines 177-179), Sec. 3 (lines 200-201), App. C (lines 1000-1002), and App. E (lines 1119-1121) to clarify this.

In App. I.3 of the revised manuscript, we extend our existing experiment on discontinuous Poisson's equation demonstrating that while DiffusionPDE and FunDPS do not suffer from the exact same failure mechanism described by Prop. 1, they still suffer from high unobserved error (∣PkerA(aˉtk−1−a+)∣\\| \mathbf{P}_{\text{ker} \mathbf{A}}(\mathbf{\bar{a}}_{t_k - 1} - \mathbf{a}_+) \\|) compared to our FunBCS method. We note that we have updated the plots to display relative ℓ2\ell_2 error (%), following the rest of our experiments; the conclusions remain the same.

We provide a brief summary of these results on the unobserved (relative) ℓ2\ell_2 error (%) across the reverse process in a table below. In addition, we provide a figure in App. I.3 with these results (Fig. 12a).

Reverse timestep tkt_kFunBCS unobserved rel. ℓ2\ell_2 (%)DDIS unobserved rel. ℓ2\ell_2 (%)FunDPS unobserved rel. ℓ2\ell_2 (%)DiffusionPDE unobserved rel. ℓ2\ell_2 (%)
10210^24.4039.51389.04389.30
10010^00.575.7133.2033.56
10−210^{-2}0.133.306.378.68
10−410^{-4}0.143.225.298.16
AuthorsOpenReview ↗

Response to Reviewer HAwq (2/2)

[W2+Q2] Guarantees for finite vs. infinite optimization time.

We thank the reviewer for raising this point. We note that Prop. 3 guarantees that after sufficient finite optimization time, FunBCS has no greater unobserved error than DDIS. The experiment provided in App. I.3 of the revised manuscript demonstrates this empirically on Poisson's equation with a discontinuous coefficient field. We have updated Remark 5 in Sec. 4 to clarify this.

Furthermore, both Cor. 1 and Thm. 1 consider idealized infinite optimization time. Cor. 1 assumes exact sampling from eq. 5 for DDIS, corresponding to the equilibrium limit of the Langevin dynamics; we have updated the manuscript in Sec. 3 (lines 215-218) to clarify this. Likewise, in Thm. 1, we consider infinite inner optimization time for FunBCS to solve the optimization problem in Eq. 11.

In Cor. 1 and Thm. 1, we evaluate the conditional means of the sampling processes of DDIS and FunBCS, respectively, at every timestep tkt_k, finding that DDIS fails to recover the correct coefficient field a+\mathbf{a}_+ whereas FunBCS does.

[W2+Q2] Guarantees for the final reconstruction instead of one step in the reverse sampling process.

We appreciate the reviewer raising this point. We agree with the reviewer that our guarantees only hold for a fixed timestep tkt_k, not across the entire reverse process starting at tkt_k. We have updated our manuscript in App. D to state this scope explicitly and distinguish the theoretical guarantees from the experimental evidence.

Nevertheless, the guarantees in Cor. 1 and Thm. 1 continue to hold at the very end of the sampling process (t1→t0t_1 \to t_0). Importantly, per Cor. 1, DDIS fails to recover the true coefficient field in expectation whenever kt1(at1(D))<1k_{t_1}(a_{t_1}^{(D)}) < 1. That is, if the denoiser fails to estimate the true coefficient field exactly, then the measurement consistency enforcement step in DDIS cannot correct this. We have updated Remark 3 in Sec. 3 to make this explicit.

In contrast, as demonstrated in Thm. 1, the measurement consistency enforcement in FunBCS can, even if k1(a1)<1/2k_1(a_1) < 1/2 (i.e., the denoiser favors the incorrect mode a−\mathbf{a}_- instead of a+\mathbf{a}_+). Consequently, while earlier DDIS updates may improve subsequent denoiser decisions as mentioned in Remark 2, our results serve to identify the error that the DDIS updates cannot remove. We have updated the discussion beneath Thm. 1 in Sec. 4 to reflect this.


We again thank the reviewer and are happy to address any remaining concerns.


References

[A] T. Lin et al. Decoupled diffusion sampling for inverse problems on function spaces. arXiv preprint arXiv:2601.23280 (2026).

[B] J. Huang et al. DiffusionPDE: Generative PDE-Solving Under Partial Observation. NeurIPS, 2024.

[C] J. Yao et al. Guided Diffusion Sampling on Function Spaces with Applications to PDEs. NeurIPS, 2025.

Thank you for your response. Most of my concerns are solved.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.