acceptodds
Under review as a conference paper at ICLR 2027

Beyond Mean Curvature: Pathwise Langevin Sampling on Learned Data Manifolds

Abstract

Underdetermined inverse problems admit multiple plausible solutions, yet learned priors entangle data-manifold geometry with training-density preferences, hindering controlled exploration. We propose geometric-prior exploration: the pretrained diffusion model defines the manifold of plausible solutions, while task potential and inverse temperature control density relative to manifold volume. To sample this target when the manifold is known only through a pretrained diffusion model, we introduce , a manifold Langevin discretization combining soft landing with a pathwise curvature correction based on the second fundamental form while avoiding explicit computation of the mean-curvature vector. We construct the required directional-curvature estimator directly from diffusion scores without forming a tangent basis or the full curvature tensor. Under inexact geometric access, we establish stationary weak-bias guarantees and show that repeated learned projection can amplify geometric error as the step size decreases. Our pathwise correction further improves the deviation of the stationary distribution from the manifold from under the standard mean-curvature correction to , where is the discretization step size. Experiments on synthetic manifolds, image inpainting, human-pose editing, and image inverse problems validate the predicted numerical behavior, demonstrate temperature-controlled exploration of task-compatible solutions, and improve reconstruction refinement at comparable runtime.

Primary area:
optimization
Keywords:
Data Manifolds, Manifold Langevin dynamics, Inexact geometry, Diffusion models, Inverse problems

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_sjkEOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies geometric-prior exploration for underdetermined inverse problems. It uses a pretrained diffusion model to approximate data-manifold geometry and targets a Gibbs distribution relative to manifold volume rather than the learned data density. The proposed IIL scheme combines tangent-projected Langevin updates, a pathwise second-fundamental-form correction, and soft landing. The authors construct directional-curvature estimates using denoiser Jacobian-vector products and finite differences, and derive stationary weak-bias bounds under inexact geometric access. Experiments cover synthetic manifolds, MNIST inpainting, Human3.6M pose editing, and terminal refinement of DAPS on FFHQ.

Critical strengths
  1. A relevant treatment of numerical error under learned geometry. The paper addresses an important distinction between accurate geometric operations and their learned approximations. The projection-bias example demonstrates that reducing the step size can amplify persistent projection errors, providing a clear motivation for studying discretization and geometric approximation jointly.

  2. A coherent connection between algorithm design and analysis. The pathwise curvature correction cancels the leading normal fluctuation associated with the realized stochastic increment, while landing stabilizes existing off-manifold deviations. The appendix analyzes these components jointly, including cross terms, localization, and geometric perturbations. The stationary analysis is a meaningful consideration beyond finite-horizon consistency.

  3. Useful implementation choices and experimental controls. The curvature estimator avoids constructing a full tangent basis or curvature tensor. The experiments include exact-versus-learned geometry comparisons, landing-strength sweeps, and equal-runtime controls for the landing-only baseline. The FFHQ evaluation also includes stronger and compute-reallocated DAPS configurations, appropriately accounting for the additional cost of geometric computations.

Critical weaknesses
  1. The algorithmic contribution needs sharper differentiation from existing geometric integrators. Increment-dependent curved updates and their approximate implementations have already been studied by Armstrong and King, while the principal IIL correction follows from the standard second-order expansion of a retraction. Soft landing also has established precedents. The manuscript acknowledges these connections, but does not sufficiently distinguish the novelty of its numerical update from the novelty of its learned-geometry implementation and analysis. Importantly, comparing IIL with a retraction truncated to the same terms would merely compare equivalent updates. What is needed is a precise mathematical positioning of IIL within the existing family of curved schemes, identifying which aspects are inherited and which represent substantive additions.

  2. The incremental contribution of the stationary theory should be made more explicit. Stationary error control under persistent geometric approximation errors is explicitly presented as a core contribution and appears to be the strongest candidate for theoretical novelty. Existing finite-horizon results do not automatically imply such guarantees, so this contribution should not be dismissed. Nevertheless, the key mechanism identified by the authors (second-order cancellation combined with normal stabilization) may also be available to a broader class of existing geometric schemes. It remains unclear whether the O(h+η)O(h+\eta) stationary weak-bias bound depends on a property specific to IIL, or follows more generally for suitable second-order curved updates equipped with landing. The authors should identify the additional assumptions and estimates needed beyond existing geometric consistency, stabilization, and Poisson-equation techniques. If the result is more general, that would itself be valuable, but the contribution should be formulated accordingly rather than attributed primarily to a distinct sampling principle.

  3. The benefits of the geometric-prior target are not fully separated from those of the numerical method. Replacing learned-density weighting with manifold-volume weighting changes the sampling objective; it is not simply a more accurate approximation of the original learned-prior posterior. This choice is plausible for controlled exploration, but greater diversity does not by itself establish better coverage of desirable, task-compatible solutions. Comparisons with DPS also involve differences in initialization and dynamics. Additional controlled experiments that vary training density while keeping geometry and task potential fixed would help isolate the intended benefit. The FFHQ refinement results demonstrate practical reconstruction improvements, but do not establish recovery of the standalone geometric-prior target.

  4. The experimental evidence could more directly validate the claimed theoretical increment. The synthetic experiments appropriately compare exact and learned geometry, but using a fixed learned model across a step-size sweep does not fully characterize dependence on geometric approximation error. Jointly varying the step size and controlled geometric-error levels on tractable manifolds would test the predicted error floor and sensitivity to projection, tangent, and curvature inaccuracies. In realistic applications, the component attribution is also incomplete: Human3.6M and FFHQ do not provide a comprehensive landing-only versus mean-curvature landing versus IIL comparison under matched conditions. Consequently, the DAPS+IIL improvements cannot be attributed specifically to the pathwise correction. The acknowledged use of development problems and unequal hyperparameter-search budgets in MNIST further limits its role as evidence of general performance advantages.

Questions for authors
  1. Does the stationary weak-bias result depend on an IIL-specific property, or does it extend to suitable existing second-order curved schemes with the same landing mechanism? Please identify the precise theoretical addition beyond the available geometric-integrator and stabilization results.

  2. Can the authors jointly vary step size and geometric approximation errors on a tractable manifold to verify the predicted error floor and error dependence, rather than comparing only exact geometry with a fixed learned approximation?

  3. On at least one realistic task, how do landing-only, mean-curvature landing, and IIL compare when initialization, geometric access, postprocessing, and computational budgets are controlled?

  4. Can the authors provide evidence that the change of prior improves meaningful solution coverage, separately from the effects of initialization and numerical discretization?

9 comments
AuthorsOpenReview ↗

A1. Scope and contribution of the stationary theory: conditions and retraction example (Q1; W2)

A1. Scope and contribution of the stationary theory (Q1; W2)

(Q1, W2 --- generality of stationary theory.) We thank the reviewer for recognizing the broader value of stationary control under persistent geometric errors.

Our stationary-analysis framework extends beyond IIL: other landing-equipped curved schemes can achieve the same O(Δt+ηeff)O(\Delta t+\eta_{\rm eff}) stationary rate, provided their geometric-operator and numerical errors satisfy additional conditions preserving normal stability and projected-generator consistency. Here ηeff\eta_{\rm eff} is a scheme-dependent aggregate of these approximation errors. Second-order geometric consistency alone does not guarantee this rate.

Conditions for the same rate O(Δt+ηeff)O(\Delta t+\eta_{\rm eff}) as IIL.

Proposition 19 separates stationary bias into normal and projected-generator errors. Let X+X^+ be one step from xx, d(x)=dist⁡(x,M)d(x)=\operatorname{dist}(x,\mathcal M), π\pi nearest-point projection, LΔt=(KΔt−I)/Δt\mathcal L_{\Delta t}=(K_{\Delta t}-I)/\Delta t the discrete generator, L\mathcal L the target generator, and ϑx\vartheta_x the cutoff. Sufficient estimates are E[d(X+)2∣x]≤(1−cΔt)d(x)2+CΔt,ηeff2+CΔt3,\mathbb E[d(X^+)^2\mid x]\le(1-c\Delta t)d(x)^2+C\Delta t\\,\eta_{\rm eff}^2+C\Delta t^3,

\le C\lVert\varphi\rVert_{C^4}(\Delta t+\eta_{\rm eff}+d(x)).$$ These estimates match Lemmas 25 and 30 (Eqs. 91 and 105) for IIL, where $c>0$ and $C$ is independent of $\Delta t$ and the admissible approximation errors. Under analogues of Assumptions 1, 5, 7, invariant-law existence, and fixed margins, Lemmas 8, 22, Proposition 19, and Poisson regularity yield $\operatorname{dist}_{\mathcal F}(\rho_{\Delta t},\mu)=O(\Delta t+\eta_{\rm eff})$. **Concrete example: approximate retraction.** Consider a smooth second-order retraction-based Langevin scheme (e.g., Bharath et al., 2025), augmented with soft landing. **To isolate retraction approximation error, we assume exact nearest projection $\pi$ and tangent projector $P$ in this illustrative example.** The update is $$\widehat X^+=x+\ell\Delta t(p-x)+[\widehat{\mathcal R}_p(w)-\widehat{\mathcal R}_p(0)],\quad p=\pi(x),\quad w=-\Delta t P(p)\nabla U(p)+\sqrt{2\Delta t/\beta}\\,P(p)\xi.$$ With $\xi\sim\mathcal N(0,I)$ (localization suppressed), let $\mathcal R_p(w)\in\mathcal M$ be the exact second-order retraction. Sufficient **additional operator-error conditions** are $$D\widehat{\mathcal R}_p(0)=D\mathcal R_p(0)=I,\quad \sup_p\lVert D^2\widehat{\mathcal R}_p(0)-D^2\mathcal R_p(0)\rVert_{\rm op}\le\varepsilon_{\mathcal R},$$ plus uniform $C^4$ bounds on the localized tangent ball. For the one-step error $e_{\Delta t}=\widehat X^+-X_{\rm ref}^+$ with shared noise/localization, Gaussian symmetry gives $$\\|\mathbb E[e_{\Delta t}\mid x]\\|\le C(\Delta t\\,\varepsilon_{\mathcal R}+\Delta t^2),\quad \mathbb E[\\|e_{\Delta t}\\|^2\mid x]\le C(\Delta t^2\varepsilon_{\mathcal R}^2+\Delta t^3).$$ Since $\mathcal R_p(w)\in\mathcal M$, $$d(\widehat X^+)^2 \le \\|(1-\ell\Delta t)(x-p)+e_{\Delta t}\\|^2.$$ Expanding this inequality and applying the moment bounds and Young's inequality gives $$\mathbb E[d(\widehat X^+)^2\mid x] \le(1-c\Delta t)d(x)^2 +C\Delta t\\,\varepsilon_{\mathcal R}^2 +C\Delta t^3.$$ The second-order reference retraction and the error-moment bounds also give projected-generator consistency with error $O(\Delta t+\varepsilon_{\mathcal R})$. Hence **the $O(\Delta t+\varepsilon_{\mathcal R})$ stationary guarantee extends to this inexact retraction scheme**, subject to the same regularity, localization, and invariant-law conditions.
AuthorsOpenReview ↗

A1. Scope and contribution of the stationary theory: generality of the stationary-analysis framework (Q1; W2)

A1. Scope and contribution of the stationary theory (Q1; W2)

(Q1, W2 --- generality of stationary theory.)

Generality of the stationary-analysis framework.

We already apply the stationary-bias analysis framework beyond IIL in our paper to HL- and hard projection (π\pi)-schemes (Theorem 34), with different one-step estimates from those required for the IIL-type rate.

  • For HL-scheme, the additive discretization defect in the one-step normal squared-distance estimate is O(Δt2)O(\Delta t^2), rather than IIL's O(Δt3)O(\Delta t^3) (Lemma 27 vs. Lemma 25). Consequently, HL achieves the weaker stationary bound O(Δt+ηH,ℓ)O(\sqrt{\Delta t}+\eta_{H,\ell}), where ηH,ℓ=εP+εH+(1+ℓ)επ\eta_{H,\ell}=\varepsilon_P+\varepsilon_H+(1+\ell)\varepsilon_\pi.
  • For hard projection π\pi-scheme, the normal distance is controlled by επ\varepsilon_\pi, but applying an inexact projection without step-size scaling introduces an O(επ/Δt)O(\varepsilon_\pi/\Delta t) term in the projected generator (Lemmas 29, 32), yielding O(Δt+εP+επ+επ/Δt)O(\Delta t+\varepsilon_P+\varepsilon_\pi+\varepsilon_\pi/\Delta t).

Thus, while the stationary-analysis framework is transferable, the resulting rates depend on both the numerical construction and the accuracy of its geometric operators.

Our focus.

Our paper addresses the challenging setting of unknown data manifolds whose geometric operators are approximated from pretrained diffusion scores, with errors that may persist as Δt→0\Delta t\to0. For IIL, ηII,ℓ=εP+εII+(1+ℓ)επ\eta_{{\rm II},\ell}=\varepsilon_P+\varepsilon_{\rm II}+(1+\ell)\varepsilon_\pi; Lemmas 23-25 and 30 establish the required estimates under simultaneous persistent errors in projection, tangent projection, and directional curvature. Our theoretical contribution therefore encompasses both a transferable stationary-error framework and its rigorous quantitative realization under inexact learned geometry.

AuthorsOpenReview ↗

A1. Numerical positioning and novelty. (W1)

A1. Numerical positioning and novelty. (W1)

(W1.) We position IIL as a pathwise curvature-corrected landing scheme for manifold Langevin sampling under inexact geometric access. Our focus is on unknown data manifolds, where projection, tangent, and curvature operators are approximated from pretrained diffusion scores, with errors that may persist as Δt→0\Delta t\to0. Our main contributions are a practical numerical construction and rigorous stationary-accuracy guarantees under these persistent geometric errors.

Distinction from existing schemes.

Armstrong-King (2022) assumes exact access to the SDE vector fields encoding the manifold geometry. Bharath et al. (2025) establish stationary accuracy under on-manifold retraction access. Jeon et al. use known constraint functions and their derivatives for landing with mean-curvature-corrected Langevin dynamics.

None establishes stationary weak-bias guarantees under persistent errors in approximate projection, tangent, and curvature operators.

Our numerical construction.

IIL uses soft landing ℓΔt(p−x)\ell\Delta t(p-x) and the same-noise directional correction −β−1Δt,IIp(v,v)-\beta^{-1}\Delta t\\,\mathrm{II}_p(v,v), where v=P(p)ξv=P(p)\xi also drives the tangent update. As shown in Eq. (11) and Proposition 6, the directional correction is the O(Δt)O(\Delta t) stochastic term of a second-order retraction expansion. An identical quadratic truncation equipped with the same landing reproduces IIL and is therefore not an independent baseline. However, the quadratic correction alone does not provide the off-manifold stabilization required under inexact geometry; exact retractions remain on the manifold by construction, but their guarantees do not automatically extend to persistent learned-operator errors.

Provable advantages.

Our analysis establishes the complementary roles of both mechanisms:

  • (i) Soft landing avoids the επ/Δt\varepsilon_\pi/\Delta t stationary-bias amplification caused by repeated approximate hard projection (Example 1; Theorem 34).

  • (ii) Same-noise pathwise curvature correction cancels the leading stochastic normal residual pathwise (Lemma 23), unlike mean-curvature landing. This distinction is sharp: Proposition 36 establishes stationary mean manifold distances of Θ(Δt)\Theta(\sqrt{\Delta t}) for HL versus Θ(Δt)\Theta(\Delta t) for IIL on the circle.

Overall.

To the best of our knowledge, IIL is the first pathwise curvature-corrected landing scheme for sampling on data manifolds. More importantly, we establish its stationary accuracy under persistent errors in diffusion-learned geometry. We construct a score-derived directional-curvature estimator with explicit approximation-error bounds (Theorem 39). Under Assumptions 1, 5, and 7 and fixed admissible landing strength, we prove first-order stationary smooth weak-bias control O(Δt+ηII,ℓ)O(\Delta t+\eta_{\mathrm{II},\ell}), where ηII,ℓ=εP+εII+(1+ℓ)επ\eta_{\mathrm{II},\ell}=\varepsilon_P+\varepsilon_{\mathrm{II}}+(1+\ell)\varepsilon_\pi (Theorem 33).

Reviewer_DXVHOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The authors propose a Langevin algorithm for sampling from a data manifold whose geometry is determined by a trained diffusion model but whose density is determined by the measurement likelihood in an inverse problem. The main technical contribution is a curvature correction that allows the algorithm to accurately sample the target distribution. The authors provide experiments on known synthetic manifolds, MNIST image inpainting, and human pose editing, and they show that their geometric prior mechanism can improve DAPS performance on FFHQ-256 inverse problems.

Critical strengths

None

Critical weaknesses

The main issue is that this paper lacks a clear and compelling motivation. The idea of a “geometric prior” makes no sense under standard Bayesian inference, and the authors do a poor job of explaining why a geometric prior is appealing and how to even define it. The results on inpainting and pose editing do not paint a clear picture of what the geometric prior is doing. For instance, Figure 4(d) shows that the proposed method provides the least sample diversity. In general, it is hard to tell what the takeaways of Figures 4 and 5 should be. This paper leaves me unconvinced that a geometric prior is at all desirable or necessary.

The second issue is that this paper is extremely difficult to read. The text is very technical and information-dense, making it difficult for a human reader to understand.

Questions for authors

None

4 comments
AuthorsOpenReview ↗

A1. Motivation and definition of the geometric prior

Thank you for reviewing our paper. We address your concerns below.

A1. Motivation and definition of the geometric prior

The idea of a “geometric prior” makes no sense under standard Bayesian inference, and the authors do a poor job of explaining why a geometric prior is appealing and how to even define it.

Our motivation is to reuse pretrained diffusion scores for geometric information while choosing the exploration target separately, decoupling plausibility from training frequency.

Limitations of learned-prior targets. Using pretrained diffusion models as learned priors is established practice, including DDRM (Kawar et al., NeurIPS 2022) and DPS (Chung et al., ICLR 2023). Their scores approximate gradients of noise-perturbed data densities (Song et al., ICLR 2021), retaining training-frequency weighting in the learned-prior target (§1, Eq. (2), p. 1). Consequently, even when two plausible solutions satisfy the same task constraints equally well, the learned prior may favor the one more frequently observed in the training data. For instance, an uncommon but natural humanoid pose may still be desirable for editing (i.e. satisfy the joint position constraints). However, sampling from the learned-prior target could yield the same common poses occurring in the training data. This limitation from using the learned prior weighting fdata⁡f_{\operatorname{data}} in diffusion posterior sampling was also mentioned before in the literature: Cohen et al. (ICLR 2024) highlight the limited semantic diversity of task-aligned posterior samples in image editing problems, while Khalafi et al. (NeurIPS 2024) study general training-distribution biases in diffusion models.

Our proposed geometric prior. Under the manifold hypothesis, the data support M\mathcal M characterizes plausible data, while the density fdataf_{\mathrm{data}} describes the relative frequency of different data points. For tasks admitting multiple plausible solutions, we want to explore alternatives satisfying the same task constraints, even if some occur less frequently in the training data. To decouple plausibility from training frequency, we remove the factor fdataf_{\mathrm{data}} while retaining the manifold-volume measure, defining the geometric-prior target (§1, Eq. (3), p. 2):

μ(dx)∝e−βU(x)vol⁡M(dx).\mu(dx)\propto e^{-\beta U(x)}\operatorname{vol}_{\mathcal M}(dx).

Here M\mathcal M specifies plausible solutions we consider, while UU encodes task preferences at hand. Under our compactness assumption on M\mathcal{M}, normalized manifold volume defines a proper prior (Appendix A.1), i.e. we propose an alternative prior that is compatible with standard Bayesian inference. By removing training-frequency weighting, our geometric prior aims to address the limited diversity of task-aligned solutions caused by training-density bias.

Theoretical and empirical verification. Theoretically, our geometric-prior target (§1, Eq. (3), p. 2) removes the training-density factor fdataf_{\mathrm{data}}. Figure 1 (§1) illustrates this distinction: varying the training density changes the learned-prior target (Eq. (2)) but leaves the geometric-prior target (Eq. (3)) unchanged. Moreover, by the Gibbs variational principle, it maximizes entropy relative to manifold volume among distributions with the same expected task potential. This provides a theoretical justification for promoting diversity among task-aligned solutions without additional training-frequency weighting.

Empirically, our experiments support the motivation for geometric-prior exploration: improving diversity among plausible, task-aligned solutions. Figure 4(c) (MNIST inpainting) shows that geometric-prior-based sampling (IIL\mathrm{II}L, Eq. (3)) achieves greater diversity than learned-prior-based methods (DPS and Ref-DPS, Eq. (2)) at comparable digit-class retention. Similarly, Figures 5(b,c) (H36M human pose editing) show greater diversity in unconstrained joints than DPS at comparable editing error over the evaluated range.

References

AuthorsOpenReview ↗

A2. Experimental interpretation and takeaways

A2. Experimental interpretation and takeaways

The results on inpainting and pose editing do not paint a clear picture of what the geometric prior is doing. For instance, Figure 4(d) shows that the proposed method provides the least sample diversity. In general, it is hard to tell what the takeaways of Figures 4 and 5 should be.

Following the motivation in A1, our experiments in Figures 4–5 evaluate controlled diversity: obtaining diverse plausible solutions while satisfying task constraints. We therefore summarize the task-specific limitations of learned-prior methods and how our geometric-prior addresses them empirically, using the proposed sampler IIL\mathrm{II}L.

ExperimentLimitations of learned-priorsEvidence and takeaway
MNIST image inpaintingLearned priors may favor digit patterns that are common in the training set over other plausible completions in the selected digit class.Fig. 4(c): IIL achieves greater missing-region diversity than DPS and Ref-DPS at comparable class retention.
H36M human pose editingLearned priors may favor common poses over less frequent but equally plausible configurations satisfying the same edit.Figs. 5(b,c): IIL achieves greater free-joint diversity than DPS at comparable editing error and runtime.

Clarification of Figures 4 and 5. Regarding the concern about Figure 4(d), visual diversity alone does not establish better performance: completions must also preserve the selected digit class and pixels from the observed part (bottom slice of the image). We measure diversity in the masked region and assess it together with task satisfaction. Figure 4(a) compares diversity against observation error; panel (b) reports the fraction of samples satisfying both identity retention and observation fidelity; and panel (c) compares diversity against identity retention. In particular, panel (c) shows greater diversity for IIL than DPS and Ref-DPS at comparable retention. These quantitative results explain the trade-offs that the examples in panel (d) cannot fully convey. Similarly, Figures 5(b,c) quantify diversity against editing error, while Figure 5(d) illustrates distinct poses satisfying the same editing constraints.

Together, these results demonstrate the practical value of our proposed sampler IIL\mathrm{II}L targeting the geometric-prior distribution (§1, Eq. (3), p. 2) for exploring diverse, task-compatible solutions. We will clarify these objectives and takeaways in §4 and the figure captions.

References

AuthorsOpenReview ↗

A3. Readability and technical contributions

A3. Readability and technical contributions

The second issue is that this paper is extremely difficult to read. The text is very technical and information-dense, making it difficult for a human reader to understand.

We regret that the reviewer found our paper difficult to read and appreciate their feedback on this matter. We contribute both (i) a practical sampler and (ii) a theoretical analysis of its sampling accuracy and we see the analysis (ii) as a central contribution: it quantifies how discretization and geometric errors affect the given sampler. This level of theoretical detail is consistent with established theory-rich research at ICLR and related venues, including Song et al. (ICLR 2021) and De Bortoli et al. (NeurIPS 2022).

Here, we would like to recall and clarify how the paper progresses from our objective to the proposed method:

  • Section 1 motivates and defines the geometric-prior target distribution.
  • Section 2 identifies difficulties in sampling it with inexact geometric operations.
  • Section 3 develops the IIL-sampler, its stationary guarantees and diffusion-score-based implementation.
  • Section 4 evaluates numerical behavior and applications.

In these Sections we tried to support our mathematical development via visual intuition and explicit (simple) toy examples:

  • Visual intuition: Figure 2 (p. 2) illustrates why curvature correction depends on the magnitude and direction of the realized step, using a parabola and a catenoid.
  • Concrete calculation: Examples 1–2 (p. 5) give explicit calculations on a line, showing projection-bias amplification and how landing avoids it. Example 3 (p. 5) uses a circle to quantify the improvement in stationary manifold adherence from correcting realized rather than average curvature. These observations motivate combining directional curvature correction with landing.
  • General analysis: The general analysis in Section 3.2 establishes, under the stated assumptions, how step size and persistent geometric inaccuracies affect both manifold deviation and stationary sampling bias. This provides quantitative understanding beyond the examples.

Changes to improve readability:

For the sake of improving readability, we have decided to make the above progression explicit in the introduction, add a plain-language interpretation before the main stationary guarantee (Theorem 3), and revise the captions of Figures 4–5 to state the task, metric conditioning, and principal takeaway for the results. We hope that these changes will make the paper more understandable for future readers.

References

AuthorsOpenReview ↗

Follow-up on our responses

Thank you again for your valuable feedback. We hope our responses have addressed your concerns, and we would be happy to hear your thoughts.

Reviewer_h4t7OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper studies sampling for underdetermined inverse problems using a geometric prior derived from a pretrained diffusion model. Instead of weighting solutions by their learned data density, it targets a Gibbs distribution relative to the manifold’s induced volume measure. The proposed IIL discretization combines soft landing with a directional second-fundamental-form correction coupled to the sampled tangent noise. The required geometry is estimated using denoisers, Jacobian-vector products, and finite differences.

Under explicit regularity, localization, and geometric-accuracy assumptions, the authors establish stationary weak-bias bounds. In particular, the pathwise correction improves the stationary off-manifold deviation from an order-square-root step-size bound for mean-curvature landing to a first-order bound. Experiments cover synthetic manifolds, MNIST inpainting, Human3.6M pose editing, and terminal refinement of DAPS on eight FFHQ inverse problems.

Critical strengths

The paper identifies and analyzes a consequential finite-step geometric effect. The distinction between average mean curvature and the curvature associated with the realized stochastic increment is clearly motivated. The circle example makes the difference concrete, and the stationary analysis quantifies how discretization and geometric approximation errors contribute to bias. This provides useful understanding beyond introducing another heuristic diffusion-guidance method.

The practical evaluation includes meaningful compute controls. The FFHQ experiments share the upstream DAPS trajectory and modify its terminal stage, while Appendix K.7 additionally considers reduced-host and stronger-native-DAPS controls using measured runtime. These comparisons strengthen the evidence that geometric refinement can improve reconstruction quality within a comparable compute budget. The gains over the native controls extend across all eight tasks in Table 1.

Critical weaknesses

The real-data experiments do not sufficiently isolate the benefit of the proposed pathwise curvature correction. The synthetic experiments compare IIL with mean-curvature landing, and MNIST includes landing-only controls. However, the FFHQ comparison changes several components together: tangent-projected drift and noise, landing, and curvature correction. The runtime controls address additional computation, but do not determine which component produces the improvement. Similarly, the directional-curvature diagnostics in Appendix K.8 demonstrate variation in the estimated geometry rather than its effect on reconstruction or exploration. This matters because the pathwise correction is the central methodological contribution. May be adding the imagenet experiment.

The empirical evidence for target-distribution accuracy is less complete than the evidence for editing and refinement. Beyond the illustrative density comparison in Figure 1, the quantitative synthetic evaluation primarily measures manifold distance and one projected potential expectation. Agreement in a single expectation does not establish correct distributional sampling: for example, the catenoid’s threefold angular potential permits incorrect mode weights without necessarily changing the expected potential. Moreover, the stationary guarantees do not establish that the finite trajectories used experimentally have mixed. The reference/source initialization and deterministic terminal refinement in the editing experiments further limit their interpretation as evidence of sampling the stated Gibbs target. The authors appropriately acknowledge several of these scope limitations, but stronger distributional validation would make the central exploration claim more convincing.

Questions for authors

Contribution attribution: At comparable measured runtime, how do DAPS with landing-only refinement and DAPS with mean-curvature landing compare with DAPS+IIL on representative FFHQ tasks? A stochastic mean-curvature implementation would be a useful control. Since Eq. (7) expresses mean curvature as an expectation of directional curvature, comparing an independent trace probe with the same-noise coupling used by IIL would also help identify the source of the benefit.

Distributional validation: On the synthetic manifolds with known target distributions, can you report mode probabilities or a distributional discrepancy using multiple test functions, alongside the potential expectation? How sensitive are these results to initialization and simulation horizon? These checks would help distinguish improved geometric adherence from accurate exploration of the intended target.

5 comments
AuthorsOpenReview ↗

A1. Isolating the Contributions of Landing and Curvature Correction (Q1; W1)

We thank the reviewer for the time and effort spent reviewing our paper.

A1. Isolating the contributions of landing and curvature correction (Q1; W1)

(Q1, W1) We thank the reviewer for highlighting the need to separate the contributions of landing, mean curvature correction, and pathwise curvature correction.

We address these contributions at three levels:

  1. Landing improves reconstruction over hard projection and DAPS.

    On the original eight FFHQ tasks at comparable runtime (Table A), hard projection (π\pi) substantially degrades PSNR, while landing-only refinement (L) outperforms the stronger DAPS baseline across all tasks, improving PSNR by 0.065-1.699 dB, demonstrating the importance of landing.

  2. Curvature correction further improves reconstruction beyond landing.

    Under the original FFHQ settings, L, HL-G1 (one-independent-probe mean-curvature landing), and IIL achieve similar PSNR (Table A). This is consistent with curvature-induced discretization errors being relatively small at the original Langevin temperature, since curvature-induced normal displacements scale as Δt/β\Delta t/\beta. Increasing the Langevin temperature (decreasing β\beta) at fixed Δt\Delta t amplifies these errors, making curvature correction more consequential. Indeed, across six higher-temperature FFHQ settings (Table B), both HL-G1 and IIL outperform L under the same landing strength. In particular, IIL improves PSNR by 0.033–0.138 dB, with positive paired 95% confidence intervals in all six settings. These results demonstrate that curvature correction provides benefits beyond landing alone, particularly at higher Langevin temperatures. Since different tasks may require different sampling temperatures, curvature correction offers greater flexibility in temperature selection while maintaining an advantage over the uncorrected sampler.

  3. Pathwise curvature correction can further improve upon mean-curvature correction.

    On FFHQ, IIL and HL-G1 achieve comparable reconstruction quality even at higher temperatures (Table B). This is consistent with our empirical findings in Appendix K.8, which show that FFHQ exhibits little variation in directional-curvature magnitudes (CVnormFFHQ=0.018\mathrm{CV}^{\mathrm{FFHQ}}_{\mathrm{norm}}=0.018), leaving limited room for additional improvements from pathwise correction.

    To further distinguish the contribution of pathwise II\mathrm{II} from mean curvature HH, we extend both the geometric diagnostics and reconstruction experiments to CIFAR-10. Compared with FFHQ, CIFAR-10 exhibits greater variation in directional-curvature magnitudes CVnormCIFAR−10=0.115\mathrm{CV}^{\mathrm{CIFAR-10}}_{\mathrm{norm}}=0.115.

    Consistently, across six CIFAR-10 inverse tasks at comparable runtime (Table C), IIL improves over independent-probe HL-G1 by 0.015-0.038 dB, accounting for 5.0-27.8% of the IIL-L gain (positive paired intervals on five of six tasks). IIL also improves over L by 0.091-0.445 dB, accounting for 3.5-42.4% of its total improvement over Stronger DAPS (positive intervals on all six tasks), further supporting point (2).

Overall, these ablations establish the distinct contributions of landing, curvature correction, and its pathwise coupling. The additional FFHQ and CIFAR-10 experiments demonstrate that curvature improves reconstruction beyond landing, while pathwise II\mathrm{II} can provide further gains over mean curvature HH, consistent with the directional-curvature differences observed in Appendix K.8.

AuthorsOpenReview ↗

A1. Supporting Experimental Results (Tables A–C)

Table A. FFHQ component attribution. Mean PSNR (dB; higher is better) across 100 test images at comparable runtime.

TaskStronger DAPSπ\piLHL-G1IIL
Random inpainting30.69825.76532.39732.40732.405
Box inpainting24.38522.66324.78724.78724.787
SR$\times$429.26425.05030.19230.18230.181
Gaussian deblurring29.50926.04330.42430.41230.412
Motion deblurring31.58425.96132.83432.82732.826
Nonlinear deblurring28.42017.27828.55428.55128.551
Phase retrieval27.34015.61827.75927.73027.730
HDR27.79318.14027.85827.85027.850

Table B. FFHQ curvature ablation at higher Langevin temperatures. PSNR differences (dB) with paired 95% CIs. Methods share step size and horizon within each setting. Results use 20 test images, 6 seeds.

Taskβ\betaHL −- L [95% CI]IIL −- L [95% CI]IIL −- HL [95% CI]
Random inpainting0.0625+0.07976 [+0.06009, +0.10018]+0.08031 [+0.06030, +0.10108]+0.00055 [-0.00041, +0.00143]
Random inpainting0.03125+0.05709 [+0.03634, +0.07991]+0.05627 [+0.03613, +0.07834]-0.00082 [-0.00280, +0.00065]
SR$\times$40.0625+0.09541 [+0.06779, +0.12833]+0.09613 [+0.06818, +0.12947]+0.00072 [+0.00029, +0.00124]
SR$\times$40.03125+0.03299 [+0.01090, +0.05842]+0.03343 [+0.01089, +0.05962]+0.00044 [-0.00052, +0.00156]
Gaussian deblurring0.0625+0.13695 [+0.10812, +0.16713]+0.13757 [+0.10848, +0.16787]+0.00062 [+0.00015, +0.00113]
Gaussian deblurring0.03125+0.09334 [+0.06374, +0.12842]+0.09681 [+0.06581, +0.13338]+0.00347 [+0.00143, +0.00563]

Table C. CIFAR-10 component attribution. Results use 25 test images across 4 seeds. Positive values indicate improvement (LPIPS sign-reversed); brackets are paired 95% CIs. Let SS denote Stronger DAPS and HLHL denote HL-G1. The two percentage columns are:

  • Curv. share: (IIL−L)/(IIL−S)(\mathrm{IIL}-L)/(\mathrm{IIL}-S), the share of IIL's total gain over Stronger DAPS contributed beyond landing.
  • II share: (IIL−HL)/(IIL−L)(\mathrm{IIL}-HL)/(\mathrm{IIL}-L), the share of the gain beyond landing attributable to using pathwise curvature instead of mean curvature.
TaskMetricL-SHL-LIIL-HL [95% CI]IIL-L [95% CI]Curv. share (%)II share (%)
Random inpaintingPSNR+0.297+0.099+0.038 [+0.022, +0.057]+0.137 [+0.076, +0.218]31.527.8
SSIM+0.00558+0.00288+0.00094 [+0.00047, +0.00151]+0.00382 [+0.00202, +0.00617]40.624.7
LPIPS+0.00787+0.00245+0.00102 [+0.00013, +0.00187]+0.00346 [+0.00115, +0.00617]30.629.3
Gaussian deblurringPSNR+2.390+0.067+0.024 [+0.010, +0.038]+0.091 [+0.048, +0.138]3.726.5
SSIM+0.05285+0.00118+0.00062 [+0.00033, +0.00092]+0.00180 [+0.00101, +0.00268]3.334.6
LPIPS+0.03547+0.00203-0.00007 [-0.00118, +0.00100]+0.00196 [-0.00014, +0.00419]5.2-3.8
Motion deblurringPSNR+2.568+0.077+0.015 [+0.001, +0.030]+0.092 [+0.038, +0.148]3.516.6
SSIM+0.04201+0.00122+0.00044 [+0.00013, +0.00079]+0.00166 [+0.00067, +0.00275]3.826.8
LPIPS+0.02590+0.00213+0.00034 [-0.00055, +0.00120]+0.00247 [+0.00036, +0.00465]8.713.7
Compressed sensingPSNR+0.604+0.423+0.022 [+0.007, +0.037]+0.445 [+0.309, +0.594]42.45.0
SSIM+0.01476+0.01629+0.00096 [+0.00046, +0.00145]+0.01725 [+0.01075, +0.02447]53.95.6
LPIPS+0.01104+0.01364+0.00043 [-0.00062, +0.00130]+0.01407 [+0.00824, +0.02038]56.03.0
Fourier reconstructionPSNR+0.454+0.089+0.024 [-0.005, +0.055]+0.113 [+0.015, +0.216]19.921.0
SSIM+0.00870+0.00320+0.00036 [-0.00060, +0.00134]+0.00356 [+0.00118, +0.00619]29.010.1
LPIPS-0.00359+0.00508+0.00046 [-0.00113, +0.00193]+0.00553 [+0.00201, +0.00934]284.88.2
Row inpaintingPSNR+0.312+0.106+0.035 [+0.017, +0.053]+0.141 [+0.060, +0.236]31.124.6
SSIM+0.00106+0.00292+0.00082 [+0.00039, +0.00131]+0.00374 [+0.00186, +0.00593]77.921.9
LPIPS+0.00030+0.00424+0.00047 [-0.00055, +0.00169]+0.00470 [+0.00197, +0.00785]94.110.0
AuthorsOpenReview ↗

A2. Distributional Validation and Sensitivity to Initialization and Horizon (Q2; W2)

A2. Distributional validation and sensitivity to initialization and horizon (Q2; W2)

(W2, Q2) We thank the reviewer for requesting additional distributional diagnostics and sensitivity analyses for initialization and simulation horizon.

We extended our synthetic experiments through three complementary analyses: distributional accuracy across multiple test functions, initialization sensitivity, and simulation-horizon sensitivity.

  1. Multiple test functions validate distributional accuracy beyond the potential expectation.

    We add eight smooth test functions per manifold and compare their empirical expectations with the known Gibbs targets. On the catenoid, cos⁡u,sin⁡u,cos⁡2u,sin⁡2u\cos u,\sin u,\cos 2u,\sin 2u probe angular modes, while t,t2,tcos⁡u,tsin⁡ut,t^2,t\cos u,t\sin u probe axial moments and angular--axial dependence, where u=atan2⁡(x2,x1)u=\operatorname{atan2}(x_2,x_1) and t=x3/2t=x_3/2. The deformed-torus and bumpy-sphere tests similarly probe spatial moments and cross-coordinate dependence.

    To complement the projected potential bias EUπ\mathcal E_U^\pi reported in Section 4.1, we measure the largest standardized expectation error across the eight additional test functions:

    := \max_{1\le j\le 8} \frac{ \left|\widehat{\mathbb E}[f_j(\pi(X))] -\mathbb E_\mu[f_j]\right| }{ \operatorname{SD}_\mu(f_j) },$$ aligned with the supremum over test functions in the integral probability metric of Section 3.2. **The results (Table D) support two main findings:** - **(i) Absolute accuracy:** IIL achieves small $D_{\max,8}=0.090$-$0.118$ across three manifolds and four seeds. - **(ii) Comparative performance:** **The projection bias amplification of hard projection ($\pi$) extends beyond the potential $U$** on all three manifolds, while IIL maintains substantially smaller discrepancies at smaller stepsize. Moreover, **IIL achieves lower mean discrepancies than HL in five of six settings evaluated**. Notably, HL approximates mean curvature by taking the trace over a learned tangent basis, whereas **IIL requires only one directional-curvature evaluation, avoiding the additional basis-wide curvature computation**.
  2. The main findings persist across initializations.

    We compare initialization from the target Gibbs distribution and the original training-data distribution (Table E). IIL maintains comparable errors under both initializations and consistently outperforms π\pi. It also achieves lower mean discrepancies than HL on the catenoid and deformed-torus, with mixed results on the bumpy sphere.

  3. The main findings persist across simulation horizons.

    Across four horizons (T/4,T/2,T,2TT/4,T/2,T,2T; Table F), IIL maintains substantially lower errors than π\pi and generally favorable performance relative to HL. Doubling the horizon from TT to 2T2T changes IIL's mean errors by only −0.022-0.022 to +0.018+0.018 across the six initialization settings, indicating limited sensitivity to horizon extension.

Overall, these additional experiments strengthen the empirical validation of our theoretical predictions beyond a single expectation and show that IIL's distributional accuracy and its comparative advantages over the π\pi-, L-, and HL-schemes are broadly preserved across the tested initializations and simulation horizons.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

General Response

We thank all reviewers for the time and effort spent reviewing our paper and for their constructive feedback. We have provided detailed responses to each reviewer and would like to briefly summarize how we have addressed the main concerns.

  1. Component attribution on realistic tasks. We clarify the distinct contributions of landing and pathwise curvature correction through controlled ablations on FFHQ and MNIST, and additional CIFAR-10 experiments. These comparisons show the benefits of landing, curvature correction beyond landing alone, and further improvements from pathwise-curvature correction over mean-curvature correction (requested by Reviewers h4t7 and sjkE).
  2. Distributional validation and geometric-error dependence. We provide additional evidence of distributional accuracy using multiple test functions, different initializations, and simulation horizons on three synthetic manifolds (requested by Reviewer h4t7). We also examine controlled geometric-operator errors across step sizes, supporting the predicted dependence on geometric approximation errors (requested by Reviewer sjkE).
  3. Motivation and theoretical contributions. We clarify the motivation for geometric-prior exploration, supported by experiments on training-density sensitivity and task-compatible solution coverage (in response to Reviewers DXVH and sjkE). We further distinguish our numerical construction and stationary guarantees under inexact learned geometry from existing curved schemes, and explain the generality of our stationary-analysis framework (requested by Reviewer sjkE).

We have also improved readability by clarifying the motivation, paper structure, and theoretical exposition, and by making the experimental objectives and takeaways more explicit (in response to Reviewer DXVH).

We hope our responses and additional evidence address the reviewers' concerns. We appreciate their valuable feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.