- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies geometric-prior exploration for underdetermined inverse problems. It uses a pretrained diffusion model to approximate data-manifold geometry and targets a Gibbs distribution relative to manifold volume rather than the learned data density. The proposed IIL scheme combines tangent-projected Langevin updates, a pathwise second-fundamental-form correction, and soft landing. The authors construct directional-curvature estimates using denoiser Jacobian-vector products and finite differences, and derive stationary weak-bias bounds under inexact geometric access. Experiments cover synthetic manifolds, MNIST inpainting, Human3.6M pose editing, and terminal refinement of DAPS on FFHQ.
Critical strengths
-
A relevant treatment of numerical error under learned geometry. The paper addresses an important distinction between accurate geometric operations and their learned approximations. The projection-bias example demonstrates that reducing the step size can amplify persistent projection errors, providing a clear motivation for studying discretization and geometric approximation jointly.
-
A coherent connection between algorithm design and analysis. The pathwise curvature correction cancels the leading normal fluctuation associated with the realized stochastic increment, while landing stabilizes existing off-manifold deviations. The appendix analyzes these components jointly, including cross terms, localization, and geometric perturbations. The stationary analysis is a meaningful consideration beyond finite-horizon consistency.
-
Useful implementation choices and experimental controls. The curvature estimator avoids constructing a full tangent basis or curvature tensor. The experiments include exact-versus-learned geometry comparisons, landing-strength sweeps, and equal-runtime controls for the landing-only baseline. The FFHQ evaluation also includes stronger and compute-reallocated DAPS configurations, appropriately accounting for the additional cost of geometric computations.
Critical weaknesses
-
The algorithmic contribution needs sharper differentiation from existing geometric integrators. Increment-dependent curved updates and their approximate implementations have already been studied by Armstrong and King, while the principal IIL correction follows from the standard second-order expansion of a retraction. Soft landing also has established precedents. The manuscript acknowledges these connections, but does not sufficiently distinguish the novelty of its numerical update from the novelty of its learned-geometry implementation and analysis. Importantly, comparing IIL with a retraction truncated to the same terms would merely compare equivalent updates. What is needed is a precise mathematical positioning of IIL within the existing family of curved schemes, identifying which aspects are inherited and which represent substantive additions.
-
The incremental contribution of the stationary theory should be made more explicit. Stationary error control under persistent geometric approximation errors is explicitly presented as a core contribution and appears to be the strongest candidate for theoretical novelty. Existing finite-horizon results do not automatically imply such guarantees, so this contribution should not be dismissed. Nevertheless, the key mechanism identified by the authors (second-order cancellation combined with normal stabilization) may also be available to a broader class of existing geometric schemes. It remains unclear whether the stationary weak-bias bound depends on a property specific to IIL, or follows more generally for suitable second-order curved updates equipped with landing. The authors should identify the additional assumptions and estimates needed beyond existing geometric consistency, stabilization, and Poisson-equation techniques. If the result is more general, that would itself be valuable, but the contribution should be formulated accordingly rather than attributed primarily to a distinct sampling principle.
-
The benefits of the geometric-prior target are not fully separated from those of the numerical method. Replacing learned-density weighting with manifold-volume weighting changes the sampling objective; it is not simply a more accurate approximation of the original learned-prior posterior. This choice is plausible for controlled exploration, but greater diversity does not by itself establish better coverage of desirable, task-compatible solutions. Comparisons with DPS also involve differences in initialization and dynamics. Additional controlled experiments that vary training density while keeping geometry and task potential fixed would help isolate the intended benefit. The FFHQ refinement results demonstrate practical reconstruction improvements, but do not establish recovery of the standalone geometric-prior target.
-
The experimental evidence could more directly validate the claimed theoretical increment. The synthetic experiments appropriately compare exact and learned geometry, but using a fixed learned model across a step-size sweep does not fully characterize dependence on geometric approximation error. Jointly varying the step size and controlled geometric-error levels on tractable manifolds would test the predicted error floor and sensitivity to projection, tangent, and curvature inaccuracies. In realistic applications, the component attribution is also incomplete: Human3.6M and FFHQ do not provide a comprehensive landing-only versus mean-curvature landing versus IIL comparison under matched conditions. Consequently, the DAPS+IIL improvements cannot be attributed specifically to the pathwise correction. The acknowledged use of development problems and unequal hyperparameter-search budgets in MNIST further limits its role as evidence of general performance advantages.
Questions for authors
-
Does the stationary weak-bias result depend on an IIL-specific property, or does it extend to suitable existing second-order curved schemes with the same landing mechanism? Please identify the precise theoretical addition beyond the available geometric-integrator and stabilization results.
-
Can the authors jointly vary step size and geometric approximation errors on a tractable manifold to verify the predicted error floor and error dependence, rather than comparing only exact geometry with a fixed learned approximation?
-
On at least one realistic task, how do landing-only, mean-curvature landing, and IIL compare when initialization, geometric access, postprocessing, and computational budgets are controlled?
-
Can the authors provide evidence that the change of prior improves meaningful solution coverage, separately from the effects of initialization and numerical discretization?
9 comments
A1. Scope and contribution of the stationary theory: generality of the stationary-analysis framework (Q1; W2)
A1. Scope and contribution of the stationary theory (Q1; W2)
(Q1, W2 --- generality of stationary theory.)
Generality of the stationary-analysis framework.
We already apply the stationary-bias analysis framework beyond IIL in our paper to HL- and hard projection ()-schemes (Theorem 34), with different one-step estimates from those required for the IIL-type rate.
- For HL-scheme, the additive discretization defect in the one-step normal squared-distance estimate is , rather than IIL's (Lemma 27 vs. Lemma 25). Consequently, HL achieves the weaker stationary bound , where .
- For hard projection -scheme, the normal distance is controlled by , but applying an inexact projection without step-size scaling introduces an term in the projected generator (Lemmas 29, 32), yielding .
Thus, while the stationary-analysis framework is transferable, the resulting rates depend on both the numerical construction and the accuracy of its geometric operators.
Our focus.
Our paper addresses the challenging setting of unknown data manifolds whose geometric operators are approximated from pretrained diffusion scores, with errors that may persist as . For IIL, ; Lemmas 23-25 and 30 establish the required estimates under simultaneous persistent errors in projection, tangent projection, and directional curvature. Our theoretical contribution therefore encompasses both a transferable stationary-error framework and its rigorous quantitative realization under inexact learned geometry.
A1. Numerical positioning and novelty. (W1)
A1. Numerical positioning and novelty. (W1)
(W1.) We position IIL as a pathwise curvature-corrected landing scheme for manifold Langevin sampling under inexact geometric access. Our focus is on unknown data manifolds, where projection, tangent, and curvature operators are approximated from pretrained diffusion scores, with errors that may persist as . Our main contributions are a practical numerical construction and rigorous stationary-accuracy guarantees under these persistent geometric errors.
Distinction from existing schemes.
Armstrong-King (2022) assumes exact access to the SDE vector fields encoding the manifold geometry. Bharath et al. (2025) establish stationary accuracy under on-manifold retraction access. Jeon et al. use known constraint functions and their derivatives for landing with mean-curvature-corrected Langevin dynamics.
None establishes stationary weak-bias guarantees under persistent errors in approximate projection, tangent, and curvature operators.
Our numerical construction.
IIL uses soft landing and the same-noise directional correction , where also drives the tangent update. As shown in Eq. (11) and Proposition 6, the directional correction is the stochastic term of a second-order retraction expansion. An identical quadratic truncation equipped with the same landing reproduces IIL and is therefore not an independent baseline. However, the quadratic correction alone does not provide the off-manifold stabilization required under inexact geometry; exact retractions remain on the manifold by construction, but their guarantees do not automatically extend to persistent learned-operator errors.
Provable advantages.
Our analysis establishes the complementary roles of both mechanisms:
-
(i) Soft landing avoids the stationary-bias amplification caused by repeated approximate hard projection (Example 1; Theorem 34).
-
(ii) Same-noise pathwise curvature correction cancels the leading stochastic normal residual pathwise (Lemma 23), unlike mean-curvature landing. This distinction is sharp: Proposition 36 establishes stationary mean manifold distances of for HL versus for IIL on the circle.
Overall.
To the best of our knowledge, IIL is the first pathwise curvature-corrected landing scheme for sampling on data manifolds. More importantly, we establish its stationary accuracy under persistent errors in diffusion-learned geometry. We construct a score-derived directional-curvature estimator with explicit approximation-error bounds (Theorem 39). Under Assumptions 1, 5, and 7 and fixed admissible landing strength, we prove first-order stationary smooth weak-bias control , where (Theorem 33).
A1. Scope and contribution of the stationary theory: conditions and retraction example (Q1; W2)
A1. Scope and contribution of the stationary theory (Q1; W2)
(Q1, W2 --- generality of stationary theory.) We thank the reviewer for recognizing the broader value of stationary control under persistent geometric errors.
Our stationary-analysis framework extends beyond IIL: other landing-equipped curved schemes can achieve the same O(Δt+ηeff) stationary rate, provided their geometric-operator and numerical errors satisfy additional conditions preserving normal stability and projected-generator consistency. Here ηeff is a scheme-dependent aggregate of these approximation errors. Second-order geometric consistency alone does not guarantee this rate.
Conditions for the same rate O(Δt+ηeff) as IIL.
Proposition 19 separates stationary bias into normal and projected-generator errors. Let X+ be one step from x, d(x)=dist(x,M), π nearest-point projection, LΔt=(KΔt−I)/Δt the discrete generator, L the target generator, and ϑx the cutoff. Sufficient estimates are E[d(X+)2∣x]≤(1−cΔt)d(x)2+CΔt,ηeff2+CΔt3,
\le C\lVert\varphi\rVert_{C^4}(\Delta t+\eta_{\rm eff}+d(x)).$$ These estimates match Lemmas 25 and 30 (Eqs. 91 and 105) for IIL, where $c>0$ and $C$ is independent of $\Delta t$ and the admissible approximation errors. Under analogues of Assumptions 1, 5, 7, invariant-law existence, and fixed margins, Lemmas 8, 22, Proposition 19, and Poisson regularity yield $\operatorname{dist}_{\mathcal F}(\rho_{\Delta t},\mu)=O(\Delta t+\eta_{\rm eff})$. **Concrete example: approximate retraction.** Consider a smooth second-order retraction-based Langevin scheme (e.g., Bharath et al., 2025), augmented with soft landing. **To isolate retraction approximation error, we assume exact nearest projection $\pi$ and tangent projector $P$ in this illustrative example.** The update is $$\widehat X^+=x+\ell\Delta t(p-x)+[\widehat{\mathcal R}_p(w)-\widehat{\mathcal R}_p(0)],\quad p=\pi(x),\quad w=-\Delta t P(p)\nabla U(p)+\sqrt{2\Delta t/\beta}\\,P(p)\xi.$$ With $\xi\sim\mathcal N(0,I)$ (localization suppressed), let $\mathcal R_p(w)\in\mathcal M$ be the exact second-order retraction. Sufficient **additional operator-error conditions** are $$D\widehat{\mathcal R}_p(0)=D\mathcal R_p(0)=I,\quad \sup_p\lVert D^2\widehat{\mathcal R}_p(0)-D^2\mathcal R_p(0)\rVert_{\rm op}\le\varepsilon_{\mathcal R},$$ plus uniform $C^4$ bounds on the localized tangent ball. For the one-step error $e_{\Delta t}=\widehat X^+-X_{\rm ref}^+$ with shared noise/localization, Gaussian symmetry gives $$\\|\mathbb E[e_{\Delta t}\mid x]\\|\le C(\Delta t\\,\varepsilon_{\mathcal R}+\Delta t^2),\quad \mathbb E[\\|e_{\Delta t}\\|^2\mid x]\le C(\Delta t^2\varepsilon_{\mathcal R}^2+\Delta t^3).$$ Since $\mathcal R_p(w)\in\mathcal M$, $$d(\widehat X^+)^2 \le \\|(1-\ell\Delta t)(x-p)+e_{\Delta t}\\|^2.$$ Expanding this inequality and applying the moment bounds and Young's inequality gives $$\mathbb E[d(\widehat X^+)^2\mid x] \le(1-c\Delta t)d(x)^2 +C\Delta t\\,\varepsilon_{\mathcal R}^2 +C\Delta t^3.$$ The second-order reference retraction and the error-moment bounds also give projected-generator consistency with error $O(\Delta t+\varepsilon_{\mathcal R})$. Hence **the $O(\Delta t+\varepsilon_{\mathcal R})$ stationary guarantee extends to this inexact retraction scheme**, subject to the same regularity, localization, and invariant-law conditions.