- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The authors propose Intrinsic Geometry Matching (IGM), a framework for learning the intrinsic geometry of stochastic process from observed data perturbations. By estimating CDC, IGM aims to recover process geometry without knowing the underlying state space or ground-truth geometry.
The authors introduce a sample-wise learning objective based on Bregman divergences and suggest using LogDet for keeping observation invariance. They further extend the framework to learn geometry from observations in different spaces. Experiments on synthetic manifolds, images, videos, and molecular dynamics demonstrate promising geometry recovery and practical applications.
Critical strengths
- The paper provides detailed mathematical principled framework. The authors develop a coherent mathematical formulation connecting stochastic processes, CDC operators, and observation equivariance. The theoretical analysis provides a clear foundation for proposed learning objective. Based on this construction, the choice of LogDet divergence is theoretically well motivated. The authors clearly establish a characterization of LogDet as the unique observation-invariant Bregman divergence under some assumptions.
- The cross-space geometry learning framework is interesting. The authors proposed framework that enables learning geometric information from observations in different spaces. In particular, the identifiability analysis for combining multiple partial observations is an interesting contribution.
Critical weaknesses
- Experiments are not well organized. The video experiments do not fully disentangle the effects of perturbation choice and geometry learning. In section 5.3, IGM is trained on temporal transitions, while the Gaussian-based baseline is trained on denoising pairs constructed from the same video dataset. Since the evaluation measures the likelihood of held-out temporal transitions, the comparison naturally favors a model trained on temporal transitions. While this comparison demonstrates the value of process-specific perturbations, it does not fully isolate the advantage of the proposed geometry learning method. In addition, It would be good to suggest additional ablation for the molecular trajectory experiment. In section 5.4, the proposed IGM bridge combines a learned conditional mean dynamics model with CDC-based endpoint conditioning. However, the baseline is latent linear interpolation (LERP), which does not use the learned mean dynamics, rather it only interpolate endpoints linearly. Therefore, it is unclear how much of the reported improvement is attributable to the learned CDC rather than the mean dynamics model.
- The theoretical scope of the framework could be clarified. The theoretical formulation relies on diffusion Markov process, while the proposed method is also applied to general data-driven perturbations, including image augmentations. It is unclear under what conditions arbitrary perturbation kernels correspond to diffusion processes with a nontrivial infinitesimal CDC. In particular, general finite-scale augmentations may not satisfy the diffusion assumptions or the chain rule used in the theoretical development.
Questions for authors
Since different Bregman divergences share the same population minimizer for conditional covariance estimation, could the authors further investigate the practical benefits of LogDet's observation-invariance under finite data and model capacity? For example, comparing equivariance error or observation consistency between LogDet and Frobenius might be interesting.
5 comments
Part B
[W2] Theoretical scope for general perturbations. This is a really interesting question about modelling assumptions. Our initial aim was to discuss the geometry of general Markov processes, which significantly generalises the manifold hypothesis. Diffusion geometry provides a theoretical model for geometry that uses only the carré du champ, and so simply requires that as input. In this sense, our finite-scale objects and methods are well defined for general perturbations, because an arbitrary perturbation kernel induces a finite-scale generator and CDC as its first two moments. When we evaluate geometric objects like the intrinsic derivative via , we do have the chain rule by construction. A separate question, which you rightly identify, is whether the underlying process we used to construct itself has the chain rule. This is really a modelling assumption, but applies in very general cases of Itô processes even when the coefficients are history-dependent (so the process is not Markov), which seems to encompass all the examples we consider here.
[Q1] Observation consistency of LogDet and Frobenius. We followed this suggestion directly (Response to All Reviewers: [A.2]) ). As the reviewer notes, all Bregman losses share the population minimiser, so any benefit of LogDet must appear with finite data and capacity. We observe the torus process of Sec. 5.1 through ill-conditioned linear distortions, with condition number up to , retrain IGM and FGM with the same data and architecture (Sec. 5.1 (ii)). We then measure the observation consistency of each method, the relative distance between the geometry learned from distorted data and the one learned from undistorted data, after pulling both back to latent coordinates. This is the equivariance error the reviewer suggests. IGM agrees with its undistorted estimate, and remains far more consistent than FGM, for every fixed loss regulariser we tested, . At the level of the loss (Response to All Reviewers: [A.1]), the regularised LogDet ranks candidate estimates almost identically in distorted coordinates while is small compared to the smallest eigenvalue of the target, and better than Frobenius at every (App. A.1). On video, LogDet also trains more stably, with gradient norms within times their median against up to for Frobenius (App. D.3, Response to All Reviewers: [A.4]).
We hope these additions address your concerns. If so, we would be grateful if you would consider raising your score, and we would be glad to answer any further questions.
Follow-up on our response
Dear Reviewer XniD, thank you again for your time and constructive feedback. We have addressed your concerns with new experiments and revisions. Since the discussion period is about to end, we would appreciate it if you could consider our improved manuscript and take these changes into account in your assessment. We would be very happy to respond to any further questions you may have.
With these revisions, we now train Frobenius and nearest-neighbour baselines on the same transitions as IGM, which disentangles the geometry-learning method from the choice of perturbations. In the molecular bridges, we separate the contribution of the learned mean from that of the CDC. Following your suggestion, we also compare the observation consistency of LogDet and Frobenius under ill-conditioned distortions. With the same finite data and capacity, LogDet stays far more consistent, and on video it also trains more stably. Finally, we clarify which parts of the theory hold for general perturbations (details in our reply above).
Part A
We thank the reviewer for the thoughtful review and the concrete suggestions. We appreciate that the reviewer finds the mathematical framework principled and coherent, the characterisation of LogDet as the unique observation-invariant Bregman divergence well motivated, and the cross-space framework interesting. The review asks us to further separate the choice of perturbations from the geometry-learning method, to separate the learned mean from the CDC in the molecular bridges, and to compare the observation consistency of LogDet and Frobenius. We have added experiments for all three. They are described in our general response ("Response to All Reviewers", Parts A and B) and summarised below, together with our answer on the theoretical scope.
[W1.1] Separating the perturbations from the geometry-learning method. We agree that evaluating on held-out transitions favours models trained on transitions. This is the point of this evaluation, which tests the choice of perturbations, as LD-RMM differs from IGM only in its perturbations (Principle 2). The two models therefore learn genuinely different geometries, IGM the intrinsic geometry of the process and LD-RMM the ambient geometry of the observations, a real-world analogue of the synthetic experiments. As there, they can still be compared by how well they capture the low-rank tangent spaces of the data manifold, which explained variance approximates (Sec. 5.3 (ii)).
The choice of loss is tested by keeping the perturbations fixed (Principle 3). With known ground truth, the advantage of LogDet over Frobenius grew with the intrinsic dimension (Sec. 5.1), which is why we adopted LogDet for the real-world experiments . On real transitions, IGM attains better held-out NLL and explained variance than FGM on video and molecular dynamics (Response to All Reviewers: [A.3]) We also consider a classical nearest-neighbour estimator of the local transition covariance, with up to 5 million reference frames, which approximates the same intrinsic object. This has a significant performance-cost trade-off, and stays behind both neural models on video while becoming substantially more expensive in time and memory as the dataset increases in size (App. D.5).
[W1.2] Mean and CDC in the MD bridges. Thank you for this suggestion, which helped us sharpen our claim (general response, B). The drift sets the mean motion, and the CDC describes the variability around it, so replacing the CDC with an isotropic or any constant covariance lowers the improvement of the bridge over LERP by only about 2 percentage points (from 30.1% to 28.2%). This comparison scores a single mean path against a single held-out trajectory, which is one realisation of a stochastic process, while the CDC acts on how such paths spread around the mean. We therefore now evaluate the bridges as a distribution, and held-out MD paths are more likely under IGM bridges than under isotropic bridges with the same mean, on all 20 test trajectories at every rank and lag (App. D.4). Claim (ii) of Sec. 5.4 now reads "this geometry improves the sampling of entire transition paths".