acceptodds
Under review as a conference paper at ICLR 2027

Diffeomorphic Optimization

Abstract

Generative models learn data distributions that reside on a low-dimensional manifold within a higher-dimensional ambient space. Optimizing differentiable objectives on such a manifold is challenging: the ambient loss landscape is high-dimensional, rugged, and non-convex. Direct gradient descent, blind to the manifold's geometry, quickly drifts off it. Our approach starts from the observation that diffusion and flow models provide a map from the data manifold to a much simpler base space in which we perform gradient descent instead. Using concepts from differential geometry, we show this is equivalent to Riemannian gradient descent on the data manifold, keeping trajectories on-manifold by construction and yielding a smoother optimization surface. We extend diffeomorphic optimization to matrix groups, such as SO(3), SE(3) and SU(N) with autograd-compatible Lie-group ODE solvers, which allows us to empirically demonstrate the effectiveness of our approach in the highly relevant task of protein design and lattice gauge theory.

Primary area:
generative models
Keywords:
Diffusion Models, Geometric Machine Learning, Differentiable Geometry

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_6YEVOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes diffeomorphic optimization: optimizing differentiable objectives through the latent space of a pretrained flow or diffusion model. The authors interpret latent-space gradient descent as Riemannian gradient descent under a transported metric, develop backpropagation methods for matrix Lie-group ODEs, and demonstrate applications to protein design, docking, Rosetta relaxation, and lattice gauge theory.

Critical strengths
  • The numerical results are broad and impressive, with potentially useful improvements over guidance, optimal control, sampling, and relaxation baselines.
  • The work provides practical technical contributions, including autograd-compatible gradients for SO(3) and adjoint-state methods for differentiating through generative ODEs on matrix Lie groups.
  • The method operates on frozen pretrained models without retraining, making it potentially useful independently of the theoretical framing.
  • The authors acknowledge several important limitations: computational cost, dependence on the generative model’s support, and the fact that latent updates are not constrained to high-density base-space regions.
  • The experiments include multiple samples, test-set evaluations, and some seed studies, which supports the reported empirical trends.
Critical weaknesses
  1. Manifold terminology is imprecise. Calling D=g(Z) a “data manifold” is not fully correct: Z is the full latent space and a diffeomorphism preserves dimension. The paper should qualify this as an approximate or learned representation of data concentration, rather than a rigorous low-dimensional manifold.
  2. Optimization claims are too strong. A unimodal base density does not imply a smoother loss landscape or easier mode switching; the reparameterized problem can be highly non-convex and poorly conditioned. These concerns have been considered in the optimization literature, see for example recent work like Diepeveen and Weber, Iso-Riemannian optimization on learned data manifolds (arXiv:2510.21033). Figure 1 is misleading because it suggests that latent reparameterization makes the problem convex, which it does not.
  3. Theorem 1 is weaker than necessary. Under the transported metric, g is an isometry, so the exponential-map correspondence should hold exactly where defined, not merely up to O(\lambda^2). This is related to the \mathbb{R}^d \to M case in Diepeveen, Pulling back symmetric Riemannian geometry for data analysis (arXiv:2403.06612); the more general N \to M case follows similarly for the exponential map. The theorem also does not establish that the Riemannian gradient is a desirable search direction. It merely tells that is an easy to compute one. In fact, the optimization paper mentioned in 2 discusses why the Riemannian gradient is not what we want.
  4. Novelty should be stated more modestly. Given that the paper is primarily an empirical extension of existing ideas for differentiating through flows, and given the theoretical gaps above, it should present itself as such and focus on what is genuinely new: the Lie-group extensions, implementation, and applications.
  5. Initialization robustness is insufficiently studied. Figure 3 shows one trajectory rather than a systematic initialization study. Since the problem is generally non-convex, results across random and difficult initializations, including failure rates, are needed to assess whether the displayed behavior is representative.
  6. Step-size selection is not justified. The appendix reports particular learning rates, but does not explain how they were chosen or how sensitive the method is to them. Unwanted stretching in the transported geometry can make the problem ill-conditioned and require small steps.
  7. Comparative claims are overstated. Guidance, reward fine-tuning, and optimal control have limitations, but so does the proposed method. The paper should present these as complementary approaches with distinct tradeoffs rather than implying that its caveats do not apply.
Questions for authors
  1. What is the authors’ motivation for claiming that the base-space reparameterization improves conditioning along the manifold, given that the reparameterized problem can be highly non-convex and poorly conditioned? Is there some structure being encoded in the models used that is not disclosed in the paper?
Reviewer_7vJiOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes a method for property optimization in high dimensional spaces using pretrained generative models. In particular, denoting the generative model f(z)f(z) that takes the latent variable zz as input and produces a sample x=f(z)x = f(z), the authors propose to optimize the objective L(x)L(x) by optimizing the latent. That is, one can simulate the gradient flow in the latent space as dzt/dt=∇zL(f(zt))dz_t/dt = \nabla_z L(f(z_t)). The authors focus on the case where the generative model ff is defined as the integral of the ODE on the SE(3) manifold as done in the generative models for protein backbones structures. For this case, they derive the coordinate form of the gradient and the formula for the adjoint integrator. They illustrate the applicability of the method on protein design tasks and lattice gauge theory problem.

Critical strengths

The main strength of this paper is the choice of the problem to tackle. Property optimization is indeed important and actual problem in the field, and progress in this direction impacts both communities of researchers and practitioners. The authors then aim to cover a wide range of applications while proposing a very general method.

Critical weaknesses

There are three major weaknesses of the paper: limited novelty of the proposed methodology, poor empirical study, poor presentation. All of them require a major revision and prevent me from recommending the acceptance of the paper. In what follows I elaborate on each issue separately.

Limited novelty. The method is already very well known and studied [1,2,3] as well as its adaptation to Lie groups [4,5]. I know of the method from [6], which might be not the first time when it was used or proposed. After that, the same method was applied to proteins [7] and extended to manifolds [8]. I think the existence of extensive literature on that topic significantly reduces the actuality of the submitted manuscript and does not bring any new ideas to the community.

Poor empirical study.

  • Peptide Design. Besides a completely unsatisfactory description of the experiments (no description of the reported metrics, no description of the ground truth), the reported results cannot be interpreted. For example, the method achieves diversity of 0.3400.340 while the baseline achieves the diversity of 0.3380.338. It is impossible to tell how significant this improvement is without reporting standard deviations and giving metrics a practical meaning.
  • Secondary Structure Modification with FrameFlow. This is a qualitative experiment, which does not seem to have a direct practical value.
  • Protein-Ligand Docking with DiffDock and Minimization of Rosetta Energy with AlphaFlow. These experiments lack comparison to baselines.
  • Lattice Gauge Theory. Besides missing baselines, the author report factorization ratio R2×2R_{2\times 2} and fraction in target sector Q0+1Q_0 + 1. I don't know how to interpret these metrics. Besides, simple HMC sampling seems to be outperforming the proposed method, which raises the question of the practical value of the considered problem.

Poor presentation.

  • Throught the paper there is a constant confusion between the objective function landscape and the density. For example figure 1, where the left most panel depicts the latent density, but the central and left panels depict the objective function. The authors claim that the method leads to a smoother optimization landscape, but this is not illustrated at all neither theoretically nor emprically on toy examples. Furthermore, the authors prove the equivalence result between the direct optimization in the state space and the optimization in the latent space, which completely confuses the reader given the original motivation to simplify the problem.
  • The beginning of Section 4 is devoted completely to deriving the gradient on SE(3). This is a textbook material [9,10].
  • Furthermore, the presentation in Section 4 is very confusing. The authors say that integrating vector fields on SE(3) is highly non-trivial and then proceed as "For example, the groups SE(3) has the product rule.". For the reader unfamiliar with these difficulties this does not help understanding at all.
  • Section 4 line 252. The authors suggest two ways to evaluate the gradient of the loss wrt the latent variable ZZ (prior space) and then they proceed to describing the coordinate form of the gradient on SO(3) and the adjoint method. The former evaluates the gradient of a function in a given point, the latter differentiates through the ODE trajectory, they're definitely not two ways to do the same thing.
  • Poor editing, eg line 412, line 471.
  • Experimental section has no explanation of metrics (not even a reference to where they are defined), eg in Table 1.
  • The Lattice Gauge Theory Section assumes the knowledge of the task specific jargon and is very obscure for the reader not familiar with the domain.

Minor things

  • Fig 1 is inconsistent – the trajectory "stucks" in a local minimum, while the desired property corresponds to local maxima.
  • line 055 "a inference"
  • The authors do not provide the code with the submission

[1] Gómez-Bombarelli, Rafael, et al. “Automatic chemical design using a data-driven continuous representation of molecules.” ACS Central Science 4.2 (2018): 268–276. arXiv:1610.02415.
[2] Liu, Xingchao, et al. “FlowGrad: Controlling the output of generative ODEs with gradients.” CVPR (2023): 24335–24344.
[3] Blasingame, Zander W., and Chen Liu. “AdjointDEIS: Efficient gradients for diffusion models.” NeurIPS (2024).
[4] Gerdes, Mathis, Pim de Haan, Roberto Bondesan, and Miranda C. N. Cheng. “Non-perturbative trivializing flows for lattice gauge theories.” arXiv:2410.13161 (2024).
[5] Wotte, Yannik P., Federico Califano, and Stefano Stramigioli. “Optimal potential shaping on SE(3) via neural ODEs on Lie groups.” The International Journal of Robotics Research (2024). arXiv:2401.15107.
[6] Ben-Hamu, Heli, et al. "D-flow: Differentiating through flows for controlled generation." arXiv preprint arXiv:2402.14017 (2024).
[7] Wang, Luran, Chaoran Cheng, Yizhen Liao, Yanru Qu, and Ge Liu. “Training free guided flow matching with optimal control.” ICLR (2025). arXiv:2410.18070.
[8] Falorsi, Luca, and Patrick Forré. “Neural ordinary differential equations on manifolds.” arXiv:2006.06663 (2020).
[9] Boumal, Nicolas. An Introduction to Optimization on Smooth Manifolds. Cambridge University Press, 2023.
[10] Solà, Joan, Jeremie Deray, and Dinesh Atchuthan. “A micro Lie theory for state estimation in robotics.” arXiv:1812.01537 (2018).

Questions for authors

My main question concerns the relation to [4]. That work already derives a Lie-algebra-valued adjoint state for continuous normalizing flows on matrix Lie groups, integrated backward in time, and states the construction in the general case. What exactly is the difference? I ask because this is the central methodological claim of the submission, and I would like a specific answer rather than a general one.

I should say that I do not expect the remaining issues to be resolved in the rebuttal. The missing baselines, the absent standard deviations, the undefined metrics, and the presentation problems in Section 1 and Section 4 require a major revision and a new round of reviewing. A response that clarifies the point above would not by itself change my assessment.

Reviewer_sUMPOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes diffeomorphic optimization: given a pretrained flow or diffusion model, viewed through its deterministic probability-flow map g from base space to data space, a differentiable cost L is minimized by gradient descent on L(g(z)) with respect to the base point z, backpropagating through the ODE solver. The authors show that, to first order in the step size, this is equivalent to Riemannian gradient descent on the image of g under the pushforward metric (Thm. 1). Their main technical contribution is extending the approach to generative models on matrix Lie groups, as used for protein backbones (SE(3)) and lattice gauge theory (SU(N)): they give an autograd-compatible expression for the SO(3) Riemannian gradient (Thm. 2) and derive an adjoint-state method on matrix Lie groups (Thm. 3). Experiments cover a toy SO(3) manifold, peptide design against OC-Flow, secondary-structure editing with FrameFlow against guidance, docking-score optimization with DiffDock against budget-matched best-of-N sampling, Rosetta energy minimization with AlphaFlow followed by Rosetta Relax, and two-dimensional SU(3) and U(1) lattice gauge theories against direct gradient descent on the links.

Critical strengths

The method is clear and makes sense. Optimizing in the base space of a pretrained generative model is a simple and general way to bias samples toward a differentiable objective while staying close to the learned data distribution, and it requires no fine-tuning or auxiliary networks. Extending this to matrix Lie groups, with a practical autograd trick for Riemannian gradients and a Lie-group adjoint method, makes the idea usable for the frame-based protein models that are standard in the field, and the experiments show it working across a broad set of applications (backbone editing, docking, energy relaxation, lattice gauge theory).

Critical weaknesses
  1. As proposed, the method requires backpropagating through a full ODE solve at every optimization step, which is very expensive in memory and compute and presumably numerically delicate. The authors acknowledge the "considerable numerical costs" (Sec. 6), and App. G shows that adjoint gradients through FrameFlow and AlphaFlow are noticeably less accurate than autograd at commonly used step sizes, while the gradient-accuracy evaluations alone took hours. Apart from a 2× speedup claim relative to OC-Flow, no wall-clock or memory numbers are given. It seems to me that this whole approach should really be done with a flow map or another few-step generator (e.g., consistency models, flow map matching, shortcut models, MeanFlow, Align Your Flow): samples can be generated in far fewer network evaluations, and backpropagating through generation becomes much cheaper and better conditioned. The authors' own ablation (App. H.2, Fig. 14), in which optimization with only 10 ODE steps still works well, suggests that a few-step map would suffice. This connection is not discussed, and none of the flow-map literature is cited.
  2. The novelty of the core method is limited. Optimizing the initial state of a flow by backpropagating through the generation is exactly D-Flow (Ben-Hamu et al., 2024), which the authors acknowledge building on, and the observation that base-space gradient descent keeps samples near the data manifold was already made both there and in earlier work on diffeomorphic counterfactuals with normalizing flows (Dombrowski et al., 2021; 2023). Thm. 1 is a fairly direct consequence of the pushforward metric and adds little beyond these works. The new contribution is therefore the extension to matrix Lie groups and the accompanying adjoint method, plus the applications. That is useful, but D-Flow is not included as a baseline, even in the AlphaFlow Rosetta experiment, which is Euclidean and where D-Flow applies directly.
Questions for authors
  1. (Experiments) Can the authors report wall-clock time and memory for the optimization in each experiment, and discuss whether the method could be run with a flow map or few-step distilled generator instead of a full ODE solve? A demonstration that the method works with a few-step map would substantially strengthen the practical case.
  2. (Clarification / experiments) Beyond the Lie-group extension, what does the method add relative to D-Flow? Can the authors include D-Flow as a baseline, at least in the Euclidean AlphaFlow experiment?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.