acceptodds
Under review as a conference paper at ICLR 2027

Coordinate-Free Automatic Differentiation

Abstract

Automatic differentiation (AD) is a cornerstone of modern machine learning. Since all AD frameworks implicitly fix a basis of Euclidean space to define gradients and Jacobians, they natively process mathematical objects that are arrays of a fixed set of coordinates. This makes it awkward to express objects such as rotations, volume forms, and curvatures that arise, for example, in applications of AI to science. Bespoke workarounds lack reusability. We show that (i) a single, basis-independent derivative operator can represent many geometric differential operations, and (ii) this operator can be expressed entirely in terms of existing AD primitives. This reusable transformation can be composed with JAX batching, compilation, and hardware acceleration. Geometry enters as algebraic data. We experimentally demonstrate its reuse across different manifold operators, its composition through a spacetime inverse-problem pipeline, and its use in optimization over different rotor subgroups.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Automatic Differentiation, Differential Geometry, Geometric Algebra

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_GiqwOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

This paper shows how to implement multivector operations from Geometric Algebra using jax primitives, so they are compatible with automatic differentiation. They demonstrate their approach on four examples from manifold learning, physics, and optimization.

Critical strengths

The chosen examples in Section 4 are interesting.

Critical weaknesses
  1. I am not sure what the novel contribution is, and how the author's work differs from other approaches to computational geometric algebra. Please see the "Questions For Authors" section.
  2. There does not appear to be any comparison to other approaches, either within the realm of geometric algebra or other more "conventional" approaches to the examples in Section 4. Why should a researcher reach for this method over everything else?
Questions for authors
  1. How does your approach differ from existing geometric algebra libraries?
  2. What precisely are the new results in Section 3?
  3. Why are there no comparisons to other libraries in Section 4? Are you doing something fundamentally different that those libraries can not handle?

I wish to emphasize that, at the moment, I am mostly just confused as to what is the claimed novelty for this submission. I have never worked with geometric algebra in a professional sense, my experience is limited to a few leisurely hours in a cafe spent with "Geometric Algebra for Computer Science" by Dorst, Fontijne, and Mann. Nonetheless, I am aware of a community of enthusiasts who don't really need to be sold on the beauty and utility of ga. What does this paper do for them?

My "Rating" is entirely preliminary. I will consider the author's responses to these questions in good-faith and update my review as appropriate.

2 comments
AuthorsOpenReview ↗

Response to Reviewer Giqw

Thank you for your review and we especially appreciate your willingness to hear about why the GA enthusiasts would be excited about this work, as we also hoped this would be effectively communicated in the paper.

Difference from GA libraries. As stated in the general comment above, existing libraries only implement geometric algebra operations, with no way of computing the multivector derivative and any of the downstream primitives. The experiments in Section 4 all require some of these primitives, and thus to re-implement them you would either have to i) write bespoke code which is specific to a certain choice of algebra, dimension, manifold, etc., or ii) re-implement our library of primitives on top of an existing GA library.

Novelty in Section 3. Section 3.1 and 3.2 are standard definitions (our favorite reference is Hestenes and Sobczyk, 1984), where we have slightly modified the definition to explicitly include the input and output blade indexing set I\mathcal{I} and O\mathcal{O} to connect it to the computational aspect that is Lemma 2 of Section 3.3. This is the only new contribution of Section 3, and formalizes the idea that one can combine geometric algebra with jvp/vjp to compute the multivector derivative. Section 3.4 and 3.5 are also standard definitions, but now are computable through compositions of the multivector derivative and the projection operators.

Novelty of Experiments. See the "Novelty of Experiments" in the general comment above.

Broader implications for the GA community Many GA enthusiasts are physicists or differential geometers who primarily use it as a tool for doing symbolic calculations to derive more compact expressions for geometric quantities (e.g. Maxwell's equation). This work bridges the gap between theory and computation and allows you to explicitly compute things like covariant derivatives of fields, curvatures on manifolds, trivialized differentials of rotor groups, and more with very few lines of code. Just as the enthusiasts leverage GA to store as much information in as few equations as possible, our implementation allows for much more compact code representations of complex geometric computations common to many AI for Science applications, among others.

Thank you for the response and clarifying the nature of the contributions. I am raising my score to a 2. I respectfully suggest this paper may find it's audience in a computational geometric algebra venue. I am not sure there will be enthusiastic interest among general machine learners.

Reviewer_QPSROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper presents a coordinate-free automatic differentiation framework based on geometric algebra and implemented using JAX primitives. It provides geometric differential operators and illustrates their reuse and composition in manifold learning, electromagnetic inverse problems, and optimization on rotor groups.

Critical strengths

The paper is clearly presented and provides a convincing general motivation for coordinate-free automatic differentiation. It's common derivative-and-projection framework unifies several geometric differential operators. It also uses standard JAX primitives, supporting composition with batching, compilation, and higher-order differentiation so it can be easily integreated into standard code and projects. A diverse set of examples illustrates the framework’s applicability across geometric learning, physics, and constrained optimization.

Critical weaknesses

Major concerns:

  • Nature of the contribution:
    The underlying mathematical machinery comes from established geometric algebra and geometric calculus. The paper’s principal contribution is to express coordinate-free derivatives through existing JAX primitives and expose them as composable transformations within JAX. This is primarily a software and integration contribution rather than a new theoretical development. Such integration can be valuable, but its significance depends on what it enables or simplifies in practice. Unclear advancement over existing JAX-based tools: Within this scope, the novelty relative to JAXGA and Numga is insufficiently established. These libraries already provide geometric algebra within JAX and support differentiation through those computations. Although the paper adds an interface for geometric differential operators, it does not clearly establish which capabilities go beyond existing tools or how much additional implementation effort its framework saves. Consequently, the substantive advance of the integration remains unclear. And this is missing, see the next point.

  • Unclear advancement over existing JAX-based tools:
    Within this scope, the novelty relative to JAXGA and Numga is insufficiently established. These libraries already provide geometric algebra within JAX and support differentiation through those computations. Although the paper adds an interface for geometric differential operators, it does not clearly establish which capabilities go beyond existing tools or how much additional implementation effort its framework saves. Consequently, the substantive advance of the integration remains unclear.

  • Insufficiently demonstrated benefit for ML:
    The paper contains several ML examples, but these primarily establish that the framework can be used successfully. They do not demonstrate an advantage over existing implementations in solution quality, computational efficiency, or implementation effort. Nor do they establish that the tasks were previously infeasible. The authors explicitly frame the experiments as demonstrations of reuse and composition rather than performance benchmarks; nevertheless, the practical significance of that reuse for ML remains insufficiently substantiated. In its current form, I would suggest publishing at an AD conference instead.

I consider these first three weakness as the major issue of this contribution: the paper does not show how or in which aspect it advances ML research.

Minor concerns:

  • Confounded surface-learning comparison: Appendix C.2.B reports 4,000 training iterations with the added curvature-integral penalty and only 150 without it, with reported runtimes of 444 and 78 seconds, respectively. Changing both the objective and the training budget prevents isolating the penalty’s effect. This comparison also evaluates two training objectives rather than the proposed framework against an existing tool. Furthermore, the optimized integrated-curvature quantity on a surface with boundary should not be interpreted as a changing Euler characteristic.

  • Limited scope of the optimization improvements:
    The conformal rotor experiment demonstrates a useful advantage over the tested fixed-chart updates near a chart singularity. However, this does not establish a general advantage over existing optimization approaches or one specific to the proposed library. BFGS also reaches essentially zero loss in the default conformal example. In the Lorentz experiment, the proposed update, static and corrected charts, and BFGS attain the same printed final loss and estimation errors. BFGS uses 40 steps versus 400 Adam steps for the other methods, although these counts cannot be translated directly into runtime comparisons because the steps have different computational costs. Hence, the setups are not comparable.

  • Limited software verifiability:
    There was no code provided to verify the contribution and the experiments This is rather unusual for such a implementation-oriented contribution.

  • Appendix code readability:
    The unusual code font makes the implementation examples difficult to read in the appendix.

Questions for authors
  • Additional functionality:
    What capabilities does the proposed framework add beyond JAXGA and Numga, and what additional implementation would those tools require to support the same examples?

  • Practical ML benefit:
    Which results establish a benefit over existing tools in functionality, implementation effort, solution quality, or computational cost? How should the optimization results be interpreted given that established alternatives often reach the same solution?

1 comment
AuthorsOpenReview ↗

Response to Reviewer QPSR

Thank you for the detailed review. We address your concerns below.

Contribution and relation to existing GA tools: As stated in the general comment above, while one can differentiate coordinates of multivectors using the usual AD primitives in either JAXGA, Numga, or other library, these do not correspond to coordinate-free differential operators. Lemma 2 of Section 3.3 is what allows us to transform these traditional AD primitives expressed as coordinates in the standard basis to the multivector derivative.

Benefit for ML applications: Our experiments merely choose a set of problems which our library solves easily. More generally, "AI for Science" is an emerging field that is filled with geometric problems of this flavor. Building in the right structure is an important aspect to solving these problems, as things like PINNs have evidenced. Furthermore, implicit in many machine learning problems is the assumption that the high dimensional data lives on some smaller low dimensional manifold. In spite of this, the tools for directly working with such manifolds is lacking.

Surface-learning comparison: The reason that we only needed 150 iterations for the control case is that the final manifold keeps shrinking as more iterations are run. For visualization purposes, we stopped it at 150 iterations so one could clearly see the spherical patch that it formed. The reason we called the added objective the Euler characteristic is because that formula is the Euler characteristic for closed manifolds; we acknowledge that the manifold we are working with is not closed (even when it is approximately a sphere), so perhaps a better name for this would be something like 'Gauss-Bonnet integral'.

Rotor Group Optimization results: As stated in the "Novelty of Experiments" section in the general comment, we do not claim that our method is better at optimizing compared to these other libraries. Rather, our framework provides a uniform way to encode these optimization spaces as data instead of code. This is what makes vectorized optimization over different subgroups possible (either via BFGS or Adam), and is the main takeaway from this experiment. The Lorentz experiment is similar; we are primarily demonstrating that the same recipe can even be used to do optimization on a non-compact group in indefinite signature.

Code: We will release the code upon acceptance as the authors agree that this is crucial to release publicly; we also will improve the code font/formatting in the Appendix B.

Reviewer_mgv6OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper addresses the computation of geometric differential operators with automatic differentiation, which natively works with coordinate arrays and carries no geometric structure. The authors propose to rely on geometric algebra and its multivector derivative, which does not depend on the choice of basis. They show that this derivative can be expressed with standard AD primitives (jvp, or a metric-aware vjp), so that it can be implemented as a composable JAX transformation, and extend it to embedded manifolds by projection onto their tangent spaces. Experiments on divergence-free flows on S2S^2 and H2H^2, curvature-based shape learning, minimal surfaces, an inverse problem in electromagnetism and optimization over rotor groups illustrate the framework.

Critical strengths
  • S1: The paper leverages geometric algebra to build, on top of classical automatic differentiation, coordinate-free geometric differential operators on embedded manifolds of any signature. Since these operators rely only on standard jvp/vjp primitives, they compose with the usual JAX transformations (vmap, jit, higher-order derivatives) and do not need to be re-derived for each space or coordinate system.

  • S2: The experiments in Section 4 show that the framework is reusable across very different settings.

Critical weaknesses
  • W1: The computational cost of the framework is not evaluated. There is no runtime or memory comparison with a standard coordinate-based implementation, although Λ(V)\Lambda(V) has dimension 2n2^n and quantities such as curvature require nested derivatives of the projector. It is therefore unclear how practical the approach is beyond the small-scale examples shown, and how it scales with the ambient and manifold dimensions.

  • W2: The paper is hard to follow for readers unfamiliar with geometric algebra. Section 3 is very dense and contains no worked example. A small example (e.g., the geometric product in 2D) and moving these definitions to the main text would make the paper more accessible to the ICLR audience.

Questions for authors
  • Q1: Could the authors report runtime and memory compared with a standard coordinate-based implementation, and discuss how they scale with the ambient and manifold dimensions?
1 comment
AuthorsOpenReview ↗

Response to Reviewer mgv6

We appreciate your feedback on the paper.

W1 / Q1: cost and scaling. Refer to the general comment above. Comparing runtime performance against a coordinate-based method would depend heavily on the exact task and exact implementation details of the other method. Generally, there is a tradeoff curve between runtime performance and generality of code; we believe that this library is very general while still being performant. We emphasize again that the thesis of this paper is to introduce a new capability, not to beat existing libraries on performance of the basic GA operations.

W2: accessibility. We agree that Section 3 is very dense as is, so we plan to expand on this section with the extra page allowance if the work is accepted. As an example of a computation of a geometric product, we give the following example.

Consider Cl(2,0,0)\mathrm{Cl}(2,0,0), which will be 4 dimensional with basis vectors 1,e1,e2,e12=e1∧e21, e_1, e_2, e_{12} = e_1\wedge e_2. To give an example of a computation of a geometric product consider X1=3e1−2e12X_1 = 3 e_1 - 2 e_{12} and X2=4−e1−5e2X_2 = 4 -e_1 - 5 e_2. Using e1e1=e2e2=1e_1e_1 = e_2e_2 = 1 and e1e2=e12=−e2e1e_1e_2 = e_{12} = -e_2e_1 (so that e12e1=−e2e_{12}e_1 = -e_2 and e12e2=e1e_{12}e_2 = e_1), we distribute the product term by term: X1X2=(3e1−2e12)(4−e1−5e2)=12e1−3e1e1−15e1e2−8e12+2e12e1+10e12e2=−3+22e1−2e2−23e12.X_1X_2 = (3 e_1 - 2 e_{12})(4-e_1 - 5e_2) = 12e_1 - 3e_1e_1 - 15e_1e_2 - 8e_{12} + 2e_{12}e_1 + 10e_{12}e_2 = -3 + 22e_1 - 2e_2 - 23e_{12}.

We also think it is informative to provide an example of computation of a multivector derivative. Again working in Cl(2,0,0)\mathrm{Cl}(2,0,0), consider the rotational field F(x)=−x2e1+x1e2F(x) = -x_2e_1 + x_1 e_2. Then, ∂I1F=e1 ∂x1F+e2 ∂x2F=e1e2−e2e1=2e12,\partial_{\mathcal I_1}F = e_1\,\partial_{x_1}F + e_2\,\partial_{x_2}F = e_1e_2 - e_2e_1 = 2e_{12}, From this we can see that the divergence (scalar part) is 00 (as we expect from a rotational field), and the curl (bivector part) is 22. A related example is the identity field, F(x)=xF(x) = x. Repeating this computation gives ∂I1F=2\partial_{\mathcal{I}_1} F = 2, which is entirely divergence (scalar part) and no curl.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General comment addressing shared concerns by reviewers

We thank the reviewers and address their shared concerns here.

Our work vs existing libraries

All three reviews ask how this work compares to existing geometric algebra (GA) libraries. We emphasize that the thrust of this work was not to introduce another library for doing GA operations, but rather to give an ML-native 'API' for differential geometry, physics, and beyond (detailed in Appendix B). As a means to this end, GA and its associated calculus are highly effective; all the quantities computed in our API are simple downstream computations from the multivector derivative, which we implement as a composable function transformation. To our knowledge, no other library (GA-based or otherwise) provides such an API, though in principle one could re-implement our contributions on top of any of the existing GA libraries that support AD. We think it is important to stress that we are not bottlenecked by using GA as a 'backend' for these computations: all the experiments in the paper can run on a Macbook (we used a GPU to speed up the minimal surface work, but this was not necessary). The table below compares capabilities, and we can add it to the paper for extra clarity.

algebraAD through JAX/TF∂I\partial_{\mathcal I} (fwd + rev)manifold operatorscurvature, integrationjit/vmap
Clifford, Kingdonyesnonononopartly
TFGA, JAXGA, Numgayesyes (coordinate gradient)nononoyes
GAlgebra (SymPy)yesn/a (symbolic)yes, symbolicyes, symbolic (charts)partlyno
Tingelstad & Egelandyesforward modenononono
This workyesyesyesyesyesyes

Novelty of Experiments

Reviewers also asked what is novel in the experiments and whether other libraries could run them. Every experiment required computing a quantity downstream of the multivector derivative; because there are no ways to compute these in other libraries, existing approaches use bespoke code tied to one algebra, dimension, or manifold. For each experiment we state what that would take.

4.1a Non-Euclidean Tilings: Existing tools like the cited Neural Conservation Law paper allow you to parameterize a divergence free vector field on flat Euclidean space. If you wanted to extend this to curved or non-Euclidean spaces, one would need to modify this recipe for each case. Meanwhile, in our framework we re-use the same code for both examples given. The purpose of this experiment is to show this; as soon as one can specify a manifold, our library can compute with the associated calculus on that manifold for free.

4.1b Backprop through the Euler Characteristic: For a surface in 3 ambient dimensions, one can compute the Gaussian curvature as a function of the first and second fundamental forms, which then integrates to the Euler characteristic; however, this formula doesn't generalize to higher dimensions. The more general notion of scalar curvature that our library implements can be defined for manifolds in any number of dimensions, so the same curvature code and Monte Carlo integration extend beyond surfaces. The goal of this experiment was to show how we could directly optimize the Gauss-Bonnet integral given some explicit representation of a manifold; we restricted to the 3d case to keep things easy to visualize.

4.2 Minimal Surfaces: The DeepCurrents code only trains 2d minimal surfaces in 3d (codimension 1). Increasing codimension requires generalizing the exterior derivative beyond a gradient (grade 0 to grade 1) and a generalized Biot-Savart kernel, both of which are trivial to compute in our library; their code computes the Biot-Savart kernel using a cross product, which only exists in 3 ambient dimensions. The only thing preventing other (n,k)(n,k) pairs in our code is that no closed form exists for the mass norm for 3≤k≤n−33 \leq k \leq n-3.

4.3 Inverse Design in Spacetime: The computation of the electromagnetic field for a charged particle with an arbitrary trajectory can be computed with standard methods. However, differentiating through such a solve so that the trajectory itself can be optimized is novel as far as we are aware. Furthermore, the core training loop of this example is only a few lines of code once you take the primitives of the library as atomic operations.

4.4 Optimization on Rotor Groups: As stated in the text, existing tools like manopt, geoopt, and geotorch can already do optimization on many of these spaces. However, these libraries implement procedures for each manifold class separately; in our library, the manifold class is specified by two pieces of data: the bivector generators {Bi}\{B_i\} of the rotor group GG, and the multivector X0X_0 which is mapped by the rotor we are optimizing. Because we can represent different spaces by data instead of code, we can vectorize the optimization across these spaces (Figure 9), which those libraries don't support.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.