acceptodds
Under review as a conference paper at ICLR 2027

E3J: An Efficient and Open-Source Backend for Euclidean Equivariant Operations on GPU and TPU

Abstract

We present \tt e3j, a fast Euclid-equivariance backend for geometric deep learning applications with JAX bindings for GPU and TPU. Leveraging both optimized CUDA and Pallas kernels and algorithmic improvements, the library achieves state-of-art throughput and runtime on both forward and backward paths. On a machine learning interatomic potential (MLIP) use case, it outperforms established backends, measuring up to 34% speed-up over \tt cuEquivariance on water box NPT simulation using MACE, while remaining fully open source. \tt e3j achieves over 80 % efficiency over the H100 maximum memory bandwidth on tensor product operations, and in many cases more than doubles throughput of message passing convolutions forward compared to previously available backends. In addition, with the release of dedicated Pallas TPU kernel, \tt e3j opens the possibility of large scale equivariant deep learning workloads on TPU architectures, which has so far been difficult to achieve. Our benchmarks show that \tt e3j also achieves over 80% of a TPUv6e memory bandwith, up to one order of magnitude more than \tt e3nn_jax. The library is available on https://github.com/instadeepai/e3jGitHub, https://pypi.org/project/e3j/PyPI and is released under an open source Apache 2.0 license.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
Euclidean, Equivariance, Geometric Deep Learning, Molecular Modelling, MLIP, Kernel Optimization, CUDA, GPU, TPU

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_nnbuOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper presents e3j, an open-source JAX library for E(3)-equivariant operations, with three sets of kernels:

deterministic CUDA kernels; Pallas GPU kernels, which are faster but non-deterministic on the backward pass and do not support fewer than 128 channels; Pallas TPU kernels.

Critical strengths

An open-source library that fills a gap, especially on TPU. At present, fast equivariant kernels on GPU depend largely on a closed-source library (cuEquivariance), and on TPU only unoptimized pure-JAX implementations exist. An open, JAX-native backend that covers both GPU and TPU is useful to practitioners. Careful benchmarking. Throughput is reported as a fraction of the hardware's peak memory bandwidth. The paper also distinguishes clearly between deterministic and non-deterministic kernels and discusses why determinism matters for NPT simulations.

Critical weaknesses

The submission violates the double-blind review policy Acknowledgments with identifying details. The paper keeps an Acknowledgments section (L500–505) that names a specific funding programme and thanks individuals by name.

Under ICLR's double-blind policy, I consider these violations sufficient grounds for rejection on their own, independent of the technical assessment below. I have notified the AC.

Questions for authors

None. The anonymity violation is decisive for my recommendation.

Reviewer_E5CBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces e3j, a fast, open-source JAX backend designed for Euclidean equivariant operations (such as tensor products and message-passing convolutions) on both GPUs and TPUs. Rather than altering the foundational O(L5)O(L^5) mathematics of Clebsch-Gordan tensor products, the authors achieve massive speedups through extreme, hardware-aware engineering optimizations—such as trace-time coefficient specialization in Pallas kernels, receiver-ordered aggregation, and edge-tiling for TPUs. In end-to-end MLIP benchmarks, e3j significantly outperforms existing open-source baselines, surpasses NVIDIA's proprietary state-of-the-art (cuEquivariance), and successfully bridges the gap for large-scale equivariant workloads on TPUs.

Overall, this is a good acceleration work. However, it lacks certain innovations in mathematics and algorithms, has limited support for mainstream SO(2) operation, and does not support second derivatives, which limite its application. Perhaps it is more suitable for conferences in the HPC and engineering fields.

Critical strengths
  • Exceptional Hardware Utilization: The forward and backward passes for tensor products achieve an impressive utilization of over 80% of the High Bandwidth Memory (HBM) on both H100 GPUs and TPUv6e architectures.
  • Breaking Memory Bottlenecks: By utilizing highly optimized fused scheduling and intelligently skipping redundant "padding edges" in dynamic graphs, e3j drastically reduces memory overhead and atomic lock conflicts. This allows MACE models to scale up to 64,000 atoms (with an average connectivity of ~40) on a single 80GB H100 GPU.
  • Strong Ecosystem Contribution: As an open-source library, e3j provides various kernel implementations (CUDA and Pallas) that not only beat closed-source competitors but also introduce production-grade acceleration for TPUs (delivering a ~5x force inference speedup compared to base e3nn).
Critical weaknesses
  • Lack of Mathematical/Algorithmic Innovation: The performance gains rely entirely on brute-force, system-level engineering rather than mathematical breakthroughs to reduce asymptotic complexity. It still executes O(L5)O(L^5) operations, which poses theoretical scalability issues for future applications requiring higher-degree polynomials.

  • Suboptimal Support for SO(2) Convolutions: While e3j's generic kernel framework can mathematically "emulate" O(L3)O(L^3) SO(2) convolutions by collapsing multiplicities via specific coefficient inputs, it lacks dedicated low-level memory scheduling for these smaller tensor shapes. Consequently, while its raw engineering speed allows it to beat pure-JAX SO(2) implementations at low degrees (L≤3L \le 3), its lack of algorithmic scaling causes it to be outperformed by native SO(2) algorithms at L≥4L \ge 4.

  • Gap in Empirical Baselines: When discussing algorithms with better complexities like SO(2), the authors only benchmark against a naive, unoptimized "pure JAX" SO(2) implementation. The lack of direct empirical comparison against equally highly-engineered kernels (such as the Triton-optimized UMA model mentioned in the text, and other works such as E2Former V2) weakens the experimental claims against the latest architectural trends.

  • Hardware and Feature Limitations:

    • Channel Constraints: The Pallas GPU kernel currently does not support channel counts below 128 due to minimum block size constraints.

    • No Double-Backward: Unlike its competitor OpenEquivariance, e3j lacks dedicated double-backward kernels, meaning it is not optimally efficient when computing forces or Hessian matrices.

    • Non-deterministic Backward Pass: To maximize cache reuse and speed, the Pallas GPU backward pass sacrifices determinism, which could introduce numerical instability risks in gradient-sensitive workflows.

Questions for authors

Nothing

Reviewer_XQUTOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

E3J provides JAX-compatible CUDA and Pallas kernels for equivariant tensor products and fused message passing on GPUs and TPUs. Sparse coefficient specialization and fused aggregation reduce memory traffic. The paper evaluates operator throughput, numerical accuracy, and MACE/NequIP inference, including a molecular-dynamics simulation benchmark.

Critical strengths
  1. The GPU/TPU implementation is a useful systems contribution, especially the dedicated TPU kernels. Benchmarks against established backends show substantial gains in practically relevant low-degree regimes.
  2. The evaluation extends beyond isolated kernels to force inference and simulations, with numerical-reference checks and explicit discussion of precision and determinism.
Critical weaknesses
  1. The headline simulation comparison isolates the convolution backend while retaining E3J's symmetric contraction in all variants. Appendix C.1 explores an optimized alternative, but Table 2 does not establish the gain over a fully optimized competing MACE stack.
  2. Training efficiency remains unestablished: first-order VJP benchmarks and recursive differentiability do not measure force-training steps requiring second derivatives. Dedicated double-backward optimization is left to future work.
Questions for authors
  1. What speedup remains against the best complete competing configuration, including its optimized symmetric contraction, with padding costs reported separately?
  2. Can you provide representative energy-and-force training-step latency and peak memory against the existing backends, including double backward?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.