acceptodds
Under review as a conference paper at ICLR 2027

FlashCart: Fast Cartesian Tensor Products for Equivariant Interatomic Potentials

Abstract

Machine-learned interatomic potentials extend atomistic simulations beyond the length- and timescales accessible to electronic-structure methods. However, the computational cost of equivariant architectures limits the local correlations they can represent in practice and therefore their achievable accuracy. Here we introduce FlashCart, which makes higher-order correlations affordable by combining generated GPU kernels with an architecture that recursively builds equivariant features and compresses them to a fixed width at each step. We express tensor products in independent Cartesian components and symbolically simplify them and their derivatives, producing fused kernels that often outperform optimized spherical counterparts. We then show that increasing correlation order improves accuracy more efficiently than increasing width, depth, or tensor rank. On SPICE-MACE-OFF, FlashCart models advance the measured accuracy–efficiency frontier: a model with million parameters achieves lower energy and force errors and faster inference than a transformer with million parameters.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
equivariant neural networks, machine-learned interatomic potentials, irreducible Cartesian tensors, tensor products, GPU kernels, many-body correlations

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_VJWyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces FlashCart, combining optimized Cartesian tensor-product kernels with a fixed-width recursive architecture to efficiently construct higher-order correlations. Its ablations show that increasing correlation order offers better accuracy–cost tradeoffs than increasing depth, width, or tensor rank. Operator benchmarks and molecular and materials evaluations demonstrate competitive accuracy and substantial speedups, advancing the measured accuracy–efficiency frontier.

Critical strengths
  1. FlashCart provides efficient Triton kernels for Cartesian tensor products, including forward, backward, and double-backward computations.

  2. The fixed-width recursive architecture constructs higher-order correlations while keeping intermediate features compact.

Critical weaknesses

The evaluation does not sufficiently separate architectural gains from kernel improvements. The symmetric contraction comparison changes both the architecture and implementation, making the reported speedups difficult to attribute. Although end-to-end scaling is evaluated over 216–4096 atoms, Figure 3 uses fixed node counts and does not establish kernel scalability across workload sizes. A breakdown and broader operator-level sweeps are needed to substantiate the performance claims.

The benefits of fixed-width compression over prior recursive constructions remain insufficiently validated. ICTP already supports Cartesian tensor products and recursive correlation construction. Benchmarks establish competitive overall accuracy, but do not quantify the accuracy–cost tradeoff introduced by fixed-width compression. A comparison with ICTP or a less compressed product basis is needed to establish the value of this architectural contribution.

Questions for authors

Q1: What are the respective contributions of the fixed-width recursive architecture and the optimized kernels to FlashCart’s accuracy–efficiency frontier? Can these contributions be quantified separately?

Q2: Why do the edge-convolution and symmetric-contraction benchmarks in Figure 3 use different node counts? How were these configurations selected, and how sensitive are the reported speedups to workload size?

Q3: Could the authors provide more insight into what currently limits the performance of the Cartesian kernels and where they see the most promising opportunities for further optimization?

Q4: Why is eSEN’s runtime estimated from published A100 measurements while the other models are benchmarked on Blackwell?

6 comments

We thank the reviewer for their feedback and suggestions. We address the weaknesses in our responses to the questions below and first indicate where each concern is discussed.

The evaluation does not sufficiently separate architectural gains from kernel improvements. The symmetric contraction comparison ...

We address the separation of architectural and kernel contributions in Q1 and the broader operator benchmarks in Q2.

The benefits of fixed-width compression over prior recursive constructions remain insufficiently validated. ICTP already supports ...

We discuss the scope of our fixed-width claim and the new comparison of reduced and retained coupling paths in Q1.

Q1: We agree that the architectural and kernel contributions should be examined separately. For this purpose, we have added a comparison of reduced and retained coupling paths in Figure G3 in Appendix G, with experimental details in Appendix F. Alongside these results, the revised manuscript and the discussion below examine the contribution of the generated kernels using the same model checkpoint (Figure G5) and the accuracy benefit of increasing correlation order (Figure 4).

Regarding the architectural contribution, our claim is that keeping the intermediate width fixed reduces computational cost and makes higher correlation orders affordable. To keep this width fixed, our ladder mixes paths and channels jointly after each product to produce dd channels per retained tensor rank. At fixed tensor ranks and feature width, each additional order requires one product and one linear map, so the computational cost grows linearly. We do not attribute an accuracy gain to compression itself.

We instead attribute the accuracy gains to using higher correlation orders, which the fixed-width ladder makes affordable. Figure 4 shows that, in the tested configurations, increasing correlation order gives larger error reductions relative to added runtime than increasing the number of feature channels, message-passing layers, or tensor rank. We also compare models at approximately fixed parameter count and find that increasing correlation order while reducing the number of feature channels lowers force error by 99–2929 % and runtime by 1010–3434 %.

However, we agree that the original manuscript did not clearly demonstrate the computational cost of retaining coupling paths. To make the benefit of path reduction clearer, we have added a comparison of reduced and retained paths in Figure G3. Both variants use the same scalar weights, Cartesian products, tensor rank cutoffs, and compiled PyTorch operations. The weights are shared across the retained contributions, making the two variants mathematically equivalent by bilinearity of the tensor products. Figure G3 shows, for example, that with 40964096 nodes, 128128 feature channels, ℓmax⁡=3\ell_{\max}=3, and ν=5\nu=5, path-reduced products make the complete VJP-of-VJP evaluation 15.8×15.8\times faster and reduce its additional peak GPU allocation by a factor of 48.048.0, averaged over three independent runs.

These results show the computational benefit of reducing paths at higher correlation orders. Furthermore, our path and channel mixing can be used with ICTP, TACE, or spherical tensor products and combined with other choices of tensor rank cutoffs and coupling paths. We do not claim in the manuscript that this mixing is specific to FlashCart or to the chosen recursion.

To examine the accuracy–cost tradeoff, we compare FlashCart with the six released GRACE-OFF models (Ref. [25] in the revised manuscript). These models apply learned linear maps between products while keeping contributions from different coupling paths separate. On the 50,19550{,}195 SPICE-MACE-OFF test structures, our 464464k-parameter model has lower mean energy and force errors than any of these models. Compared with the largest GRACE-OFF model (1.61.6M parameters), these errors are 3939 % and 5151 % lower, respectively, and evaluation is 3.1×3.1\times faster. We note that FlashCart was trained on SPICE-MACE-OFF, while GRACE-OFF used a subset of SPICE-v2 comprising molecules with a formal net charge of zero. The results are reported in Figure 1 and Table G1 of the revised manuscript, with evaluation details presented in Appendix F.

Finally, Figure G5 (Figure G2 in the original manuscript) measures the contribution of the generated kernels with the Cartesian representation, architecture, and a model checkpoint with 5.6M parameters held fixed. With both implementations compiled, the generated kernels make inference 8.2×8.2\times and the training benchmark 5.4×5.4\times faster than the corresponding PyTorch operations.

Q2: We thank the reviewer for raising this point and have added Figures G1 and G2 in Appendix G, with benchmark details in Appendix F.

We chose 10241024 nodes for edge convolutions and 40964096 nodes for node-local products to represent realistic system sizes and provide enough parallel work for the GPU. Convolutions process contributions from every edge, while node-local products act on each node, so the two operations provide different amounts of parallel work. The additional benchmarks in Figures G1 and G2 support these choices, showing little further reduction in FlashCart's runtime per node beyond these system sizes.

We agree that the original manuscript did not sufficiently examine how kernel runtime and memory depend on the number of nodes, feature channels, and neighbors per node. The additional benchmarks cover 256256–65,53665{,}536 nodes for edge convolutions and 256256–262,144262{,}144 nodes for node-local products. We use 6464, 128128, and 256256 feature channels and maximum tensor ranks ℓmax⁡∈{2,3,4}\ell_{\max}\in\{2,3,4\}. For convolutions, we also compare 1616, 3232, 6464, and 128128 neighbors per node. We use 128128 feature channels for the largest systems.

For edge convolutions, FlashCart remains faster on average than the baseline implementations for the forward, VJP, and VJP-of-VJP. For the VJP-of-VJP, for example, it is 1.88×1.88\times faster than cuEquivariance, 3.02×3.02\times faster than OpenEquivariance, and 6.33×6.33\times faster than FlashTP. These averages use the 6060 configurations for which all four implementations completed the VJP-of-VJP, with three independent runs for each configuration. At 65,53665{,}536 nodes, 128128 feature channels, 6464 neighbors per node, and ℓmax⁡=2\ell_{\max}=2, FlashCart also completes the VJP-of-VJP while all three baselines run out of GPU memory.

For node-local products, the additional benchmarks use ν=3\nu=3 because cuEquivariance could not evaluate the VJP-of-VJP at ν=4\nu=4 for maximum tensor ranks 33 and 44 (Figure 3(b)). At ν=3\nu=3, the runtime difference decreases as the number of nodes increases, and cuEquivariance becomes faster than the full FlashCart ladder at larger system sizes. For example, at 65,53665{,}536 nodes and 128128 feature channels, the VJP-of-VJP takes 0.380.38 μ\mus/node for path-reduced products and 1.371.37 μ\mus/node for the full ladder, compared with 1.151.15 μ\mus/node for cuEquivariance, averaged over the three maximum tensor ranks and three independent runs. At 262,144262{,}144 nodes, forward evaluation also takes 0.1070.107 μ\mus/node with cuEquivariance and 0.1140.114 μ\mus/node with FlashCart's path-reduced products.

An important difference is that cuEquivariance evaluates a fused symmetric contraction, while our full ladder evaluates products and learned linear maps separately. The linear maps increase runtime at low correlation orders, but compress paths and channels to keep the number of intermediate features fixed as the order increases. Figure 3(b) shows results for FlashCart up to ν=6\nu=6 at every tested rank, and we have also tested orders up to ν=8\nu=8 in our internal benchmarks. By comparison, cuEquivariance completed all three evaluations only up to ν=3\nu=3 at maximum tensor ranks 33 and 44 in our benchmarks. Furthermore, Figure G3 shows the computational benefit of reducing paths between correlation orders, as discussed in Q1. We have also expanded the corresponding discussion in Section 6 and Appendix G.

Q3: At higher tensor ranks, more tensor components and coupling paths must be evaluated. The challenge is to organize these calculations so that inputs and intermediate results can be reused without requiring too much temporary storage. We see opportunities in adapting how the calculations are grouped into kernels for different tensor ranks, numbers of feature channels, and numbers of neighbors.

We also see opportunities to exploit further symmetries in the recursive products. Together with more effective symbolic simplification, this could reduce computational cost and the size of the generated code. Code growth also affects initial compilation at higher ranks, although compiled kernels are cached and reused. The current schedules and compilation costs are described in Section 4 and Appendices C and D.

Beyond these performance optimizations, extending the kernels beyond natural parity would broaden the range of couplings and architectures they support.

We want to note that we already discuss some of these limitations in the manuscript. We outline the more specific directions for further kernel optimization here to keep the Limitations paragraph concise.

Q4: To the best of our knowledge, the checkpoints for the eSEN-3.2M and eSEN-6.5M SPICE-MACE-OFF models are not publicly available, so we could not benchmark these models on our NVIDIA RTX PRO 6000 Blackwell GPUs.

We hope these clarifications and additional results address the reviewer's concerns.

Sincerely yours,

The authors

For the Q4, how about aligning experiments on A100?

Reviewer_yEoAOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces FlashCart, combining fused Cartesian tensor-product kernels with an architecture for constructing higher-order correlations. For the kernel implementation, rank-ℓ\ell irreducible Cartesian tensors are stored using 2ℓ+12\ell+1 independent components in a non-orthogonal basis. Tensor products and their derivatives are symbolically simplified and translated into fused GPU kernels. For the architecture, FlashCart follows the atomic cluster expansion framework but constructs higher-order features through recursive channel-wise tensor products and learned fixed-width compression. Validation includes comparisons with optimized spherical implementations, architectural ablations, molecular and materials benchmarks, and molecular dynamics.

Critical strengths
  1. A useful contribution to Cartesian equivariant computation. The fused kernels support both inference and force training. The work clearly distinguishes full 3ℓ3^\ell Cartesian storage from compact 2ℓ+12\ell+1 representations. Figure G2 demonstrates substantial speedups using the same model checkpoint.

  2. Competitive accuracy-efficiency tradeoffs. FlashCart achieves lower energy and force errors than PET-L with substantially fewer parameters and faster evaluation. Parameter-matched ablations support increasing correlation order, while three-seed evaluations and molecular dynamics tests strengthen the empirical evidence.

Critical weaknesses
  1. The advantage over existing recursive ACE constructions is insufficiently isolated. Although related methods are discussed, there is no direct comparison with recursive approaches such as GRACE. Comparisons with MACE's symmetric contractions do not isolate these effects, since the parameterizations differ. Similar compression could also be applied to Cartesian constructions such as ICTP and TACE, although both employ element-dependent coupling coefficients that mix all channels. However, both the closed-form formulation of ICTP and the Cartesian tensor product followed by ICTD decomposition in TACE can theoretically achieve channel compression, which warrants further discussion, an orthogonal transformation can be applied to contract the CG coefficients in advance, allowing us to retain the compact 2ℓ+12\ell+1-component Cartesian representation while directly reusing existing operators in spherical space.

  2. Representation and implementation benefits remain difficult to separate. Compact Cartesian couplings are related to spherical couplings by fixed basis transformations. Symmetrization and trace removal are incorporated into symbolic construction rather than performed explicitly at runtime. A matched spherical implementation would clarify which benefits arise from the coordinates versus kernel design. Related compact representations, such as MACE-ICTC, also merit discussion.

Questions for authors

My questions are largely aligned with the weaknesses outlined above. Overall, I lean toward accepting this paper, as I believe that further discussion of Cartesian and spherical representations would benefit the community's understanding of these approaches. However, before acceptance, I believe the authors should provide a more detailed discussion of the weaknesses outlined above.

5 comments

We thank the reviewer for their feedback and suggestions and address their concerns below.

W1: We thank the reviewer for raising this point and have added a comparison of reduced and retained coupling paths to the revised manuscript (see Figure G3 in Appendix G). We have also added GRACE-OFF results to the revised manuscript (see Figure 1 and Table G1). We discuss the results in Section 6 and Appendix G and provide the experimental details in Appendix F of the revised manuscript. We address the reviewer's concerns and discuss the additional results in more detail below.

Indeed, GRACE also builds higher-order correlations through recursive products. It restricts intermediate tensor ranks and coupling paths and applies learned linear maps between products, while keeping contributions from different coupling paths separate. Our approach goes further by mixing paths and channels jointly to produce dd channels per retained tensor rank after each product. At fixed tensor ranks and number of feature channels, each additional correlation order requires one pairwise product and one linear map, so the computational cost grows linearly with the correlation order. As the reviewer notes, this mixing can also be used with ICTP, TACE, or spherical tensor products and combined with other choices of tensor ranks and coupling paths. We do not claim in the manuscript that this mixing is specific to FlashCart or to the chosen recursion.

Our claim is that keeping the intermediate width fixed reduces computational cost and makes higher correlation orders affordable. We do not attribute an accuracy gain to compression itself and instead examine the accuracy benefit of increasing correlation order (Figure 4). At approximately fixed parameter count, we also find that increasing correlation order while reducing the number of feature channels lowers force error by 99–2929 % and runtime by 1010–3434 %.

We agree that the original manuscript did not clearly demonstrate the computational cost of retaining coupling paths, which may explain the request for comparisons with other recursive constructions. We therefore compare reduced and retained paths in Figure G3 with 40964096 nodes and 128128 feature channels, using the same scalar weights, Cartesian products, tensor rank cutoffs, and compiled PyTorch operations. The weights are shared across the retained contributions, so the two variants are mathematically equivalent by bilinearity of the tensor products. A representative result is a 15.8×15.8\times reduction in runtime and a 48.0×48.0\times reduction in additional peak GPU allocation for the VJP-of-VJP evaluation at maximum tensor rank 33 and correlation order 55. For more details, we refer to Figure G3 and Appendices F and G.

In addition to MACE, we now also compare FlashCart with GRACE-OFF (Ref. [25] in the revised manuscript), which uses a different parameterization of higher-order correlations. We evaluated all six released models on the same 50,19550{,}195 SPICE-MACE-OFF test structures and found that our 464464k-parameter model has lower mean energy and force errors than any of the GRACE-OFF models. Compared with the largest GRACE-OFF model (1.61.6M parameters), our 464464k-parameter model has 3939 % lower mean energy error and 5151 % lower mean force error and is 3.1×3.1\times faster. We note that FlashCart was trained on SPICE-MACE-OFF, while the released GRACE-OFF models were trained on a subset of SPICE-v2 comprising molecules with a formal net charge of zero.

Table G1 now includes results for all six GRACE-OFF models, and Figure 1 shows the three models with two message-passing layers. Appendix F describes the corresponding evaluation procedure.

Finally, we thank the reviewer for pointing us to TACE, which is now discussed in Appendix A.

W2: We agree that it is important to clarify the roles of representation and implementation. We discuss them separately below, beginning with the representation.

As stated in Section 2 and Appendix B, the Cartesian and spherical couplings are related by a fixed invertible linear map and thus have the same expressive power. We also explicitly state in Section 6 and demonstrate in Figure C1(c) in Appendix C that the operation counts show no overall arithmetic advantage over sparse spherical coupling and therefore do not explain the observed runtime improvements. Our performance claims concern the generated kernels, including memory access and neighbor aggregation.

The basis transformations can indeed be applied to coupling coefficients in advance or to the inputs and outputs of existing spherical operators. Cartesian-3j, for example, constructs Cartesian coupling coefficients by transforming Wigner-3j3j coefficients (Ref. [53] in the revised manuscript). In FlashCart, we instead use Cartesian tensor algebra to derive explicit equations for each independent output component through contractions, symmetrization, and trace removal. Our contribution is to simplify and differentiate these equations symbolically and translate the resulting forward and derivative expressions directly into fused kernels (Sections 3 and 4).

Regarding the requested matched spherical implementation, we already compare FlashCart with OpenEquivariance, a spherical kernel generator similar in spirit to ours (Figure 3(a)). It uses 2ℓ+12\ell+1 components per irreducible tensor and generates explicit forward and derivative kernels by unrolling the sums over nonzero Clebsch–Gordan coefficients. It also fuses the products with neighbor aggregation. Across the 3030 edge-convolution configurations described in Appendix F, FlashCart is on average 1.9×1.9\times, 2.1×2.1\times, and 3.1×3.1\times faster for the forward, VJP, and VJP-of-VJP, respectively. We have also added Figure G1 in Appendix G to show how runtime and memory depend on the number of nodes, feature channels, and neighbors per node. In these additional benchmarks, FlashCart is on average 2.01×2.01\times, 1.94×1.94\times, and 3.02×3.02\times faster than OpenEquivariance for the forward, VJP, and VJP-of-VJP, respectively.

As for the implementation benefits, Figure G5 in the revised manuscript (or Figure G2 in the original manuscript) compares the generated kernels with the corresponding PyTorch operations using the same compact Cartesian representation and the same checkpoint with 5.65.6M parameters. With both implementations compiled, inference is 8.2×8.2\times and the training benchmark 5.4×5.4\times faster with the generated kernels. The representation, architecture, and model parameters are unchanged.

Finally, we have added MACE-ICTC to the related work in Appendix A.

We hope these clarifications and additional results address the reviewer's concerns.

Sincerely yours,

The authors

We thank the reviewer for the time and effort they have spent evaluating our work. As the end of the discussion period approaches, we would appreciate their feedback on whether our response addresses the concerns raised in the review.

Sincerely yours,

The authors

Reviewer_xzaqOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors provide symbolic construction of Cartesian tensor product and the corresponding receiver-sorted edge convolution and further constraints on the maximum order after the tensor product.

Critical strengths
  1. The writing is very clear, and the speedup results are comparable to cuEquivariance and OpenEquivariance.
  2. Using the Cartesian tensor product shows consistently good performance on the PET-MAD benchmark, which includes famous datasets like MPtrj, Matbench, etc.
Critical weaknesses
  1. A general question about the motivation for the Cartesian tensor product: The Cartesian tensor product uses symmetric traceless tensors as bases, and they cannot capture the anti-symmetric information, e.g., vector cross product. However, the Clebsch-Gordan tensor product (CG-TP) is proven to be the most general equivariant bilinear form by Schur's Lemma. If the speedup is only comparable to CG-TP, then what is the benefit of using the Cartesian tensor product instead of CG-TP?
  2. I would like to see the runtime comparison to FlashTP [1] on Figure 3.
  3. I would also like to see more runtime ablations w.r.t. batch size (number of graphs), multiplicities, and node degrees with various avg.
  4. It is recommended to use ℓ\ell instead of ll.

[1] Lee, Seung Yul, et al. "FlashTP: Fused, Sparsity-Aware Tensor Product for Machine Learning Interatomic Potentials." ICML. 2025.

Questions for authors

See weaknesses.

8 comments

We thank the reviewer for their feedback and suggestions and address their concerns below.

W1: We agree that it is important to clarify the motivation for using Cartesian tensor products. As stated in Section 2 and Appendix B, Cartesian and spherical couplings with the same ranks and parities are related by a fixed invertible change of basis and have the same expressive power. We also explicitly state in Section 6 and demonstrate in Figure C1(c) that the Cartesian basis provides no overall arithmetic advantage over sparse spherical coupling. Our motivation is to use Cartesian tensor algebra as the starting point for kernel generation. The Cartesian products follow from contractions, symmetrization, and trace removal, giving explicit polynomials with rational coefficients in the 2ℓ+12\ell+1 independent components. We can then simplify and differentiate these expressions using exact symbolic arithmetic and translate the results directly into fused forward, VJP, and VJP-of-VJP kernels (Sections 3 and 4).

Regarding performance, our results show that FlashCart is faster on average than the spherical baselines. Across the 3030 edge-convolution configurations described in Appendix F, FlashCart is 1.6×1.6\times, 1.1×1.1\times, and 1.9×1.9\times faster on average than cuEquivariance for the forward, VJP, and VJP-of-VJP, respectively. The corresponding factors compared with OpenEquivariance are 1.91.9, 2.12.1, and 3.13.1. For the VJP-of-VJP, FlashCart also uses about half their peak GPU memory. We emphasize that our performance claims concern the generated kernels, including memory access and neighbor aggregation, rather than the choice of basis itself.

As for the cross product, it is well defined in Cartesian coordinates through the Levi–Civita symbol. Its output remains unchanged under spatial inversion and therefore has even parity, whereas the rank-11 tensors used in FlashCart change sign and have odd parity. We do not include this coupling because we restrict the products to natural parity, as stated in Section 2 and the Limitations paragraph. ICTP makes the same restriction, following MACE. The released GRACE-OFF models (see Ref. [25] in the revised manuscript) and recent NequIP and Allegro foundation models (see https://arxiv.org/abs/2607.28461) also retain only natural-parity features. Finally, as noted in Appendix B, our approach can be extended to opposite-parity outputs using Levi–Civita contractions and explicit parity labels.

W2: We thank the reviewer for suggesting the comparison with FlashTP, which was missing from the original manuscript and is now included in Figure 3(a). Across all 3030 edge-convolution configurations, FlashCart is on average 5.7×5.7\times, 5.6×5.6\times, and 6.6×6.6\times faster than FlashTP for the forward, VJP, and VJP-of-VJP, respectively, and is faster in every tested configuration. We have added the reference and discussion in Section 6 and Appendix A and the benchmark details in Appendix F.

W3: We agree that these benchmarks were missing from the original manuscript and have added Figures G1 and G2 in Appendix G. In FlashCart, we batch graphs by concatenating their atomic positions and species and adjusting the edge indices accordingly. The kernels process the resulting node and edge features together, so we vary the total number of nodes and edges in the additional benchmarks. These benchmarks use 6464, 128128, and 256256 feature channels and maximum tensor ranks ℓmax⁡∈{2,3,4}\ell_{\max}\in\{2,3,4\}. For convolutions, we also compare 1616, 3232, 6464, and 128128 neighbors per node. The calculations cover 256256–65,53665{,}536 nodes for convolutions and 256256–262,144262{,}144 nodes for products, with the largest system sizes using 128128 feature channels. Appendix F presents the benchmark details.

For edge convolutions, FlashCart remains faster on average for the forward, VJP, and VJP-of-VJP. For the VJP-of-VJP, for example, it is 1.88×1.88\times faster than cuEquivariance, 3.02×3.02\times faster than OpenEquivariance, and 6.33×6.33\times faster than FlashTP. These averages use the 6060 configurations for which all four implementations completed the VJP-of-VJP, with three independent runs for each configuration. At 65,53665{,}536 nodes, 128128 feature channels, 6464 neighbors per node, and ℓmax⁡=2\ell_{\max}=2, FlashCart also completes the VJP-of-VJP, while all three baselines run out of GPU memory during this evaluation.

We limited the additional node-local benchmarks to ν=3\nu=3 because cuEquivariance could not evaluate the VJP-of-VJP at ν=4\nu=4 for maximum tensor ranks 33 and 44 (Figure 3(b)). At ν=3\nu=3, its fused symmetric contraction becomes faster than the full FlashCart ladder at larger system sizes. For example, with 65,53665{,}536 nodes and 128128 feature channels, the VJP-of-VJP takes 0.380.38 μ\mus/node for path-reduced products and 1.371.37 μ\mus/node for the full ladder, compared with 1.151.15 μ\mus/node for cuEquivariance, averaged over the three maximum tensor ranks and three independent runs.

An important difference is that cuEquivariance evaluates a fused symmetric contraction, while our full ladder evaluates products and learned linear maps separately. The linear maps may increase runtime at low correlation orders, but they compress paths and channels to keep the number of intermediate features fixed as the order increases. Figure 3(b) shows results for FlashCart up to ν=6\nu=6 at every tested rank, and we have also tested orders up to ν=8\nu=8 in internal benchmarks. By comparison, cuEquivariance completed all three evaluations only up to ν=3\nu=3 at maximum tensor ranks 33 and 44 in our benchmarks. We have expanded this discussion in Section 6 and Appendix G, including the initial kernel preparation and compilation times.

Finally, to examine the cost of retaining coupling paths directly, we have also added Figure G3. It compares mathematically equivalent reduced and retained paths using the same scalar weights and compiled PyTorch operations. For example, with 40964096 nodes, 128128 feature channels, ℓmax⁡=3\ell_{\max}=3, and ν=5\nu=5, path-reduced products make the complete VJP-of-VJP evaluation 15.8×15.8\times faster and reduce its additional peak GPU allocation by a factor of 48.048.0, averaged over three independent runs. At ℓmax⁡=4\ell_{\max}=4 and ν=5\nu=5, the retained variant runs out of GPU memory for the forward, VJP, and VJP-of-VJP.

W4: We have replaced ll with ℓ\ell for tensor ranks throughout the text and equations.

We hope these clarifications and additional results address the reviewer's concerns.

Sincerely yours,

The authors

Reviewer_xzaqOpenReview ↗

Response to Authors

Thank you for your detailed rebuttal. I believe most of my questions are resolved.

However, for the first point, I still have some questions that need to be clarified. I don't see how kernel tracing offers a conceptual advantage over vanilla implementations of CG-TP, like the e3nn package, which also uses torch.fx for symbolic tracing and kernel buildup. Regarding efficiency, could you provide the exact time complexity w.r.t. the maximum angular degree and the number of degrees?

Comments on the paper

2 comments

We thank the reviewers for their feedback and suggestions. We found that most of the weaknesses and questions raised concern the contributions of the architecture and kernels and the coverage of the benchmarks. To address these points, we have clarified the scope of our contributions and added further benchmarks to the revised manuscript. We summarize the main points below:

  • We clarify that we introduce fixed-width mixing into the recursion to make higher correlation orders affordable. After each product, we mix coupling paths and feature channels back to a fixed width, so the number of intermediate features does not grow with correlation order. We attribute the accuracy gains to increasing correlation order, rather than compression itself. Furthermore, we attribute the improved computational efficiency of the kernels to their implementation rather than the choice of basis alone.

  • We introduce Figure G3, which compares reduced and retained coupling paths using the same scalar weights and compiled PyTorch operations. We find that, for example, with 40964096 nodes, 128128 feature channels, ℓmax⁡=3\ell_{\max}=3, and ν=5\nu=5, path reduction makes the complete VJP-of-VJP evaluation 15.8×15.8\times faster and reduces its additional peak GPU allocation by a factor of 48.048.0. We also discuss the existing evidence on the accuracy benefit of increasing correlation order (Figure 4) and the contribution of the generated kernels using the same model checkpoint (Figure G5).

  • We introduce Figures G1 and G2, which examine runtime and memory for different numbers of nodes and feature channels and, for edge convolutions, neighbors per node. We have also added FlashTP to Figure 3(a) and the additional convolution benchmarks introduced in the revised manuscript. For example, across the 6060 configurations, FlashCart evaluates the VJP-of-VJP on average 1.88×1.88\times, 3.02×3.02\times, and 6.33×6.33\times faster than cuEquivariance, OpenEquivariance, and FlashTP, respectively.

  • For node-local products at ν=3\nu=3, cuEquivariance becomes faster than the full FlashCart ladder at larger system sizes. This is partly because cuEquivariance evaluates the symmetric contraction as a fused operation, while our ladder evaluates products and learned linear maps separately. The linear maps increase runtime at low orders but keep the number of intermediate features fixed as the order increases, making higher orders affordable. FlashCart reaches ν=6\nu=6 at every tested rank in Figure 3(b), while cuEquivariance is already unable to evaluate the VJP-of-VJP at ν=4\nu=4 for maximum tensor ranks 33 and 44.

  • We evaluate GRACE-OFF models on the same SPICE-MACE-OFF test structures and add the results to Figure 1 and Table G1. Compared with the largest GRACE-OFF model (1.61.6M parameters), our 464464k-parameter model has 3939 % lower mean energy error and 5151 % lower mean force error and is 3.1×3.1\times faster. We note that FlashCart was trained on SPICE-MACE-OFF, while GRACE-OFF used a subset of SPICE-v2 comprising molecules with a formal net charge of zero. The authors selected this subset using the same criteria as for MACE-OFF to make the models comparable (see Ref. [25] in the revised manuscript).

Below, we respond to each comment and indicate where the corresponding changes appear in the manuscript. All changes to the manuscript are highlighted in blue.

Sincerely yours,

The authors

Following a suggestion in the discussion, we have repeated the runtime and memory measurements on an NVIDIA A100 GPU (80 GB), using the same model checkpoints and benchmark systems as in Figures 1 and G5. We have added Figures G6 and G7 to Appendix G, with experimental details in Appendix F and discussion in Section 6 and Appendix G.

We found that FlashCart remains on the observed accuracy–efficiency frontier for runtime and memory on both GPU architectures. Compared with the published A100 results for eSEN-3.2M and eSEN-6.5M, our model with 5.65.6M parameters is 7.8×7.8\times and 15.5×15.5\times faster, respectively, with lower mean energy and force errors.

Sincerely yours,

The authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.