- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces FlashCart, combining optimized Cartesian tensor-product kernels with a fixed-width recursive architecture to efficiently construct higher-order correlations. Its ablations show that increasing correlation order offers better accuracy–cost tradeoffs than increasing depth, width, or tensor rank. Operator benchmarks and molecular and materials evaluations demonstrate competitive accuracy and substantial speedups, advancing the measured accuracy–efficiency frontier.
Critical strengths
-
FlashCart provides efficient Triton kernels for Cartesian tensor products, including forward, backward, and double-backward computations.
-
The fixed-width recursive architecture constructs higher-order correlations while keeping intermediate features compact.
Critical weaknesses
The evaluation does not sufficiently separate architectural gains from kernel improvements. The symmetric contraction comparison changes both the architecture and implementation, making the reported speedups difficult to attribute. Although end-to-end scaling is evaluated over 216–4096 atoms, Figure 3 uses fixed node counts and does not establish kernel scalability across workload sizes. A breakdown and broader operator-level sweeps are needed to substantiate the performance claims.
The benefits of fixed-width compression over prior recursive constructions remain insufficiently validated. ICTP already supports Cartesian tensor products and recursive correlation construction. Benchmarks establish competitive overall accuracy, but do not quantify the accuracy–cost tradeoff introduced by fixed-width compression. A comparison with ICTP or a less compressed product basis is needed to establish the value of this architectural contribution.
Questions for authors
Q1: What are the respective contributions of the fixed-width recursive architecture and the optimized kernels to FlashCart’s accuracy–efficiency frontier? Can these contributions be quantified separately?
Q2: Why do the edge-convolution and symmetric-contraction benchmarks in Figure 3 use different node counts? How were these configurations selected, and how sensitive are the reported speedups to workload size?
Q3: Could the authors provide more insight into what currently limits the performance of the Cartesian kernels and where they see the most promising opportunities for further optimization?
Q4: Why is eSEN’s runtime estimated from published A100 measurements while the other models are benchmarked on Blackwell?
6 comments
Q2: We thank the reviewer for raising this point and have added Figures G1 and G2 in Appendix G, with benchmark details in Appendix F.
We chose nodes for edge convolutions and nodes for node-local products to represent realistic system sizes and provide enough parallel work for the GPU. Convolutions process contributions from every edge, while node-local products act on each node, so the two operations provide different amounts of parallel work. The additional benchmarks in Figures G1 and G2 support these choices, showing little further reduction in FlashCart's runtime per node beyond these system sizes.
We agree that the original manuscript did not sufficiently examine how kernel runtime and memory depend on the number of nodes, feature channels, and neighbors per node. The additional benchmarks cover – nodes for edge convolutions and – nodes for node-local products. We use , , and feature channels and maximum tensor ranks . For convolutions, we also compare , , , and neighbors per node. We use feature channels for the largest systems.
For edge convolutions, FlashCart remains faster on average than the baseline implementations for the forward, VJP, and VJP-of-VJP. For the VJP-of-VJP, for example, it is faster than cuEquivariance, faster than OpenEquivariance, and faster than FlashTP. These averages use the configurations for which all four implementations completed the VJP-of-VJP, with three independent runs for each configuration. At nodes, feature channels, neighbors per node, and , FlashCart also completes the VJP-of-VJP while all three baselines run out of GPU memory.
For node-local products, the additional benchmarks use because cuEquivariance could not evaluate the VJP-of-VJP at for maximum tensor ranks and (Figure 3(b)). At , the runtime difference decreases as the number of nodes increases, and cuEquivariance becomes faster than the full FlashCart ladder at larger system sizes. For example, at nodes and feature channels, the VJP-of-VJP takes s/node for path-reduced products and s/node for the full ladder, compared with s/node for cuEquivariance, averaged over the three maximum tensor ranks and three independent runs. At nodes, forward evaluation also takes s/node with cuEquivariance and s/node with FlashCart's path-reduced products.
An important difference is that cuEquivariance evaluates a fused symmetric contraction, while our full ladder evaluates products and learned linear maps separately. The linear maps increase runtime at low correlation orders, but compress paths and channels to keep the number of intermediate features fixed as the order increases. Figure 3(b) shows results for FlashCart up to at every tested rank, and we have also tested orders up to in our internal benchmarks. By comparison, cuEquivariance completed all three evaluations only up to at maximum tensor ranks and in our benchmarks. Furthermore, Figure G3 shows the computational benefit of reducing paths between correlation orders, as discussed in Q1. We have also expanded the corresponding discussion in Section 6 and Appendix G.
Q3: At higher tensor ranks, more tensor components and coupling paths must be evaluated. The challenge is to organize these calculations so that inputs and intermediate results can be reused without requiring too much temporary storage. We see opportunities in adapting how the calculations are grouped into kernels for different tensor ranks, numbers of feature channels, and numbers of neighbors.
We also see opportunities to exploit further symmetries in the recursive products. Together with more effective symbolic simplification, this could reduce computational cost and the size of the generated code. Code growth also affects initial compilation at higher ranks, although compiled kernels are cached and reused. The current schedules and compilation costs are described in Section 4 and Appendices C and D.
Beyond these performance optimizations, extending the kernels beyond natural parity would broaden the range of couplings and architectures they support.
We want to note that we already discuss some of these limitations in the manuscript. We outline the more specific directions for further kernel optimization here to keep the Limitations paragraph concise.
Q4: To the best of our knowledge, the checkpoints for the eSEN-3.2M and eSEN-6.5M SPICE-MACE-OFF models are not publicly available, so we could not benchmark these models on our NVIDIA RTX PRO 6000 Blackwell GPUs.
We hope these clarifications and additional results address the reviewer's concerns.
Sincerely yours,
The authors
For the Q4, how about aligning experiments on A100?
We thank the reviewer for their feedback and suggestions. We address the weaknesses in our responses to the questions below and first indicate where each concern is discussed.
We address the separation of architectural and kernel contributions in Q1 and the broader operator benchmarks in Q2.
We discuss the scope of our fixed-width claim and the new comparison of reduced and retained coupling paths in Q1.
Q1: We agree that the architectural and kernel contributions should be examined separately. For this purpose, we have added a comparison of reduced and retained coupling paths in Figure G3 in Appendix G, with experimental details in Appendix F. Alongside these results, the revised manuscript and the discussion below examine the contribution of the generated kernels using the same model checkpoint (Figure G5) and the accuracy benefit of increasing correlation order (Figure 4).
Regarding the architectural contribution, our claim is that keeping the intermediate width fixed reduces computational cost and makes higher correlation orders affordable. To keep this width fixed, our ladder mixes paths and channels jointly after each product to produce d channels per retained tensor rank. At fixed tensor ranks and feature width, each additional order requires one product and one linear map, so the computational cost grows linearly. We do not attribute an accuracy gain to compression itself.
We instead attribute the accuracy gains to using higher correlation orders, which the fixed-width ladder makes affordable. Figure 4 shows that, in the tested configurations, increasing correlation order gives larger error reductions relative to added runtime than increasing the number of feature channels, message-passing layers, or tensor rank. We also compare models at approximately fixed parameter count and find that increasing correlation order while reducing the number of feature channels lowers force error by 9–29 % and runtime by 10–34 %.
However, we agree that the original manuscript did not clearly demonstrate the computational cost of retaining coupling paths. To make the benefit of path reduction clearer, we have added a comparison of reduced and retained paths in Figure G3. Both variants use the same scalar weights, Cartesian products, tensor rank cutoffs, and compiled PyTorch operations. The weights are shared across the retained contributions, making the two variants mathematically equivalent by bilinearity of the tensor products. Figure G3 shows, for example, that with 4096 nodes, 128 feature channels, ℓmax=3, and ν=5, path-reduced products make the complete VJP-of-VJP evaluation 15.8× faster and reduce its additional peak GPU allocation by a factor of 48.0, averaged over three independent runs.
These results show the computational benefit of reducing paths at higher correlation orders. Furthermore, our path and channel mixing can be used with ICTP, TACE, or spherical tensor products and combined with other choices of tensor rank cutoffs and coupling paths. We do not claim in the manuscript that this mixing is specific to FlashCart or to the chosen recursion.
To examine the accuracy–cost tradeoff, we compare FlashCart with the six released GRACE-OFF models (Ref. [25] in the revised manuscript). These models apply learned linear maps between products while keeping contributions from different coupling paths separate. On the 50,195 SPICE-MACE-OFF test structures, our 464k-parameter model has lower mean energy and force errors than any of these models. Compared with the largest GRACE-OFF model (1.6M parameters), these errors are 39 % and 51 % lower, respectively, and evaluation is 3.1× faster. We note that FlashCart was trained on SPICE-MACE-OFF, while GRACE-OFF used a subset of SPICE-v2 comprising molecules with a formal net charge of zero. The results are reported in Figure 1 and Table G1 of the revised manuscript, with evaluation details presented in Appendix F.
Finally, Figure G5 (Figure G2 in the original manuscript) measures the contribution of the generated kernels with the Cartesian representation, architecture, and a model checkpoint with 5.6M parameters held fixed. With both implementations compiled, the generated kernels make inference 8.2× and the training benchmark 5.4× faster than the corresponding PyTorch operations.