acceptodds
Under review as a conference paper at ICLR 2027

LunaTP: Enabling Efficient and Scalable Tensor Product for Equivariant MLIPs

10 views

Abstract

Equivariant machine learning interatomic potentials (MLIPs) promise atomistic simulations with accuracy approaching that of quantum-mechanical calculations. However, the high computational cost of their core Clebsch–Gordan tensor product (CGTP) convolutions severely limits model scalability and efficiency. In this paper, we rethink the parallel execution of CGTP convolutions and introduce LunaTP. Built on node-centric execution, LunaTP exploits path-group parallelism and multi-level reuse to accelerate CGTP convolutions and higher-order derivative computation while preserving the underlying mathematical formulation. On an NVIDIA H100 GPU, LunaTP achieves kernel speedups of up to 250.0 over e3nn and 17.1 over OpenEquivariance. It also delivers end-to-end training throughput gains of up to 28.7 over e3nn and 3.11 over OpenEquivariance. Finally, LunaTP demonstrates scalability and robust performance across models, workload scales, and GPUs, delivering substantial acceleration for training and atomistic simulation with high-accuracy equivariant MLIPs. Our full source code will be made publicly available.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
CGTP convolution, Equivariant MLIPs, GPU optimization

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_6L1aOpenReview ↗
Rating
4
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

LunaTP introduces fused GPU kernels for Clebsch–Gordan tensor-product convolutions while preserving the original formulation and coupling paths. Its node-centric execution accumulates contributions from incoming edges on chip before writing the results to global memory. Register-aware path grouping balances input reuse against accumulator storage, while task ordering accounts for differences in node degree and path cost. The generated kernels exploit CG sparsity and reuse shared inputs, products, and coefficients.

The implementation extends these optimizations to backward, double-backward, and triple-backward computations. Evaluation covers kernel performance, numerical accuracy, end-to-end training, and molecular dynamics across multiple models, GPUs, and numerical precisions.

Critical strengths

Broad applicability and practical derivative support. LunaTP accelerates existing CGTP-based models without restricting their coupling paths or requiring retraining. The optimization addresses both computation and graph reduction, rather than accelerating isolated tensor products alone. Higher-order derivative support extends its usefulness from inference to force and Hessian training. Numerical comparisons against e3nn cover the forward pass and derivatives through third order in FP32 and FP64. The NequIP-XL training experiment additionally uses the same initialization and training settings for both backends, obtaining similar convergence and validation errors differing by less than 0.3% across energy, force, stress, and Hessian targets. These checks provide meaningful evidence of correctness beyond the performance measurements.

Extensive evaluation on contemporary SOTA models and optimized baselines. Training benchmarks cover eight architectures: MACE-L, NequIP-XL, TACE-L, SevenNet-Omni, EquFlashV2, NequIX, Prophet, and BAM. Comparisons include OpenEquivariance, cuEquivariance, and FlashTP, rather than relying only on gains over e3nn. Additional experiments examine different GPUs, precisions, system sizes, graph densities, angular degrees, and channel widths. Matched-batch and maximum-batch results are clearly distinguished, helping separate execution improvements from larger-workload benefits. The MD benchmarks further demonstrate practical gains on TACE-L and NequIP-XL, reporting higher speed than OpenEquivariance at the reported large-system workloads.

Critical weaknesses

Further memory optimization remains an opportunity. The paper reports that memory consumption is comparable to FlashTP and OpenEquivariance from forward through double-backward, although LunaTP provides further savings for triple-backward. The convolution still consumes path weights generated by the radial MLP. Given the demonstrated speed improvements, extending fusion to radial-weight generation could offer additional memory savings by avoiding materialized edge weights. I would welcome discussion of combining LunaTP with streaming ideas from Sobek. This is a potential extension rather than a demonstrated flaw or a result established by the current paper.

Questions for authors

Beyond avoiding materialized radial weights, what similarities and differences exist between Sobek and LunaTP in graph reduction, on-chip accumulation, computational reuse, and higher-order differentiation? Which aspects of Sobek’s streaming design could complement LunaTP’s current optimizations?

4 comments

We thank the reviewer for highlighting LunaTP’s broad applicability and extensive evaluation. Below, we discuss further memory optimization and respond to the questions about Sobek.

Weakness & Question: Memory optimization and complementarity with Sobek

We thank you for the insightful comment. We agree that extending the fusion scope to radial-weight generation can further reduce memory usage.

Sobek uses streaming execution to integrate radial projection, tensor products, and graph aggregation, avoiding the materialization of full per-edge weights and their gradients. This approach can complement LunaTP's computational reuse. In addition, Sobek's strategy of saving or recomputing node aggregates under a workspace memory budget could help control temporary memory usage as the fusion scope expands.

We have implemented a preliminary version that fuses the final Linear projection of the radial network with TP convolution. Specifically, the kernel computes radial projections in edge blocks and temporarily stores the resulting weight blocks in shared memory. The same kernel then performs CG contractions and local neighborhood aggregation, avoiding the global storage of full per-edge weights.

We evaluated the fused version on an H100 GPU (FP32, Linear input dimension 64). LunaTP + Linear and LunaTP (fused Linear) denote the separate and fused implementations, respectively. Entries report runtime (ms) / peak memory (MiB).

CaseStageSobekLunaTP + LinearLunaTP (fused Linear)
C2Forward6.2291 / 57.10.3790 / 325.60.2811 / 56.8
C2Backward2.2182 / 72.70.5154 / 609.20.9292 / 71.6
C2Double-bwd34.7733 / 212.92.3674 / 1107.42.7377 / 87.1
C2Triple-bwd—6.2892 / 1899.37.2570 / 105.7
C4Forward21.2547 / 112.21.3149 / 982.01.0206 / 112.0
C4Backward10.3002 / 135.61.9119 / 1884.53.3581 / 135.4
C4Double-bwd119.2751 / 279.08.1955 / 3595.910.0302 / 162.0
C4Triple-bwd—21.9084 / 6360.028.0072 / 199.4
C6Forward22.5446 / 100.01.4378 / 897.81.0932 / 99.8
C6Backward16.7077 / 122.32.2736 / 1784.33.5164 / 121.8
C6Double-bwd168.7816 / 267.58.8048 / 3347.910.5208 / 147.6
C6Triple-bwd—24.0410 / 5886.527.7071 / 180.5
C8Forward25.1624 / 65.11.1192 / 613.10.8736 / 65.0
C8Backward18.6864 / 79.31.5587 / 1231.54.5521 / 79.1
C8Double-bwd142.0085 / 219.46.3129 / 2286.38.3332 / 96.0
C8Triple-bwd—18.3149 / 4001.421.4277 / 117.5
C10Forward31.1948 / 65.71.3048 / 626.21.0752 / 66.0
C10Backward23.5297 / 79.81.9599 / 1259.65.0179 / 80.7
C10Double-bwd182.4347 / 222.66.9170 / 2338.59.9779 / 98.1
C10Triple-bwd—22.1552 / 4123.828.5375 / 120.7

Across these configurations, LunaTP (fused Linear) reduces peak memory usage by 82.6%–97.1% and achieves 1.21–1.35× forward speedups over the separate implementation. Compared with Sobek, it achieves 2.39–29.01× speedups from forward through double backward with comparable or lower peak memory usage. In double backward, peak memory usage is reduced by 41.9%–59.1%. The fused version also supports triple backward.

LunaTP (fused Linear) is still at an early stage of optimization, and its higher-order backward passes currently take longer than those of the unfused version. We found that fusion increases register pressure, while shared-memory accesses and synchronization between tasks also introduce additional overhead. Nevertheless, these results show that fusing radial projection with TP convolution can significantly reduce LunaTP's memory usage. We are continuing to optimize this kernel and commit to open-sourcing the implementation for the community.

Q1: Similarities and differences between Sobek and LunaTP

Both Sobek and LunaTP use node-centric reduction and on-chip accumulation, but differ in work partitioning, computational reuse, and higher-order differentiation, as discussed below.

Graph reduction: Both implementations use node-centric reduction. Sobek partitions forward tasks by node and channel block, while LunaTP additionally partitions work across path groups, allowing different path groups for the same node to execute in parallel.

On-chip accumulation: Both implementations keep output accumulators live during neighborhood traversal and accumulate neighbor contributions on chip. LunaTP constrains path-group size using a per-channel output component budget. This preserves input reuse within each group while controlling register usage to maintain sufficient parallelism.

Computational reuse: Sobek processes pairs of paths that share inputs during double backward to reduce repeated loads. LunaTP reuses inputs through path grouping and further reuses intermediate products within each path. It also combines product terms that share coefficients to reduce redundant computation.

Higher-order differentiation: Both implementations organize reductions according to the differentiation target and use dedicated double-backward kernels. LunaTP extends path grouping and within-path reuse to derivative computation. It fuses derivative branches that share inputs and assigns independent branches to different CUDA streams to overlap their execution. These mechanisms are further extended to dedicated triple-backward kernels.

In terms of feature support, Sobek supports only FP32 and requires all input and output node irreps to have the same multiplicity, with an edge irrep multiplicity of 1. In contrast, LunaTP does not have these restrictions.

A detailed comparison of these implementations has been added to Appendix A.

Please let us know if you have any feedback to our explanation or other questions. We will be very happy to respond.

Thank you for the detailed explanation and clarification. I look forward to the release of LunaTP and its broader adoption by the community. In particular, we look forward to releasing a general-purpose interface for Sobek-style fused operations, as Sobek's existing interface is not sufficiently general, limiting its widespread adoption by the community. Existing engines that employ Sobek-inspired approaches are tailored to specific MLIP architectures rather than providing broadly reusable implementations.

We thank the reviewer for the positive feedback and suggestion. We will ensure that LunaTP provides flexible, general-purpose interfaces that are easy for the community to use.

Reviewer_hNKGOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes LunaTP, which parallelizes the edge computation of each CG-TP for each node and parallelizes path computation via grouping. The results show the effectiveness of the method compared to famous CG-TP implementation packages, including cuEquivariance, FlashTP, and e3nn.

Critical strengths

The idea is clear, and the speedup in Figure 4 compared to FlashTP and cuEquivariance is really outstanding.

Critical weaknesses
  1. FlashTP [1] has already used parallelism for edge computation and path aggregation. The paper's contributions largely build against FlashTP, especially when proposing node-centric parallelism and path-group parallelism. The methodological contribution is marginal. However, the microbenchmark performance and engineering are indeed impressive. So I don't think this diminishes the value of the paper.
  2. I would like to see more ablation tests, e.g., various batch sizes or various node degrees, especially with high std.
  3. In Figure 5, the speedup over MACE is marginal compared to OpenEquivariance. Why?

[1] Lee, Seung Yul, et al. "FlashTP: Fused, Sparsity-Aware Tensor Product for Machine Learning Interatomic Potentials." ICML. 2025.

Questions for authors
  1. Please discuss and cite [2] [3] [4]
  2. The notation could be improved. Equation 1 is standard, but the notation later on, like Cm1m2pC_{m_1 m_2}^p, is not. You need to define it w.r.t. the original notation. Also, the inline equation at line 286 is informal. Please revise.

[2] Li, Yunyang, et al. "E2former: An efficient and equivariant transformer with linear-scaling tensor products." Advances in Neural Information Processing Systems 38 (2026): 23593-23630.

[3] Lin, Yuchao, et al. "Tensor Decomposition Networks for Fast Machine Learning Interatomic Potential Computations." Advances in Neural Information Processing Systems 38 (2026): 88471-88495.

[4] Xie, YuQing, et al. "Asymptotically Fast Clebsch-Gordan Tensor Products with Vector Spherical Harmonics." arXiv preprint arXiv:2602.21466 (2026).

4 comments

We appreciate the reviewer’s positive assessment of LunaTP’s clear design and strong performance. Below, we clarify our contributions and address the concerns and suggestions.

W1: Contributions beyond FlashTP

FlashTP uses sparse CG computation, kernel fusion, and cross-path input reuse. Building on these techniques, LunaTP further explores performance optimizations for node-centric execution. The main implementation differences are summarized below.

AspectFlashTPLunaTP
Execution strategyEdge-parallelNode-centric
Work partitioningEdge × aggregated paths × channel blockNode × budgeted path group × channel block
Task orderingEdge indexCG-term count, node degree
Data and computation reuseCross-path inputsCross-path inputs, products, coefficients
Graph aggregationGlobal atomicsOn-chip accumulation
Kernel compilationAOTJIT
Third-order derivativesNoYes

In terms of feature support, FlashTP requires channel counts to be multiples of 32 and edge irrep multiplicity to be 1. LunaTP imposes no channel alignment constraint and supports edge irrep multiplicities greater than 1.

We have provided a detailed comparison of LunaTP and FlashTP in Appendix A.

W2: Batch-size and node-degree ablations

We added three ablation studies on batch size, node degree, and degree distribution, all using configuration C5. The base graph contains 512 nodes, each with degree 32. The results are summarized below.

(1) Batch-size ablation.

We keep the structure of each graph fixed and increase the batch size.

StageBatch sizeSpeedup vs. OpenEq
Forward1 → 1283.7× → 3.8×
Backward1 → 646.1× → 8.1×
Double-bwd1 → 643.7× → 3.9×
Triple-bwd1 → 1612.4× → 14.2×

(2) Node-degree ablation.

We fix the batch size at 1 and the node count at 512, and increase the degree of each node from 16 to 256.

StageNode degreeSpeedup vs. OpenEq
Forward16 → 2563.7× → 3.9×
Backward16 → 2565.7× → 7.1×
Double-bwd16 → 2563.4× → 3.8×
Triple-bwd16 → 25612.6× → 12.8×

(3) Degree-distribution ablation.

We fix the batch size at 1 and keep the node and edge counts unchanged. With a mean degree of 32, we increase the degree standard deviation from 0 to 64.

StageDegree std.Speedup vs. OpenEq
Forward0 → 643.7× → 3.4×
Backward0 → 646.1× → 6.3×
Double-bwd0 → 643.7× → 3.8×
Triple-bwd0 → 6412.4× → 12.0×

LunaTP maintains its performance advantage from forward through triple backward across different batch sizes, node degrees, and degree distributions. The full results and analysis have been added to Appendix D.4.

W3: Limited end-to-end gains on MACE

MACE-L uses a relatively small CGTP irrep configuration (Lmax⁡L_{\max}=2,channel=128), resulting in a lower computational cost for CGTP convolutions. Under Matched EFS, e3nn has a step time of 1504.5 ms, with 609.1 ms spent on CGTP convolutions. OpenEq reduces CGTP time to 50.2 ms, accounting for only 5.1% of its total step time (980.1 ms). At this point, non-CGTP computation dominates the training time. According to Amdahl's law, even eliminating the remaining CGTP time would yield at most a 1.05× speedup. LunaTP further reduces CGTP time to 14.5 ms, achieving a 1.04× speedup over OpenEq.

Q1: Discussion of additional related work

We thank the reviewer for pointing out these related works. We have added citations and discussions of all three in Section 6.

E2Former [2] uses spherical harmonic translation expansions and Wigner 6j6j recoupling to reorganize edge-wise tensor products into node-level operations and neighbor aggregation. TDN [3] approximates the CG coefficient tensor with a low-rank canonical polyadic (CP) decomposition and shares weights across paths to reduce computational cost. Xie et al. [4] use vector spherical harmonics and fast spherical harmonic transforms to achieve asymptotic acceleration of the complete CGTP.

These works improve computational efficiency at the algorithm level. In contrast, LunaTP optimizes CGTP execution on GPUs while preserving its original mathematical formulation. We have clarified the relationship between these approaches and LunaTP in the revised manuscript.

Q2: Clarification of notation and equations

Thank you for the careful review. We have explicitly defined the simplified notation for CG coefficients in Section 4.3 and revised the informal mathematical expressions throughout the manuscript.

Thank you again for your review. Please let us know if you have any questions. We would be happy to respond.

[2] Li, Yunyang, et al. "E2former: An efficient and equivariant transformer with linear-scaling tensor products." Advances in Neural Information Processing Systems 38 (2026): 23593-23630.

[3] Lin, Yuchao, et al. "Tensor Decomposition Networks for Fast Machine Learning Interatomic Potential Computations." Advances in Neural Information Processing Systems 38 (2026): 88471-88495.

[4] Xie, YuQing, et al. "Asymptotically Fast Clebsch-Gordan Tensor Products with Vector Spherical Harmonics." arXiv preprint arXiv:2602.21466 (2026).

Reviewer_hNKGOpenReview ↗

Response to Authors

Thanks for the rebuttal, and I believe my questions are resolved. I raised the score accordingly

Thank you for increasing your score! We are happy that our response addressed your concern, and we thank you again for your constructive feedback.

Reviewer_UqYGOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents LunaTP, a GPU implementation of Clebsch–Gordan tensor-product convolution for equivariant interatomic potentials. It utilizes node-based aggregation, path grouping and reuse of intermediate products, and extends these ideas to higher-order derivatives.

Critical strengths

The work targets a real bottleneck in equivariant MLIPs. I also find the support of double and triple backward quite interesting, as MD simulations or phonons really require such capabilities.

Critical weaknesses

Although this work looks to be an very interesting one, I still have some concerns mainly regarding its performance benchmarks. See below

Questions for authors
  1. The evaluations mostly include the H100, A100, RTX3090. Could the authors also discuss whether the same optimizations are expected to remain effective on Blackwell GPUs?
  2. The MD speed up evaluation is only limited to silicon. It would be nice to have an MD benchmark on a liquid system because liquid has more sophisticated changing local environments. Also, I want to see if this new implementation changes the numerical stability of energy/force prediction in inference and if they will change the stability of long MD simulations
  3. The evaluation focuses on FP32 and FP64. Could the authors clarify whether LunaTP supports FP16 or BF16 mixed-precision execution, and whether the reported FP32 baselines permit TF32 computation where applicable? I understand that lower precision may affect force and Hessian accuracy, so a discussion of the supported precision modes and their numerical limitations would be useful even without additional benchmarks.
  4. Section 6 discusses the main related approaches, but the differences between their implementations remain difficult to follow for readers outside GPU optimization. A compact comparison table showing the execution strategy, data reuse, derivative support, and supported precision would help. In particular, the paper should make clearer which design choices are inherited from earlier implementations and which are specific to LunaTP.
3 comments

We thank the reviewer for recognizing the practical value of our work. We address the benchmarking concerns and questions below.

Q1: Effectiveness on Blackwell GPUs

LunaTP is implemented in CUDA using standard GPU instructions, and its optimizations are not tied to a specific GPU architecture. We therefore expect these optimizations to remain effective on Blackwell.

To assess their effectiveness, we evaluated LunaTP on an NVIDIA B300 GPU (288 GB) using the same benchmark suite. The table below reports speedups relative to e3nn.

BackendForwardBackwardDouble backwardTriple backward
e3nn1.00×1.00×1.00×1.00×
FlashTP7.7×7.3×14.1×—
OpenEq19.9×11.5×21.3×5.8×
LunaTP68.7×67.6×105.8×73.0×

These results show that LunaTP retains its performance advantage on Blackwell. We have added these results to Appendix E.2 (Figure 10) of the revised manuscript.

Q2: Liquid-system benchmarks, prediction accuracy, and long-term MD stability

(1) MD benchmark on a liquid system.

We have added MD benchmarks for TACE-L and NequIP-XL on liquid water. LunaTP achieves speedups of 11.2× and 19.6× over e3nn, and 2.0× and 2.4× over OpenEq, respectively. We additionally evaluated a biomolecular system, achieving corresponding speedups of 2.1× and 2.2× over OpenEq.

These results are included in Section 5.2 and Figure 6 of the revised manuscript.

(2) Numerical stability during inference.

We evaluated the numerical stability of MACE-MP-L, NequIP-OAM-XL, and TACE-OAM-L during inference. We evaluated energy, force, and stress errors against e3nn FP64 on 5,000 random sAlex[1] structures, and force/stress rotational equivariance errors on a 500 structure. The table below reports the mean RMSEs.

ModelBackendEnergy (eV)Force (eV/Å)Stress (eV/ų)Force equivariance (eV/Å)Stress equivariance (eV/ų)
MACE-MP-Le3nn FP328.13e-63.83e-61.70e-74.42e-61.39e-7
LunaTP FP328.20e-63.85e-61.70e-74.39e-61.42e-7
LunaTP FP643.44e-153.21e-151.16e-168.70e-153.25e-16
NequIP-OAM-XLe3nn FP321.64e-51.66e-61.02e-71.99e-65.49e-8
LunaTP FP321.64e-51.67e-61.02e-72.00e-65.46e-8
LunaTP FP643.82e-159.62e-163.43e-173.87e-151.76e-16
TACE-OAM-Le3nn FP324.08e-61.66e-67.12e-82.33e-67.18e-8
LunaTP FP324.21e-61.66e-67.10e-82.32e-67.16e-8
LunaTP FP642.48e-157.34e-163.10e-174.49e-152.05e-16

Across all three models, LunaTP FP32 and e3nn FP32 yield comparable prediction and rotational equivariance errors. We have also added results on Matbench Discovery, where LunaTP and e3nn achieve comparable predictive performance (Table 13).

Full results and analysis are provided in Appendix G of the revised manuscript.

(3) Long-term MD simulations.

We performed 300 ps NVE simulations with MACE-MP-L and NequIP-OAM-XL on 81 systems from TM23[2,3]. The table reports the mean and maximum absolute deviations from the initial total energy (Mean jump and Max jump), averaged over 81 systems in eV/atom.

ModelBackendMean jumpMax jump
MACE-MP-Le3nn3.789e-47.772e-4
MACE-MP-LLunaTP3.794e-47.772e-4
NequIP-OAM-XLe3nn4.913e-49.431e-4
NequIP-OAM-XLLunaTP4.909e-49.432e-4

For both models, LunaTP and e3nn show similar energy deviations and averaged energy curves, indicating comparable energy conservation.

We also performed 1000 ps NVE simulations of a 2000-atom biomolecular system with the LunaTP backend. We used MACE-MH-1 (OMat), SevenNet-Omni-l5 (OMat), and NequIP-OAM-XL. The results are summarized below.

ModelMean jumpMax jump
MACE-MH-1 (OMat)5.590e-48.481e-4
SevenNet-Omni-l5 (OMat)4.643e-48.156e-4
NequIP-OAM-XL4.731e-48.231e-4

All three models maintained stable total energy over 1000 ps. The maximum jump was below 8.5e-4 eV/atom for each model.

The full energy curves and analysis are provided in Appendix H (Figure 15) of the revised manuscript.

Q3: FP16/BF16 support and TF32 settings

(1) FP16/BF16 support and numerical limitations.

Thank you for the suggestion. Our original evaluation focused on FP32 and FP64. During the rebuttal period, we implemented and tested FP16/BF16 mixed-precision variants of LunaTP. These kernels use FP32 for reductions and FP16 or BF16 for other operations.

We evaluated NequIP-OAM-XL’s predictions against e3nn FP64 and tested the rotational equivariance of its force and stress outputs. The table below reports the mean RMSEs.

ConfigurationEnergy (eV)Force (eV/Å)Stress (eV/ų)Force equivariance (eV/Å)Stress equivariance (eV/ų)
LunaTP FP643.82e-159.62e-163.43e-173.87e-151.76e-16
LunaTP FP321.64e-51.67e-61.02e-72.00e-65.46e-8
LunaTP BF16 (mixed)1.74e-22.10e-31.31e-41.52e-36.20e-5
LunaTP FP16 (mixed)2.14e-32.63e-41.66e-51.94e-47.66e-6

The results show that FP16 and BF16 mixed precision yield errors roughly two and three orders of magnitude larger than FP32, respectively. FP16 (5 exponent bits, 10 fraction bits) has a narrower dynamic range and is more prone to overflow and underflow. BF16 (8 exponent bits, 7 fraction bits) has a range close to FP32 but lower precision than FP16, typically producing larger rounding errors. These errors propagate through forward and derivative computations, affecting predictions and their derivatives.

We therefore recommend FP32 or FP64 for precision-sensitive applications, including Hessian calculations, phonon calculations, and NVE MD. FP16/BF16 mixed precision may be considered for coarse energy ranking, RDF, and coordination distributions.

(2) TF32 settings.

We disabled TF32 for all backends in all experiments to maintain consistent precision settings. We have clarified this in Appendix B of the revised manuscript.

Q4: Implementation comparison and clarification of contributions

Thank you for the suggestion. We have added a comparison of LunaTP with other methods in Appendix A (Table 1), summarized below.

Implementation design

cuEqFlashTPOpenEqSobekLunaTP
Execution strategy—Edge-parallelEdge-parallelNode-centricNode-centric
Work partitioning—Edge × aggregated paths × channel blockEdge segmentNode × channel blockNode × budgeted path group × channel block
Task ordering—Edge indexEdge indexNode indexCG-term count, node degree
Data and computation reuse—Cross-path inputsCross-path inputsCross-path inputsCross-path inputs, products, coefficients
Graph aggregation—Global atomicsGlobal atomics / on-chip partial sumsOn-chip accumulationOn-chip accumulation
Fusion scopeTP + aggregationTP + aggregationTP + aggregationRadial projection + TP + aggregationTP + aggregation (+ envelope)
Kernel compilation—AOTJITJITJIT

Feature support

AspectcuEqFlashTPOpenEqSobekLunaTP
Second-order derivativesYesYesYesYesYes
Third-order derivativesYesNoYesNoYes
Arithmetic precisionFP32, FP64FP32, FP64FP32, FP64FP32FP32, FP64
Nonuniform multiplicitiesYesYesYesNoYes
Channel alignmentNoneMultiple of 32NoneNoneNone
Edge irrep multiplicity≥11≥11≥1

Like prior work, LunaTP uses sparse CG computation, kernel fusion, cross-path input reuse, and JIT code generation. Building on these techniques, we further explore performance optimizations for node-centric execution:

  • Node-centric execution: LunaTP organizes CGTP convolutions around nodes and performs neighborhood aggregation on chip to reduce global memory updates. Path-group parallelism further increases intra-node parallelism.
  • Work partitioning: LunaTP partitions work across nodes, path groups, and channel blocks. A per-channel output component budget constrains path-group size to balance input reuse and register usage.
  • Task ordering: LunaTP sorts path groups by their total number of nonzero CG terms and nodes within each group by degree, both in descending order, to mitigate load imbalance.
  • Data and computation reuse: LunaTP reuses inputs across paths and intermediate products within each path. It also combines product terms that share coefficients to reduce redundant memory accesses and computation.
  • Higher-order derivative optimization: LunaTP extends these execution and reuse mechanisms to double-backward and triple-backward kernels. Independent derivative branches are assigned to different CUDA streams to overlap their execution.

We’re happy to discuss these points with you, and please feel free to let us know if anything is unclear!

[1] Schmidt at al. Improving machine-learning models in materials science through large datasets. Materials Today Physics.

[2] Fu et al. Learning Smooth and Expressive Interatomic Potentials for Physical Property Prediction. ICML 2025.

[3] Owen et al. Complexity of many-body interactions in transition metals via machine-learned force fields from the {TM23} data set. npj.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revisions

We thank all reviewers for their thoughtful and constructive feedback. We have revised the manuscript accordingly and highlighted the changes in blue. The main revisions are summarized below.

  1. Performance evaluation (Reviewers UqYG, hNKG): Added B300 benchmarks (Appendix E.2), liquid-water and biomolecular inference benchmarks (Section 5.2), and ablations on batch size, node degree, and degree distribution (Appendix D.4).

  2. Accuracy and stability (Reviewer UqYG): Added numerical accuracy, rotational equivariance, and Matbench Discovery evaluations (Appendix G), alongside NVE simulations on TM23 and a biomolecular system (Appendix H).

  3. Contributions and presentation (Reviewers UqYG, hNKG, 6L1a): Expanded related work and implementation comparisons (Section 6; Appendix A), clarified LunaTP’s contributions and notation (Appendix A; Section 4.3), and specified inference protocols (Appendix B.4) and TF32 settings (Appendix B).

Sincerely,

The authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.