- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper targets the training cost of based interatomic potentials. The dense matrix products in these models are already efficient, but current implementations write large intermediate edge tensors to memory for the steps around them, namely gathering, rotation, recombination, layout changes and inverse rotation. TrimSO(2) folds these steps into neighbouring operations. Rotation happens while gathering node features, recombination happens inside the activation, and inverse rotation happens during the reduction onto nodes. The kernels support the backward and double backward passes needed for force training, and a similar idea is applied to the node feed forward networks. On eSEN, UMA and EquiformerV3 trained on OMat24 in FP32, TrimSO(2) gives to faster training, to lower peak memory and to less HBM traffic than the official PyTorch code, while matching its outputs and gradients to about .
Critical strengths
-
The gains are consistent across three recent models and two GPUs, and the numerical checks are thorough. Outputs, gradients and training states match the reference closely, and the architectures are unchanged, so the method can be dropped into existing training code.
-
The paper handles the double backward pass needed to train on forces, including the rotation derivatives. This is the hard part, and it is often what stops existing fusion or compilation tools from being used for these models.
Critical weaknesses
-
The only baseline is the official PyTorch code in eager FP32. This is a relevant baseline, since torch.compile does not support the double backward pass and the official training setups run eagerly. Still, a to gain from fusing gather, scatter and elementwise operations over eager PyTorch is roughly what one would expect, and the paper does not test a stronger option such as a traced and compiled path, as the authors themselves note. All runs also disable TF32 and mixed precision, while models like UMA are pretrained in BF16. With faster low precision matrix products the balance of the step changes, so the reported speedups may not carry over to how these models are actually trained.
-
The technical idea is close to existing GNN systems work. Avoiding intermediate edge tensors by fusing gather, transform and scatter is used in Graphiler, FlashTP, cuEquivariance and OpenEquivariance. The new part is carrying this through the layout changes and the double backward, but the paper does not explain what made this hard beyond existing designs. Also, much of the EquiformerV3 gain comes from the node feed forward reordering, which is not specific to (Table 7 goes from to when it is added).
-
HBM traffic drops for UMA but the step time only drops . A breakdown of the remaining time would show where the limit now is.
-
On H100, the full TrimSO(2) step for UMA (244.86 ms) is slower than the previous ablation stage (238.37 ms) in Table 7.
Questions for authors
- How does the forward pass of TrimSO(2) compare with fairchem's Triton kernels for UMA inference?
- Why is the full method slower than the previous stage for UMA on H100?
- Need to show training speedups with TF32 or BF16 enabled.
- Need to show validation errors after the 40,000 step eSEN run, and results on graphs with uneven neighbour counts.
4 comments
5. Training performance on graphs with uneven neighbor counts. On A100 we compared four neighbor-count distributions with EquiformerV3. Each graph has 256 atoms and 16,384 directed edges, averaging 64 neighbors per atom. Each setting ran independently three times, timing 100 steps per run. The table reports the mean time and the highest peak memory over the three runs.
| Neighbor-count SD | PyTorch (ms/step) | TrimSO(2) (ms/step) | Speedup | PyTorch peak memory (GiB) | TrimSO(2) peak memory (GiB) |
|---|---|---|---|---|---|
| 0 | 1548.60 | 530.46 | 2.92× | 50.61 | 16.03 |
| 16 | 1553.69 | 537.27 | 2.89× | 50.61 | 16.03 |
| 32 | 1553.21 | 533.96 | 2.91× | 50.61 | 16.03 |
| 48 | 1553.72 | 536.28 | 2.90× | 50.61 | 16.03 |
With the total edge count fixed, TrimSO(2) achieves 2.89–2.92× training speedup as the neighbor-count standard deviation ranges from 0 to 48, and peak memory is about 32% of official PyTorch. Peak memory is the same in all four settings (50.61 and 16.03 GiB), as expected. With the total edge count fixed, the peak is set by the tensors allocated per edge, independent of how those edges are distributed across nodes. Time is also essentially unaffected, so uneven neighbor counts introduce no pronounced load imbalance.
6. Comparison with a compiled baseline. We used NequIP-style tracing to place the energy, force, and stress computations in a single graph, then compiled that graph and the parameter-gradient computation it requires with PyTorch Inductor. We measured full training steps for eSEN, UMA, and EquiformerV3 on A100 and H100 (Appendix C.2, Table 6). The compiled baseline was faster than eager in all six comparisons. Over this baseline, TrimSO(2) still achieves 1.16–2.62× end-to-end speedup and 1.73–2.75× lower peak allocated memory.
7. UMA HBM traffic and kernel time. We broke down HBM traffic and GPU kernel time during UMA training on A100.
| Kernel category | PyTorch HBM read+write (GiB) | TrimSO(2) HBM read+write (GiB) | PyTorch kernel time (ms) | TrimSO(2) kernel time (ms) |
|---|---|---|---|---|
| Elementwise kernels | 979.47 | 15.60 | 704.96 | 16.64 |
| Matrix multiplication | 54.02 | 36.83 | 309.66 | 241.92 |
| Other kernels | 37.53 | 55.94 | 46.57 | 60.85 |
| Total | 1071.02 | 108.37 | 1061.19 | 319.41 |
Before the optimization, elementwise kernels accounted for 91.5% of total traffic and 66.4% of total kernel time, and these operators were bandwidth-bound. After the optimization, their HBM read+write falls from 979.47 to 15.60 GiB and their time from 704.96 to 16.64 ms. Matrix multiplication now accounts for 75.7% of total kernel time, and the bottleneck shifts from bandwidth to compute, which is why bandwidth falls by a larger factor than time. Reducing matrix-multiplication cost further, for example with mixed precision, could improve efficiency further.
8. SO(2) dataflow design and the contribution of the node-side optimization. The difficulty in optimizing SO(2) convolution is that the dense matrix products and the surrounding operations use different coordinate frames and feature layouts. The matrix products organize their inputs by local angular order. The activation must obtain both the positive and negative angular-order components of the product output. Node aggregation first has to rotate messages from different edges back into a common global frame. TrimSO(2) uses these dependencies: it generates the angular-order layout the matrix products require while gathering, reads and recombines the paired components in the activation, and rotates back and accumulates edge messages in the reduction. This preserves the efficient dense matrix products and avoids separate writes and re-reads of entire edge tensors for these transformations. We extend the same dataflow design to backward and double backward, accelerating the full training computation.
The node-side equivariant feedforward optimization does not depend on the angular-order structure specific to SO(2), but also follows the same principle. Channel mixing that can be done per tile stays on chip, avoiding a write-back of intermediates. The three edge-side optimizations alone give 1.82× end-to-end speedup, and adding the node-side optimization brings it to 2.83× (A100, 686.16 → 441.90 ms/step). The three edge-side optimizations apply to all three models, and the node-side optimization provides further improvements for models where the feedforward network is an appreciable cost.
We again thank Reviewer NZQP for the thorough comments. Your comments have helped us improve the clarity of the paper. Please let us know if any part of our response remains unclear, and we would be glad to explain further.
The authors have provided additional experiments that addressed my concerns. The overall performance and acceleration remain consistent in the new experiments. I wil therefore keep my score.
Thank you
We are glad that our response addressed your concerns! Thank you again for your time and constructive feedback.
We thank Reviewer NZQP for the detailed and professional feedback. The revision adds each of the measurements you listed. Our responses to your questions are provided below.
1. Comparison with FAIRChem UMA inference kernels. We compared TrimSO(2) against two optimized inference settings from FAIRChem,
umas_fast_gpuandNitro. For the 1,000-atom copper system, the median end-to-end latencies are as follows.umas_fast_gpu(ms)In the H100 size sweep,
Nitrois fastest for 10–500 atoms, and TrimSO(2) is 1.30× and 1.28× faster thanNitroat 1,000 and 5,000 atoms. The full settings and results are in Appendix C.7, Table 23 and Figure 14.2. UMA ablation timing on H100. Thank you for pointing this out. The original three runs varied considerably across these stages due to server fluctuations: the run-to-run standard deviation of Edge → Node is about 19% of its mean, and the difference between 238.37 ms and 244.86 ms falls within measurement variation. We re-measured both stages three times each on the same H100 with the same workload (OMat24 batches, at most 700 atoms per batch). Edge → Node averaged 303.66 ms/step and full TrimSO(2) averaged 286.87 ms/step, so enabling the node-side optimization reduces time by 5.53%, and the reversal did not recur. The revised Table 8 uses the re-measured values. We re-checked the remaining results and found no comparable variation.
3. UMA training performance at different precisions. We compared UMA-S-1.2 training with FP32, TF32, and BF16 autocast on the same A100. Both implementations use the same random initial state and OMat24 batch sequence, with at most 700 atoms per batch.
With TF32 and BF16 autocast, official PyTorch is 1.14× and 1.61× faster than its own FP32 setting. At matched precision, TrimSO(2) is 2.93×, 3.80×, and 1.88× faster than official PyTorch in FP32, TF32, and BF16 respectively. Even in FP32, TrimSO(2) is 1.82× faster than the fastest of the three official settings, BF16 autocast.
4. eSEN validation error after 40,000 training steps on OMat24. We trained eSEN 30M for 40,000 steps with PyTorch and with TrimSO(2), and evaluated the resulting checkpoints on the OMat24 validation set (Appendix C.6.4, Table 20).
The per-atom energy, force, and stress MAE are close between the two implementations, with relative differences of 1.9%, 0.4%, and 1.5% respectively.