- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces FlashSO(2), which accelerates the SO(2) convolution introduced by Passaro & Zitnick (eSCN).
The authors provide three fused kernels for rotation and gated activation, and apply the Ozaki scheme for precision emulation to the two SO(2) convolution blocks.
Their results show that the proposed implementation achieves a rotation equivariance error similar to that of the original FP32 implementation and accelerates MALOQ training and UMA-M inference by 3.9× and 3.3×, respectively.
Critical strengths
Clear results. The throughput gain is significant, and the paper tackles an important problem with current SO(2) convolution layers.
Critical weaknesses
-
While the paper measures equivariance error, training loss, and Hamiltonian MAE, important metrics are missing. For example, material properties obtained from the Hamiltonian matrix for MALOQ, as well as energy, force, and stress errors from the MLIP model, are not reported.
-
Limited accuracy comparison between FlashSO(2) and the original FP32 implementation using FP32-trained models. This is important because we can use pretrained checkpoints for faster inference while preserving accuracy.
-
No training loss results for UMA-M. Conservative force-field training requires double backward, and although training throughput is reported, training loss and accuracy comparisons with FP32 are missing.
Questions for authors
- Can the authors provide metrics for energy, forces, and/or other material properties with FlashSO(2), in addition to training loss and rotation equivariance error?
- Can the authors separately report the performance contribution of each of the three fused kernels (S1, S2, and S3)?
- Can the authors provide error metrics for UMA-M and MALOQ using existing FP32-trained checkpoints with FlashSO(2), to verify whether using this method leads to any accuracy degradation?
- Can the authors provide training loss curves and accuracy metrics for conservative MLIP models trained with FlashSO(2) and compare them with FP32 models?
4 comments
3. Can the authors provide error metrics for UMA-M and MALOQ using existing FP32-trained checkpoints with FlashSO(2), to verify whether using this method leads to any accuracy degradation?
Yes. FlashSO(2) keeps the weights in FP32, so FP32-trained checkpoints load directly. Running the FP32-trained weights of the UMA-S architecture from point 1 with Emu2 does not change any held-out MAE by more than 0.02%. For the released UMA-M 1.1 checkpoint, Emu2 changes the energy and force MAEs against DFT on 20,000 frames of the OMat24 validation set by less than 0.01% (7.59 meV/atom and 46.4 meV/Å with both FP32 and Emu2).
For MALOQ, Emu2 inference of the FP32-trained weights changes the MAE of the Hamiltonian entries by 0.5% (median over the 32 conformers of Table R2). The revised paper will include these errors for MALOQ and UMA-M 1.1.
4. Can the authors provide training loss curves and accuracy metrics for conservative MLIP models trained with FlashSO(2) and compare them with FP32 models?
Training UMA requires on the order of a hundred H200 GPUs for weeks (28 days for UMA-M and 10 days for UMA-S) [1], so we trained the UMA-S architecture of point 1 on the much smaller MPtrj dataset instead. Forces and stresses come from autograd, so each of the 319,084 training steps runs double backward through the SO(2) layer. Emu2 matches FP32 within the statistical noise (Table R1), and its training loss tracks the FP32 curve over the whole run, ending only 0.3% above it. Table R4 gives the training loss of both runs at different steps along the run.
Table R4. Training loss of the UMA-S architecture with FP32 and with Emu2 (same recipe, seed, initial weights and batches), as the median over the 20 epochs before each step.
| training step | FP32 | Emu2 | Emu2 vs FP32 |
|---|---|---|---|
| 80,000 | 1.2160 | 1.2178 | +0.15% |
| 160,000 | 1.0045 | 1.0057 | +0.12% |
| 240,000 | 0.7807 | 0.7823 | +0.20% |
| 319,084 (end) | 0.6567 | 0.6586 | +0.28% |
We hope that these additional experiments address your concerns, and we would be glad to hear whether anything remains unclear.
References
[1] B. M. Wood et al., UMA: A family of universal models for atoms, NeurIPS (2025).
Thank you for the detailed response and additional experiments. The new results address my main concerns regarding accuracy preservation, the performance breakdown of the fused kernels, and conservative MLIP training.
I appreciate the authors' efforts during the rebuttal period. I am satisfied with the responses and will raise my score.
We are glad that our response addressed your concerns. And thank you for the careful review and the constructive suggestions, and for raising your score.
Thank you for the careful review of our paper. We have run new experiments to address all three weaknesses (see point 1 for the first, point 3 for the second, and point 4 for the third). All your questions are answered in order below.
Notation. FP32-REF is the original PyTorch implementation of the SO(2) layer in FP32. FP32-FL denotes our fused kernels with the convolution GEMMs in FP32, i.e., without emulation. Emu2 is the emulated setting of the paper with two INT8 slices, in which our fused kernels run the GEMMs with integer tensor cores. Everything outside the SO(2) layer remains in FP32 in all our settings. All timings are on one GH200.
1. Can the authors provide metrics for energy, forces, and/or other material properties with FlashSO(2), in addition to training loss and rotation equivariance error?
For UMA, we trained the UMA-S architecture (without MoLE) from scratch on MPtrj in FP32 and with Emu2 under identical settings (the same recipe, seed, initial weights and batches). Against DFT on 80,105 held-out frames, Emu2 exhibits an MAE change of +0.5%, +0.1% and −0.3% for the energy, force and stress with respect to FP32, respectively. All these values remain within the statistical noise (Table R1). Relaxations of 2,000 materials do not separate the two models either. For MALOQ, we derived orbital energies from the predicted Hamiltonians. With Emu2, the median errors of the HOMO, LUMO and gap against DFT stay within 0.3 meV of FP32 (Table R2). The revised paper will include these orbital energies as well.
Table R1. Energy, force and stress errors against DFT on 80,105 frames held out from training, with the final weights of each run evaluated in FP32. Brackets give 95% bootstrap intervals over materials.
Table R2. Median absolute errors of orbital energies against DFT over the 32 held-out conformers, in meV, with the weights trained with Emu2.
2. Can the authors separately report the performance contribution of each of the three fused kernels (S1, S2, and S3)?
Table R3 gives the time of each operation in one SO(2) layer of UMA-M, including the three fused kernels. Fusion cuts S1 from 18.8 to 5.6 ms, S2 from 4.4 to 1.2 ms and S3 from 6.2 to 0.9 ms. With Emu2, S1, S2 and S3 drop further to 1.7, 0.6, and 0.5 ms, because they pass INT8 slices instead of FP32 data. The largest gain comes from the convolution GEMMs between the kernels, which emulation runs 6.8× faster (45.0 → 6.6 ms). The revised paper will include the more detailed breakdown of Table R3.
Table R3. GPU time in ms of each operation in one SO(2) layer of UMA-M with 2,304 atoms and 179,712 edges. The percentages correspond to the shares of the summed time of the five operations.