- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents a technique for accelerating the inference of equivariant machine learning interatomic potentials (MLIPs). The proposed method is inspired by, though not identical to, early exit techniques in Transformer models. Specifically, a router module placed on the model features before each tensor product layer determines whether to continue the computation for each edge feature, thereby reducing the overall number of operations. To train this router, the authors introduce a knowledge distillation scheme that uses the force discrepancy between the full model (teacher) and the routed model (student) as supervision.
Experiments are conducted on the Allegro architecture, using both a pretrained Allegro foundation model fine-tuned on the target dataset and a model trained from scratch on the target dataset. As a result, the method achieves approximately twofold inference speedup with a modest decrease in energy and force prediction accuracy. The authors additionally present an interpretability analysis, which visually shows which parts of a given material system exit the tensor product computation early and which do not.
Critical strengths
- The idea of letting the model decide for itself whether to continue the computation is interesting, and the authors effectively bring the concept of early exit into equivariant MLIPs.
- The authors comprehensively evaluate the method beyond energy and force prediction accuracy, using physically meaningful metrics such as NVE energy conservation and bulk modulus.
Critical weaknesses
- The evaluation is conducted only on two specialized systems, CO/Pt and CDP, rather than on standard benchmarks, which makes it difficult to intuitively compare the proposed method with existing approaches.
- Although the paper is generally understandable, there is considerable room for improvement in the writing and figures, as detailed in the next section.
Questions for authors
I have the following questions and suggestions, which I believe could strengthen the work:
- The proposed method is implemented on Allegro, an edge-centric and strictly local architecture. How would the method behave on atom-centric models such as NequIP?
- The evaluation is limited to two specialized systems. I suggest additional experiments on standard benchmarks, which could be designed in two settings: (i) evaluating on the data used to pretrain Allegro-OAM-L, and (ii) fine-tuning Allegro-OAM-L on well-known benchmark datasets not included in its pretraining data. In addition, while Figure 5 (a-b) presents routing statistics, these are limited to the specific systems. Could the authors report such statistics on general benchmarks covering diverse materials, as suggested above?
- It would be informative to include a baseline that distills the full model into a smaller Allegro model with the same number of layers but fewer channels, without routing. Such a comparison would clarify whether adaptive routing offers a better accuracy-efficiency trade-off than conventional distillation into a smaller model.
- Equivariant MLIPs typically do not stack many layers, which seems to leave limited room for adaptive depth allocation. In this context, the approximately twofold speedup, while meaningful, appears somewhat marginal. Could the authors discuss the practical significance of adaptive depth for equivariant MLIPs?
- The interpretability analysis is interesting, but its takeaway is unclear. It is not evident whether the routing patterns confirm that the model behaves in a physically reasonable manner, or whether they suggest directions to further improve accuracy or speed. Could the authors elaborate on what insights researchers can draw from this analysis?
In addition, I have the following suggestions regarding presentation:
- The manuscript is readable overall, but the writing would benefit from substantial polishing.
- Figure 1 and Figure 2 (a) do not sufficiently convey how the proposed method works. In addition, the two figures appear redundant and could be merged into a single, clearer overview figure.
- The inverted y-axis in Figure 3 is confusing, even with the explanation in the caption.
- The font sizes in Figures 3 and 4 should be increased.
- The colorbar in Figure 3 does not indicate the values at both ends of its range.
- The 3D plots in Figure 5 (d) hinder readability; 2D plots would be more effective.
- Line widths, marker sizes, or axis ranges in Figures 7-10 of the Appendix could be adjusted for better readability.
Overall, the idea of adaptive equivariant computation is interesting. However, the contribution remains marginal in its current form, and the presentation requires substantial improvement. I therefore recommend a weak rejection at this stage.
Post-rebuttal update: The additional experiments and revisions in the rebuttal have addressed most of my concerns, and I have updated my score accordingly.