acceptodds
Under review as a conference paper at ICLR 2027

Scaling Distributed Machine Learning Interatomic Potential Inference with DistMLIP2

Abstract

Atomistic simulation has been the primary workhorse in computational materials and drug discovery over the past 60 years. However, the quantum mechanical calculations pivotal for the effective simulation of atomistic systems, such as density functional theory (DFT), remain too slow for practical use in large scale systems. In recent years, machine learning interatomic potentials (MLIPs), machine learning models trained on DFT energy and force labels, are capable of reaching DFT-levels of accuracies while performing orders of magnitude faster inference than traditional DFT workflows. Distributing MLIP inference across many GPUs poses as a powerful method for scaling simulation sizes and time scales even further. In this work, we present DistMLIP2, where we introduce four elements enabling a new state of the art in distributed MLIP inference: runtime skin recomputation, partial graph reconstruction, hierarchical partitioning and partition cycling, and graph computation pruning. We implement four widely used MLIPs into DistMLIP2: UMA, MACE, Orbv3, and NequIP, and show DistMLIP2 pareto-dominates the previous state of the art distributed inference methods such as ALCHEMI, LAMMPS-KOKKOS, and DistMLIP when controlling for MLIP size, architecture, kernels, and inference settings. Most notably, DistMLIP2 achieves 2.4x faster inference time than ALCHEMI while simulating 2.16x more atoms compared to LAMMPS. With partition cycling enabled, we show that DistMLIP2 can simulate systems containing 4-5x the atom count of its previous limit.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
machine learning interatomic potential, molecular dynamics simulation, atomistic simulation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_byrWOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces DistMLIP2, an inference platform that allows running several popular machine learning interatomic potentials, or machine learning force fields, across many GPUs, with new improvements to improve speed and the scale of systems that can be run. They compare against other distributed inference methods on several simulation systems.

Critical strengths

This paper introduces an update to DistMLIP that includes multiple additional optimizations that show improvements in inference efficiency and capacity over existing implementations. This release also integrates multiple different MLIPs into one distributed inference platform, with MACE, NequIP, UMA, and Orbv3, which have not previously been integrated into one framework.

Critical weaknesses

This paper doesn't include error bounds, estimates, or uncertainties for the timing and capacity information that's been reported. Given that each of these factors can vary significantly system to system, and vary even for multiple runs of the same system, it makes it difficult to assess with confidence the significance of various performance improvements.

Second, the described approach for partial graph reconstruction seems a bit unclear in its current description, and potentially incorrect. When reference coordinates are reset for dirty atoms but the edges are kept for clean atoms, how does this interact with the fact that references might be updated asynchronously? Couldn't this result in interactions between atoms that move substantially being missed, at least in the description written right now?

Finally, while this paper makes strong claims about performance improvements and capacity improvements, Table 2 is the only table that directly compares to other distributed inference approaches, and here the exact setup of each simulation and each condition for each result is unclear. How many atoms was each system run under? Are all methods timed under identical conditions? What's the precise set of timings and their boundaries being measured -- wall clock or something else? How well are the options available for each method tuned? These questions make it difficult to actually compare the cost-performance trade-off across distributed settings for different systems, a central claim of the paper. This is especially important given that unlike what's claimed in the text, Figure 3B does actually seem to show that time cost increases substantially for MACE, and visibly for NequIP based on partition cycles per node. For full assessment, these details will be helpful.

Questions for authors

How exactly is timing being measured and defined? How many different times were these systems run, and how condition dependent are the results? How were baseline settings optimized? Can matched baseline results (or at least a baseline result) be provided to the strong and weak scaling curves? How are these results expected to change on different hardware with different memory bandwidth and compute core differences?

3 comments
AuthorsOpenReview ↗

Author Response 1

We would like to thank the reviewer for the review! We’re pleased to see that you enjoyed our optimizations showing “improvements in inference efficiency and capacity over existing implementations”. Here are our point-by-point responses to your questions.

This paper doesn't include error bounds, estimates, or uncertainties for the timing and capacity information that's been reported. Given that each of these factors can vary significantly system to system, and vary even for multiple runs of the same system, it makes it difficult to assess with confidence the significance of various performance improvements.

In the following table, we show the 25th percentile and 75th percentile bounds for our major timing results:

WorkloadMethod4 GPUs8 GPUs16 GPUs32 GPUs
H₂O iceAlchemi + MACE0.465–0.4670.435–0.4360.499–0.5010.740–0.766
H₂O iceDistMLIP + MACE1.371–1.5001.894–2.016——
H₂O iceDistMLIP2 + MACE0.385–0.3870.330–0.3320.344–0.3470.386–0.393
H₂O iceDistMLIP2 + NequIP0.352–0.3540.359–0.3630.388–0.3930.471–0.475
H₂O iceDistMLIP2 + ORB1.333–1.3411.312–1.3141.312–1.3151.334–1.337
H₂O iceDistMLIP2 + UMA0.241–0.2420.298–0.3000.302–0.3030.247–0.249
H₂O iceFAIR-Chem + UMA0.278–0.2790.275–0.2760.284–0.2850.285–0.286
H₂O iceLAMMPS + MACE0.283–0.2840.307–0.3120.315–0.3340.349–0.395
H₂O iceLAMMPS + NequIP0.541–0.5500.780–0.7920.881–0.8901.061–1.069
Li₃PO₄Alchemi + MACE0.456–0.4570.442–0.4460.516–0.5190.735–0.776
Li₃PO₄DistMLIP + MACE1.383–1.5211.846–1.967——
Li₃PO₄DistMLIP2 + MACE0.388–0.3900.388–0.3890.406–0.4090.460–0.464
Li₃PO₄DistMLIP2 + NequIP0.350–0.3520.352–0.3550.392–0.3970.475–0.481
Li₃PO₄DistMLIP2 + ORB1.318–1.3201.339–1.3441.308–1.3101.316–1.318
Li₃PO₄DistMLIP2 + UMA0.238–0.2390.298–0.2990.306–0.3080.317–0.319
Li₃PO₄FAIR-Chem + UMA0.261–0.2620.278–0.2790.283–0.2870.286–0.288
Li₃PO₄LAMMPS + MACE0.285–0.2880.301–0.3160.318–0.3410.360–0.414
Li₃PO₄LAMMPS + NequIP0.538–0.5440.734–0.7440.872–0.8821.036–1.046
GaNAlchemi + MACE0.445–0.4460.481–0.4840.688–0.7230.811–0.841
GaNDistMLIP + MACE1.334–1.4831.979–2.067——
GaNDistMLIP2 + MACE0.360–0.3610.360–0.3640.362–0.3640.439–0.442
GaNDistMLIP2 + NequIP0.415–0.4180.369–0.3720.405–0.4120.491–0.495
GaNDistMLIP2 + ORB1.334–1.3381.315–1.3191.310–1.3131.330–1.337
GaNDistMLIP2 + UMA0.246–0.2460.282–0.2830.288–0.2900.318–0.320
GaNFAIR-Chem + UMA0.278–0.2790.263–0.2640.291–0.2940.310–0.313
GaNLAMMPS + MACE0.294–0.2960.316–0.3290.347–0.3730.437–0.453

The table shows that the timing metrics are almost always quite tight. In fact, DistMLIP2 has a more consistent timing spread of 0.88% across runs while the average of all other methods is 3.36%.

We will include this table in the manuscript.

For the capacity results, we slowly increased the number of atoms in the supercell by a fixed interval until the GPU went out of memory. As a result, there is no notion of uncertainty or error in this metric. However, there does exist variance in the overall capacities due to node densities and edge counts that arise from simulating different systems. This is why we benchmarked 5 different real-world systems (H2O ice, Li3PO4, GaN, a metal organic framework, and a capsid protein) to show DistMLIP2 pareto dominates previous methods across all systems and models.

AuthorsOpenReview ↗

Author Response 2

Second, the described approach for partial graph reconstruction seems a bit unclear in its current description, and potentially incorrect. When reference coordinates are reset for dirty atoms but the edges are kept for clean atoms, how does this interact with the fact that references might be updated asynchronously? Couldn't this result in interactions between atoms that move substantially being missed, at least in the description written right now?

Interactions between atoms that move substantially are not missed. This is because edges are only kept for clean atoms if there are no adjacent dirty atoms. Every dirty atom causes a double-sided edge update to and from all neighboring atoms, including clean atoms. This holds true even when the clean and dirty atoms are in different partitions. Therefore, our edges are always correctly maintained despite asynchronous updates of reference positions. We outline this in section 3.2, but we will clarify this further in the final draft.

In order to empirically show correctness, we performed additional experiments looking at energy discrepancies for partial graph reconstructions when compared to single GPU, full graph baselines. We find that the differences are indeed identical and within floating-point precision.

SystemGPUsMax Energy discrepancy (eV/atom)Max force discrepancy (eV/Å)
Li₃PO₄22.1300e-085.7220e-06
Li₃PO₄42.1175e-085.9605e-06
Li₃PO₄81.2397e-085.9605e-06
Li₃PO₄161.2952e-086.1989e-06

The above table shows the max energy and force errors for MACE + DistMLIP2 of the partial graph reconstruction compared to a single GPU, full graph reconstruction baseline. We ran each system three times, each for 10k steps at 0.5 fs timesteps. Temperature is 1000 K. Energy discrepancies consistently hover around 10e-9 eV/atom while force discrepancies hover around 10e-6 eV/angstrom.

Finally, while this paper makes strong claims about performance improvements and capacity improvements, Table 2 is the only table that directly compares to other distributed inference approaches, and here the exact setup of each simulation and each condition for each result is unclear. How many atoms was each system run under?

The exact atomic counts can be found in Table 2 under the columns denoted “C”. The C column represents the capacity of the simulation in millions of atoms as mentioned in the caption of Table 2.

Are all methods timed under identical conditions? What's the precise set of timings and their boundaries being measured -- wall clock or something else?

In Section 4.2, we discuss the timing conditions – we benchmark 100 steps of MD after 10 GPU warmup steps. These are complete end-to-end simulation measurements and are the most realistic timing setting for real world simulation use cases.

How well are the options available for each method tuned? These questions make it difficult to actually compare the cost-performance trade-off across distributed settings for different systems, a central claim of the paper.

The inference options for each of the baselines are found in Table 6 in the appendix. We used the fastest available inference settings for each of the models (e.g. flash-UMA’s Triton kernels, cuEquivariance, torch compile, OpenEquivariance) for the most realistic and fair head to head comparisons. Our results solidly emphasize DistMLIP2’s pareto dominance compared to the strongest possible baselines.

This is especially important given that unlike what's claimed in the text, Figure 3B does actually seem to show that time cost increases substantially for MACE, and visibly for NequIP based on partition cycles per node. For full assessment, these details will be helpful.

As mentioned in Section 4.2 and the Figure 3 caption, the Figure 3 results are for our partition cycling experiments – these are fundamentally a different feature proposed and implemented into DistMLIP2 that is used to increase the overall simulation capacity given a fixed GPU memory budget. Naturally, the increase in overall capacity implies more computation required to calculate energy and forces. Hence the increase in simulation time. We want to emphasize, however, that the partition cycling results in Figure 3b are showcasing a completely different feature and are not to be compared to the results in Table 2, which highlight the primary distributed inference capabilities of DistMLIP2.

If you have any further questions on this, please do let us know so we can get them sorted out. Partition cycling is a novel feature that is enabled by our hierarchical partitioning strategy only to be used within constrained GPU memory settings. For standard distributed inference settings and comparisons, we would like to point you to Table 2.

AuthorsOpenReview ↗

Author Response 3

How exactly is timing being measured and defined? How many different times were these systems run, and how condition dependent are the results? How were baseline settings optimized? Can matched baseline results (or at least a baseline result) be provided to the strong and weak scaling curves? How are these results expected to change on different hardware with different memory bandwidth and compute core differences?

As mentioned in Section 4.2, we perform 10 warmup MD steps for the GPU before timing 100 MD steps. This is the standard protocol followed in previous works [1-3]. For this work, we only had access to H100 GPUs; however, we expect scaling curves to perform similarly for different hardware and chipsets up to a first order approximation. Because we matched compute, atomic system, and used the fastest inference settings across all experiments and baselines, we also expect DistMLIP2 to continue being the state of the art system in capacity and runtime even when run on different hardware.

For a comparison against a baseline method, we run weak and strong scaling against ALCHEMI using the MACE model. For reference, we also include the DistMLIP2 + MACE results:

GPUsWeak: ALCHEMIWeak: DistMLIP2Strong: ALCHEMIStrong: DistMLIP2
175.06075.628419.677†411.750
293.01778.392254.868213.341
4105.13380.491150.027112.051
8125.89480.46698.63461.900
16175.66785.93782.78041.186
32211.15494.55072.73839.696

DistMLIP2’s strong scaling and weak scaling is 1.8x and 2.2x more efficient, respectively, than ALCHEMI’s.

[1] DistMLIP: A Distributed Inference Platform for Machine Learning Interatomic Potentials, ICLR 2026, ​​https://arxiv.org/abs/2506.02023

[2] Scalable Parallel Algorithm for Graph Neural Network Interatomic Potentials in Molecular Dynamics Simulations, JCTC 2024, https://pubs.acs.org/jctcce/article-abstract/20/11/4857/169287/Scalable-Parallel-Algorithm-for-Graph-Neural

[3] Scalable foundation interatomic potentials via message-passing pruning and graph partitioning, npj Computational Materials, https://www.nature.com/articles/s41524-026-02001-4

Once again, we would like to thank you for the review and hope we were able to alleviate each of your concerns! We hope we were able to establish DistMLIP2 as the de facto state of the art distributed MLIP inference platform. If you have any further questions, please let us know!

Reviewer_kryeOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

In the manuscript "Scaling Distributed Machine Learning Inter-atomic Potential Inference With DISTMLIP2", the authors introduce another iteration or implementation of the original DistMLIP framework. The manuscript outlines several performant features, which will of significant interest to the community. For example, hierarchical partitioning, and partial graph reconstruction should be essential for any performant GPU GNN implementation. Scaling behaviour of their implementation is excellent and and the flatlining between 16-32 GPU is not unseen before, though I am bit unconvinced that it is just because of the reasoning provided. The added support for Orbv3 over DistMLIP is also noted.

Critical strengths

Technical aspects of the paper are great, the presented performance is probably on the Pareto front for the category.Most of their ideas and inferences are correct and validated prior in different papers or software packages, yet the authors have managed to weave a coherent story.

Critical weaknesses

However I am still not convinced that this reiteration of DistMLIP crosses the threshold of innovation warranting a new paper. The suggested improvements have also been realized in the community in different forms, or functionalities. Therefore I dont think that this paper crosses the threshold for a new paper.

Questions for authors

Questions:

  1. How are the scaling laws if you run it in pure CPU mode? Is it possible?
  2. How easy/difficult it is to port a new GNN to DistMLIP2? Any community example available?
2 comments
AuthorsOpenReview ↗

Author Response 1

We would like to thank the reviewer for the review! We are pleased to hear the reviewer recognizes the significant interest to the community that our work will bring! However, it seems as though there’s been a fundamental misunderstanding of our work that we would like to address.

However I am still not convinced that this reiteration of DistMLIP crosses the threshold of innovation warranting a new paper.

We emphatically disagree with your claim that our work does not cross the threshold for a new paper. Our contributions revolve around MLIP inference systems advancements and have similar scope and impact to previously published works such as [1], [2] and [3]. [1] introduces graph-level partitioning for MLIPs. [2] introduces a specialized kernel for equivariant message passing operations in MLIPs. [3] introduces message passing pruning and implements a graph-level partitioning scheme. In this work, we introduce multiple novel advancements, including specialized research-grade kernels and new algorithms, towards accelerating distributed inference.

We agree that this is not a standard MLIP training paper. However, we emphasize the importance of research-grade systems-level contributions to the field of molecular simulation. We have shown repeatedly that there still remains huge upside for accelerating MLIPs through optimizing and innovating upon the platforms in which they run inference. DistMLIP2 represents that perspective and makes a leap forward with the introduction of non-obvious, non-trivial, and novel algorithms and efficient implementations of those algorithms.

We do agree with you that the platform will have significant impacts on how scientists perform molecular simulation however.

[1] DistMLIP: A Distributed Inference Platform for Machine Learning Interatomic Potentials, ICLR 2026, ​​https://arxiv.org/abs/2506.02023

[2] FlashTP: Fused, Sparsity-Aware Tensor Product for Machine Learning Interatomic Potentials, ICML 2025, https://openreview.net/pdf?id=wiQe95BPaB

[3] Scalable foundation interatomic potentials via message-passing pruning and graph partitioning, npj Computational Materials, https://www.nature.com/articles/s41524-026-02001-4

The suggested improvements have also been realized in the community in different forms, or functionalities. Therefore I dont think that this paper crosses the threshold for a new paper.

If you provide previous works that you feel overlap with DistMLIP2, we can discuss the specific novelty of our method. Upon a very deep literature search after reading the review, the most similar prior work to any of the methods introduced in DistMLIP2 is the partial graph reconstruction algorithm in Yao et al. (2004) [1], but there are fundamental differences between the algorithms. First, the granularity of updates is different, with Yao et al. doing so at the cell level while we do so on the atomic level. This finer granularity minimizes the update size for each timestep. In addition, our neighbor lists are distributed across devices, with each GPU only getting a fraction of the list, while the algorithm proposed by Yao et al. constructs a shared list over devices. For partial updates, this incurs additional complexity around ownership and scope of edge updates that we outline in the paper. The complexity is compounded once integrated with the hierarchical, n-hop partitioning scheme in DistMLIP2 as well.

We will include these differences in the manuscript.

Ultimately, it would be great if you could provide examples of prior works that undermine the novelty of our current work. We can then review the evidence and then respond. It will also allow us to be more clear with how our work differentiates itself from prior distributed inference methods in the manuscript.

[1] Improved neighbor list algorithm in molecular simulations using cell decomposition and data sorting method, Computer Physics Communications, 2004. https://www.sciencedirect.com/science/article/abs/pii/S0010465504002097?via%3Dihub#SEC006

AuthorsOpenReview ↗

Author Response 2

How are the scaling laws if you run it in pure CPU mode? Is it possible?

Our parallel algorithms are implemented in the form of highly optimized, memory-efficient CUDA kernels. As a result, DistMLIP2’s primary implementation is GPU-only; however, it is theoretically possible to implement the algorithms on CPU. Due to the importance of GPU computation, we consider a CPU re-implementation of the DistMLIP2 platform to be out of scope.

How easy/difficult it is to port a new GNN to DistMLIP2? Any community example available?

DistMLIP2 is a highly modular codebase designed to be easily extended, repackaged, or even integrated into other systems. As a result, porting a new GNN to DistMLIP2 is quite straightforward. We estimate that it would take only 1-3 hours to set up and evaluate a new MLIP into DistMLIP2.

We plan on open sourcing the entire DistMLIP2 platform as well as the 4 implemented MLIPs as example implementations for the community. We are also committed to continuously maintaining the library, introducing new MLIPs, and addressing community questions, for the future and open source science community.

We would like to once again thank the reviewer for the review! We are pleased to see we are aligned on the impact and significance of our work and hope we were able to clear up any misunderstandings regarding its novelty and paper-worthiness. Please let us know if you have any other concerns!

Reviewer_Y4X5OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposed four elements for distributed MLIP inference: runtime skin recomputation, partial graph reconstruction, hierarchical partitioning and partition cycling, and graph computation pruning. The paper implemented their techniques into UMA, MACE, Orbv3, and NequIP, and demonstrating the improved performance.

Critical strengths
  1. The four techniques for distributed MLIP inference are intuitive and reasonable.
  2. The high-level motivation (speed up distributed MLIP) is good.
  3. The overall experiments are showing the improved performance of the proposed idea.
Critical weaknesses
  1. I highly appreciate the technical advances made by this paper for fast MLIP inference; however, I am inclined to think that this article is a technical report rather than a research paper.
  2. I noticed that Table 5 calls the 1.6M-parameter NequIP “untrained.” This seems inconsistent with the statements in the main results.
  3. In my understanding, the introduced partial graph reconstruction and graph pruning may change the graph topology and may induce the performance variation in prediction. But the main results did not report performance. Can the authors quantify numerical deviations across full and partial graph rebuilds and partitioning configurations, and clarify whether the reported differences are limited to floating-point precision?
Questions for authors
  1. What is the paper’s main research contribution beyond implementing and combining existing techniques, and how does it advance prior distributed MLIP inference systems?

  2. Was the NequIP model in Table 5 trained? If so, on what data and with what procedure?

  3. Can you report energy and force deviations from a reference implementation across full and partial graph rebuilds and partitioning configurations? Are any differences larger than floating-point error?

  4. Why did the authors not compare with other native distributed MLIPs, such as Allegro and SevenNet?

7 comments
AuthorsOpenReview ↗

Author Response 1

We would like to thank the reviewer for the review. We are glad you enjoyed the intuitive techniques, high level motivation, and strong results. Below are our responses to each of the weaknesses and questions raised. However, there appear to be some fundamental misunderstandings we would like to clear up.

I highly appreciate the technical advances made by this paper for fast MLIP inference; however, I am inclined to think that this article is a technical report rather than a research paper.

We vehemently disagree with your inclination that our work is a technical report rather than a research paper. Our contributions are novel ML systems level contributions similar in scope, style, and impact as published research papers such as [1], [2], and [3]. [1] introduces graph-level partitioning for MLIPs. [2] introduces a specialized kernel for equivariant message passing operations in MLIPs. [3] introduces message passing pruning and implements a graph-level partitioning scheme. In this work, we introduce multiple novel advancements, including specialized research-grade kernels and new algorithms, towards accelerating distributed inference.

We believe that our algorithms and kernels outlined in Section 3 are highly non-obvious and non-trivial improvements over current MLIP inference systems and kernels. We also achieve very strong results that have the potential to greatly accelerate a large amount of scientific simulation workloads.

[1] DistMLIP: A Distributed Inference Platform for Machine Learning Interatomic Potentials, ICLR 2026, ​​https://arxiv.org/abs/2506.02023

[2] FlashTP: Fused, Sparsity-Aware Tensor Product for Machine Learning Interatomic Potentials, ICML 2025, https://openreview.net/pdf?id=wiQe95BPaB

[3] Scalable foundation interatomic potentials via message-passing pruning and graph partitioning, npj Computational Materials, https://www.nature.com/articles/s41524-026-02001-4

I noticed that Table 5 calls the 1.6M-parameter NequIP “untrained.” This seems inconsistent with the statements in the main results.

The “untrained” label in Table 5 is a typo from an earlier version of the manuscript. We will promptly remove it. Thank you for spotting this!

The Nequip model we used was trained on 1.58 million structures from the MPTrj dataset [1] to be compliant with the MatBench Discovery [2] benchmark.

[1] https://docs.materialsproject.org/services/ml-and-ai-applications/mptrj [2] https://www.nature.com/articles/s42256-025-01055-1

AuthorsOpenReview ↗

Author Response 2

In my understanding, the introduced partial graph reconstruction and graph pruning may change the graph topology and may induce the performance variation in prediction. But the main results did not report performance. Can the authors quantify numerical deviations across full and partial graph rebuilds and partitioning configurations, and clarify whether the reported differences are limited to floating-point precision?

The graph topology within the cutoff radius is identical between partial graph reconstruction and full graph reconstruction modes. It is only the topology of the atoms outside of the standard cutoff radius but within the skin radius that may differ. However, this discrepancy is inconsequential as those atoms are not actually used in the energy and force calculation.

A theoretical analysis of the adjacency list conditions confirm this: suppose we have two atoms with positions x,yx, y where ∣∣x−y∣∣<r||x - y|| < r. Then, since reference positions are updated when an atom moves more than rskin2\frac{r_\text{skin}}{2} from their reference positions, ∣∣xref−yref∣∣<r+rskin||x^\text{ref} - y^\text{ref}|| < r + r_\text{skin}, so there will exist an edge between the atoms. Note that the time at which reference positions are updated is not a factor, as the deviation bound always holds, so the graph is always correct even when only a subset of positions are updated at every timestep. We use both-sided updates to ensure that this remains true when only one atom is marked as dirty.

Although extraneous edges do exist in this setup, they are pruned using the strict cutoff rr before being passed to the MLIP. This does not affect the correctness of the energy and force outputs as MLIPs use a decaying cutoff that sets the weight of all edges with length longer than rr to 0.

We perform further experiments on energy and force deviations for DistMLIP2 full and partial graph reconstruction compared to the single GPU baselines. We find that the energy and force predictions are indeed identical and within floating-point precision.

SystemGPUsReconstructionMax Energy discrepancy (eV/atom)Max force discrepancy (eV/Å)
Li₃PO₄4Full2.1845e-085.7220e-06
Li₃PO₄4Partial2.1175e-085.9605e-06
Li₃PO₄8Full1.4086e-086.1393e-06
Li₃PO₄8Partial1.2397e-085.9605e-06
Li₃PO₄16Full1.0936e-085.4836e-06
Li₃PO₄16Partial1.2952e-086.1989e-06
Table 4: Max error for MACE in using partial graph reconstruction and full graph construction for all integration steps compared to single GPU baseline. We ran each system (~3k atoms per system) three times, each for 10k steps at 0.5 fs timesteps. Temperature is 1000 K.

Energy discrepancies consistently hover around 1e-8 eV/atom while force discrepancies hover around 1e-6 eV/angstrom, within numerical precision, indicating the correctness of our approach across a challenging simulation environment.

We will add these results and theoretical analysis to the final version of the manuscript.

What is the paper’s main research contribution beyond implementing and combining existing techniques, and how does it advance prior distributed MLIP inference systems?

The methods that comprise the core functionality of DistMLIP2 are both novel and non-obvious. DistMLIP2 is not simply combining pre-existing techniques. We will make the writing more clear to highlight the novelties of the system within the manuscript.

If you provide previous works that you feel overlap with DistMLIP2, we can discuss the specific novelty of our method. Upon a very deep literature search after reading your review, the most similar prior work to any of the methods introduced in DistMLIP2 is the partial graph reconstruction algorithm in Yao et al. (2004) [1], but there are fundamental differences between the algorithms. First, the granularity of updates is different, with Yao et al. doing so at the cell level while we do so on the atomic level. This finer granularity minimizes the update size for each timestep. In addition, our neighbor lists are distributed across devices, with each GPU only getting a fraction of the list, while the algorithm proposed by Yao et al. constructs a shared list over devices. For partial updates, this incurs additional complexity around ownership and scope of edge updates that we outline in the paper. The complexity is compounded once integrated with the hierarchical, n-hop partitioning scheme in DistMLIP2 as well.

We assert that we provide multiple novel algorithmic innovations to the field of distributed molecular inference. We will include these differences in the manuscript.

[1] Improved neighbor list algorithm in molecular simulations using cell decomposition and data sorting method, Computer Physics Communications, 2004. https://www.sciencedirect.com/science/article/abs/pii/S0010465504002097?via%3Dihub#SEC006

AuthorsOpenReview ↗

Author Response 3

Was the NequIP model in Table 5 trained? If so, on what data and with what procedure?

The Nequip model we used was trained on the MPTrj training set for 800 epochs. The first 500 epochs were spent with a 1:5 ratio of energy to force loss weights at 0.02 lr. For the last 300 epochs, we decreased lr to 0.001 and used a 1:1 ratio of energy to force loss weights with TF32 disabled to convergence. This is in line with both the original Nequip paper and other prior efforts on Nequip training.

Can you report energy and force deviations from a reference implementation across full and partial graph rebuilds and partitioning configurations? Are any differences larger than floating-point error?

We tested energy and force deviations for partial and full graph reconstruction across multiple systems and runs against a single GPU baseline. Results can be found in the table above. We find that DistMLIP2, under a wide range of configurations, does not exhibit differences larger than floating point error with baseline implementations. We will include these results in the manuscript.

Why did the authors not compare with other native distributed MLIPs, such as Allegro and SevenNet?

Our goal is not to develop and benchmark individual models like Allegro or SevenNet, but rather to develop and benchmark against other distributed inference platforms. Both the Allegro and SevenNet models use the native LAMMPS multi-GPU implementation that we have already benchmarked against in Table 2.

However, for completeness, since the SevenNet distributed inference method does differ slightly from native LAMMPS multi-GPU inference, we also implement the SevenNet model into DistMLIP2 and benchmark the SevenNet native distributed implementation against DistMLIP2. We report results in the table below. T stands for throughput in microseconds/atom-timestep (lower is better) while C stands for capacity in millions of atoms (higher is better). We use the same evaluation protocol as described in Section 4.2 of the paper.

ImplementationGPUsGaN TGaN CIce TIce CLi3PO4 TLi3PO4 CMOF TMOF CCapsid TCapsid C
SevenNet DistMLIP2412.7380.0716715.2280.0601913.2310.0673912.0850.072OOMOOM
SevenNet DistMLIP286.4080.13787.6470.11796.6550.13916.0450.1498OOMOOM
SevenNet DistMLIP2163.3120.27753.9500.22553.4250.26613.1200.29161.7730.2494
SevenNet Native417.6250.0489620.8490.0448818.2420.0530416.8700.0594OOMOOM
SevenNet Native89.1120.110410.8620.089869.8580.10218.6520.1123OOMOOM
SevenNet Native164.6680.22015.6720.1724.9190.20434.5380.22683.6760.2494

In summary, SevenNet + DistMLIP2 is up to 2.07x faster than SevenNet + LAMMPS while simulating up to 1.46x more atoms and maintaining equivalence within numerical precision in the energy and force outputs.

Once again, we would like to thank the reviewer for the review! As we’ve shown above, DistMLIP2 is the state of the art distributed MLIP inference platform built upon multiple novel systems-level algorithms and kernels. We hope we were able to address all of your concerns. If not, please do let us know!

Comments on the paper

1 comment

Thank you for the response during the rebuttal period. We are more resolute in our belief that there have been significant misunderstandings regarding our work. We are also glad to say that we are very familiar with all of the works you have provided and are thus even more confident in the novelty of our own work.

Here are our point by point responses:

Regarding partial graph reconstruction:

We have already discussed the differences between Yao et al. (2004) and our work -- as well as why our work adds novelty beyond the 2004 Yao paper in our first response above. We would like to direct the reviewer to that response.

Regarding runtime skin recomputation:

Our contribution lies in taking advantage of the two distance calculations typically performed by standard MLIP inference -- the first distance calculation is involved in constructing the neighborlist, the second calculation is involved in gathering the necessary position-to-edge gradients for force backpropagation. We allow the standard r_c + r_{skin} radius to go stale while actively applying the new r_c cutoff during the second distance calculation. These are not "conventional Verlet neighor-list techniques implemented in LAMMPS" because our technique is not related to MD/LAMMPS whatsoever. It is a technique that requires a slight modification of the MLIP forward pass itself -- something that an implementation in LAMMPS cannot and does not perform.

Regarding hierarchical partitioning and partition cycling:

On this topic, the reviewer mentions "Pytorch Geometric also mentions hierarchical sampling (https://github.com/pyg-team/pytorch_geometric/releases?after=1.7.2#release-2.4.0) which sounds quite close to graph pruning." The method in PyTorch Geometric's codebase is not at all similar to what we introduced here. Hierarchical partitioning implemented in DistMLIP2 involves two levels of communication (where node-level partitions are expanded by multiple hops while retaining one-hop exchanges between GPU subworkers). We also perform partition balancing that is pruning-aware using Hungarian matching over estimated per-GPU post-pruning node/edge work. Partition cycling, a feature enabled by our hierarchical partitioning method, allows for more logical partitions than physical workers. Using hierarchical partitioning, we are able to expand the full model depth before running each logical partition sequentially on the GPU fleet.

The "hierarchical partitioning" mentioned in the PyTorch Geometric docs is a general, not MD-specific, vanilla implementation used for training mini batches of graphs for GNNs.

Regarding KIM/TorchML:

KIM does not support partition cycling, amortizing cached, layer-dependent computation graphs for re-use across multiple timesteps, or the 2-level hierarchical partitioning we mention above (where multi-hop worker partitions are combined with single hop GPU subpartitions, enabling different communication frequencies between the two levels). The idea of "constructing of successive edge graphs with progressively smaller computational regions", we agree, is not novel. We are not claiming novelty here though.

We hope we were able to clear up any final misunderstandings! If you have any other questions or confusions, please let us know!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.