acceptodds
Withdrawn from ICLR 2027

When Data Ordering Helps or Hurts: A Matched-Support Study of Interatomic Potentials

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Curriculum learning reorganizes training data to guide the optimization of machine-learned interatomic potentials. Its effect on joint energy and force accuracy remains unclear because reported gains often combine ordering with changes in sample selection and exposure. We isolate ordering on OC20 using matched training pools and one-pass budgets, holding initialization, within-model loss weights, and evaluation conditions fixed. Five orderings are compared across four models and four in- and out-of-distribution validation splits. These comparisons reveal that ordering changes the Energy–Force balance even under the same training objective. Difficulty-based and coverage-to-tail curricula increase mean Force MAE by 10.1–16.6% versus uniform on complete validation splits, with penalties also present under constant learning rates. Increasing the pool from 200k to 500k improves the uniform Force baseline but preserves substantial ordering costs. At 500k, coverage-first reduces PaiNN Energy MAE by 24.3% while increasing Force MAE by 0.14%. No tested curriculum jointly lowers both mean errors relative to uniform. Data ordering therefore changes the Energy–Force balance under fixed exposure and loss weights, making curriculum value depend on the prediction objective and training regime.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Curriculum learning, Machine-learned interatomic potentials, Data ordering, Energy–force trade-offs, Scientific machine learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_kbDpOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how training-data ordering affects interatomic potentials while holding the training pool, sample exposure, initialization and within-model optimization settings fixed. It compares five orderings across four architectures on OC20, with a larger-budget extension. The main finding is that several intuitive curricula worsen force prediction, while occasional gains depend on the model, budget and prediction objective.

This is a useful controlled empirical study. However, its contribution currently remains a regime-specific observation rather than a sufficiently general explanation of when ordering helps or hurts. Stronger mechanistic analysis and broader training-regime validation would substantially improve the paper.

Critical strengths

The matched-support design addresses a real ambiguity: curriculum gains can otherwise reflect changes in data selection or repeated exposure. Experiments cover four architectures and four ID/OOD validation splits. Complete-split evaluation strengthens the evidence beyond the initial validation subsets. Reporting energy and force separately exposes regressions hidden by a single metric. The negative results provide a useful caution against assuming that intuitive curricula improve training.

Critical weaknesses

The practical scope is narrow. Every example is presented once, starting from random initialization. The experiments establish ordering effects in this regime, but do not show whether they persist under multi-epoch training, convergence-oriented budgets or fine-tuning. A common shuffled continuation would help distinguish persistent effects from transient endpoint bias.

The tested difficulty measure limits the conclusion. Reference-force RMS measures label amplitude rather than learning difficulty. Failure of ascending and descending force-magnitude orders does not establish the limitations of teacher-error, uncertainty-based or adaptive curricula.

Mechanistic insight remains limited. Reordering examples also changes minibatch composition and gradient statistics. The study does not separate these effects from inter-batch order or explain the observed energy–force responses. Moreover, three architectures predict forces directly, limiting interpretation in terms of energy-value versus energy-gradient accuracy.

The empirical contribution needs stronger generalization. The study demonstrates several failures but provides no predictive criterion for when ordering helps. It uses one dataset, three seeds in the core study and one seed at 500k. The highlighted 24.3% energy improvement therefore requires replication.

Questions for authors

Do the differences persist after identical shuffled continuation or multi-epoch training? How do curricula based on teacher prediction error compare with force-RMS ordering? Can fixed-minibatch permutations separate batch-composition effects from presentation order? Can gradient diagnostics and additional seeds explain and reproduce the model-dependent energy–force changes?

Reviewer_bcGeOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper discusses whether presentation order alone, with the training multiset, one-pass exposure, initialization scheme, and loss weights held fixed, changes the energy/force accuracy of MLIPs on OC20 S2EEF. The findings center on that uniform ordering is never beaten by the difficulty or coverage-to-tail schedules. Coverage-first is roughly neutral on forces but costs energy. No ordering jointly improves both errors, and the penalties persist when the pool grows.

Critical strengths
  1. The experimental control is great for the curriculum learning regime. The authors deliberately keep identical multisets, identical exposure count, identical optimizer config across orderings within each architecture and paired comparison against uniform at the seed/split level. This isolates permutation effects.
  2. The authors use four architectures spanning invariant to higher-order equivariant, four distribution-shift splits, and full-split evaluation make the negative results credible.
  3. The join energy-force view is a great addition: the observation that hard -> easy improves EquiformerV2 energy while curing forces is the kind of trade-off that a force-only evaluation would hide.
Critical weaknesses
  1. The training regime is far from how MLIPs are actually trained. The paper does not address that its conclusions would transfer to real cases. Specifically, every model sees each of 200k records exactly once. PaiNN receives 6252 optimizer updates in total. SchNet 8336, GemNet-OC 12500. The resulting errors are 2-3x worse that published values for these architectures on the OC20-2M subset. The models are in the very early phase of training. Curriculum learning typically involves multiple epochs with pacing functions or reweighing in a strict one-pass regime. The terminal checkpoint is dominated by the distribution of the last stages. So the narrative "ordering hurts" shall be reduced to :ending training on a non-representative checkpoints would hurt". The paper acknowledges the design is "narrower" than standard curricula. However, the title and abstract are stated as general claims about data ordering for MLIPs.

  2. The authors do not investigate any mechanism. Sec 3.4 promises stage-end diagnostic and Appendix H defines them. However, no stage-end curves or per-stage deltas appear anywhere in the paper. For instance, the authors propose hypotheses, recency/distribution shift at the final stage, is directly testable with what the authors already have (stage-end MAE for each ordering), and additionally with cheap controls such as a short uniform cool-down after a staged schedule. The paper reports that ordering hurts but fails to offer evidence about why, limiting the practical aspect of this paper.

Questions for authors

See above weaknesses.

Reviewer_D5bEOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies interatomic potentials. These can be equated to a network that takes atom types and 3D positions and predicts the energy of the structure. This network also predicts the forces, that are 3D vectors, literal derivatives of the position w.r.t the energy, ie ∂E/∂positions. Specifically, this paper studies a specific dataset, OC20, which contains snapshots of small molecules (the adsorbates) on a catalyst surface. The dataset is a series of snapshots derived from a geometric relaxation of the molecules, from high energy states with large forces to lower states with near zero forces. The objective of using ML on OC20 is to learn the adsorption energy landscape. In standard quantum-mechanics, the theory linking shapes to energy is Density Functional Theory (DFT). ML can be used to reproduce DFT potentials approximately but very quickly, which allows for the efficient search of new chemical compounds, e.g. catalysts.

In this study, the authors take a 200k pool of frames, they train one pass over it in 4 stages of 50k frames. They compare 5 ordering of the frames:

  • Uniform shuffle
  • easy -> hard and hard->easy sorted by force magnitude
  • "coverage first" as a round-robin over bins created from easily computed descriptors like atom counts, energy per atom, atomic-number statistics,
  • "coverage to tail" which is a mix of the last two.

They find that uniform shuffle is nearly always best or near best. Sorted ordering improves Mean Absolute Error by 10-17%. No ordering improves both energy and force.

Critical strengths

1- The core methodological point is sound. Many curriculum papers change which samples are seen, or how often, along with the order. Holding the multiset and exposure fixed (Eq. 1) cleanly isolates the effect of ordering.

2- The comparison is broad: the authors used 4 architectures spanning the invariant-to-equivariant range, 4 ID/OOD splits. They evaluate on the full splits (~1M frames each), and they report both energy and force.

3- Full per-cell tables are in the appendix and code is released. It's mostly a negative result, but honestly reported.

Critical weaknesses

1- The results is as expected. Without shuffling, final weights are biased towards whatever comes last. In classical ML theory, this is why SGD shuffles its inputs.

2- There are some known prior work on this: Wu et al ICLR 2021. Also the literature on random reshuffling for SGD, essentially the question “is a permutation better than shuffling?” is a very general question on ML.

3- The models are undertrained. Published Equiformer reaches 19.4 meV/Å force and 278 meV energy ; in this article uniform Equiformer 64–79 meV/Å and 0.66–0.93 eV, roughly 3.5x worse. This is only an early-training snapshot, not the full study?

4- The authors ban change in frequency (section 3.1). The classical Bengio-Hacohen-Weinshall form starts with an easy subset and is grown while keeping the easy case in the mix, forcing a change in frequency. Pure sorted permutation in one pass is not a good case for SGD, so not a good strategy here either, and so in practice this curriculum would not be used. This is not enough to conclude that "curricula hurt".

5- Easy->Hard and Hard->Easy are based on force RMS, but high force frames correspond to early, unrelaxed geometries while low RMS correspond to relaxed geometries, not necessarily on the inherent difficulty of the frame. Going easy->hard finished on unrelaxed frames while the opposite is true for hard->easy. This is distribution shift, essentially. Maybe authors should assess difficulty differently, e.g. with a reference model ?

6- This is biased but I found the text hard to read with expressions I'm not familiar with: “bounded cells”, “training identities”, “seed-local pool”, “matched-support” etc.

[Wu 2021] Wu, Dyer & Neyshabur, When Do Curricula Work? (ICLR 2021) arXiv preprint arXiv:2012.03107 (2020).

[Gürbüzbalaban 2021] Gürbüzbalaban, Mert, Asu Ozdaglar, and Pablo A. Parrilo. "Why random reshuffling beats stochastic gradient descent." Mathematical Programming 186.1 (2021): 49-84.

[Mishchenko 2020] Mishchenko, Konstantin, Ahmed Khaled, and Peter Richtárik. "Random reshuffling: Simple analysis with vast improvements." Advances in neural information processing systems 33 (2020): 17309-17320.

Questions for authors

1- How does this work relate to Wu et al. (ICLR 2021)? What does it show beyond “random ordering is hard to beat”?

2- Can you test the non-stationarity explanation directly, for example by correlating each ordering’s penalty with a divergence between the stage-4 distribution and the validation distribution?

3- Do the conclusions hold in a multi-epoch regime closer to published OC20 training, or with a growing-pool pacing curriculum that relaxes the matched-frequency constraint?

4- How do your uniform baselines compare with published MAEs for the same architectures, and how far from convergence are they?

5- Was coverage-to-tail designed as the proposed method, with the success criteria fixed before the experiments?

6- Does a model-based difficulty score behave differently from force RMS, given that force RMS mostly tracks the relaxation step?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.