acceptodds
Under review as a conference paper at ICLR 2027

UMA-DOS: Transfer Learning from Pretrained Atomistic Models for Density of States Prediction

Abstract

The Density Of States (DOS) is a fundamental spectral property of crystalline materials that governs a wide range of chemical, electronic, and optical properties. Despite its importance, fast but accurate DOS prediction based on machine learning remains challenging because it involves a high-dimensional structured target, the physically meaningful features of which depend on subtle relations between crystal structure and energy-resolved electronic states. Existing machine-learning approaches primarily focus on learning strong structural representations from crystal graphs, but often rely on DOS supervision alone and do not fully exploit the transferable information already learned by large pretrained atomistic models. In this work, we present UMA-DOS, a transfer-learning framework for total DOS prediction built on pretrained UMA backbones. UMA-DOS uses a pretrained atomistic encoder to represent crystal structure, aggregates node-level features into a graph-level representation, and maps this representation to an energy-resolved DOS through a structured spectral decoder with explicit shape–amplitude factorization. Under a unified Materials Project evaluation pipeline, UMA-DOS achieves the strongest controlled performance among the reproduced baselines, with its clearest advantages appearing in shape-sensitive measures of spectral fidelity. Transfer ablations further show that the value of pretraining is realized only through downstream adaptation: frozen transfer underperforms scratch training, whereas end-to-end fine-tuning yields the best results. These findings show that pretrained atomistic representations can transfer effectively to total DOS prediction when paired with a shape-aware spectral decoder and adapted end-to-end.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Density of States, transfer learning, DOS prediction

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_DPv1OpenReview ↗
Rating
3
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents UMA-DOS, which adapts pretrained atomistic models to predict total DOS using a spectral decoder and separate shape and amplitude predictions. My main questions are listed below.

Critical strengths

See Questions

Critical weaknesses

See Questions

Questions for authors
  1. How does DOSTransformer perform with the same shape–amplitude factorization and training losses? The current comparison mixes changes to the encoder, target, and loss, so the source of the improvement is unclear.

  2. The low-data experiments should also include UMA-DOS trained from scratch at each training fraction.

  3. Were any test structures, or similar structures, included in UMA’s pretraining data? Overlap would not necessarily mean DOS-label leakage, but it would change how we interpret the results. A split by composition or structure type would be useful.

  4. I could not tell whether DOS is reported per atom or per cell. If a cell is duplicated, DOS per cell should double in amplitude, while DOS per atom should stay the same. Mean pooling and normalized weighted pooling can give the same representation for both cells. How would the amplitude head distinguish them? Is atom count used elsewhere?

Reviewer_BsxxOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposed UMA-DOS, which uses a pretrained UMA and uses a transfer learning framework to reuse pretrained UMA representations for DOS prediction through a structured spectral decoder with explicit shape amplitude factorisation. Experiments on the total DOS dataset obtained from the Material Project show that UMA-DOS outperforms two SOTA DOS prediction models.

Critical strengths
  • This paper is well motivated, and the author studies an important problem. The DOS prediction dataset is always limited, and using foundational pretrained models to use this for DOS prediction is an interesting direction.
  • Experimental results show that from Table 1, UMA-DOS outperforms the other existing SOTA models, and also from Figure 3, shows that for the few-shot experiment, UMA-DOS outperforms the existing baseline models.
Critical weaknesses

In this work the author uses only two baseline models (Mat2Spec and DOSTransformer) in Table 1 and Figure 3, but these models do not compare against the UMA backbone used in these baseline models. Thus, it is unclear whether the gain comes from using a strong pretrained model or from the proposed UMA-DOS model.

Questions for authors

Please see the weakness

Reviewer_CZb6OpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates whether atomistic models pretrained on energies and forces can transfer effectively to electronic density-of-states prediction. UMA-DOS combines a pretrained UMA encoder with crystal-level pooling and a convolutional spectral decoder. It predicts normalized spectral shape and overall amplitude separately, aiming to capture local spectral structure while accommodating variation in DOS magnitude. The experiments compare scratch training, frozen transfer, ELoRA adaptation, and full fine-tuning under a shared Materials Project evaluation protocol. Full fine-tuning performs best among the UMA-DOS configurations and improves several spectral-quality metrics over reproduced baselines. Additional MAD-DOS experiments and architectural ablations provide further evidence for the approach.

Critical strengths

-The transfer study addresses a meaningful question about reusing representations learned for energies and forces to predict electronic spectra. Comparing scratch, frozen, parameter-efficient, and fully fine-tuned models helps clarify when pretraining is useful.

-The decoder and shape–amplitude ablations support the main design choices. They show that the prediction head contributes to performance alongside the pretrained encoder.

-The evaluation takes preprocessing differences between prior methods seriously. Shared splits, three-seed central comparisons, and the additional MAD-DOS experiment strengthen the empirical evidence.

Critical weaknesses

-The headline nMAE advantage depends on the baseline’s target parameterization. Table 7 shows that the original peak-normalized DOSTransformer achieves lower nMAE than UMA-DOS, 0.098 versus 0.112, although UMA-DOS performs better on SAM and overlap. The main comparison instead uses DOSTransformer trained on absolute-scale targets. A baseline that retains normalized-shape supervision and learns a separate amplitude head would better isolate the proposed model’s advantage.

-The low-data experiments establish the performance of the complete system, but do not isolate the contribution of pretraining. Figure 3 compares different model families without including scratch UMA-DOS at each training fraction. The observed advantage could also reflect the decoder, loss, or shape–amplitude decomposition.

Questions for authors

-How does DOSTransformer perform when it retains its normalized-shape objective and gains a learned amplitude head under comparable training conditions?

-How do pretrained and scratch UMA-DOS compare at 10%, 25%, and 50% of the training data? Does the benefit of pretraining increase as supervision becomes more limited?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.