acceptodds
Under review as a conference paper at ICLR 2027

DeMiR: Density-Matrix-Informed Representations for Molecular Property Prediction

Abstract

The Hohenberg–Kohn theorem, which states that all physical quantities in the ground state are functionals of the electron density, suggests that much of a molecule's behavior is encoded in its electronic structure. This study proposes DeMiR, a model that treats electron density matrix prediction as a pretraining task. DeMiR reconstructs cc-pVDZ/B3LYP density matrices for molecules ranging in size from QM9-scale molecules to drug-like molecules with up to 146 atoms, achieving median NMAE values of 1.0% and 2.16%, respectively. At the atomic level, the model's embeddings contain electronic structure information: per-atom electronic observables are predicted with an of 0.94 by a linear probe, whereas Uni-Mol reaches 0.84 and a randomly initialized model of the same architecture only 0.63. The embeddings also remove 44.9% of the residual error left by a geometry-based descriptor, showing that this information is not linearly accessible from an explicit geometric descriptor. At the molecular level, transfer depends on the label type. Where electronic structure directly determines the property, transfer holds: on an external experimental pKa test set, fine-tuned DeMiR achieves an MAE of 1.143, significantly outperforming a randomly initialized control with the same architecture and training budget (1.397), and performing on par with or better than chemical foundation models pretrained on far larger datasets. In contrast, for labels such as pIC50 and ADMET, DeMiR shows no consistent advantage over existing general-purpose foundation models, as two-dimensional substructures and physicochemical descriptors have already saturated predictive performance. This study presents the scope within which a connection between electron density prediction capability and molecular property prediction capability holds, along with its boundaries.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Molecular Representation Learning, Quantum Chemistry, Electron Density, Density Matrix, Pretraining, Equivariant Neural Networks

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_VbnYOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes DeMiR, which uses density-matrix prediction to pretrain molecular representations. It combines an equivariant encoder with a physics-constrained prediction head and evaluates density reconstruction, atom-level electronic information, and downstream property prediction. The results show useful transfer for pKa, alongside limited benefits on ADMET tasks.

Critical strengths
  1. The paper goes beyond reporting reconstruction accuracy: its atom-level probes and geometric controls provide meaningful evidence that pretraining makes electronic information more accessible in the learned representations.

  2. The external pKa evaluation, including a matched randomly initialized model, supports the practical value of pretraining. The explicit discussion of limited ADMET gains also provides useful insight into the approach’s boundaries.

Critical weaknesses
  1. The comparison with dedicated 3D molecular models is insufficient. Although several baselines are included, Uni-Mol is the principal 3D reference in the molecular-property experiments. This provides limited evidence of competitiveness against stronger geometric approaches. Relevant additions include Uni-Mol2 (Ji et al., NeurIPS 2024), which reports improved QM9 performance over Uni-Mol, and ViSNet (Wang et al., Nature Communications 2024), an equivariant geometric network evaluated on quantum-chemical property prediction. Comparisons should use matched splits and conformers, with frozen representation probing clearly distinguished from end-to-end prediction. Without these comparisons, the advantage over the selected baselines does not sufficiently establish DeMiR’s broader value relative to existing 3D models.

  2. The ablations do not sufficiently isolate what drives the transfer gains. The random-initialization and pooling controls are useful, but the pretraining procedure combines density-matrix supervision, SP2 projection, and auxiliary real-space density and dipole losses. Their individual contributions to downstream performance remain unclear. In particular, the paper does not establish whether predicting the full density matrix provides additional benefits over simpler quantum-property pretraining using the same encoder. Targeted ablations with matched data and training budgets should assess downstream transfer, not only reconstruction accuracy.

Questions for authors
  1. Can the authors include Uni-Mol2 and ViSNet under comparable evaluation settings to establish how DeMiR compares with stronger 3D molecular models?

  2. How much do SP2 projection and the auxiliary losses contribute to downstream performance? Does full density-matrix supervision outperform simpler quantum-property pretraining with the same architecture and budget?

1 comment

We thank the reviewer for the kind assessment.

Q1. Uni-Mol2 and ViSNet. We added both, with frozen and end-to-end results in separate tables.

In the frozen setting, every model gets the same RDKit conformers as DeMiR, the atom probe uses heavy atoms only because Uni-Mol2 reads no hydrogens, and physical properties follow the Table 6 protocol.

FrozenAtom probe R²pKaFreeSolvHexadecaneMOF
DeMiR0.9391.9120.8950.16668.4
Uni-Mol0.8422.1561.4330.30477.2
Uni-Mol2 84M0.7672.1411.3170.19370.6
Uni-Mol2 1.1B0.7702.2321.2750.18574.5
ViSNet (SPICE2 MLIP)0.9651.6490.7390.18666.2

DeMiR beats both Uni-Mol2 models everywhere. ViSNet has no public foundation weights, so we used a released ViSNet trained on SPICE2 energies and forces (InstaDeep's visnet-organics checkpoint). That is quantum-chemical pretraining on drug-like molecules, and it beats DeMiR on four of five frozen readouts.

End-to-end (3 seeds), we trained ViSNet (6.7M parameters, our encoder has 6.4M) from scratch on each task.

Test MAEpKaFreeSolvHexadecaneMOF
DeMiR, fine-tuned1.129 ± 0.0580.642 ± 0.0420.131 ± 0.00264.3 ± 2.4
Random init, fine-tuned1.441 ± 0.0320.689 ± 0.0600.206 ± 0.02365.3 ± 4.4
ViSNet, from scratch1.132 ± 0.0310.543 ± 0.0220.151 ± 0.02282.1 ± 12.0

DeMiR ties ViSNet on pKa, wins on hexadecane and MOF, and loses on FreeSolv. ViSNet ran on a different GPU type, so we do not test the difference. Overall, DeMiR is well above Uni-Mol and Uni-Mol2, below the SPICE2-pretrained ViSNet on frozen read-outs, and level with ViSNet trained end to end; it is also the only one that predicts a density matrix. Tables 1, 2, and 6 all use frozen probes, and the revised captions say so.

Q2. SP2 and auxiliary losses. Each model below continues from the same stage-1 checkpoint for 6,000 iterations.

ModelAtom probe R²Dipole vector R²pKa (3 seeds)NMAE QM9 / QMugs
Stage-1 start0.9030.8831.205 ± 0.0132.03 / 5.01%
No SP20.9110.9001.227 ± 0.0303.38 / 20.2%
SP20.9430.9121.159 ± 0.0311.39 / 2.91%
SP2 + auxiliary (DeMiR recipe)0.9400.9121.143 ± 0.0481.04 / 2.17%

SP2 raises the atom probe from 0.911 to 0.943 and improves pKa by 0.084 (95% CI [0.027, 0.143]). The auxiliary losses add nothing measurable.

Q2 (second part). Full DM vs. simpler QM properties. Details are in the General response. DM gives more atom-level information and better frozen QM9 probes, while scalar targets transfer better to pKa when both are pretrained on QM9 only.

Reviewer_732iOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes DeMiR, a molecular representation model pretrained to predict electron density matrices from molecular geometry. It combines an equivariant encoder, an atom-pair density-matrix decoder, and a physics-constrained projection enforcing electron-count conservation and idempotency. The authors demonstrate accurate density reconstruction and improved encoding of atomic electronic properties. Downstream improvements are strongest for quantum-chemical properties and selected experimental properties such as pKa, while advantages on ADMET tasks are limited.

Critical strengths

Physically grounded representation learning. Predicting density matrices provides a principled electronic-structure pretraining objective. The physical projection enforces meaningful constraints, and the atomic probing experiments demonstrate that the resulting embeddings encode electronic observables more effectively than several baselines.

Careful empirical evaluation. The paper includes controlled comparisons against random initialization, existing molecular representations, and geometric descriptors. The authors also acknowledge where their approach does not improve downstream performance, particularly on ADMET tasks.

Critical weaknesses

Insufficient positioning against relevant prior work. The paper overlooks Two-Stage Pretraining for Molecular Property Prediction in the Wild (Wijaya, Guo, Sun et al., 2025), which already investigates improving molecular representations through pretraining on computationally derived quantum-chemical properties, including DFT-derived HOMO, LUMO, and dipole moments, before transferring to experimentally measured properties. Related work such as Quantum-Informed Molecular Representation Learning Enhancing ADMET Property Prediction (2024) is also relevant. Although DeMiR uses density-matrix supervision rather than scalar quantum properties, the underlying motivation is closely aligned. These precedents deserve discussion to establish the distinction and novelty of the proposed approach.

Limited conceptual novelty and incremental contribution. The central approach follows an established paradigm of improving molecular representations through quantum-chemical pretraining, replacing scalar electronic-property targets with the electron density matrix. While the density-matrix decoder and physical projection are technically meaningful, it is unclear whether this constitutes a sufficiently substantial conceptual advance in molecular representation learning. More importantly, the paper does not directly establish whether density-matrix supervision offers meaningful advantages over simpler quantum-property pretraining objectives. The limited downstream improvements outside electronic-structure-related properties further weaken the argument for the broader significance of the proposed representation.

Questions for authors
  1. How does DeMiR compare conceptually and empirically with prior quantum-informed molecular pretraining approaches, particularly MoleVers (Wijaya, Guo, Sun et al., 2025)?

  2. What is the specific benefit of predicting the full density matrix rather than simpler electronic observables such as HOMO, LUMO, or dipole moment? A controlled comparison using the same encoder, training data, and computational budget would help clarify whether the increased complexity of density-matrix supervision is justified.

2 comments

We thank the reviewer for pointing to MoleVers and for the precise request for a matched comparison. We ran the matched comparison you asked for.

Q1 / W1. MoleVers and Kim et al. (2024). Both are now in the related work section, with Fallani et al. (2025). MoleVers first pretrains with masked atom prediction and denoising at large noise scales, then regresses HOMO, LUMO, and dipole computed with DFT (Psi4) for 130K GDB-17 molecules, optionally adding pairwise property rankings from an LLM. Kim et al. pretrain on quantum properties for ADMET. DeMiR regresses the full one-electron state, whereas these methods regress a few summaries of it, and the comparison above measures what the full state gives under matched conditions.

Q2. Full DM vs. HOMO / LUMO / dipole. The QM-property pretraining ablation regresses DFT HOMO, LUMO, and dipole on the same molecules, encoder, and steps. These are the three DFT labels MoleVers uses in its second stage (General response, first two rows of the table). The matrix gives more atom-level information (atom probe 0.930 vs. 0.906, dipole vectors 0.929 vs. 0.756, carbon polarization 0.893 vs. 0.645) and better frozen QM9 probes on the 15 targets the QM-property model never saw (1.00 vs. 1.30). On pKa, the QM-property model transfers better when both are pretrained on QM9 only (1.123 vs. 1.300). DeMiR, pretrained on QM9 + QMugs, reaches 1.129, which suggests that drug-like pretraining data matters at least as much as the target. We have narrowed the abstract accordingly. The General response also covers coordinate denoising. It is close to the denoising half of MoleVers' first stage, though our noise is small (σ = 0.04 Å) and we do not add masked atom prediction.

W2. Limited gains outside electronically governed properties. We agree with your concern, and the paper notes this as a limitation: on ADMET, fine-tuning moves the encoder as much as on pKa but helps only two of the twelve tasks. The new results agree: denoising and scalar targets transfer to molecules about as well as DM, and only the DM target adds atom-level directional information.

My concerns are somewhat addressed. I will keep my score.

Reviewer_at9qOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces a new method to build a learned molecule representation. Instead of a pre-training task using only the molecule geometry the proposed method DeMiR aims to predict the ground state electron density matrix from the geometry in pre-training. It is argued that this is a more informative training task, enabling the model to learn a richer representation. The representation resulting from training on QM9 and QMUGS molecules is then evaluated by predicting different molecular properties from the resulting embedding.

While using more electronic structure information as a pre-training task seems to be a well founded idea, it is not very novel. Careful comparisons with existing work would be necessary to justify how this new method is better.

Critical strengths

The idea to use density matrix prediction as pre-training for representation learning is an interesting approach to achieve a more informative embedding.

Critical weaknesses

Using electronic structure prediction as a pre-training task is not a novel idea. Especially the HELM [1] model introduces a very similar approach but instead of learning the density matrix, the Hamiltonian is learned. Direct comparison on the same data would be necessary to show the impact of this new method compared to methods like HELM. Also comparing the achieved quality of the predicted density matrices on the same data to the existing density matrix prediction methods in the literature would help evaluate the presented method.

Additionally there are no ablations for many of the choices made in the architecture and method (see questions).

  1. Kaniselvan, M., Miller, B., Gao, M., Nam, J. & Levine, D. Learning from the Electronic Structure of Molecules across the Periodic Table. International Conference on Learning Representations 2026, 133458–133482 (2026).
Questions for authors
  • Is it necessary for good representation learning results to use the density matrix purification methods that are discussed?
  • Are virtual nodes really needed?
  • How does the proposed architecture compare to what is in the literature? Could one for example train HELM on density matrix prediction and compare?
  • The method is compared to using the randomly initialized model without pre-training. Could also a direct comparison be made with the same model but trained on an only geometry pre-training task on the same data?
2 comments

We thank the reviewer for the four questions. We ran an experiment for each.

Q1. Is SP2 needed? Yes, in DeMiR's second stage. We trained three stage-2 models from the same stage-1 checkpoint for 6,000 iterations each.

ModelAtom probe R²Dipole vector R²pKa (3 seeds)NMAE QM9 / QMugs
Stage-1 start0.9030.8831.205 ± 0.0132.03 / 5.01%
No SP20.9110.9001.227 ± 0.0303.38 / 20.2%
SP20.9430.9121.159 ± 0.0311.39 / 2.91%
SP2 + auxiliary (DeMiR recipe)0.9400.9121.143 ± 0.0481.04 / 2.17%

SP2 raises the atom probe from 0.911 to 0.943 and improves pKa (−0.084, 95% CI [−0.143, −0.027], paired bootstrap over 278 molecules). Without it, density error on QMugs rises to 20%. The auxiliary losses add nothing measureable. Each comes from a single pretraining run.

Q2. Are virtual nodes needed? Not for the representation. Removing them at stage 2 gives atom probe 0.968 vs. 0.965, and pKa stays within noise. That model does start from a stage-1 encoder trained with virtual nodes, but the HELM-style model below never had any and still reaches 0.949. Virtual nodes help training speed and density accuracy, and the revision says so. For the camera-ready version, we will add a DeMiR model trained from scratch without virtual nodes.

W1 / Q3. HELM. HELM uses electronic-structure pretraining to improve interatomic potentials. Our question is which electronic information an encoder carries and when it helps on experimental properties. HELM's code has not been released (the arXiv version lists it as TBA), so we reimplemented its readout (on-site blocks from nodes, off-site blocks from edges, gated nonlinearity, no virtual nodes) and trained it on our DM target and data instead of HELM's Hamiltonian target.

  • Its density is more accurate than our decoder's: median NMAE 2.65% vs. 3.27% at 40k, both without SP2.
  • It has the highest carbon polarization of any model (0.912). Both models trained on the DM target beat denoising and scalar targets on this measure, so this advantage comes from the target and does not depend on our decoder.
  • Despite its better density, it transfers poorly to pKa (1.544 ± 0.024).
  • For scale, a superposition-of-atoms guess scores 12.7% NMAE on QM9 and DeMiR scores 1.00%. Published DM predictors use other molecules, bases, and metrics, so we do not rank against them. dm-PhiSNet (Yescas-Ramos et al., 2026), for example, is trained and tested on configurations of six small molecules and reports a one-shot energy error of about 6 × 10⁻³ kcal/mol on ethanol.

Novelty. We agree that electronic-structure pretraining is not new, and we now cite HELM, MoleVers, Kim et al. (2024), and Fallani et al. (2025). Our contribution is a matched comparison of matrix, scalar, geometric, and combined targets. It shows that the density matrix adds directional, atom-level information and a usable density, but it does not transfer to molecules better than denoising.

Q4. Geometry-only pretraining. We trained the same encoder with coordinate denoising (σ = 0.04 Å) on the same data and budget (General response, table). Denoising matches or beats DM on the atom probe and on pKa, so we have dropped the claim that DM supervision drives pKa transfer. DM is still ahead on directional information: atomic dipole vectors 0.929 vs. 0.860, carbon polarization 0.893 vs. 0.738.

Denoising also needs far less data. The table gives atom probe R² after pretraining on nested QM9 subsets for 4k steps each (Fig. 7 in the revised PDF).

Pretraining molecules1071,07110,710107,108107,108, 40k steps
DM0.7300.8250.8430.8630.930
Denoising0.9080.9460.9530.9540.957

DM keeps improving with more data and more steps, and the gap falls from 0.178 to 0.027, but it never passes denoising within our budget.

Reviewer_at9qOpenReview ↗

Response to rebuttal

Thank you for the honest and thorough response, and for providing the additional experiments. While most of my concerns were answered with this revision, the introduced method does not consistently outperform the added baselines. Coordinate denoising is also far more data-efficient. Since these results undermine the premise that density-matrix pretraining is especially useful, I will keep my score.

Comments on the paper

1 comment

We thank the reviewers for their careful reviews. All three reviewers asked for a controlled comparison of pretraining targets (Reviewers at9q Q4, 732i Q2, VbnY Q2), and two requested design ablations (Reviewers at9q Q1 to Q3, VbnY Q2). We ran both, and the results led us to drop one claim.

Controlled comparison (new Sec. 4.4). Every model uses the same eSCN encoder, QM9 split, optimizer, and batch size, and trains for 40k steps. Only the target and its output head differ. We reran all downstream evaluations on one machine with fixed seeds, so DeMiR and random-init pKa differ slightly from the submission (1.129 / 1.441 vs. 1.143 / 1.397).

Target (QM9, 40k steps)Atom probe R²Atomic dipole vector R²Carbon polarization R²QM9 frozen MAE (rel. to DM)pKa MAE (20k ckpt)
Density matrix (DM)0.9300.9290.8931.001.300 ± 0.029
HOMO / LUMO / dipole0.9060.7560.6451.231.123 ± 0.019
Coordinate denoising0.9570.8600.7380.981.117 ± 0.017
DM + denoising0.9480.911ᵃ—1.001.125 ± 0.028
HELM-style readout, DM target0.949—0.9121.091.410ᵇ
Random init0.6340.5230.499—1.441 ± 0.032
DeMiR (reference)0.9390.9080.849—1.129 ± 0.058

ᵃ 20k checkpoint. ᵇ 1 seed. pKa uses the 20k checkpoint, the one fine-tuned for every model (40k where run: DM 1.392, dual 1.120, HELM-style 1.544). Definitions and the 20k table are in Appendix H.

pKa at 40k, with all three models fine-tuned on a second machine: DM 1.370 ± 0.023, HOMO / LUMO / dipole 1.131 ± 0.032, coordinate denoising 1.079 ± 0.066 (vs. DM: −0.239, 95% CI [−0.312, −0.166]; −0.291, 95% CI [−0.378, −0.207]). We compare these three only with each other.

Three conclusions:

  1. DM beats scalar quantum targets at the atom level (Reviewers 732i, VbnY), most clearly on atomic dipole vectors (0.929 vs. 0.756).
  2. Coordinate denoising matches or beats DM on the atom probe and on pKa (Reviewer at9q), so we withdraw the claim that the DM target drives pKa transfer. The DM model still leads on atomic dipole vectors, and both models trained on the DM target lead on carbon polarization.
  3. Training on both targets costs nothing in pKa or density accuracy. The combined model matches denoising on pKa and the DM model on density accuracy (median NMAE 4.78% vs. 4.73%).

Revised claims. (i) DM pretraining makes atom-level electronic information, especially its directional part, more linearly accessible than scalar quantum targets do. (ii) In DeMiR's second stage, the SP2 projection improves it and keeps the density accurate on QMugs (Reviewer at9q, Q1). (iii) Adding DM supervision to geometric pretraining costs nothing downstream. We no longer claim DM is the best target for property transfer.

Revisions. New Sec. 4.4, an ablation appendix (SP2, auxiliary losses, virtual nodes, HELM-style readout), Uni-Mol2 and ViSNet baselines, related work on HELM, MoleVers, Kim et al. (2024) and Fallani et al. (2025), and a revised abstract, contributions, and conclusion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.