- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes DeMiR, which uses density-matrix prediction to pretrain molecular representations. It combines an equivariant encoder with a physics-constrained prediction head and evaluates density reconstruction, atom-level electronic information, and downstream property prediction. The results show useful transfer for pKa, alongside limited benefits on ADMET tasks.
Critical strengths
-
The paper goes beyond reporting reconstruction accuracy: its atom-level probes and geometric controls provide meaningful evidence that pretraining makes electronic information more accessible in the learned representations.
-
The external pKa evaluation, including a matched randomly initialized model, supports the practical value of pretraining. The explicit discussion of limited ADMET gains also provides useful insight into the approach’s boundaries.
Critical weaknesses
-
The comparison with dedicated 3D molecular models is insufficient. Although several baselines are included, Uni-Mol is the principal 3D reference in the molecular-property experiments. This provides limited evidence of competitiveness against stronger geometric approaches. Relevant additions include Uni-Mol2 (Ji et al., NeurIPS 2024), which reports improved QM9 performance over Uni-Mol, and ViSNet (Wang et al., Nature Communications 2024), an equivariant geometric network evaluated on quantum-chemical property prediction. Comparisons should use matched splits and conformers, with frozen representation probing clearly distinguished from end-to-end prediction. Without these comparisons, the advantage over the selected baselines does not sufficiently establish DeMiR’s broader value relative to existing 3D models.
-
The ablations do not sufficiently isolate what drives the transfer gains. The random-initialization and pooling controls are useful, but the pretraining procedure combines density-matrix supervision, SP2 projection, and auxiliary real-space density and dipole losses. Their individual contributions to downstream performance remain unclear. In particular, the paper does not establish whether predicting the full density matrix provides additional benefits over simpler quantum-property pretraining using the same encoder. Targeted ablations with matched data and training budgets should assess downstream transfer, not only reconstruction accuracy.
Questions for authors
-
Can the authors include Uni-Mol2 and ViSNet under comparable evaluation settings to establish how DeMiR compares with stronger 3D molecular models?
-
How much do SP2 projection and the auxiliary losses contribute to downstream performance? Does full density-matrix supervision outperform simpler quantum-property pretraining with the same architecture and budget?
We thank the reviewer for the kind assessment.
Q1. Uni-Mol2 and ViSNet. We added both, with frozen and end-to-end results in separate tables.
In the frozen setting, every model gets the same RDKit conformers as DeMiR, the atom probe uses heavy atoms only because Uni-Mol2 reads no hydrogens, and physical properties follow the Table 6 protocol.
DeMiR beats both Uni-Mol2 models everywhere. ViSNet has no public foundation weights, so we used a released ViSNet trained on SPICE2 energies and forces (InstaDeep's visnet-organics checkpoint). That is quantum-chemical pretraining on drug-like molecules, and it beats DeMiR on four of five frozen readouts.
End-to-end (3 seeds), we trained ViSNet (6.7M parameters, our encoder has 6.4M) from scratch on each task.
DeMiR ties ViSNet on pKa, wins on hexadecane and MOF, and loses on FreeSolv. ViSNet ran on a different GPU type, so we do not test the difference. Overall, DeMiR is well above Uni-Mol and Uni-Mol2, below the SPICE2-pretrained ViSNet on frozen read-outs, and level with ViSNet trained end to end; it is also the only one that predicts a density matrix. Tables 1, 2, and 6 all use frozen probes, and the revised captions say so.
Q2. SP2 and auxiliary losses. Each model below continues from the same stage-1 checkpoint for 6,000 iterations.
SP2 raises the atom probe from 0.911 to 0.943 and improves pKa by 0.084 (95% CI [0.027, 0.143]). The auxiliary losses add nothing measurable.
Q2 (second part). Full DM vs. simpler QM properties. Details are in the General response. DM gives more atom-level information and better frozen QM9 probes, while scalar targets transfer better to pKa when both are pretrained on QM9 only.