- Rating
- 3(+2)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2(+1)
Read review
Paper summary
This paper proposes Pangaea, a framework for automated task-specific 3D molecular descriptor design. It organizes descriptor construction into a Unified Construction Space and uses an LLM-based Incumbent Revision Agent to search for effective pipelines. The method is evaluated on a low-data QM9 setting and an Fe Mössbauer benchmark, where it reports improvements over hand-crafted descriptors, black-box search methods, and several deep-learning baselines. The overall idea is interesting, but I have some concerns about the experimental validity, especially potential target-information leakage in QM9 and insufficiently strong baselines.
Critical strengths
- The decomposition of 3D descriptor construction into Formation, Extraction, and Compression provides a useful structured search space. The LLM-based search over this space is also an interesting direction.
- The paper additionally includes comparisons with black-box optimizers and several ablations of the search strategy.
Critical weaknesses
-
Potential target-information leakage in QM9. Pangaea uses electronic-structure-derived inputs, including DFT-based quantities, while many QM9 targets are themselves quantum-chemical properties. This raises the concern that some prediction targets may already be strongly encoded in the input features. Without carefully ruling this out, it is difficult to attribute the reported gains to the proposed descriptor-search method itself.
-
The baseline comparison is not sufficiently strong. The paper mainly compares against SOAP, LMBTR, GRIND, and several neural models such as SchNet, ViSNet, MolSpectra, and MABNet. Stronger and more recent molecular representation or foundation-model baselines should be included under matched input-information conditions.
Questions for authors
-
Can the authors analyze whether any QM9 prediction target can be directly or approximately recovered from the electronic-structure-derived input features? This is important to rule out information leakage.
-
Can the authors include stronger and more recent molecular property prediction baselines under comparable input conditions?
5 comments
The results show that Pangaea maintains strong predictive performance without electronic-structure inputs. Specifically, Pangaea (struct-search) achieves the best mean rank across all three learners, while Pangaea (full-search, struct-only) ranks second for Ridge and SVR. Moreover, Pangaea (struct-search) achieves an average relative MAE reduction of 23.6%, closely matching that obtained in the full-input setting.
Among the 12 tasks, performance degradation is observed for the task. We therefore further investigate this target's sensitivity to electronic inputs. Specifically, we examine its relationship with DFT-derived features under two settings.
-
Geometric features (G): Geometry, composition, and topology descriptors, along with 12 closed-form functionals covering common statistical aggregations and spatial descriptors (including summary statistics, dipole-like and second-order moments, and the Politzer functional), applied to a geometric control point cloud defined by atomic coordinates and atomic numbers.
-
Electronic features (E): The same 12 closed-form functionals applied to six DFT-derived point clouds, including ADCH charges, four ESP extrema sources, and basin attractors.
The following table summarizes the improvements achieved by adding electronic features (E) to geometric features (G) across two machine-learning models. The results show that is the only target that benefits substantially from electronic features, consistent with its direct reconstruction from ADCH charges and atomic coordinates. In contrast, shows modest, model-dependent improvements, while the remaining ten targets exhibit little to no consistent predictive benefit from electronic features.
\hline \text{Target} & \text{SVR} & \text{XGBoost} \\\\ \hline \mu & +100\\% & +91\\% \\\\ c_{v} & +19\\% & +4\\% \\\\ \text{Other 10 targets} & -52\\%\text{ to }+9\\% & -9\\%\text{ to }+6\\% \\\\ \hline \end{array}$$ For a broader range of molecular property prediction tasks, obtaining target properties often requires additional calculations following DFT geometry optimization. For example, Mössbauer parameters require calculations of the electron density and electric-field gradient at the nucleus [1], whereas NMR chemical shifts require magnetic-shielding calculations [2]. In such cases, DFT-derived electronic-structure information may provide useful molecular descriptors [3]. With this consideration, we include DFT-derived information as an optional descriptor source in *Pangaea*, providing flexibility to explore additional molecular descriptors for a wider range of prediction tasks. [1] Pápai, M.; Vankó, G. On Predicting Mössbauer Parameters of Iron-Containing Molecules with Density-Functional Theory. J. Chem. Theory Comput. 2013, 9 (11), 5004–5020. https://doi.org/10.1021/ct4007585. [2] Willoughby, P. H.; Jansma, M. J.; Hoye, T. R. A Guide to Small-Molecule Structure Assignment through Computation of ¹H and ¹³C NMR Chemical Shifts. Nat. Protoc. 2014, 9 (3), 643–660. https://doi.org/10.1038/nprot.2014.042. [3] Kiryanov, I. I.; Mukminov, F. H.; Tulyabaev, A. R.; Khalilov, L. M. Prediction of ¹³C NMR Chemical Shifts by Artificial Neural Network. I. Partial Charge Model as Atomic Descriptor. Chemom. Intell. Lab. Syst. 2016, 152, 62–68. https://doi.org/10.1016/j.chemolab.2016.01.010.2. Strength of baselines (W2 & Q2)
We understand the reviewer’s concerns regarding the strength and fairness of the baseline comparisons. In the original manuscript, we compare our method with four representative molecular representation models. To strengthen the comparison, we include three additional, more recent and advanced models as baselines under comparable input conditions: DGT (Nature Communications, June 2026), EPT (Nature Communications, February 2026), and C-FREE (ICML, July 2026). We evaluate these models in the small-sample QM9 setting adopted in this work, using the same data splits and labeled samples for downstream tasks as in the original manuscript. For C-FREE and EPT, we fine-tune the models using their official pretrained weights. The results are presented below.
\hline \text{Method} & \mu & \alpha & \varepsilon_{\mathrm{HOMO}} & \varepsilon_{\mathrm{LUMO}} & \Delta\varepsilon & R^{2} & \text{ZPVE} & U_{0} & U & H & G & c_{v} & \text{Mean rank} \\\\ \hline \text{SchNet (NIPS 2017)} & \text{0.515} & \text{0.491} & \text{22.51} & \text{20.13} & \text{26.21} & \text{10.09} & \text{15.71} & \text{1050} & \text{1049} & \text{1050} & \text{1050} & \text{0.228} & \text{6.5} \\\\ \text{ViSNet (Nat. Comm. 2024)} & \text{0.397} & \mathbf{0.298} & \text{10.56} & \text{9.977} & \text{13.91} & \text{99.31} & \text{1.375} & \text{891} & \text{888} & \text{887} & \text{882} & \text{0.137} & \text{3.7} \\\\ \text{MolSpectra (ICLR 2025)} & \text{0.222} & \text{0.608} & \text{17.76} & \text{18.51} & \text{15.21} & \text{13.89} & \text{11.49} & \text{9101} & \text{5379} & \text{4098} & \text{4103} & \text{0.192} & \text{6.3} \\\\ \text{MABNet (ICML 2025)} & \text{0.316} & \text{0.370} & \text{9.008} & \text{11.08} & \text{16.80} & \text{12.28} & \text{2.152} & \text{683} & \text{648} & \text{692} & \text{688} & \text{0.158} & \text{3.8} \\\\ \text{DGT (Nat. Comm. 2026)} & \text{0.504} & \text{0.595} & \text{15.46} & \text{17.09} & \text{17.47} & \text{483.20} & \text{13.680} & \text{7.6e4} & \text{7.5e4} & \text{7.5e4} & \text{7.5e4} & \text{0.268} & \text{7.6} \\\\ \text{EPT (Nat. Comm. 2026)} & \mathbf{0.211} & \text{0.364} & \text{18.76} & \text{18.49} & \text{23.23} & \text{7.60} & \text{17.470} & \text{1343} & \text{1339} & \text{1563} & \text{1475} & \text{0.175} & \text{5.6} \\\\ \text{C-FREE (ICML 2026)} & \text{1.030} & \text{8.560} & \text{11.70} & \text{9.35} & \text{14.30} & \text{121.00} & \text{2.000} & \text{4.6e4} & \text{5.9e4} & \text{5.9e4} & \text{5.4e4} & \text{1.990} & \text{6.9} \\\\ \textit{Pangaea}\text{ (SVR, full-search, struct-only)} & \text{0.791} & \text{0.477} & \text{8.13} & \text{9.22} & \text{12.29} & \text{6.21} & \text{0.448} & \text{240} & \text{241} & \text{245} & \text{241} & \mathbf{0.134} & \text{2.7} \\\\ \textit{Pangaea}\text{ (SVR, struct-search)} & \text{0.752} & \text{0.464} & \mathbf{7.82} & \mathbf{9.15} & \mathbf{11.68} & \mathbf{5.73} & \mathbf{0.420} & \mathbf{199} & \mathbf{199} & \mathbf{199} & \mathbf{200} & \text{0.150} & \mathbf{1.9} \\\\ \hline \end{array}$$ As shown in the table, *Pangaea* achieves lower MAEs than all seven deep-learning baselines on 10 of the 12 tasks in the full-search, struct-only setting and on 9 of the 12 tasks in the struct-search setting. These settings yield the second-best and best mean ranks, respectively, demonstrating *Pangaea*’s effectiveness for molecular property prediction in the small-sample setting. **Manuscript revisions.** We add DGT, EPT, and C-FREE as baselines in Table 1 (p. 7).
1. Potential target-information leakage (W1 & Q1)
We acknowledge the reviewer’s concern that electronic-structure-derived inputs may already encode QM9 target properties, making it difficult to attribute the reported improvements to the descriptor-search method itself.
To assess the contribution of descriptor search without electronic-structure information derived from DFT calculations, we conduct experiments under two settings: Pangaea (full-search, struct-only) searches the full source library and then selects the highest-scoring pipeline that uses only structural inputs and no DFT-derived electronic-structure information, whereas Pangaea (struct-search) restricts the search to structural sources, excluding DFT-derived electronic-structure information from the outset. The two tables below report predictive performance and mean ranks across the 12 tasks, respectively. (LMBTR K2 encodes two-body atomic distances, while K3 additionally captures three-body angular information. GRIND (vdW only) uses only the van der Waals interaction field, likewise excluding explicit electronic-structure information.)
\hline \text{Learner} & \text{Setting} & \mu & \alpha & \varepsilon_{\mathrm{HOMO}} & \varepsilon_{\mathrm{LUMO}} & \Delta\varepsilon & R^{2} & \text{ZPVE} & U_{0} & U & H & G & c_{v} \\\\ \hline \text{XGBoost} & \text{Best baseline} & \mathbf{0.500} & 0.983 & 8.88 & \mathbf{9.66} & 12.54 & 20.36 & 0.943 & \mathbf{1613} & \mathbf{1613} & \mathbf{1613} & \mathbf{1604} & 0.403 \\\\ & \textit{Pangaea}\text{ (full-search, struct-only)} & 0.675 & 0.954 & 9.12 & 9.80 & \mathbf{12.36} & 16.23 & 0.981 & 1673 & 1690 & 1670 & 1667 & 0.423 \\\\ & \textit{Pangaea}\text{ (struct-search)} & 0.679 & \mathbf{0.952} & \mathbf{8.83} & 9.99 & 12.53 & \mathbf{15.81} & \mathbf{0.882} & 1685 & 1668 & 1658 & 1649 & \mathbf{0.396} \\\\ \hline \text{Ridge} & \text{Best baseline} & \mathbf{0.534} & 0.679 & 9.05 & 11.85 & 15.37 & 15.01 & 0.409 & 381 & 381 & 381 & 381 & 0.227 \\\\ & \textit{Pangaea}\text{ (full-search, struct-only)} & 0.771 & 0.630 & \mathbf{8.73} & 12.28 & 15.43 & 8.43 & 0.423 & 269 & 307 & 270 & \mathbf{211} & 0.209 \\\\ & \textit{Pangaea}\text{ (struct-search)} & 0.740 & \mathbf{0.590} & 9.17 & \mathbf{11.81} & \mathbf{14.83} & \mathbf{7.71} & \mathbf{0.379} & \mathbf{211} & \mathbf{211} & \mathbf{211} & \mathbf{211} & \mathbf{0.198} \\\\ \hline \text{SVR} & \text{Best baseline} & \mathbf{0.542} & 0.496 & \mathbf{7.79} & 9.33 & 12.35 & 9.91 & 0.480 & 435 & 435 & 435 & 435 & 0.160 \\\\ & \textit{Pangaea}\text{ (full-search, struct-only)} & 0.791 & 0.477 & 8.13 & 9.22 & 12.29 & 6.21 & 0.448 & 240 & 241 & 245 & 241 & \mathbf{0.134} \\\\ & \textit{Pangaea}\text{ (struct-search)} & 0.752 & \mathbf{0.464} & 7.82 & \mathbf{9.15} & \mathbf{11.68} & \mathbf{5.73} & \mathbf{0.420} & \mathbf{199} & \mathbf{199} & \mathbf{199} & \mathbf{200} & 0.150 \\\\ \hline \end{array}$$ $$\begin{array}{lccc} \hline \text{Setting} & \text{SVR} & \text{Ridge} & \text{XGBoost} \\\\ \hline \text{Topo. \\& Global (no-DFT)} & 5.2 & 5.0 & 5.5 \\\\ \quad \text{+SOAP} & 5.4 & 5.2 & 5.4 \\\\ \quad \text{+LMBTR (K2)} & 4.5 & 4.1 & 3.9 \\\\ \quad \text{+LMBTR (K2+K3)} & 4.0 & 3.6 & 2.7 \\\\ \quad \text{+GRIND (vdW only)} & 5.3 & 6.2 & 5.7 \\\\ \quad \text{+}\textit{Pangaea}\text{ (full-search, struct-only)} & 2.4 & 2.3 & 2.9 \\\\ \quad \text{+}\textit{Pangaea}\text{ (struct-search)} & \mathbf{1.3} & \mathbf{1.6} & \mathbf{1.9} \\\\ \hline \end{array}$$