- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper investigates the robustness of 3D molecular prediction models to coordinate perturbations. The authors identify a shift–utility gap: although coordinate perturbations can substantially degrade the overall predictive performance of 3D models, the magnitude of representation shift does not reliably indicate whether a 3D model will outperform a 2D model on an individual sample.
Based on this observation, the authors propose CORE, a dynamic expert-routing framework that combines 2D and 3D molecular predictions with representation-shift descriptors, prediction disagreement, and a bounded 2D-anchored robust pathway.
The authors validate their findings on OpenADMET and six external molecular regression datasets, demonstrating the distinction between representation-shift detection and sample-level predictive utility. Experiments with Uni-Mol and Frad illustrate the vulnerability of 3D models to coordinate perturbations. Additional experiments, including ablation studies and evaluations on alternative conformers, show that CORE improves robustness over static expert fusion under severe coordinate randomization, while offering limited advantages under clean coordinates and more realistic conformer variations.
Critical strengths
- The paper identifies an important distinction between representation shift and sample-level predictive utility. Experiments across multiple molecular datasets show that representation shift can be detected reliably, yet its magnitude does not consistently indicate whether a 3D model outperforms a 2D model for individual molecules.
- The paper provides extensive experiments across multiple datasets, including evaluations with different 3D encoders, ablation studies, and statistical significance tests. The proposed CORE framework also demonstrates statistically significant improvements over static expert fusion under severe coordinate randomization. These experiments lend credibility to the empirical findings, although the practical advantages of CORE under more realistic conformational variations remain limited.
Critical weaknesses
- CORE primarily combines existing techniques, including representation-shift detection and dynamic expert routing, without introducing a substantially new methodological idea. More importantly, the ablation studies show no statistically significant improvement over a simpler two-expert dynamic gating baseline. This raises concerns about whether the additional architectural complexity is necessary or provides meaningful benefits, weakening the paper's methodological contribution.
- Although CORE significantly outperforms static expert fusion under severe coordinate randomization, this perturbation type is included during training and may not reflect realistic molecular conformational variations. As shown in Figure 4, CORE does not significantly outperform static fusion on held-out ETKDG and MMFF conformers. Consequently, it remains unclear under which realistic conditions CORE should be preferred over simpler alternatives and what substantial practical advantages it offers. This limits the significance and applicability of the proposed approach.
Questions for authors
- Could the authors clarify the key advantages of CORE and why practitioners should prefer it over simpler alternatives, such as static expert fusion or a two-expert dynamic gate? Given that CORE shows no statistically significant improvement over the simpler dynamic gate and offers limited advantages on held-out ETKDG/MMFF conformers, what specific deployment scenarios would justify its additional complexity? Which experimental results most strongly support this practical advantage?
2 comments
Thank you for the detailed response and the additional analyses. I appreciate the authors' transparency in distinguishing the benefits of adaptive routing from the incremental contribution of Full CORE.
The new results largely confirm my original concerns. While adaptive routing provides meaningful improvements over static fusion under severe coordinate randomization, Full CORE does not demonstrate a statistically significant advantage over the simpler two-expert gate. Furthermore, under the more realistic held-out ETKDG and MMFF conformer settings, static fusion remains competitive and achieves higher average performance.
I appreciate the authors' willingness to clarify these limitations and narrow their practical recommendations. However, the additional evidence does not substantially address my concerns regarding the methodological novelty and practical necessity of the full framework. The empirical observation about the distinction between representation shift and predictive utility remains valuable, but the added complexity of Full CORE is not sufficiently justified by the reported results.
Therefore, I will maintain my original rating.
Response to Reviewer i7yp
For a practitioner, we would start with the two-expert gate if adaptive routing is needed, and with static fusion if the main objective is accuracy on ETKDG/MMFF conformers. Even under severe randomization, where Full showed its clearest gain over Static, Full achieved a macro-R² of 0.570981 versus 0.569236 for the two-expert gate. The incremental difference was +0.001745 (BH-adjusted q = 0.716582).
We examined this choice further in a paired comparison on identical outer-test molecule–endpoint records. All three ETKDG conformers and the MMFF-lowest conformer were held out from training and model selection.
We also examined the error a practitioner might encounter from an unfavorable conformer. For each molecule–endpoint pair, we took the largest absolute error across its four conformers, then averaged within each endpoint. Against each baseline, Full had a lower mean worst-conformer error on five of nine endpoints. All 18 endpoint-wise intervals included zero. These exploratory intervals use an approximate correction for repeated-CV dependence and do not provide simultaneous coverage across comparisons.
Under the severe randomization condition included in training, Full retained more accuracy than endpoint-specific static fusion (random-coordinate ΔR² = +0.024463, BH-adjusted q = 0.013301). The two-expert gate captured most of that numerical gain. The evidence supports adaptive routing relative to static fusion in this stress test. The direct comparison with the two-expert gate provides insufficient evidence that the third expert adds enough predictive benefit to justify its complexity. We will make that distinction explicit in the discussion and limit our recommendation for Full CORE accordingly.