Mixed performance relative to LoRA ensembles: The experimental results do not establish that BLL consistently outperforms LoRA ensembles. For eSEN-md LoRA is better on ECE, Spearman, and NLL. For OrbMol LoRA is better across all reported metrics.
The OOD evaluation on OMOL25 is not genuinely OOD for the backbone: The OMol25 foundation backbones are pretrained on the full OMol25 training set, including configurations later designated as OOD-size and OOD-force for the BLL. Furthermore, the BLL are fit using entirely frozen features from these pretrained backbones. The experiments cannot establish how the methods handle genuine OOD data.
Lack of UQ validation in MD or active learning: The motivation emphasizes the importance of reliable UQ for molecular dynamics or active learning, however, the experiments primarily evaluate static force predictions.
Limited applicability to direct force MLIPs: The formulation only applies to direct force prediction architectures which can have problematic MD instabilities.
Inconsistent training protocols of BLL: The experimental setup differs without explanation substantially between MPtrj and OMol25. On MPtrj, the authors optimize the penultimate force projection jointly with the BLL whereas on OMol25 they fit BLL using entirely frozen features.
Fairness of LoRA ensemble comparison: The LoRA ensembles presented as baseline models modify the message-passing layers while BLL only modifies the force head or its final projection. These differences affect the learned representations and force prediction accuracy potentially affecting the UQ comparison. The results when BLL performs better on UQ tasks (eSEN-sm, orb-v3-direct) the presented LoRA models show significant MAE errors. The comparison may reflect differences in underlying force predictors rather than UQ methods alone.
Unclear and inconsistent calibration protocol: It is difficult to understand what was used for the calibration datasets in the experiments. In particular, Table 1 evaluates the BLL models and the ensemble of three pretrained backbones on OMol25 Neutral Val. The ensemble of three pretrained backbones was recalibrated on the id test while Appendix H states that the BLL recalibration parameters are fit on Neutral Val. This indicates that the BLL models are evaluated on the same dataset used for calibration then compared to an ensemble calibrated on a different dataset. This introduces bias in the ECE and NLL comparison and undermines any conclusion about BLL providing better results than the ensemble.
Lack of clarity: The paper has unclear explanation of calibration datasets, evaluation splits, and baselines. There are also grammatical errors and lack of variable definition. These make it difficult to assess the validity of the experiments.
Minor:
L036: missing space
L056: define N
L090: define N
L078: define MCMC acronym
L124: define D
L129: define E, reuse of N
L196: “either part of all of it” did you mean “either part or all of it”?
L205: missing space
L329: space before comma
Should define OOD-Mid and OOD-Far in maintext
Use of OOD-Size/OOD-size/OOD Size
Numbering of equations in the appendix instead of the main text