- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
mdRNAcat is a molecular dynamics dataset for RNA, which contains RNA systems simulated with REST2 enhanced sampling. Each system uses 32 REST2 states and 200-300 ns of simulation per state. The main feature is that it provides not only RNA coordinates, but also instantaneous atomic forces and forces recomputed under the unscaled reference Hamiltonian. The lowest REST2 state is stored every 1 ps, while configs from all REST2 states are stored every 100 ps. This dataset can be a resource for force matching, learned molecular potentials, coarse-grained RNA models, etc.. The dataset was validated through seq and structure-motif coverage of PDB RNA data, vs conventional MD, replica-exchange acceptance & energy overlap, qualitative RNA structural behavior, etc..
Critical strengths
- This dataset fills a gap in RNA machine learning. The RNA dynamics problem is important, especially bridging with the functional RNA design and interactions with other molecules. Most of the current ML models deal with static RNA structures, facing the challenge of data scarcity. This dataset may fill in the RNA dynamics dataset gap, facilitating the further development of dynamics-aware RNA ML models.
- The biophysics perspective of this paper is generally sound (to me). Firstly the dataset contains rich metadata (coordinates, forces, scaled/reference forces, exchange information, structure annotations, etc.). Secondly this dataset use REST2 to reach configs beyond normal 300K sampling and then recompute forces for those configs under the unscaled Hamiltonian, which is useful for training potentials on configs that are rare in ordinary MD. Thirdly, the authors make the effort to diagnose REST2 behavior, including the report of transition probability, energy overlaps, conformational landscapes at different temperature, per-system energy-overlap plots, etc.. The mean transition probability of 0.183 is close to the desired regime for RE. That is useful evidence that the simulations are not obviously broken.
Critical weaknesses
[1] ML utility is not demonstrated.
- There is no enough ML elements. Following my comments in the strengths part, although I think the biophysics is generally grounded and well motivated, there is no model trained on mdRNAcat. There is no baseline, no train/val/test split (even no such recommendation on the split), no seq generalization experiment, no family/structural-level OOD benchmark. As a result, the manuscript demonstrates the generated MD data, but doesn't yet show that the dataset works well for the ML problem that motivate the paper. As in this ML conference, I would expect to see some downstream ML-based/AI4Science style experiments. I think this would also reveal practical problems in the dataset that trajectory-level analysis may not expose. The current manuscript looks more suitable for biophysics or RNA journal like NAR.
[2] Physical validity & coverage of the simulated ensembles are insufficiently established
- The schema defines as the number of RNA atoms, and coordinates and forces are stored as . But the simulations use explicit water and ions. If only RNA coordinates are stored, then two identical RNA conformations can experience different forces because the surrounding water and ions are different.
- The biochemical env may be unrealistic for many RNAs. The source dataset of the simulation is not balanced. Many riboswitches are included (22 out of 48 systems). However, the simulation protocol describes RNA in TIP4P-D water with KCl and doesn't describe retrained , bound ligands, and other biologically important cofactors. This is important because many RNA 3D structures (especially riboswitches, which have different states) strongly depend on , specific ions, bound metabolites, cofactors, etc.. If those components were removed from experimental structures, the resulting ensemble may not correspond to the native biological ensemble. The authors used structural behavior of the trajectories as evidence for dataset validity, and in my opinion the authors should report: 1) which heteroatoms were present in the PDB; 2) which were retained/removed; 3) whether the structure is apo or ligand-bound; 4) whether known structural ions were retained. Otherwise, conformation diversity may come from destabilizing RNA by removing the native env.
- The enhanced-sampling validation is too weak. The main REST2-versus-MD comparison uses one system, 6qn3, and one conventional MD trajectory (with the report that conventional MD samples only 47.6% of the area by that of the REST2 in a 2-dim TICA projection). The problems are: 1) it's only one RNA; 2) only one conventional MD run; 3) “area covered in TICA space” depends strongly on the projection, binning, and definition of occupied area; 4) broader exploration does not demonstrate convergence and that the newly sampled states are physically correct.
- The coverage metric used 3-mer TF-IDF followed by truncated SVD + Euclidean distance, which mainly captures local nucleotide composition, not obviously a strong measure of biological/structural diversity. Similarly, the structural coverage is based on 13 hand-crafted motif descriptors.
Questions for authors
- Any ML elements planned to be included into the manuscript?
- Which atoms are included in the stored coordinates; whether the forces include solvent/ion interactions; whether solvent and ion configurations can be recovered?
- Why is 3-mer TF-IDF an appropriate metric for RNA sequence-space coverage?
- I'm confused on the dataset selection. Why the initial procedure produces 105 candidate structures then only 43 are selected? Why the others are removed? What do the "outside computational scope", "system size out of scope", "difficult to parameterize" mean? Why a lot of data is simply "not prioritized"? Any clarification?
- How do you deal with the , bound metabolites, ions, other heteroatoms, etc. before simulation?
- For the 22 riboswitch systems, were their cognate ligands retained? If not, should these trajectories be interpreted as apo RNA simulations?
- How should users correctly reweight higher-state samples to the reference Hamiltonian?
3 comments
Response to Questions 3 to 6
-
We thank the reviewer for this observation. We agree that 3-mer TF-IDF representations and the selected structural motif descriptors cannot comprehensively characterize the biological and structural diversity of RNA. However, we would like to clarify that these metrics were primarily used to guide the selection and prioritization of additional RNA systems, rather than to establish an exhaustive or optimal measure of RNA sequence and structural coverage. Our objective was to use simple, reproducible descriptors to identify underrepresented regions of sequence-composition and structural-motif space, thereby promoting diversity within the available computational budget. We do not claim that these metrics are uniquely optimal, nor that they capture all biologically relevant relationships between RNA systems. Indeed, mdRNAcat is an evolving resource that has already been expanded since the original submission. Its utility is not dependent on any particular coverage metric: the publicly available structures and trajectories allow users to independently assess the dataset's diversity and suitability for their applications using alternative sequence, structural, or functional representations. We have clarified the intended purpose and limitations of these descriptors in the revised manuscript.
-
We agree with the Reviewer that the original manuscript did not explain the selection sufficiently. We have revised the text to better explain our reasoning. We additionally simulated six further systems and added them to the dataset. Some systems were entirely removed for either of two reasons: too large sections of the RNA not captured in PDB to accurately reconstruct and residue modification that were integral to the structure not parameterized in the used force fields of the project. While it is possible in both cases to reconstruct some similar structure, we are limited in how many systems can be simulated by the amount of available compute resources, which is why those systems were deprioritized. The same is true for larger systems which require more than 32 replicas to simulate with sufficient exchange rates. While it is possible to simulate them, we chose to focus on systems with less compute requirements to sample more different systems.
-
We thank the reviewer for highlighting the importance of biochemical context and experimental design when interpreting the conformational ensembles of any biomolecule. We agree that ligand binding, structural divalent ions, and other cofactors can substantially affect RNA stability and conformational preferences. In particular, some riboswitches adopt different conformational states depending on the presence of their cognate ligands. Our dataset was designed to provide a homogeneous collection of RNA conformational ensembles und a standardized biochemical environment, rather than to reproduce the specific experimental conditions associated with every deposited structure. To this end, the simulations were performed without bound ligands or retained structural divalent ions, using a consistent explicit-solvent KCl solution environement. This choice ensures the maximum consistency across systems for downstream learning tasks, but necessarily limits the interpretation of the resulting ensembles. In particular, the riboswitch trajectories should be interpreted as simulations of the corresponding apo RNA systems under the specified simulation conditions, rather than a representation of their complete native ligand-bound environments. We acknowledge that removing the stabilizing ions or cofactors distort conformational preferences, likely destabilizing experimentally observed structures. However, the correct treatment of structural divalent ions in particular is not trivial and continues to be a subject of frontier research in this area. We opted to build a dataset that would facilitate models learning the intramolecular interactions and non-divalent ions solvent effects as a reasonable first scientific first objective, leaving these important effects for subsequent special treatment.
-
Yes, they are apo RNA simulations, as explained in more detail in the answer to question 5.
Response to Question 7 and remaining Critical Weaknesses
- We thank the reviewer for this question. We would like to distinguish between statistical reweighting of conformational ensembles and the evaluation of molecular forces under the reference Hamiltonian. REST2 is designed to sample the correct equilibrium distribution at the reference thermodynamic state (λ = 1) through replica exchange. Consequently, no additional reweighting is required for equilibrium observables calculated from this state, subject to adequate convergence. Importantly, mdRNAcat already provides two sets of force labels for the sampled configurations: the forces corresponding to their respective REST2-scaled Hamiltonians, and forces independently re-evaluated under the reference Hamiltonian (λ = 1). Thus, configurations from higher thermodynamic states can be used directly with reference-Hamiltonian force labels, without any further force rescaling. We deliberately retain these additional configurations and force evaluations to facilitate applications such as force-based learning, conformational exploration, and out-of-distribution evaluation. They are not, however, automatically distributed according to the reference equilibrium ensemble. If users wish to incorporate higher-state configurations into equilibrium averages, an additional statistical reweighting procedure, such as MBAR, would be required, using the appropriate Hamiltonian energies and sufficient overlap between states. We have clarified this distinction and the interpretation of the provided force labels in the dataset documentation.
Additionally, we thank the reviewer for the observations on critical weakness 4. As discussed in our response to the fist two points of reviewer 1, REST2 is a well-established enhanced-sampling method whose methodological advantages have been extensively investigated and validated in the literature (Wang et al., 2011). While conventional MD efficiently samples local fluctuations within metastable conformational minima, it frequently fails to overcome substantial free-energy barriers, limiting exploration of the global conformational landscape. REST2 is specifically designed to alleviate these limitations. Our comparison was therefore intended to illustrate the enhanced conformational exploration accessible through mdRNAcat, not to independently revalidate REST2 or establish its computational efficiency. We agree that the occupied area in a two-dimensional TICA projection depends on the projection, binning and definition of occupied space. The reported comparison for 6qn3 should therefore be understood as an illustrative example rather than a universal measure of sampling efficiency. To provide a broader characterization beyond this single example, we have additionally included per-system RMSD plots of the reference thermodynamic state (λ = 1) for all RNA systems contained in the dataset in the Appendix. These allow readers to examine the extent of structural deviations sampled across the entire dataset, independently of the particular TICA projection and discretization used in the original comparison. Finally, we emphasize that enhanced conformational exploration, thermodynamic convergence and physical accuracy are distinct questions. Neither the TICA comparison nor the additional RMSD analyses are presented as proof of convergence or experimental validity, which require separate assessments. We have clarified these distinctions in the revised manuscript.
Response to Question 1
We thank the reviewer for the indepth critique and suggestions on our paper and want to answer the questions provided to us, as well as commenenting on the raised weaknesses. The below numbering will reflect the numbering used in the questions and reference the critical weaknesses associated which the question.
Yes. In response to the reviewers' suggestions, we have added a downstream ML application in Section 5.4 and Appendix A.10 of the revised manuscript. We also thank the reviewer for the suggestions raised under Cirtical Weaknesses 1. We agree that the original manuscript would have benefited from an explicit downstream learning experiment with appropriate baselines and an assessment of generalization. We have now addressed these points in Section 5.4 and Appendix A.10 of the revised manuscript. As detailed in our response to Reviewer 1, we evaluated RNA all-atom reconstruction using three established approaches (ABC2A, RAFA and RFdiffusion3NA), supplemented by geometric and training controls. Importantly, we also fine-tuned a pretrained model on mdRNAcat and evaluated its generalization to previously unseen RNA systems, thereby directly addressing the reviewer's concerns about the absence of model training and empirical evidence of dataset utility. To ensure a meaningful evaluation, we selected holdout systems absent from both the fine-tuning data and the model's original training corpus, and fixed the evaluation protocol before examining the holdout results. Of six selected systems, four satisfied the predefined evaluation criteria, while two were reported as attrition rather than replaced. On the four evaluable systems, fine-tuning reduced the reconstruction error to 0.344 Å, compared with 0.555 Å for the original pretrained checkpoint and 0.558 Å for an EMA-matched control without additional training. Regarding train/validation/test splits, mdRNAcat is deliberately released as a general-purpose resource rather than a task-specific benchmark. Different learning objectives require different splitting strategies: for example, within-system conformational learning, generalization to unseen sequences, and transfer across structural families cannot meaningfully be evaluated using the same universal split. We therefore advocate splits appropriate to the intended application, with system-level separation for generalization experiments and explicit precautions against leakage between correlated trajectory frames. We acknowledge that our limited holdout experiment does not establish comprehensive family- or structural-level generalization. Nevertheless, the revised manuscript now provides a concrete ML application with multiple baselines, controlled fine-tuning and an independent holdout evaluation, demonstrating the practical utility of mdRNAcat without conflating a foundational dataset contribution with the development of a comprehensive RNA ML benchmark.
Originally the dataset stored only the coordinates and forces of the RNA atoms in the system. In response to the Reviewers question, we now also added the ions and the solvent as well, so all can be recovered, albeit at a lower frame rate due to storage limitations. Expanding on the question with the points raised in Critical Weaknesses 1: While it is true, that two identical RNA conformations can be subjected to differenct forces with explicit solvent, we disagree with the Reviewer that it is something to avoid. Contrary, we think that it is an important physical property of all-atom explicit water simulations. Any Machine Learning task learning the forces of such a simulation should also learn these properties, no single solute conformation could give the definite forces on the solute. Nonetheless to allow deeper analysis of the forces and wider applications, we added the ion and water coordinates and forces to the replica h5 files and adjusted the script to reflect that change as mentioned above.