- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
Modern RNA track is dominated by sequence/static-structure supervision, while functional RNA behavior is inherently ensemble- and dynamics-dependent. RiboMotion addresses a real gap by providing roughly 100$\mu$s of explicit-solvent MD across 1,039 RNA systems, with 6 downstream tasks, and the demo that multi-frame MD supervision can improve an RNA conformation generator.
Critical strengths
- The RNA dynamics problem is important to bridge to the functional design, and RiboMotion directly addresses that issue.
- RiboMotion have tried broader scales including different simulation regimes, different distinct systems, etc.; The data splitting well noted the similarity issues & temporal orders; the Protenix experiment is compelling with careful experiments controls.
Critical weaknesses
-
The scientific meaning of the benchmark depends strongly on the reliability and sampling quality of the MD trajectories, but this is not validated sufficiently for a benchmark of RNA dynamics. Most systems are represented by only one approximately 80–100 ns trajectory. Independent simulations are available for only nine Core RNAs, and there is no independent-replica panel for the 309 RNA–ligand systems. Experimental validation is also very limited: it mainly consists of one UUCG construct, for which the source NMR structures agree better with the eNOE measurements than the MD ensemble. The chosen RNA force field is widely used, but RNA force fields, including ff99+bsc0+χOL3, have known limitations for some conformations and interactions. The paper itself also acknowledges that the simulated ensembles depend on the force field and simulation conditions. This issue is consequential because these simulations define the supervision and the ground-truth targets. Many biologically relevant RNA transitions can occur on timescales longer than the simulated windows. Therefore, it is unclear whether the models are learning transferable RNA conformational dynamics or mainly reproducing finite-window observables generated by this particular force field and simulation protocol. In my view, this distinction is fundamental for a benchmark whose main claim is to enable learning of RNA dynamics.
-
Several benchmark design choices make the reported generalization difficult to interpret. For the Core distance task, entry into the evaluation population depends on diagnostics computed from the target trajectory itself, including half-window RMSF agreement, contact-occupancy agreement, and pair-distance consistency. Thus, the primary benchmark evaluates models on a population that has been retrospectively selected using the answer trajectories. I understand the motivation of removing poorly sampled targets, but this creates a target-conditioned evaluation population rather than a fully prospective benchmark. This is especially difficult to justify because Appendix F.5 already shows results on all 97 technically valid test RNAs. The RNA–ligand split also does not enforce ligand-scaffold separation. Therefore, the reported test performance does not clearly establish generalization to unseen ligand chemotypes. In addition, the primary hydrogen-bond MacroAP is unconventional: systems without positive events receive AP = 0, giving a maximum score of only 20/39 on the test set, while false positives inside these systems do not affect their AP. This makes the absolute value and model ranking difficult to interpret, especially because the supplementary system-weighted AP and Brier metrics give a different view of performance. Together, these design choices make it difficult to determine how well the benchmark measures the intended out-of-distribution prediction of RNA dynamics.
Questions for authors
- Clarification / benchmark design. Why are trajectory-consistency criteria used to determine entry into the primary Core distance benchmark rather than only to define diagnostic strata? Given the results in Appendix F.5, what is the reason for not using all 97 technically valid test RNAs as the primary evaluation population?
- Additional evaluation that would affect my assessment. Can the authors report a ligand-scaffold-disjoint evaluation while also preserving RNA-level separation? In particular, how much of the RNA–ligand performance remains when the test ligands have unseen chemical scaffolds?
- Sampling uncertainty. How much of the reported benchmark error can be attributed to finite-trajectory sampling variability? The independent-replica analysis on nine RNAs is useful but small, and no equivalent replica analysis is available for the ligand cohort. Can the authors provide a broader replica analysis, or otherwise quantify the sampling uncertainty relative to the model improvements?
- Event-label robustness. How sensitive are the stacking- and hydrogen-bond-disruption results to the chosen persistence threshold? For example, do the relative model rankings remain similar when the persistence criterion is approximately 0.5 ns, 1 ns, and 2 ns?
- Ligand trajectory integrity. How many RNA–ligand systems are affected by the whole-ligand translation correction described in Appendix A.3, and how much do the ligand-distance results change if these systems are excluded? The manuscript states that the available exports do not contain valid periodic boxes and that physical validation of these corrections would require the original boxed trajectories