- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This work addresses zero‑shot RNA‑modification‑site prediction by leveraging molecular side‑information of modified nucleosides. Conventional multi‑task RNA‑modification predictors rely on modification‑specific output heads and cannot score unseen modification types without target‑specific labelled training data. The authors augment the MultiRM benchmark with standardized molecular descriptors (Morgan fingerprints, physicochemical scalars, parent‑base information) for each modified ribonucleoside and design a target‑conditioned compatibility framework. The model uses modification embedding to condition cross‑attention‑based RNA context aggregation as well as final site‑level scoring, avoiding modification‑dedicated output heads, so it can evaluate held‑out modification types purely from their molecular descriptors.
Extensive inductive leave‑one‑modification‑out (LOMO) evaluations are conducted with provenance‑aware leakage prevention. The authors further adopt stricter source‑separated and grouped‑substructure hold‑out protocols, plus external zero‑shot transfer tests on ac4C which is entirely absent from MultiRM. Controlled ablations indicate that molecular substructure fingerprints deliver most transferable gain, while physicochemical scalar features contribute little. Target‑conditioned cross‑attention for RNA context selection brings extra improvements beyond feeding molecular descriptors only into the final scorer. Performance gain correlates positively with chemical similarity between the held‑out modification and training‑set modifications. As target‑specific labelled data increases, the benefit from molecular side‑information diminishes, showing this mechanism is most valuable under zero‑shot or low‑label regimes. Gains are preserved when swapping the BiLSTM encoder for the stronger pretrained AIDO.RNA backbone.
Critical strengths
-
This paper puts forward a practical zero‑shot setting for RNA‑modification‑site prediction: predicting sites for unseen modification types given only their molecular chemical structure, without any site labels of that modification for training. It fills a gap compared with existing multi‑type RNA modification predictors which require target‑specific supervision.
-
Provenance‑aware inductive LOMO is carefully designed to eliminate data leakage; the authors remove samples originating exclusively from the held‑out modification, instead of only masking labels. Multiple hold‑out variants (source‑separated, grouped‑substructure removal) and external real‑world test on ac4C strengthen empirical evidence. Multiple control baselines (parent‑only descriptors, permuted fingerprints, nearest‑target transfer, independent pooling) cleanly isolate contributions of each component.
-
Systematic ablations decompose contributions from molecular substructure fingerprints, physicochemical features, target‑conditioned context aggregation. The paper characterizes how performance gain changes with chemical similarity and with increasing amounts of target‑specific labels. Replacing the RNA encoder backbone verifies that the zero‑shot gain is not an artifact of weak sequence encoders.
Critical weaknesses
-
The macro AUC gain brought by full molecular descriptors over parent‑only baseline is around +0.019 under LOMO setting. Although statistically significant via bootstrap resampling, absolute improvement is limited. Two held‑out modifications (m5C, inosine) even show negative gains. The authors should further discuss practical utility given this modest uplift.
-
When structurally‑related modification groups are fully removed (strict substructure hold‑out), performance gain shrinks to merely +0.003. This suggests the method cannot deliver reliable predictions for completely novel modification families with no related sub‑structures present in training corpus. The practical boundary of generalization should be highlighted more prominently in the main text.
-
It is natural to ask: can off‑the‑box RNA LLMs perform comparable zero‑shot modification‑site prediction via prompting with modification chemical information? The absence of such baseline weakens the demonstration of the proposed specialized framework’s necessity.
-
The work uses fixed RDKit‑computed Morgan fingerprints as molecular representation. It remains untested whether alternative molecular featurization choices change zero‑shot transfer performance.
Questions for authors
-
The macro AUC gain brought by full molecular descriptors over parent‑only baseline is around +0.019 under LOMO setting. Although statistically significant via bootstrap resampling, absolute improvement is limited. Two held‑out modifications (m5C, inosine) even show negative gains. The authors should further discuss practical utility given this modest uplift.
-
When structurally‑related modification groups are fully removed (strict substructure hold‑out), performance gain shrinks to merely +0.003. This suggests the method cannot deliver reliable predictions for completely novel modification families with no related sub‑structures present in training corpus. The practical boundary of generalization should be highlighted more prominently in the main text.
-
It is natural to ask: can off‑the‑box RNA LLMs perform comparable zero‑shot modification‑site prediction via prompting with modification chemical information? The absence of such baseline weakens the demonstration of the proposed specialized framework’s necessity.
-
The work uses fixed RDKit‑computed Morgan fingerprints as molecular representation. It remains untested whether alternative molecular featurization choices change zero‑shot transfer performance.