- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
Cross-family generalization for RNA secondary-structure prediction is challenging and important. This paper introduces ThermoToken, which constructs a thermodynamic response representation from perturbed BPP maps and combines it with RiNALMo features and the current pairing state to iteratively revise secondary structure. The method is elegant and shows some improvements from 0.7193 to 0.7455 in F1 metric.
Critical strengths
-
I like the idea of using thermodynamic sensitivity to enhance the prediction of RNA secondary-structure prediction. Instead of using a single base-pair probability (BPP) map, the method incorporates maps obtained under controlled energy-parameter perturbations. This offers a plausible way to capture complementary information about competing pairing alternatives.
-
The study demonstrates the efficacy of alleviating the gap under family shift. ThermoToken improves over the matched RiNALMo baseline on all nine held-out RNA families, increasing family-macro flexible F1 from 0.7193 to 0.7455.
-
Some other advantages include lightweight adaptation and useful component analysis. The ablations examine thermodynamic inputs and iterative updates separately, while the loop-level analysis connects prediction gains to improved structural organization.
Critical weaknesses
-
The central novelty is not sufficiently isolated. The ablation compares no thermodynamic input against the full nine-channel representation, but lacks a reference-BPP-only baseline. Consequently, it remains unclear whether the gains arise from perturbation responses specifically or simply from adding thermodynamic information.
-
Limited evidence for robustness. The main experiments use a single random seed, without uncertainty estimates or significance testing. This matters particularly for small test families, such as 23S rRNA and Telomerase RNA.
-
Incomplete controlled comparisons. Several competing methods’ results are taken from prior publications rather than reproduced under an identical protocol. The matched RiNALMo comparison is informative, but broader superiority claims require stronger controls.
-
Efficiency is insufficiently characterized. The small number of trainable parameters does not capture the cost of computing nine BPP maps. Runtime and memory measurements across sequence lengths would clarify the practical tradeoff.
-
Last but not least, the evaluation scope is kind of narrow. Evidence centers on Archive-II and one frozen backbone. Additional datasets or backbones would strengthen claims about the general applicability of thermodynamic response features.
Questions for authors
When I look at the related work section, I find that most of the work cited is from before 2025. Are there any more recent papers that tackle this problem or relate to this field?
6 comments
Is performance stable across different training seeds?
We evaluated Full Response and P0-native across three matched training seeds (42, 43, and 44). The nine-family Macro Flexible F1 scores for Full Response are 0.7455, 0.7458, and 0.7422, giving a mean of 0.7445 ± 0.0020 (sample SD). The corresponding P0-native scores are 0.7250, 0.7247, and 0.7265, with a mean of 0.7254 ± 0.0010. Full Response yields a positive paired gain for each seed; the mean gain is +0.0191, with a paired 95% t-confidence interval of [0.0117, 0.0264]. Tables R2–R3 provide the family-wise results and descriptive uncertainty summaries. Seed 42 is the configuration reported as the original main result in the manuscript (Tables S5–S6).
Table R2. Full Response Flexible F1 for each RNA family across training seeds; seed 42 corresponds to the original main result.
| RNA family | Seed 42 | Seed 43 | Seed 44 |
|---|---|---|---|
| 5S | 0.9216 | 0.9201 | 0.9190 |
| 16S | 0.7585 | 0.7602 | 0.7566 |
| 23S | 0.8513 | 0.8502 | 0.8509 |
| Group I | 0.6780 | 0.6785 | 0.6742 |
| SRP | 0.7249 | 0.7265 | 0.7214 |
| tRNA | 0.9501 | 0.9523 | 0.9456 |
| tmRNA | 0.8145 | 0.8150 | 0.8104 |
| Telomerase | 0.1922 | 0.1930 | 0.1876 |
| RNase P | 0.8184 | 0.8167 | 0.8145 |
| Macro F1 | 0.7455 | 0.7458 | 0.7422 |
Table R3. Family-wise means, sample standard deviations, and 95% t-confidence intervals across three seeds (n = 3; reported as descriptive uncertainty summaries).
| RNA family | Mean | Sample SD | 95% t-confidence interval |
|---|---|---|---|
| 5S | 0.9202 | 0.0013 | [0.9170, 0.9235] |
| 16S | 0.7584 | 0.0018 | [0.7538, 0.7630] |
| 23S | 0.8508 | 0.0005 | [0.8495, 0.8521] |
| Group I | 0.6769 | 0.0023 | [0.6710, 0.6827] |
| SRP | 0.7242 | 0.0026 | [0.7177, 0.7308] |
| tRNA | 0.9493 | 0.0034 | [0.9408, 0.9578] |
| tmRNA | 0.8133 | 0.0025 | [0.8070, 0.8195] |
| Telomerase | 0.1909 | 0.0029 | [0.1837, 0.1981] |
| RNase P | 0.8165 | 0.0019 | [0.8117, 0.8214] |
Have more representative RNA secondary-structure prediction baselines been included?
Following your suggestion, we expanded the comparison to include RNAstructure, CONTRAfold, MXfold2, RNAformer, SPOT-RNA, RNA-FM, and UFold. Their nine-family Macro Flexible F1 scores are 0.5850, 0.6322, 0.7282, 0.3840, 0.6036, 0.6130, and 0.3970, respectively, compared with 0.7455 for ThermoToken. Table 1 reports the family-wise results under the common Archive-II test partitions and Flexible F1 scorer. Trainable external baselines use the same family-held-out training and validation protocol, with the target family excluded from both; each method retains its corresponding checkpoint and native decoding procedure (Section 4.1; Table 1).
What are the end-to-end inference time and memory requirements, and where does most of the cost arise?
We profiled inference time and memory across sequence lengths with batch size 1 and caching disabled. The three learned refinement steps introduce only a small additional computational overhead, whereas nine-channel BPP generation accounts for the largest share of inference time and increasingly dominates as sequence length grows. Detailed runtime and memory measurements are reported in Table R4.
Table R4. Inference time and memory measurements across RNA length groups (batch size = 1, no caching, three measurements per RNA).
| Metric | Short | Medium | Long |
|---|---|---|---|
| Mean length (nt) | 98.30 | 198.80 | 377.80 |
| Nine-channel BPP (ms) | 32.84 | 160.44 | 544.64 |
| RiNALMo backbone (ms) | 27.65 | 29.27 | 33.95 |
| Data transfer (ms) | 0.17 | 0.34 | 0.57 |
| ThermoToken refinement (ms) | 10.14 | 12.47 | 15.43 |
| Official decoder (ms) | 3.25 | 14.21 | 46.14 |
| End-to-end latency (ms) | 74.05 | 216.73 | 640.73 |
| Share of time spent on BPP (%) | 43.57 | 72.73 | 82.99 |
| Peak host PSS (MB) | 6140.95 | 6348.49 | 6420.94 |
| Peak host RSS (MB) | 7619.66 | 7848.63 | 7921.27 |
| Peak allocated GPU memory (MB) | 2643.34 | 3245.01 | 4558.07 |
| Peak reserved GPU memory (MB) | 2870.00 | 3466.00 | 5817.00 |
Is the same improvement trend also observed with the RNA-FM backbone?
The RNA-FM experiment provides a second-backbone assessment of the same trend. Native, P0-only, P0×9, and Full Response achieve nine-family Macro Flexible F1 scores of 0.6130, 0.7183, 0.7197, and 0.7213, respectively . Full Response improves over Native by approximately 0.1084 and over P0×9 by approximately 0.0016, showing that thermodynamic-response-conditioned adaptation also improves predictions with RNA-FM representations (Table S7).
How have recent related studies been more fully discussed in the manuscript?
Following your suggestion, we have expanded Sections 2.1–2.2 to incorporate recent studies on structural diversity, dataset design, experimentally sourced evaluation, and hybrid physics–learning approaches. Specifically, Section 2.1 now discusses recent developments: “Recent studies further highlight the importance of structural diversity and dataset design: eFold examines diverse RNA structures (de Lajarte et al., 2026), RNASSTR studies expanded sequence–structure data (Langeberg et al., 2026), and Szikszai et al. (2026) analyze model generalization using experimentally sourced structures.” In Section 2.2, we further added: “MoEFold2D combines learned and physics-based predictors under distribution shift, providing another complementary strategy (Qiu, 2025).” These additions broaden the original literature discussion and better contextualize ThermoToken's focus on thermodynamic-response-conditioned adaptation for cross-family generalization.
Reference
[1]Penić, R. J., Vlašić, T., Huber, R. G., Wan, Y., and Šikić, M. (2025). RiNALMo: General-purpose RNA language models can generalize well on structure prediction tasks. Nature Communications, 16, 5671. DOI: 10.1038/s41467-025-60872-5
[2]de Lajarte, A. A., Martin des Taillades, Y. J., Aruda, J., Bongrand, P., Fuchs Wightman, F., Salazar, D., Allan, M. F., Kalicki, C., L'Esperance-Kerckhoff, C., Kashi, A., Jossinet, F., and Rouskin, S. (2026). Diverse database and machine learning model to narrow the generalization gap in RNA structure prediction. Science Advances, 12(9), eadz4967. DOI: 10.1126/sciadv.adz4967
[3]Langeberg, C. J., Kim, T., Nagle, R., Rajinikanth, A., Meredith, C., Garuadapuri, D. A., Doudna, J. A., and Cate, J. H. D. (2026). Improving RNA secondary structure prediction through expanded training data. RNA, 32(11), 1687–1700. DOI: 10.1261/rna.081259.126
[4]Szikszai, M., Wang, T.-Y., Krueger, R., Mathews, D. H., Ward, M., and Aviran, S. (2026). Deep learning for RNA secondary structure determination: Gauging generalizability and broadening the scope of traditional methods. RNA, 32(4), 428–442. DOI: 10.1261/rna.080846.125
We appreciate your positive assessment of the thermodynamic-sensitivity idea and your suggestions on isolating its contribution, assessing stability, broadening comparisons, and characterizing computational cost. We added controlled ablations, three-seed results, additional baseline evaluations, and end-to-end runtime and memory measurements.
Do thermodynamic perturbation responses provide information beyond the reference BPP?
To isolate the contribution of perturbation responses, we compared Native, P0-native, P0×9, and Full Response with the RiNALMo backbone fixed. P0×9 repeats the reference BPP across nine channels and therefore matches the input-channel count of Full Response. The nine-family Macro Flexible F1 scores are 0.7193, 0.7250, 0.7347, and 0.7455, respectively (Table R1). Full Response improves over the native backbone by 0.0262 and over the channel-matched P0×9 control by 0.0108. This comparison isolates the additional learnable information contributed by the perturbation responses beyond reference BPP and channel count (Table S3).
Table R1. Thermodynamic-response ablation based on RiNALMo.