acceptodds
Under review as a conference paper at ICLR 2027

RNAStructFold: MSA-free RNA 3D Structure Prediction via RNA Language-Model Attention Priors

Abstract

Accurate RNA three-dimensional structure prediction remains an open problem, and the strongest current methods (e.g., RhoFold+, RoseTTAFold2NA) depend heavily on multiple sequence alignments (MSAs) to reach their best accuracy. MSA construction is computationally expensive and fundamentally limited for RNAs with few or no homologs, such as orphan viral RNAs, orphan non-coding RNAs, and many regulatory riboswitches. RhoFold+'s own ablations report up to a 10 Å RMSD improvement with increasing MSA depth, underscoring this dependence on homolog availability. We introduce RNAStructFold, which replaces the alignment-dependent input branch of RhoFold+ with structural priors derived from a pretrained RNA language model (ERNIE-RNA), treating single-sequence inference as a first-class objective rather than a degraded fallback. Residue embeddings are obtained via learnable layer fusion, and attention maps are projected into pair representations that initialize the Evoformer; a lightweight confidence head additionally predicts global RMSD and TM-score without requiring reference structures at inference. To guard against coevolutionary information re-entering through the pretrained encoder, we filter ERNIE-RNA's pretraining corpus to exclude sequences homologous to our evaluation sets. RNAStructFold is trained across an extended sequence range (16–960 nt), broadening applicability to longer RNAs while eliminating MSA search overhead. On RNA-Puzzles and CASP15, RNAStructFold achieves TM-scores and RMSDs close to RhoFold+ on average, with clear improvements on several individual targets (PZ14, PZ17, PZ18, R1116, R1107), while trailing on a small number of outliers, and yields roughly 2–3× faster inference. On a newly curated OrphanRNA benchmark of low-homology viral RNAs, riboswitches, and long RNA structures, performance is more mixed, with the largest gains on medium- and long-length (>200 nt) targets. These results indicate that language-model-derived structural priors can substitute for a substantial fraction of MSA-derived evolutionary signal, offering a scalable, faster alternative for RNA tertiary structure prediction in low-homology and long-sequence regimes.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
RNA 3D structure prediction+RNA language models+MSA-free structure prediction+structural biology+attention-derived priors+Evoformer+geometric deep learning+confidence prediction

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_9Tu3OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

RNAStructFold replaces the MSA-based input branch of RhoFold+ with ERNIE-RNA residue and attention-derived pair features, enabling MSA-free RNA 3D structure prediction. The method is faster but has lower average structural accuracy than RhoFold+.

Critical strengths
  1. The method provides a straightforward way to integrate RNA language-model residue and pair features into an established folding architecture.

  2. Several individual RNA targets show substantial improvements over RhoFold+.

Critical weaknesses
  1. RNAStructFold has lower mean accuracy than RhoFold+ on all three evaluated benchmarks, including a substantial deficit on the intended low-homology OrphanRNA setting.

  2. The representation contribution is not isolated. Section 4.1 changes the training set from 1,916 representatives to 8,242 structures, while Section 3 simultaneously replaces the input representation. As acknowledged in Section 6, the language model, pair features, training-set composition, and optimization all change together.

  3. The claimed runtime advantage lacks raw timings, hardware specifications, and a reproducible timing protocol.

  4. Replacing MSA with a Language Model is not a novel idea.

Questions for authors

see above

Reviewer_8STVOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents RNAStructFold, an MSA-free RNA 3D structure prediction framework built on RhoFold+. The proposed method replaces the original MSA-based input representations with pretrained ERNIE-RNA features. Specifically, it employs learnable layer fusion for residue representations and attention-derived pair initialization for the folding network. An auxiliary head is introduced to predict global structural quality metrics.

The experiments compare RNAStructFold against RhoFold+ on RNA-Puzzles, CASP15, and OrphanRNA. Although the proposed method reports approximately 2.1–2.7× inference speed-ups and improvements on selected RNA targets, its average structural prediction accuracy is consistently lower than that of RhoFold+.

Critical strengths
  1. An MSA-free RNA 3D structure prediction framework is practically relevant, especially for RNA sequences with limited detectable homologs.

  2. The proposed integration of ERNIE-RNA representations into an established folding architecture is straightforward and potentially useful.

Critical weaknesses

1. Limited methodological novelty

My primary concern is the limited methodological novelty of the proposed approach.

The main contribution consists of replacing the MSA/RNA-FM input representations in RhoFold+ with pretrained ERNIE-RNA hidden states and attention maps. The downstream folding architecture, including the Evoformer, IPA-based structure module, coordinate conversion, and EGNN refinement, is largely inherited from RhoFold+.

Prior work, particularly DRfold2[1], has already demonstrated MSA-free RNA 3D structure prediction using pretrained RNA language models. Although RNAStructFold uses a different language model and feature integration strategy, the manuscript does not sufficiently establish what fundamentally new modeling insight is introduced beyond this existing paradigm.

I therefore consider the methodological contribution incremental, and the paper currently lacks convincing evidence that its specific architectural modifications provide a distinctive advantage.

2. Insufficient comparisons with existing MSA-free methods

A major weakness of the experimental evaluation is the absence of comparisons with relevant existing MSA-free RNA structure prediction methods.

The manuscript primarily compares RNAStructFold against RhoFold+, which relies on alignment-derived information. While this comparison is useful for examining the consequences of removing the MSA input from the original architecture, it is insufficient to establish the competitiveness of RNAStructFold among existing single-sequence prediction approaches.

Given the substantial overlap in the problem setting, a quantitative comparison with DRfold2 and other relevant methods on common benchmark targets is essential. Without such comparisons, it is difficult to assess whether the proposed framework offers any meaningful advantage over established MSA-free alternatives in either structural accuracy or computational efficiency.

3. Limited empirical evidence supporting the proposed approach

The reported results do not convincingly demonstrate the effectiveness of the proposed method.

According to Tables 1–3, RNAStructFold consistently underperforms RhoFold+ in average TM-score:

  • RNA-Puzzles: 0.226 vs. 0.240
  • CASP15: 0.100 vs. 0.134
  • OrphanRNA: 0.067 vs. 0.124

Across all 47 reported evaluation records, the average TM-score decreases from 0.181 to 0.151.

The degradation on OrphanRNA is particularly concerning because the motivation for MSA-free prediction is especially relevant to RNA targets with limited homologous information. On this benchmark, RNAStructFold does not achieve any wins exceeding the stated TM-score comparison threshold. Although several individual RNA targets show substantial improvements, these examples do not establish consistent advantages across diverse targets.

At the same time, the paper lacks essential ablation studies. In particular, the authors do not separately evaluate the contributions of each component. These missing experiments make it difficult to determine whether attention-derived pair features provide meaningful structural information beyond simpler language-model representations.

Reference:

[1] Li Y, Feng C, Zhang X, et al. DRfold2 is a deep learning-based tool that enables efficient and accurate RNA structure prediction[J]. PLoS Biology, 2026, 24(2): e3003659.

Questions for authors
  1. Comparison with current methods: Why are established and explicitly cited MSA-free RNA 3D structure prediction methods not included in the quantitative evaluation? Can the authors provide comparisons on common RNA-Puzzles or CASP targets using consistent evaluation metrics?

  2. Methodological contribution: What is the principal algorithmic novelty beyond connecting ERNIE-RNA representations to the existing RhoFold+ folding architecture? Can the authors demonstrate that attention-derived pair representations provide improvements over simpler sequence-embedding-based alternatives?

  3. Ablation studies: What are the respective contributions of residue features, attention-derived pair features, learnable layer fusion, and the auxiliary prediction head? Are these improvements maintained when the training dataset and computational budget are controlled?

  4. Computational efficiency: Can the authors provide raw runtime measurements, hardware configurations, and an explicit separation of MSA retrieval time from neural network inference time?

Reviewer_u2KjOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper addresses the problem of RNA 3D structure prediction. The authors propose replacing the multiple sequence alignments (MSAs) with representations of a pretrained RNA language model as inputs for structure prediction. In particular, they introduce RNAStructFold, an MSA-free adaptation of the existing RhoFold+ model, in which the MSA and RNA-FM input branch is replaced by features from the pretrained RNA language model ERNIE-RNA. The rest of the architecture is retained from RhoFold+, with the addition of an auxiliary head to predict global RMSD and TM-score. The proposed RNAStructFold is compared to RhoFold+ on RNA-Puzzles, CASP15, and OrphanRNA benchmarks. The results show that the proposed model performs worse overall than the baseline on all three datasets, while obtaining improvements on certain targets and achieving reported inference speed-ups up to 2.7x.

Critical strengths

The paper clearly states the limitations of the following: the approach, reproducibility of the results, and the evaluation setting.

Critical weaknesses

In its current form, I recommend rejection because the paper contains several critical weaknesses: the paper does not provide sufficient details about the training and evaluation settings to understand the results; the originality and contributions are not clear; the experimental evaluation is very limited, comparing only to one baseline among many existing methods; and the results are not convincing, as the proposed method performs worse than the baseline across all evaluated datasets, while only bringing inference speedups without any details about the time-measurement setup. I support my assessment of the paper by elaborating on each point below.

Presentation

The paper is incomplete in its current form. The writing does not state the training budget, model dimensions, pretraining details of the ERNIE-RNA model [4], how the RhoFold+ [1] baseline was run, or the inference time evaluation setup. The authors clearly state in several places that certain descriptions are omitted, yet it is unclear why information about the paper's experiments would be unavailable. Excluding experimental details makes it hard to follow, understand, and assess the contributions of the paper.

Originality, Contributions, and Soundness

First of all, the authors state an MSA-free RNA structure prediction model as a contribution, and they cite existing methods that already have this capability [2,3], but they do not explain the benefits over these methods, nor do they compare them experimentally. Therefore, it is not clear what the proposed method provides with respect to related approaches. Second, "An evaluation of accuracy and computational cost." is presented as a contribution, but the method performs worse than the only baseline on all datasets, and the inference speed-ups of 2.1-2.7x are reported without any details about the measurement. In fact, the authors clearly state that the inference time evaluation does not include raw timing measurements, hardware specifications, caching conditions, or a breakdown of the execution, and that the overall speedup of 2.54x is reported without the aggregation rule specified. Therefore, the message of the experiments is not clear. If the goal was to show a trade-off between performance and inference time, this would require specifying the experimental setting and providing further interpretation of the results. Finally, the proposed RNAStructFold retains the downstream folding and coordinate-refinement modules of RhoFold+ [1] and replaces its MSA/RNA-FM input branch with features from the pretrained ERNIE-RNA model [4]. The authors also add the auxiliary quality prediction head, but they clearly state that it is only described as a part of the architecture and never evaluated in any of the experiments. Changes restricted to the input representation could constitute a contribution if the empirical experiments clearly demonstrated their benefits, which is not the case as argued above. Therefore, the current form only makes minor changes to existing models without showing consistent benefits, making the originality and the novelty of the approach unclear.

Additionally, the experimental evaluation is very limited. The method is only compared to RhoFold+, excluding all other RNA structure prediction methods in the literature, for example [2,3]. Furthermore, the RhoFold+ model provides an ablation without the MSA module in the corresponding paper, and the method is never compared to this alternative. Given that modifying the RhoFold+ model to work without MSAs as input is stated as a core contribution of the paper, it is not clear why this comparison is omitted. Based on the current presentation (which lacks details as explained above), RhoFold+ is never retrained on the training dataset proposed in the paper. Therefore, all the comparisons to the baseline follow from a setting where it is impossible to separate the influence of the model and the training data.

[1] Shen, Tao, et al. "Accurate RNA 3D structure prediction using a language model-based deep learning approach." Nature Methods 21.12 (2024): 2287-2298.

[2] Li, Yang, et al. "Integrating end-to-end learning with deep geometrical potentials for ab initio RNA structure prediction." Nature Communications 14.1 (2023): 5745.

[3] Li, Yang, et al. "DRfold2 is a deep learning-based tool that enables efficient and accurate RNA structure prediction." PLoS Biology 24.2 (2026): e3003659.

[4] Yin, Weijie, et al. "ERNIE-RNA: an RNA language model with structure-enhanced representations." Nature Communications 16.1 (2025): 10076.

Questions for authors
  1. Could you provide details about the experimental setup, especially the inference time measurement?

  2. Could you compare your approach to other MSA-free alternatives? This could include cited methods: RhoFold+ [1] without input MSAs, DRfold [2], and DRfold2 [3]. Importantly, you should clearly state and assess the influence of different training data.

  3. Could you clarify what the benefits of your approach are? For example, you could provide an analysis of the trade-offs between performance and inference time if that is presented as one of the main contributions.

  4. Could you further analyze the experimental results? As of now, the proposed model performs worse overall on all the datasets, while showing improvements only in certain cases. Further analyses should isolate the characteristics of the instances where the model performs better/worse and explain the reasons behind it.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.