acceptodds
Under review as a conference paper at ICLR 2027

RNAcoder: Tokenizing RNA Structures with All-Atom Autoencoders

Abstract

Tokenizing protein structures, by reconciling atomic resolution and the efficiency of coarse-grained modeling, has led to notable successes in homology search, protein design and property prediction. Extending tokenization to RNA holds great potential to capture its structural richness despite data scarcity. Yet, existing approaches overlook two key features of RNA: the centrality of base pairs and dinucleotides as building blocks, and the prevalence of chemical modifications. We introduce RNAcoder, a modification-aware, all-atom RNA structure tokenizer. Unlike previous approaches, RNAcoder tokenizes dinucleotides and base pairs alongside single nucleotides, each through a strictly local autoencoder. RNAcoder's learned tokens align with known Leontis-Westhof base pair families and the semantics of chemical modifications, although never trained on these labels. Integrated into coarse-grained graphs to represent whole RNA structures, RNAcoder tokens improve performance on small molecule binding site prediction, RNA-protein interface prediction, and ligand chemical class prediction. Our code is available anonymously at: https://anonymous.4open.science/r/RNAcoder-233E

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
RNA, RNA Structure, Structural biology, Tokenization, Autoencoders, Representation learning, Graph neural networks, Geometric deep learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_RyM9OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This manuscript proposes RNAcoder, a modification-aware framework for learning local all-atom representations of RNA nucleotides, base pairs, and consecutive dinucleotides. The approach uses EquiformerV3 encoders and MLP decoders to reconstruct atomic coordinates and chemical identities in the proposed RNAtom37 layout, producing continuous embeddings that serve as node and edge features in downstream RNA graphs. The authors report accurate local reconstruction and improvements over hand-crafted 2.5D graph features on small-molecule binding-site prediction, protein-interface prediction, and pocket-conditioned ligand-class prediction. Explicitly encoding nucleotide-pair geometry and accommodating chemical modifications are useful RNA-specific design choices, with ablations supporting the contribution of pair-level representations. However, improvements over other pretrained representations are task-dependent, and the downstream evaluations rely on experimentally resolved RNA structures, including protein-bound conformations for interface prediction. These results do not establish whether the learned representations remain useful with predicted structures or across different binding states, leaving their applicability beyond experimentally characterized RNAs unresolved. Overall, the manuscript provides evidence for a useful local geometric representation within structure-conditioned benchmarks, while the practical scope of its broader biological applications requires clearer justification.

Critical strengths

RNA-specific representation of nucleotide-pair geometry

Jointly encoding base pairs and consecutive dinucleotides is a meaningful RNA-specific design choice, allowing inter-nucleotide geometry to be represented explicitly as graph-edge features rather than relying solely on individual nucleotide embeddings. The ablation in Table 3 supports the usefulness of these pair-level representations: removing them reduces the reported AUC by 4.1, 1.7, and 6.2 percentage points on RNA-Site, RNA-Prot, and RNA-Ligand, respectively.

Explicit support for chemical modifications

RNAtom37 provides a common atomic reconstruction layout for standard and supported modified nucleotides, while the training objective explicitly distinguishes modification subtypes. This is a useful representation contribution because it retains chemical distinctions that would be lost by reducing every nucleotide to its canonical A/C/G/U identity.

Evidence of downstream utility

The evaluation goes beyond reconstruction accuracy to include three downstream tasks, comparisons with other pretrained representations, and ablations of the encoder and bottleneck design. The strongest result is on RNA-Prot, where RNAcoder achieves an AUC of 70.5±0.3%70.5\pm0.3\%, compared with 65.2±0.8%65.2\pm0.8\% for RiboSphere and 64.1±0.6%64.1\pm0.6\% for the stronger 2.5D baseline. These results provide evidence that the proposed local features can be useful within the evaluated structure-conditioned setting.

Critical weaknesses

The practical scope is insufficiently established

All three downstream evaluations assume experimentally resolved RNA structures, and RNA-Ligand additionally assumes an identified three-dimensional binding pocket. For RNA-Prot, the interface labels are derived from RNA–protein complexes, so the evaluation concerns identifying contacts from RNA conformations observed in the bound state. Removing protein atoms from the model input does not establish performance on an independently determined unbound RNA conformation. Consequently, the results demonstrate retrospective, structure-conditioned prediction, but do not establish applicability when the relevant RNA conformation is unavailable or changes upon binding. This does not invalidate the task, but it leaves the intended deployment setting and the significance of the broader biological applications insufficiently clear. The authors should identify concrete applications in which the required input structures are available; if applications beyond experimentally characterized conformations are intended, a focused evaluation using predicted structures or matched bound/unbound structures would help substantiate that scope.

The effective biological diversity of the training data remains unclear

Table 5 reports approximately 1.74 million nucleotide training samples extracted from 2,170 RNA units, but these counts alone do not establish the number of independent RNA families or structural environments represented. The tokenizer uses a temporal split rather than a family-based split, while the downstream benchmarks use structural clustering. The reported sampling strategy usefully addresses imbalance among local geometric categories, but does not by itself establish coverage across RNA families and protein-binding contexts. A breakdown of the training data by RNA family and binding context, counting both RNA units and sampled local environments, would make the effective diversity more interpretable. Where sample sizes permit, reporting existing downstream results separately for major RNA groups would also help determine whether the gains extend across biological contexts rather than primarily reflecting recurrent motifs in well-represented groups.

Questions for authors

The benefit of learned tokenization over directly supplied geometry is not fully isolated

The 2.5D baselines use nucleotide identities and discrete edge types, whereas RNAcoder supplies learned features derived from all-atom geometry. The pair-token ablation therefore supports adding pair-level information, but does not fully distinguish the benefit of pretraining from that of providing richer geometric inputs. Likewise, accurate coordinate reconstruction establishes fidelity to the input, but does not alone establish an advantage over directly computed geometric descriptors. A targeted comparison using local torsions, distances, and relative base orientations as features in the same downstream graph framework would clarify the additional value of the learned tokenizer. This would address the representation contribution more directly than adding further unrelated downstream tasks.

Reviewer_twUTOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces RNAcoder, a set of three strictly local, all-atom autoencoders for RNA structure. One encodes single nucleotides, one encodes base pairs and one encodes dinucleotides (consecutive nucleotides). Each subunit is represented as a 5 Å atom radius graph. An SE(3)-invariant EquiformerV3 encoder maps it to a continuous token (d = 32 for residues, d = 64 for pairs), and an MLP decoder reconstructs local-frame coordinates, atom types and residue type.

To make modified nucleotides decodable, the authors introduce RNAtom37, a fixed 37-slot atom nomenclature covering 23 standard and modified residue types, analogous to AlphaFold’s atom37. Training uses PDB non-redundant structures with a time split and word2vec-style subsampling.

Validation comes in three parts:

Reconstruction: sub-0.2 Å RMSD and high residue and modification classification accuracy. Latent analysis: UMAP clustering and a linear probe (99% balanced accuracy) for Leontis-Westhof (LW) families, a linear probe for the isodiscrepancy index (IDI, r = 0.96), separation of evolutionarily aligned base pairs, and embedding arithmetic for 2′-O-methylation. Downstream tasks: the tokens are used as node and edge features in an RGCN on three RNAglib tasks (RNA-Site, RNA-Prot, RNA-Ligand). They beat the hand-crafted 2.5D graph baselines, beat RiboSphere and BiHiTo on RNA-Prot, but trail them on RNA-Site and RNA-Ligand. A BiHiTo-node plus RNAcoder-edge hybrid is the most balanced. Ablations remove the base-pair and dinucleotide tokens, swap the encoder, and compare VAE, VQ-VAE and FSQ bottlenecks.

Critical strengths
  1. RNA-appropriate local tokenization with modification support. Treating base pairs and dinucleotides as first-class tokens matches how RNA structure is organized: pairing and stacking drive folding. The “no bp/dn” ablation supports that these tokens carry downstream signal, for example RNA-Site AUC 74.8 → 78.9 and RNA-Ligand balanced accuracy 47.9 → 54.0. The strictly local objective is also a reasonable answer to RNA data scarcity, compared with whole-structure tokenizers. This is the core reason the paper could be of interest.

  2. RNAtom37 and released code. A canonical, modification-aware atom layout is reusable beyond this paper, for example as a target for structure prediction or generation. Together with the anonymized code and a detailed appendix, it improves reproducibility and gives the work value independent of the downstream numbers.

Critical weaknesses
  • RNAcoder improves over the hand-crafted 2.5D graphs, but against the pretrained tokenizers it wins only on RNA-Prot. It loses to BiHiTo and RiboSphere on RNA-Site (71.0 vs 75.3/74.1 balanced accuracy) and to BiHiTo on RNA-Ligand (54.0 vs 69.2). The abstract and conclusion (“consistently improves”, “significant”) overstate this.
  • No statistical tests are reported, and the claim that variance is “roughly halved” does not hold across tasks.
  • The comparison to 2.5D graphs is confounded by architecture. RNAcoder uses an RGCN with fused edge-feature MLPs, while the 2.5D RGCN uses only relation IDs. A control that feeds LW one-hot or simple continuous geometric descriptors through the same edge-feature pathway is needed. Natural descriptors include the six rigid-body base-pair parameters, backbone torsions, or NtC dinucleotide classes (Černý et al., 2020).
  • Without that control, it is unclear whether the gain comes from learned tokens or simply from richer edge featurization.
  • BiHiTo and the hybrid model are absent from the hyperparameter search (Table 13), so it is unclear they were tuned comparably.
  • LW families are themselves defined deterministically from local geometry (FR3D). Any near-lossless local encoding (RMSD around 0.05 Å) should therefore make them linearly decodable, so the 99% probe mostly confirms reconstruction fidelity rather than emergent biology. LW labels also drove the training subsampling.
  • The nearest-neighbour test is over only four candidates, all of which differ mainly by base identity. That makes success close to guaranteed.
  • The IDI probe lacks a description of how train/test pairs were split. IDI takes a limited set of values per family and identity combination, which raises a memorization concern.
Questions for authors

Do the RNAglib downstream structures (RNA-Site, RNA-Prot, RNA-Ligand test sets) overlap with the structures used to train the tokenizers? If so, what is the performance when they are excluded? In the “no bp/dn” ablation, what replaces the base-pair and backbone edge features: LW relation types, a single relation type, or nothing? Were BiHiTo and the BiHiTo+RNAcoder hybrid tuned with the same search protocol as Table 13?

Reviewer_uJ7UOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes RNAcoder, a set of three strictly local all-atom autoencoders. Each one encodes a single nucleotide, a base pair, or a dinucleotide into a continuous token (d = 32 or 64). The encoder is an invariant EquiformerV3 over a 5 Å atom radius graph. The decoder is an MLP that reconstructs coordinates in a canonical local frame, using a new 37-slot atom layout ("RNAtom37") designed to cover 19 modified nucleotides. The authors report three things:

  • Reconstruction: sub-0.2 Å RMSD.
  • Latent-space analyses: linear decodability of Leontis–Westhof (LW) families, correlation with isostericity (IDI), separation of evolutionarily aligned base pairs, and word2vec-style analogies for 2′-O-methylation.
  • Downstream use: the tokens serve as node and edge features in an RGCN on three RNAglib tasks (RNA-Site, RNA-Prot, RNA-Ligand), replacing nucleotide identity and LW edge labels.
Critical strengths

Strengths

1. Biologically well-motivated tokenization units. The paper is the first learned tokenizer to treat base pairs and dinucleotides (stacking steps) as tokens alongside single nucleotides. This matches the two main forces in RNA folding. The ablation shows that removing these edge tokens hurts all three downstream tasks.

2. Modification awareness via RNAtom37. The 37-slot nomenclature puts standard and 19 modified nucleotides into one fixed tensor layout, which fills a real gap left by existing RNA tokenizers. It is also useful as a standalone resource, for example as a reconstruction target for RNA structure prediction or generation.

Critical weaknesses

weaknesses

1. The downstream claims are overstated, and the key control is missing. The paper claims RNAcoder "consistently improves" performance, but this holds only against the hand-crafted 2.5D graphs.

  • Against existing tokenizers, RNAcoder loses on 2 of 3 tasks. On RNA-Site, BiHiTo reaches 75.3 balanced accuracy vs. 71.0 for RNAcoder. On RNA-Ligand, it is 69.2 vs. 54.0.
  • The ablation never tests the core thesis, that learned base-pair and dinucleotide tokens beat hand-crafted descriptors. Two controls are needed: RNAcoder residue tokens combined with standard Leontis–Westhof edge types, and continuous geometric edge features such as 3DNA base-pair/step parameters or NtC dinucleotide classes. Without them, the gains may come simply from giving the GNN continuous geometry.

2. The interpretability claims lack baselines, and one is inaccurate.

  • The 99% Leontis–Westhof probe needs a baseline. The base-pair tokens are computed from coordinates in a base-centric frame, and the family labels are a deterministic function of that geometry. A probe on raw coordinates or on the relative pose of the two bases would likely score similarly.
  • The abstract says the tokens capture modification semantics without being trained on these labels. In fact, the residue loss explicitly supervises all 19 modification subclasses.
  • The analogy experiment covers only 2′-O-methylation, a sugar modification separate from the base, so additive behavior is almost expected. The nearest-neighbor search is also run over just 4 candidates that are already highly similar to each other.
Questions for authors
  1. How does RNAcoder residue tokens + LW edge types perform, and how do 3DNA or NtC continuous edge features perform?
  2. Were BiHiTo and the hybrid tuned with the same search as the other models?
  3. What do the LW and IDI probes achieve on raw local-frame coordinates or relative-pose features?
  4. Does the analogy result hold for base modifications, and with nearest-neighbour search over all 23 centroids?
  5. How is the single-residue frame defined when P(i+1) or P(i) is missing, and does the encoder ever see P(i+1)?
1 comment

RNAcoder tokenizes base pairs and dinucleotides alongside single nucleotides, which fits RNA’s folding physics well, and RNAtom37 is a useful resource for modified nucleotides in its own right. My main objection was that the submitted ablation did not isolate the paper’s own thesis; the rebuttal fixes this with properly tuned controls against Leontis–Westhof edge types and 3DNA/NtC descriptors, where RNAcoder is ahead on RNA-Site and RNA-Ligand. The compositionality results are also now convincing, with the arithmetic surviving removal of L_atom and extending to base modifications, and the predicted-structure and COVID experiments widen the paper’s relevance considerably. My one remaining concern is RNA-Ligand, which was rebuilt mid-rebuttal with a leak fix and a baseline-input change applied together, leaving a large asymmetric swing unexplained. That is a presentation issue rather than a flaw in the method. I recommend acceptance, conditional on the RNA-Ligand ablation, the revised abstract, and the recalibrated downstream claims appearing in the camera-ready version

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.