acceptodds
Under review as a conference paper at ICLR 2027

RiboWeave: Integrating Secondary and Tertiary Structure Knowledge into Sequence Representations for RNA Aptamer Design

Abstract

RNA aptamers are single-stranded nucleic acid molecules that bind targets with high affinity and specificity, with applications in diagnostics, targeted therapeutics, and drug delivery. High-throughput screening yields aptamer candidates, but validating screened sequences is time-consuming and labor-intensive, motivating computational models to accelerate aptamer discovery. Screening data provide a natural source of supervision, yet typically contain only sequences and enrichment measurements without matched structural information. Moreover, modeling RNA-protein interactions requires integrating information across modalities and granularities, as sequence, secondary structure, and tertiary structure contribute at both local and whole-molecule levels. We present RiboWeave, a framework that transfers structural knowledge into sequence representations through nucleotide-level pairing and geometry prediction and molecule-level contrastive alignment with structural representations. Its Hybrid version combines the resulting representations through task-specific fusion, and all RiboWeave models require only RNA sequences at inference. Cross-modal evaluation reveals both shared and modality-specific structural information, while joint supervision balances the two structural modalities. For downstream activity prediction, RiboWeave Hybrid achieves the highest macro across CD3 and cMYC under cluster-disjoint splits, outperforming both single-modality cross-granularity variants and single-granularity capacity controls. In a common generation framework, RiboWeave Hybrid achieves the highest novel-sequence fractions while maintaining strong predicted activity and high sequence diversity across CD3 and cMYC. These results support the utility of multimodal and multigranular structural supervision for sequence-only aptamer activity prediction and computational design.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
RNA Representation Learning, Structural Knowledge Transfer, Multimodal Integration, Multi-Granularity Learning, AI-Driven RNA Aptamer Design

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_BmmhOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

RiboWeave adapts pretrained RNA-FM representations using secondary- and tertiary-structure supervision at two granularities. Nucleotide-level models predict base pairing and geometric targets; molecule-level models align sequence representations with structure-encoder representations using contrastive learning. A Hybrid model fuses the two jointly supervised sequence encoders for downstream activity prediction and latent-space sequence generation. Structural information is used during upstream training, while downstream inference requires RNA sequences only. Experiments include structural readouts, enrichment-derived pseudo-activity prediction for CD3, cMYC, and RBD, and computational generation evaluated with a separately trained predictor panel. Hybrid improves prediction and sequence novelty on CD3 and cMYC; performance on RBD remains weak, and generated candidates are not experimentally validated.

Critical strengths

Keeping inference sequence-only makes the approach useful for screening datasets without matched structures. The improvements over frozen RNA-FM on CD3 and cMYC show that upstream structural supervision can help in this setting.

The evaluation includes several useful controls: cluster-disjoint partitions, pairing-only and geometry-only hybrids, readout-capacity controls, supervision permutations, and predicted-structure-input comparisons (Tables 13–14). Guidance and final scoring use disjoint partitions, and Tables 16–17 examine sensitivity to the evaluator family.

Critical weaknesses

I am not convinced that the full framework adds enough beyond a simpler structure-supervised model. The main ingredients are supervised structural prediction, sequence–structure InfoNCE, pooling, and feature fusion. The paper does include ablations, but the key comparison is very close: Table 13 gives a CD3/cMYC macro R² of 0.5408 for Hybrid and 0.5393 for the secondary-structure-only Pair-hybrid, a difference of 0.0015. Table 12 also shows that Nuc-pair is already competitive and Nuc-joint does not improve its macro score. The duplicated-feature controls address readout capacity, but a continued-pretraining control using the same upstream sequences and training budget is missing. Variation across downstream seeds also does not show whether the small gain survives independent upstream training runs. These results leave the benefit of combining both structural modalities and both granularities unclear.

The generation experiments provide limited evidence of aptamer function. Table 18 evaluates novelty, diversity, and predicted enrichment-derived pseudo-activity, without measuring binding or functional activity. The disjoint evaluator partition and leave-matched-family-out analysis are useful safeguards. However, the evaluators still share the RNA-FM foundation and screening-label source, so agreement among them may not establish accuracy on novel generated sequences. On CD3, Hybrid's mean predicted score is essentially tied with Nuc-joint (0.8226 versus 0.8228); on cMYC, its Top10 score is slightly lower (0.7959 versus 0.7967). RBD performance is weak, and established aptamer generators are not compared under the same protocol. I therefore see evidence for improved computational surrogate metrics, with much less support for the claimed design utility.

Questions for authors

Does the Hybrid–Pair-hybrid difference in Table 13 persist across independent upstream training runs? How does Hybrid compare with continued RNA-FM pretraining on the same upstream sequences under a matched training budget?

How does final-evaluator accuracy vary with similarity to the evaluator-training partition? Results specifically in the novelty range of the generated candidates would help assess whether their predicted activity scores are reliable.

2 comments
AuthorsOpenReview ↗

Response by Authors (Part 1/2)

We thank the reviewer for the thoughtful and constructive feedback. We appreciate their recognition of the usefulness of sequence-only inference, the benefits of upstream structural supervision over RNA-FM, and the safeguards included in our evaluation. We address the related weaknesses and questions together below.

W1 & Q1

I am not convinced that the full framework adds enough beyond a simpler structure-supervised model. The main ingredients are supervised structural prediction, sequence–structure InfoNCE, pooling, and feature fusion. The paper does include ablations, but the key comparison is very close: Table 13 gives a CD3/cMYC macro R² of 0.5408 for Hybrid and 0.5393 for the secondary-structure-only Pair-hybrid, a difference of 0.0015. Table 12 also shows that Nuc-pair is already competitive and Nuc-joint does not improve its macro score. The duplicated-feature controls address readout capacity, but a continued-pretraining control using the same upstream sequences and training budget is missing. Variation across downstream seeds also does not show whether the small gain survives independent upstream training runs. These results leave the benefit of combining both structural modalities and both granularities unclear.

Does the Hybrid–Pair-hybrid difference in Table 13 persist across independent upstream training runs? How does Hybrid compare with continued RNA-FM pretraining on the same upstream sequences under a matched training budget?

We agree that the original controls did not fully isolate whether the observed gains came from combining structural modalities and supervision granularities, rather than from additional training or a particular upstream checkpoint. To address these concerns, we added three complementary analyses:

  1. Hybrid vs. Pair-hybrid across independent upstream runs and alternative splits. We trained three independent upstream model groups (U0, U1, U2) and evaluated each on three cluster-disjoint splits. Fig. 6 reports the mean paired Hybrid–Pair-hybrid difference across splits with nominal 95% tt-based CIs (n=3n=3, df=2df=2):
TaskUpstreamMean paired ΔR2\Delta R^2Nominal 95% tt CI
CD3U0+0.001134[−0.005559, +0.007827]
CD3U1+0.000796[−0.006979, +0.008571]
CD3U2+0.001370[−0.003400, +0.006140]
cMYCU0+0.003695[+0.001999, +0.005391]
cMYCU1+0.000974[−0.003527, +0.005475]
cMYCU2+0.003190[+0.000904, +0.005477]
Macro2U0+0.002414[−0.001683, +0.006511]
Macro2U1+0.000885[−0.001753, +0.003524]
Macro2U2+0.002280[−0.000244, +0.004804]

Mean differences favor Hybrid in all nine comparisons, but the CIs include zero for all three groups on CD3 and Macro2; on cMYC, they exclude zero for U0 and U2. Overall, these results indicate a modest but consistent mean advantage of Hybrid over Pair-hybrid, although the statistical evidence is not uniform across comparisons.

  1. Continued-pretraining control. We used the same upstream RNA streams and seed (20261108), with matched selected epoch budgets (15 epochs for Nuc-joint and 56 for Mol-joint). We visualized the results in Figure 3c.
ComparisonTaskΔR2\Delta R^2Paired 95% CI
Nuc-joint - continued MLMCD3+0.006562[0.002686, 0.010522]
Nuc-joint - continued MLMcMYC+0.008635[0.003039, 0.014280]
Nuc-joint - continued MLMMacro2+0.007599[0.004143, 0.011067]
Mol-joint - continued MLMCD3+0.016240[0.011707, 0.020774]
Mol-joint - continued MLMcMYC+0.018380[0.012152, 0.024799]
Mol-joint - continued MLMMacro2+0.017310[0.013479, 0.021261]

The paired 95% CIs exclude zero, supporting a benefit from explicit structural supervision beyond continued sequence-only adaptation.

  1. Independent capacity controls. To more directly address the capacity concern, we replaced the duplicated-feature controls in the original Fig. 3b and Table 14 with independent-encoder capacity controls. Specifically, Nucleotide-capacity combines two independently trained Nuc-joint encoders (U0+U2), while Molecule-capacity combines two independently trained Mol-joint encoders (U0+U2).
ModelCD3 R2R^2cMYC R2R^2RBD R2R^2Macro2 R2R^2Macro3 R2R^2
Hybrid0.6686 ± 0.00090.4131 ± 0.00170.0070 ± 0.00140.5408 ± 0.00100.3629 ± 0.0005
Independent Nucleotide-capacity (U0+U2)0.6646 ± 0.00090.4115 ± 0.00500.0049 ± 0.00130.5380 ± 0.00270.3603 ± 0.0016
Independent Molecule-capacity (U0+U2)0.6640 ± 0.00070.4061 ± 0.00090.0066 ± 0.00200.5350 ± 0.00050.3589 ± 0.0006

Hybrid performs better than both same-granularity independent-encoder controls, suggesting that its gain is not explained solely by combining two independently adapted encoders.

AuthorsOpenReview ↗

Response by Authors (Part 2/2)

W2 & Q2

The generation experiments provide limited evidence of aptamer function. Table 18 evaluates novelty, diversity, and predicted enrichment-derived pseudo-activity, without measuring binding or functional activity. The disjoint evaluator partition and leave-matched-family-out analysis are useful safeguards. However, the evaluators still share the RNA-FM foundation and screening-label source, so agreement among them may not establish accuracy on novel generated sequences. On CD3, Hybrid's mean predicted score is essentially tied with Nuc-joint (0.8226 versus 0.8228); on cMYC, its Top10 score is slightly lower (0.7959 versus 0.7967). RBD performance is weak, and established aptamer generators are not compared under the same protocol. I therefore see evidence for improved computational surrogate metrics, with much less support for the claimed design utility.

How does final-evaluator accuracy vary with similarity to the evaluator-training partition? Results specifically in the novelty range of the generated candidates would help assess whether their predicted activity scores are reliable.

We agree that our current generation results are computational proxies rather than experimentally validated aptamer function. We added the following analyses.

Evaluator reliability in the novelty range. We stratified natural-holdout sequences by maximum identity to the evaluator-training partition and compared these bins with the generated-candidate distribution. We added the results in Table 19.

TargetMaximum identityGenerated (%)Natural test (%)Natural test R2R^2 [95% CI]
CD3≤60%\leq60\%13.2327.430.654 [0.619, 0.687]
CD3(60,65]%(60,65]\%55.1950.910.640 [0.612, 0.667]
CD3(65,70]%(65,70]\%26.1418.340.599 [0.547, 0.646]
CD3>70%>70\%5.443.320.593 [0.449, 0.708]
cMYC≤60%\leq60\%24.2526.400.438 [0.400, 0.474]
cMYC(60,65]%(60,65]\%54.0751.750.385 [0.356, 0.411]
cMYC(65,70]%(65,70]\%18.3918.450.293 [0.244, 0.339]
cMYC>70%>70\%3.293.410.315 [0.201, 0.410]

This indicates that evaluator accuracy does not collapse in the sequence-similarity range occupied by most generated candidates, but it does not establish accuracy on the generated distribution itself because generated sequences lack experimental labels.

Comparison with GRAPE-LM. We also retained the GRAPE-LM generation framework and replaced its frozen RNA-FM encoder with RiboWeave encoders, while matching data splits, training settings, and the eight-evaluator panel across five seeds. For brevity, we show the RNA-FM baseline, the best-performing RiboWeave encoder for the reported score metrics, and Hybrid; complete results are reported in Table 25 and 26.

TargetEncoderMean predicted scoreTop10 score
CD3RNA-FM (GRAPE-LM)0.8274 ± 0.00380.9220 ± 0.0029
CD3Nuc-joint0.8327 ± 0.00500.9225 ± 0.0031
CD3Hybrid0.8308 ± 0.00620.9221 ± 0.0030
cMYCRNA-FM (GRAPE-LM)0.7025 ± 0.00190.8052 ± 0.0035
cMYCNuc-geo0.7096 ± 0.00230.8094 ± 0.0063
cMYCNuc-joint0.7087 ± 0.00230.8133 ± 0.0041
cMYCHybrid0.7064 ± 0.00250.8074 ± 0.0026

Novelty and diversity remain broadly comparable at the predefined 75% threshold. These results show that structure-informed representations can improve predicted activity within the GRAPE-LM framework, although Hybrid is not uniformly the best representation in this setting. A possible explanation is that GRAPE-LM operates directly on flattened nucleotide-level token embeddings, making it more compatible with Nuc-model.

External computational assessment. Although experimental validation is not available, we additionally evaluated generated candidates with Boltz-2.

We first calibrated Boltz-2 on published aptamers against random and shuffled controls. For cMYC, ipTM distinguished known positives from controls with AUROC 0.951, whereas CD3 was not informative (AUROC 0.380), so we restricted analysis to cMYC.

MetricHybrid Top1RNA-FM Top1Hybrid Top10RNA-FM Top10
ipTM ↑0.7159 ± 0.21430.6206 ± 0.15960.6689 ± 0.10950.6514 ± 0.1120
Interchain PAE (Å) ↓22.73 ± 1.8423.98 ± 1.0923.33 ± 0.8323.30 ± 1.15

Hybrid Top1 is numerically better on both ipTM and interchain PAE, although the variance is substantial and we do not claim statistical significance. In the highest-ipTM predicted complexes, the RNA-FM candidate mainly contacts the cMYC surface, whereas the Hybrid candidate forms a loop-like conformation that partially encloses a helical segment of cMYC, resulting in a more interlocked predicted interface (Fig. 9).

We regard Boltz-2 as exploratory computational evidence, not experimental validation of binding affinity or specificity.

Reviewer_tZu4OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

RiboWeave adapts RNA-FM using secondary- and tertiary-structure supervision through nt-level prediction and molecule-level contrastive alignment. Hybrid model combines 2 representations for sequence-only aptamer prediction and generation. It improves enrichment-score prediction on CD3 and cMYC. Generated sequences show greater novelty and modestly higher predicted scores; experimental activity is untested.

Critical strengths
  • The evaluation includes meaningful safeguards (clustering over variable regions, structural homology filtering, supervision permutations, as well as separate generation and evaluator training partitions).

  • Structural supervision consistently improves several representations over RNA-FM. The generation cross-scoring and evaluator-exclusion analyses also address preferential scoring by matching evaluator families.

Critical weaknesses
  • Limited methodological novelty and incremental benefit from tertiary information. Structural prediction, contrastive alignment, residual adaptation and fusion are established components. S-PLM already transfers structural information into sequence representations through multi-view contrastive learning. RiboWeave contribution is principally their combination for RNA aptamers. The full hybrid improves macro R^2 from 0.5393 to 0.5408 over pair-hybrid (Table 13), leaving the practical value of the added tertiary supervision uncertain.

  • The controls do not fully isolate the source of improvement. Duplicating one representation matches fusion-head capacity but does not match 2 independently adapted encoders. Permuted supervision can damage pre-trained features, so outperforming it does not establish an advantage over ordinary continued pre-training. Reported variability covers downstream seeds with upstream checkpoints fixed. Moreover, molecule-level structural evaluation uses jointly learned structure encoders containing nucleotide identities as references; therefore, it does not independently separate structural information from sequence similarity.

  • The design evidence is indirect. Separate evaluator data and cross-scoring are useful, but all evaluator families derive from RNA-FM and learn the same enrichment labels. Agreement can therefore co-exist with shared errors on novel sequences. Natural-holdout accuracy does not establish reliability in the generated sequence distribution. Without experimental validation (not obligatory) or a direct comparison with GRAPE-LM, the results establish improved computational proxies rather than improved aptamer discovery.

Questions for authors
  • Does hybrid reliably outperform pair-hybrid across independent upstream training runs and alternative cluster splits? Please report paired confidence intervals for this difference.

  • Can the gains be reproduced against continued-MLM adaptation and a fusion of 2 independently trained pairing-supervised encoders with comparable training budgets? What are the additional training and inference costs?

  • Does molecule-level transfer persist with sequence-only structure-encoder controls and independent structural similarity measures, rather than learned reference embeddings?

  • How does the method compare with structRFM for representation learning and GRAPE-LM for generation under the same protocol? Can candidates matched for sequence novelty be tested experimentally for binding and specificity?

3 comments
AuthorsOpenReview ↗

Response by Authors (Part 1/3)

We thank the reviewer for the thoughtful and constructive feedback. We appreciate the reviewer's recognition of our evaluation safeguards, the consistent benefits of structural supervision over RNA-FM, and the evaluator analysis. We address the related weaknesses and questions together below.

W1 & Q1:

Limited methodological novelty and incremental benefit from tertiary information. Structural prediction, contrastive alignment, residual adaptation and fusion are established components. S-PLM already transfers structural information into sequence representations through multi-view contrastive learning. RiboWeave contribution is principally their combination for RNA aptamers. The full hybrid improves macro R2R^2 from 0.5393 to 0.5408 over pair-hybrid (Table 13), leaving the practical value of the added tertiary supervision uncertain.

Does Hybrid reliably outperform Pair-hybrid across independent upstream training runs and alternative cluster splits? Please report paired confidence intervals for this difference.

We agree that structural prediction, contrastive alignment, residual adaptation, and fusion are established techniques, and that S-PLM is an important precedent for sequence–structure contrastive transfer. Our contribution is to systematically investigate secondary- and tertiary-structure supervision at nucleotide and molecule levels, assess their individual and complementary effects, and study their transfer to sequence-only aptamer prediction and generation.

We will expand the related-work discussion, including S-PLM and other structure-aware protein/RNA models, and more clearly distinguish established methodological components from the contribution of our study.

We also agree that the original Hybrid–Pair-hybrid difference is small. We therefore trained three independent upstream model groups (U0, U1, U2) and evaluated each on three cluster-disjoint splits (split seeds 20261108–20261110). The following table reports the mean paired $ \Delta R^2 $, defined as R2R^2(Hybrid) − R2R^2(Pair-hybrid), across splits for each upstream group, with nominal 95% t-based confidence intervals (n = 3, df = 2); Appendix Fig. 6 visualizes the same results.

TaskUpstream configurationMean paired ΔR2\Delta R^2Nominal 95% tt CI
CD3U0+0.001134[−0.005559, +0.007827]
CD3U1+0.000796[−0.006979, +0.008571]
CD3U2+0.001370[−0.003400, +0.006140]
cMYCU0+0.003695[+0.001999, +0.005391]
cMYCU1+0.000974[−0.003527, +0.005475]
cMYCU2+0.003190[+0.000904, +0.005477]
Macro2U0+0.002414[−0.001683, +0.006511]
Macro2U1+0.000885[−0.001753, +0.003524]
Macro2U2+0.002280[−0.000244, +0.004804]

Mean differences favor Hybrid in all nine comparisons. However, the intervals include zero for all three upstream groups on CD3 and Macro2 R2R^2; on cMYC, they exclude zero for U0 and U2.

Overall, these results indicate a modest but consistent mean advantage of Hybrid over Pair-hybrid, although the statistical evidence is not uniform across comparisons. We will revise the corresponding claim accordingly. For reproducibility, the upstream checkpoint epochs are listed in Table 15.

W2 & Q2 & Q3:

The controls do not fully isolate the source of improvement. Duplicating one representation matches fusion-head capacity but does not match 2 independently adapted encoders. Permuted supervision can damage pre-trained features, so outperforming it does not establish an advantage over ordinary continued pre-training. Reported variability covers downstream seeds with upstream checkpoints fixed. Moreover, molecule-level structural evaluation uses jointly learned structure encoders containing nucleotide identities as references; therefore, it does not independently separate structural information from sequence similarity.

Can the gains be reproduced against continued-MLM adaptation and a fusion of 2 independently trained pairing-supervised encoders with comparable training budgets? What are the additional training and inference costs?

Does molecule-level transfer persist with sequence-only structure-encoder controls and independent structural similarity measures, rather than learned reference embeddings?

Thank you for your constructive suggestions. We agree that the original controls do not fully isolate these factors, and we added targeted experiments.

Continued-MLM control. We compared structural supervision with continued-MLM adaptation using the same upstream RNA streams and seed (20261108), with epoch-matched checkpoints: 15 epochs for the nucleotide-level comparison and 56 epochs for the molecule-level comparison. Both methods used backbone-mean representations and the same downstream prediction protocol with five downstream seeds.

AuthorsOpenReview ↗

Response by Authors (Part 2/3)

We report the mean paired difference below and visualize the results in Figure 3c.

ComparisonTaskΔR2\Delta R^2Paired 95% CI
Nuc-joint - continued MLMCD3+0.006562[0.002686, 0.010522]
Nuc-joint - continued MLMcMYC+0.008635[0.003039, 0.014280]
Nuc-joint - continued MLMMacro2+0.007599[0.004143, 0.011067]
Mol-joint - continued MLMCD3+0.016240[0.011707, 0.020774]
Mol-joint - continued MLMcMYC+0.018380[0.012152, 0.024799]
Mol-joint - continued MLMMacro2+0.017310[0.013479, 0.021261]

The 95% CIs are percentile intervals from 10,000 paired test-cluster bootstrap replicates. Within each replicate, we averaged the paired differences across the five fixed downstream seeds and, for Macro2, equally across CD3 and cMYC. These intervals quantify test-set sampling uncertainty conditional on the fitted models.

All six paired 95% CIs exclude zero, supporting benefits from explicit structural supervision beyond additional sequence-only adaptation.

Independent-encoder controls. Both Hybrid and Pair-hybrid fuse two independently trained upstream encoders under the same downstream fusion design. Their robustness across multiple upstream training seeds and alternative cluster-disjoint splits is analyzed in our response to Q1. To more directly address the capacity concern, we replaced the duplicated-feature controls in Fig. 3b and Table 14 with independent-encoder capacity controls. Specifically, Nucleotide-capacity combines two independently trained Nuc-joint encoders (U0+U2), while Molecule-capacity combines two independently trained Mol-joint encoders (U0+U2). All models use the original data split, five downstream seeds, and the same fusion architecture.

ModelCD3 R2R^2cMYC R2R^2RBD R2R^2Macro2 R2R^2Macro3 R2R^2
Hybrid0.6686 ± 0.00090.4131 ± 0.00170.0070 ± 0.00140.5408 ± 0.00100.3629 ± 0.0005
Independent Nucleotide-capacity0.6646 ± 0.00090.4115 ± 0.00500.0049 ± 0.00130.5380 ± 0.00270.3603 ± 0.0016
Independent Molecule-capacity0.6640 ± 0.00070.4061 ± 0.00090.0066 ± 0.00200.5350 ± 0.00050.3589 ± 0.0006

Hybrid achieves higher mean performance than both independent same-granularity capacity controls across CD3, cMYC, Macro2, and Macro3. These controls suggest that Hybrid's gain is not explained solely by using two independently adapted encoders, and support the value of combining representations learned at different supervision granularities.

Additional costs. Relative to RNA-FM’s 99.522M parameters, Nuc-joint and Mol-joint add 4.356M (4.38%) and 6.570M (6.60%) sequence-side parameters, respectively; Hybrid’s final fusion head contains 1.081M parameters. Additional training computation comes primarily from structural supervision. Inference uses only RNA sequences and does not require structure prediction or execution of the structure encoders.

Independent structural evaluation on molecule-level model. We agree that evaluation against jointly learned structure embeddings does not independently establish preservation of structural similarity. We therefore evaluated sequence-derived representations against RNAforester similarity for 2D and US-align TM-score for 3D, and added two complementary controls: continued-MLM adaptation on the same upstream RNAs and Mol-joint-noID, which replaces nucleotide identity features in both structure encoders with constant vectors while retaining structural connectivity and geometry. To reduce computational cost, we used a sampled query–gallery protocol. Details are provided in Appendix.

ModelReadoutRNAforester Spearman (2D)US-align Spearman (3D)
RNA-FMBackbone mean0.24280.1068
Mol-pairTrack$_{2D}$0.29960.1305
Mol-geoTrack$_{3D}$0.29800.1335
Mol-jointTrack$_{2D}$ / Track$_{3D}$0.33790.1412
Mol-joint-noIDTrack$_{2D}$ / Track$_{3D}$0.34960.0689
Continued-MLMBackbone mean0.29360.0905

Mol-pair, Mol-geo, and Mol-joint outperform RNA-FM on both independent structural measures, with Mol-joint ranking highest among the three. Mol-joint also exceeds continued-MLM under the same upstream RNA set and epoch budget, supporting a contribution from structural supervision beyond continued sequence-only adaptation.

Removing nucleotide identity largely preserves the 2D correlation: Mol-joint-noID is comparable to Mol-joint. In contrast, its 3D correlation decreases substantially, suggesting that nucleotide identity plays a more important role in preserving 3D structural rankings under this configuration.

Overall, these results provide evidence that molecule-level structural supervision transfers information related to both 2D and 3D structural similarity into sequence-derived representations.

AuthorsOpenReview ↗

Response by Authors (Part 3/3)

W3 & Q4:

The design evidence is indirect. Separate evaluator data and cross-scoring are useful, but all evaluator families derive from RNA-FM and learn the same enrichment labels. Agreement can therefore co-exist with shared errors on novel sequences. Natural-holdout accuracy does not establish reliability in the generated sequence distribution. Without experimental validation (not obligatory) or a direct comparison with GRAPE-LM, the results establish improved computational proxies rather than improved aptamer discovery.

How does the method compare with structRFM for representation learning and GRAPE-LM for generation under the same protocol? Can candidates matched for sequence novelty be tested experimentally for binding and specificity?

We agree that experimental validation would substantially strengthen the evidence for generated aptamers. Such wet-lab validation is not currently available, so we will clarify that our present design results are computational rather than experimentally validated.

Comparison with structRFM. We added structRFM to the activity-prediction comparison in revised Fig. 3a and Table 13. RiboWeave Hybrid achieves higher performance on CD3, cMYC, Macro2, and Macro3, while both methods remain weak on RBD.

ModelCD3 R2R^2cMYC R2R^2RBD R2R^2Macro2 R2R^2Macro3 R2R^2
RiboWeave Hybrid0.6686 ± 0.00090.4131 ± 0.00170.0070 ± 0.00140.5408 ± 0.00100.3629 ± 0.0005
structRFM0.6510 ± 0.00270.3848 ± 0.00310.0079 ± 0.00100.5179 ± 0.00190.3479 ± 0.0015

Comparison with GRAPE-LM. We retained the GRAPE-LM generation framework and replaced its frozen RNA-FM encoder with RiboWeave encoders, while matching data splits, training settings, and the eight-evaluator panel across five seeds. For brevity, we show below the RNA-FM baseline, the best-performing RiboWeave encoder for the reported score metrics, and Hybrid; complete results are reported in Table 25 and 26.

TargetEncoderMean predicted scoreTop10 score
CD3RNA-FM (GRAPE-LM)0.8274 ± 0.00380.9220 ± 0.0029
CD3Nuc-joint0.8327 ± 0.00500.9225 ± 0.0031
CD3Hybrid0.8308 ± 0.00620.9221 ± 0.0030
cMYCRNA-FM (GRAPE-LM)0.7025 ± 0.00190.8052 ± 0.0035
cMYCNuc-geo0.7096 ± 0.00230.8094 ± 0.0063
cMYCNuc-joint0.7087 ± 0.00230.8133 ± 0.0041
cMYCHybrid0.7064 ± 0.00250.8074 ± 0.0026

Novelty and diversity remain broadly comparable at the predefined 75% threshold. These results show that structure-informed representations can improve predicted activity within the GRAPE-LM framework, although Hybrid is not uniformly the best representation in this setting. A possible explanation is that GRAPE-LM operates directly on flattened nucleotide-level token embeddings, making it more compatible with representations shaped by nucleotide-level structural supervision.

External computational assessment with Boltz-2. As an external computational assessment, we evaluated generated candidates with Boltz-2. We first calibrated Boltz-2 on published aptamers against random and shuffled controls. For cMYC, ipTM distinguished known positives from controls with AUROC 0.951, whereas CD3 was not informative (AUROC 0.380), so we restricted analysis to cMYC.

MetricHybrid Top1RNA-FM Top1Hybrid Top10RNA-FM Top10
ipTM ↑0.7159 ± 0.21430.6206 ± 0.15960.6689 ± 0.10950.6514 ± 0.1120
Interchain PAE (Å) ↓22.73 ± 1.8423.98 ± 1.0923.33 ± 0.8323.30 ± 1.15

Hybrid Top1 is numerically better on both metrics, but variability is substantial and we do not claim statistical significance. In the highest-ipTM predicted complexes, the RNA-FM candidate mainly contacts the cMYC surface, whereas the Hybrid candidate forms a loop-like conformation partially enclosing a helical segment of cMYC, producing a more interlocked predicted interface (Fig. 9). This qualitative difference may suggest a more intimate interaction geometry for the Hybrid Top1 candidate, although it is based solely on predicted structures and should not be interpreted as evidence of stronger binding. At the Top10 level, the two methods are broadly comparable.

We regard Boltz-2 as exploratory computational evidence, not experimental validation of binding affinity or specificity.

Reviewer_LPjTOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

It is beneficial to have generic RNA sequence encoders that provide representations that yield accurate few-shot learning for predicting the outcome of experiments on particular aptamer sequences. On the other hand, there is a limitation of sequence-only modeling to capture certain crucial 3D interactions, etc. This paper provides an approach that is quite attractive: train the encoder using loss functions and output heads that are based on 3D/2D structural data, but use the sequence-only encoder for downstream supervised tasks without relying on structures.

The results show that this provides impressive performance improvements vs. alternative recent RNA LM approaches.

Critical strengths

The paper calls attention to a problem domain that might be unfamiliar to many ICLR readers.

The proposed technique is elegant. It seems very appealing to use these sophisticated structure-based losses only in a 'mid-training' stage before small task-specific fine tuning data is available.

The ablations and results are convincing, modulo my questions below about error bars.

Critical weaknesses

It was challenging for me to understand the degree of novelty of the particular modeling components. I understand that the combination of them is novel, but my impression is that some of these building blocks are well-established in the world of 3D modeling for DNA/RNA/proteins. There should be more citations in section 3.2. I also found it confusing that RNA-FM isn't described in detail, given that you build on top of it.

The evaluation only considers offline settings where train-test splits are made from pre-collected data. It would be much more useful if the model was validated in new prospective design experiments.

I am skeptical of the validity of the error bars in figure 3. It appears that these just correspond to noise from multiple fine-tuning runs, not from statistical estimation variability resulting from a test set of size N. The error bars strike me as very small. If you were to account for these other sources of estimation variability, would the difference between the systems' performance be statistically significant. Similarly, it was confusing to me that there were no error bars in Figure 4.

Questions for authors

Please respond to my 'critical weaknesses' points above.

2 comments
AuthorsOpenReview ↗

Response by Authors (Part 1/2)

We thank the reviewer for the thoughtful and constructive feedback. We appreciate their recognition of the problem setting, the appeal of structural supervision before downstream applications, and the strength of the results. We address questions below.

W1:

It was challenging for me to understand the degree of novelty of the particular modeling components. I understand that the combination of them is novel, but my impression is that some of these building blocks are well-established in the world of 3D modeling for DNA/RNA/proteins. There should be more citations in section 3.2. I also found it confusing that RNA-FM isn't described in detail, given that you build on top of it.

We agree that RiboWeave's novelty does not lie in individual components such as structural prediction or contrastive alignment. Our contribution is to systematically investigate secondary- and tertiary-structure supervision at nucleotide and molecule levels, assess their individual and complementary effects, and test whether the resulting representations transfer to sequence-only aptamer prediction and generation.

We will expand Section 2 to better position RiboWeave relative to prior structure-aware representation learning. Among protein models, S-PLM uses sequence–structure contrastive learning, SaProt and ProSST incorporate discretized structural information through structure-aware or quantized tokens, and PST introduces structural graph modules into a pretrained protein language model. Among RNA models, ERNIE-RNA injects RNA pairing priors into attention, while structRFM uses secondary-structure-guided RNA pretraining. We will distinguish these established components from our contribution and also expand the description of RNA-FM.

W2:

The evaluation only considers offline settings where train-test splits are made from pre-collected data. It would be much more useful if the model was validated in new prospective design experiments.

We interpret 'prospective design experiments' as prospective experimental validation of newly designed aptamers, rather than further retrospective evaluation on pre-collected datasets. We agree that synthesizing generated candidates and experimentally measuring binding activity and specificity would provide stronger evidence of design utility. However, such wet-lab validation is not yet available.

As an external computational assessment, we evaluated generated candidates with Boltz-2. We first calibrated Boltz-2 on published aptamers against random and shuffled controls. For cMYC, ipTM distinguished known positives from controls with AUROC 0.951, whereas CD3 was not informative (AUROC 0.380),so we restricted analysis to cMYC.

MetricHybrid Top1RNA-FM Top1Hybrid Top10RNA-FM Top10
ipTM ↑0.7159 ± 0.21430.6206 ± 0.15960.6689 ± 0.10950.6514 ± 0.1120
Interchain PAE (Å) ↓22.73 ± 1.8423.98 ± 1.0923.33 ± 0.8323.30 ± 1.15

Hybrid Top1 is numerically better on both metrics, but variability is substantial and we do not claim statistical significance. In the highest-ipTM predicted complexes, the RNA-FM candidate mainly contacts the cMYC surface, whereas the Hybrid candidate forms a loop-like conformation partially enclosing a helical segment of cMYC, producing a more interlocked predicted interface (Fig.9). This qualitative difference may suggest a more intimate interaction geometry for the Hybrid Top1 candidate, although it is based solely on predicted structures and should not be interpreted as evidence of stronger binding. At the top-10 level, the two methods are broadly comparable.

We regard Boltz-2 as exploratory computational evidence, not prospective validation of binding affinity or specificity. We will report these results in the appendix and retain experimental validation as an important next step.

AuthorsOpenReview ↗

Response by Authors (Part 2/2)

W3:

I am skeptical of the validity of the error bars in figure 3. It appears that these just correspond to noise from multiple fine-tuning runs, not from statistical estimation variability resulting from a test set of size N. The error bars strike me as very small. If you were to account for these other sources of estimation variability, would the difference between the systems' performance be statistically significant. Similarly, it was confusing to me that there were no error bars in Figure 4.

We agree that the original Fig. 3 error bars capture downstream-training variability, not finite-test-set uncertainty. We therefore performed 10,000 paired test-cluster bootstrap resamples, retaining all sequences within each sampled cluster and sharing each resample across methods and five fixed downstream seeds. We recomputed each seed's R2R^2, averaged the paired differences, and used the 2.5th/97.5th percentiles as nominal 95% CIs.

Fig. 7 and the table report ΔR2=RHybrid2−Rcomparator2\Delta R^2=R^2_{\mathrm{Hybrid}}-R^2_{\mathrm{comparator}} with paired test-cluster 95% CIs. We replaced the original duplicated-representation capacity controls with fusions of two independently trained encoders (U0+U2) at the nucleotide and molecule levels, and added structRFM as an additional baseline. Macro2 equally averages the CD3 and cMYC differences within each replicate.

ComparatorCD3 ΔR2\Delta R^2 [95% CI]cMYC ΔR2\Delta R^2 [95% CI]Macro2 ΔR2\Delta R^2 [95% CI]
RNA-FM0.015207 [0.011063, 0.019436]0.014487 [0.007917, 0.020982]0.014847 [0.011041, 0.018735]
ERNIE-RNA0.014873 [0.008080, 0.021665]0.017205 [0.007405, 0.026770]0.016039 [0.010093, 0.021903]
RNAErnie0.013787 [0.006929, 0.020933]0.028158 [0.017957, 0.038352]0.020972 [0.014816, 0.027181]
RiNALMo-Mega0.015087 [0.008029, 0.022145]0.023474 [0.013294, 0.033524]0.019281 [0.013098, 0.025464]
structRFM0.017578 [0.010642, 0.024534]0.028243 [0.017695, 0.038854]0.022911 [0.016561, 0.029292]
Nuc-pair0.005933 [0.001661, 0.010207]0.003028 [-0.003676, 0.009633]0.004481 [0.000553, 0.008487]
Nuc-geo0.008450 [0.004358, 0.012622]0.004469 [-0.002189, 0.011089]0.006459 [0.002595, 0.010275]
Nuc-joint0.005614 [0.001954, 0.009239]0.004249 [-0.001162, 0.009660]0.004932 [0.001707, 0.008206]
Mol-pair0.008383 [0.004617, 0.012252]0.010042 [0.003981, 0.016189]0.009213 [0.005721, 0.012822]
Mol-geo0.015778 [0.011186, 0.020312]0.023314 [0.016210, 0.030483]0.019546 [0.015310, 0.023692]
Mol-joint0.004869 [0.002247, 0.007533]0.009536 [0.005304, 0.013693]0.007203 [0.004720, 0.009620]
Pair-hybrid0.002104 [-0.000948, 0.005220]0.001069 [-0.004025, 0.006242]0.001587 [-0.001355, 0.004649]
Geo-hybrid0.006023 [0.002706, 0.009415]0.003797 [-0.001606, 0.009229]0.004910 [0.001784, 0.008017]
Independent nucleotide capacity0.004066 [0.000593, 0.007502]0.001590 [-0.003837, 0.007017]0.002828 [-0.000408, 0.005998]
Independent molecule capacity0.004624 [0.002089, 0.007175]0.006971 [0.002891, 0.010932]0.005797 [0.003407, 0.008093]

After accounting for finite-test-set uncertainty, Hybrid remains significantly better than all language-model baselines and most structure-supervised models on Macro2, while the 95% CIs for Pair-hybrid and the independent nucleotide-capacity control include zero.

For Fig. 4, we added mean ± SD across five independent generation seeds, reported in Table 22, to show generation-run variability.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank all reviewers for their thoughtful and constructive feedback. We appreciate their recognition of the importance of sequence-only RNA modeling, the benefits of upstream structural supervision, and the safeguards in our evaluation. Based on the reviews, we clarified the scope of our contribution, strengthened the controls, and revised several claims.

Contribution and positioning. We position RiboWeave as a study of secondary- and tertiary-structure supervision at nucleotide and molecule levels, their individual and complementary effects, and their transfer to sequence-only aptamer prediction and generation. We also expanded related work on structure-aware protein models (S-PLM, SaProt, ProSST, PST) and RNA models (ERNIE-RNA, structRFM), and added a clearer description of RNA-FM.

Robustness and stronger controls. We evaluated Hybrid and Pair-hybrid across three independent upstream model groups and three alternative cluster-disjoint splits. Hybrid shows positive mean gains in all nine task–upstream-group comparisons; cMYC gains are statistically resolved for U0 and U2, while Macro2 confidence intervals include zero. We therefore interpret the Hybrid advantage as modest rather than uniformly significant.

To address finite-test-set uncertainty, we performed 10,000 paired test-cluster bootstrap resamples and report 95% CIs for ΔR2=RHybrid2−Rcomparator2\Delta R^2=R^2_{\mathrm{Hybrid}}-R^2_{\mathrm{comparator}}. On Macro2, Hybrid remains significantly better than all language-model baselines and most structure-supervised models, while the 95% CIs for Pair-hybrid and the independent nucleotide-capacity control include zero. We also added mean ± SD across five generation seeds to Fig. 4.

We further added matched continued-MLM controls using the same upstream RNA streams. Nuc-joint improves Macro2 by +0.007599+0.007599 (95% CI [0.004143,0.011067][0.004143,0.011067]) and Mol-joint by +0.017310+0.017310 (95% CI [0.013479,0.021261][0.013479,0.021261]), supporting a benefit from explicit structural supervision beyond continued sequence-only adaptation. We also replaced duplicated-feature capacity controls with independent-encoder controls; Hybrid remains above both same-granularity dual-encoder controls.

Independent structural evaluation. We added RNAforester-based 2D and US-align-based 3D similarity evaluations, together with continued-MLM and Mol-joint-noID controls. Mol-pair, Mol-geo, and Mol-joint all exceed RNA-FM on both independent structural measures, with Mol-joint achieving the highest correlations among these three models. Mol-joint also outperforms continued-MLM on both measures. Removing nucleotide identity largely preserves 2D correlation but substantially reduces 3D correlation.

Additional baselines and generation analysis. We added structRFM to activity prediction; Hybrid achieves Macro2 R2=0.5408±0.0010R^2=0.5408\pm0.0010 versus 0.5179±0.00190.5179\pm0.0019 for structRFM. We also evaluated RiboWeave representations within the GRAPE-LM generation framework under matched splits, training settings, and evaluator panel. Structure-informed representations improve predicted activity, although Hybrid is not uniformly best; nucleotide-level representations are particularly competitive in GRAPE-LM.

We also stratified evaluator reliability by sequence identity to the evaluator-training set. Most generated candidates lie at ≤70%\leq70\% identity, where natural-holdout R2R^2 remains positive for both CD3 and cMYC. This supports evaluator reliability in the relevant similarity range, while not establishing accuracy on the generated distribution itself.

External computational assessment. We agree that prospective binding/specificity experiments would substantially strengthen the design claims. Such wet-lab validation is not currently available, so we now clearly describe the generation results as computational evidence. As an external computational assessment, we calibrated Boltz-2 on published aptamers and controls (cMYC ipTM AUROC 0.951; CD3 0.380) and restricted generated-candidate analysis to cMYC. Hybrid Top1 is numerically better than RNA-FM Top1 in ipTM and interchain PAE, but variability is substantial and Top10 results are broadly comparable. In the highest-ipTM predicted complexes, the Hybrid Top1 forms a more interlocked interface with cMYC than the RNA-FM Top1, suggesting a more intimate predicted interaction geometry. We therefore treat Boltz-2 only as exploratory computational evidence.

Overall, the additional analyses and controls substantially strengthen the empirical support for RiboWeave and improve the rigor and completeness of the revised manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.