- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
RiboWeave adapts pretrained RNA-FM representations using secondary- and tertiary-structure supervision at two granularities. Nucleotide-level models predict base pairing and geometric targets; molecule-level models align sequence representations with structure-encoder representations using contrastive learning. A Hybrid model fuses the two jointly supervised sequence encoders for downstream activity prediction and latent-space sequence generation. Structural information is used during upstream training, while downstream inference requires RNA sequences only. Experiments include structural readouts, enrichment-derived pseudo-activity prediction for CD3, cMYC, and RBD, and computational generation evaluated with a separately trained predictor panel. Hybrid improves prediction and sequence novelty on CD3 and cMYC; performance on RBD remains weak, and generated candidates are not experimentally validated.
Critical strengths
Keeping inference sequence-only makes the approach useful for screening datasets without matched structures. The improvements over frozen RNA-FM on CD3 and cMYC show that upstream structural supervision can help in this setting.
The evaluation includes several useful controls: cluster-disjoint partitions, pairing-only and geometry-only hybrids, readout-capacity controls, supervision permutations, and predicted-structure-input comparisons (Tables 13–14). Guidance and final scoring use disjoint partitions, and Tables 16–17 examine sensitivity to the evaluator family.
Critical weaknesses
I am not convinced that the full framework adds enough beyond a simpler structure-supervised model. The main ingredients are supervised structural prediction, sequence–structure InfoNCE, pooling, and feature fusion. The paper does include ablations, but the key comparison is very close: Table 13 gives a CD3/cMYC macro R² of 0.5408 for Hybrid and 0.5393 for the secondary-structure-only Pair-hybrid, a difference of 0.0015. Table 12 also shows that Nuc-pair is already competitive and Nuc-joint does not improve its macro score. The duplicated-feature controls address readout capacity, but a continued-pretraining control using the same upstream sequences and training budget is missing. Variation across downstream seeds also does not show whether the small gain survives independent upstream training runs. These results leave the benefit of combining both structural modalities and both granularities unclear.
The generation experiments provide limited evidence of aptamer function. Table 18 evaluates novelty, diversity, and predicted enrichment-derived pseudo-activity, without measuring binding or functional activity. The disjoint evaluator partition and leave-matched-family-out analysis are useful safeguards. However, the evaluators still share the RNA-FM foundation and screening-label source, so agreement among them may not establish accuracy on novel generated sequences. On CD3, Hybrid's mean predicted score is essentially tied with Nuc-joint (0.8226 versus 0.8228); on cMYC, its Top10 score is slightly lower (0.7959 versus 0.7967). RBD performance is weak, and established aptamer generators are not compared under the same protocol. I therefore see evidence for improved computational surrogate metrics, with much less support for the claimed design utility.
Questions for authors
Does the Hybrid–Pair-hybrid difference in Table 13 persist across independent upstream training runs? How does Hybrid compare with continued RNA-FM pretraining on the same upstream sequences under a matched training budget?
How does final-evaluator accuracy vary with similarity to the evaluator-training partition? Results specifically in the novelty range of the generated candidates would help assess whether their predicted activity scores are reliable.
2 comments
Response by Authors (Part 2/2)
W2 & Q2
The generation experiments provide limited evidence of aptamer function. Table 18 evaluates novelty, diversity, and predicted enrichment-derived pseudo-activity, without measuring binding or functional activity. The disjoint evaluator partition and leave-matched-family-out analysis are useful safeguards. However, the evaluators still share the RNA-FM foundation and screening-label source, so agreement among them may not establish accuracy on novel generated sequences. On CD3, Hybrid's mean predicted score is essentially tied with Nuc-joint (0.8226 versus 0.8228); on cMYC, its Top10 score is slightly lower (0.7959 versus 0.7967). RBD performance is weak, and established aptamer generators are not compared under the same protocol. I therefore see evidence for improved computational surrogate metrics, with much less support for the claimed design utility.
How does final-evaluator accuracy vary with similarity to the evaluator-training partition? Results specifically in the novelty range of the generated candidates would help assess whether their predicted activity scores are reliable.
We agree that our current generation results are computational proxies rather than experimentally validated aptamer function. We added the following analyses.
Evaluator reliability in the novelty range. We stratified natural-holdout sequences by maximum identity to the evaluator-training partition and compared these bins with the generated-candidate distribution. We added the results in Table 19.
| Target | Maximum identity | Generated (%) | Natural test (%) | Natural test [95% CI] |
|---|---|---|---|---|
| CD3 | 13.23 | 27.43 | 0.654 [0.619, 0.687] | |
| CD3 | 55.19 | 50.91 | 0.640 [0.612, 0.667] | |
| CD3 | 26.14 | 18.34 | 0.599 [0.547, 0.646] | |
| CD3 | 5.44 | 3.32 | 0.593 [0.449, 0.708] | |
| cMYC | 24.25 | 26.40 | 0.438 [0.400, 0.474] | |
| cMYC | 54.07 | 51.75 | 0.385 [0.356, 0.411] | |
| cMYC | 18.39 | 18.45 | 0.293 [0.244, 0.339] | |
| cMYC | 3.29 | 3.41 | 0.315 [0.201, 0.410] |
This indicates that evaluator accuracy does not collapse in the sequence-similarity range occupied by most generated candidates, but it does not establish accuracy on the generated distribution itself because generated sequences lack experimental labels.
Comparison with GRAPE-LM. We also retained the GRAPE-LM generation framework and replaced its frozen RNA-FM encoder with RiboWeave encoders, while matching data splits, training settings, and the eight-evaluator panel across five seeds. For brevity, we show the RNA-FM baseline, the best-performing RiboWeave encoder for the reported score metrics, and Hybrid; complete results are reported in Table 25 and 26.
| Target | Encoder | Mean predicted score | Top10 score |
|---|---|---|---|
| CD3 | RNA-FM (GRAPE-LM) | 0.8274 ± 0.0038 | 0.9220 ± 0.0029 |
| CD3 | Nuc-joint | 0.8327 ± 0.0050 | 0.9225 ± 0.0031 |
| CD3 | Hybrid | 0.8308 ± 0.0062 | 0.9221 ± 0.0030 |
| cMYC | RNA-FM (GRAPE-LM) | 0.7025 ± 0.0019 | 0.8052 ± 0.0035 |
| cMYC | Nuc-geo | 0.7096 ± 0.0023 | 0.8094 ± 0.0063 |
| cMYC | Nuc-joint | 0.7087 ± 0.0023 | 0.8133 ± 0.0041 |
| cMYC | Hybrid | 0.7064 ± 0.0025 | 0.8074 ± 0.0026 |
Novelty and diversity remain broadly comparable at the predefined 75% threshold. These results show that structure-informed representations can improve predicted activity within the GRAPE-LM framework, although Hybrid is not uniformly the best representation in this setting. A possible explanation is that GRAPE-LM operates directly on flattened nucleotide-level token embeddings, making it more compatible with Nuc-model.
External computational assessment. Although experimental validation is not available, we additionally evaluated generated candidates with Boltz-2.
We first calibrated Boltz-2 on published aptamers against random and shuffled controls. For cMYC, ipTM distinguished known positives from controls with AUROC 0.951, whereas CD3 was not informative (AUROC 0.380), so we restricted analysis to cMYC.
| Metric | Hybrid Top1 | RNA-FM Top1 | Hybrid Top10 | RNA-FM Top10 |
|---|---|---|---|---|
| ipTM ↑ | 0.7159 ± 0.2143 | 0.6206 ± 0.1596 | 0.6689 ± 0.1095 | 0.6514 ± 0.1120 |
| Interchain PAE (Å) ↓ | 22.73 ± 1.84 | 23.98 ± 1.09 | 23.33 ± 0.83 | 23.30 ± 1.15 |
Hybrid Top1 is numerically better on both ipTM and interchain PAE, although the variance is substantial and we do not claim statistical significance. In the highest-ipTM predicted complexes, the RNA-FM candidate mainly contacts the cMYC surface, whereas the Hybrid candidate forms a loop-like conformation that partially encloses a helical segment of cMYC, resulting in a more interlocked predicted interface (Fig. 9).
We regard Boltz-2 as exploratory computational evidence, not experimental validation of binding affinity or specificity.
Response by Authors (Part 1/2)
We thank the reviewer for the thoughtful and constructive feedback. We appreciate their recognition of the usefulness of sequence-only inference, the benefits of upstream structural supervision over RNA-FM, and the safeguards included in our evaluation. We address the related weaknesses and questions together below.
W1 & Q1
We agree that the original controls did not fully isolate whether the observed gains came from combining structural modalities and supervision granularities, rather than from additional training or a particular upstream checkpoint. To address these concerns, we added three complementary analyses:
Mean differences favor Hybrid in all nine comparisons, but the CIs include zero for all three groups on CD3 and Macro2; on cMYC, they exclude zero for U0 and U2. Overall, these results indicate a modest but consistent mean advantage of Hybrid over Pair-hybrid, although the statistical evidence is not uniform across comparisons.
The paired 95% CIs exclude zero, supporting a benefit from explicit structural supervision beyond continued sequence-only adaptation.
Hybrid performs better than both same-granularity independent-encoder controls, suggesting that its gain is not explained solely by combining two independently adapted encoders.