- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This work introduces RfamFlow, which uses discrete flow matching to conditionally generate RNA families. The core innovation is nested classifier-free guidance, which leverages Rfam’s hierarchical classification information to improve the quality of RNA family generation with small sample sizes. The experiments include RNA sequence generation, mask reconstruction, and zero-shot fitness prediction, with particularly notable improvements observed for families with small sample sizes.
Critical strengths
-
The research problem is valuable. To address the issue of uneven data distribution across RNA families, the use of hierarchical information for conditional generation is a well-designed approach.
-
Ablation experiments support the core method. On small-sample families, Nested CFG improved the bit score from 15.5 to 24.4 compared to Flat CFG, demonstrating the effectiveness of hierarchical guidance.
Critical weaknesses
-
Lack of functional validation. The quality of the generated sequences relies primarily on computational metrics such as cmsearch, which do not sufficiently demonstrate that the generated sequences possess the expected biological functions.
-
Limited generalization ability. The method relies on the known taxonomic hierarchy of Rfam and has not yet been shown to effectively handle RNA from unknown families or RNA lacking reliable taxonomic information.
Questions for authors
-
Is Nested CFG still effective when the RNA family being tested does not appear in the training set?
-
Is it possible to incorporate independent structural predictions or functional assessments to verify that the generated sequences not only exhibit high family similarity but also possess reasonable biological functions?
3 comments
Thanks for your response. I will keep my positive score since the performance of these tricks is good. However, I will not raise my score since the novelty of this work is limited.
Thank you for your response and for your continued support. Your comments on functional validation and unseen families helped us improve this work.
Thank you for recognizing the value of the problem and our use of hierarchical information, and for pointing out the issues. We have uploaded a revised manuscript with new experiments. Both base models are now retrained with the same three-level label input (top, sub, and family) and differ only in how the labels are dropped during training and in the guidance, which makes the comparison between flat and nested clearer. All related tables and figures are recomputed.
1. Functional validation (weakness 1, question 2). We didn't run wet-lab experiments. We provide two assessments that do not rely on cmsearch.
(a) Biophysical features (Section 4.2, Figure 4A-B, Appendix C.3). We fold the generated sequences with RNAfold and compare the per-family GC content and MFE with the natural sequences of the family. Over all families, RfamFlow-large correlates with the natural medians at r >= 0.96 for GC and r >= 0.94 for MFE, whereas RfamGen drops to 0.34 and 0.39 on the small cohort.
(b) Structure of individual sequences (Section 4.2, Figure 4C, Appendix C.3.2). We fold each generated hit and its nearest natural sequence of the same family with RNAfold and compare the two MFE structures by tree-edit similarity. Figure 4C shows five pairs whose sequence similarity is only 0.76-0.84 and whose structure similarity is 0.93-0.97.
2. Unseen families and RNA without a reliable hierarchy (weakness 2, question 1). We added a new experiment on unseen families (Section 4.2, Figure 5, Appendix C.5). We drew 40 random families that have no non-seed sequence, so the model never saw them in training, and generated 100 sequences for each from the top and sub labels alone. For each generated sequence we find its 20 nearest natural seeds in RNA-FM and ERNIE-RNA embedding space and measure the share that have the same top and sub type. Nested guidance raises this share from 45% without guidance to 63% (RNA-FM), and from 36% to 51% (ERNIE-RNA). It rises for 40 and 39 of the 40 families. The cmsearch validity rises from 0.63 to 0.91, and the hit ratio from 11.1% to 14.0%. The flat model with guidance reaches 35% / 28%, a validity of 0.35, and a hit ratio of 8.2%. This holds at the level of RNA sub-types, and we make no claim at the level of the individual family.
Additionally, for RNA lacking a reliable sub-type or family label, we generated from the top label alone, which shows that the coarse level is informative on its own. The 20 nearest natural seeds of the generated sequences share the top type for 62% / 55% of Intron sequences, against 2% of all seeds, and for 71% / 77% of Cis-reg sequences, against 26% (RNA-FM / ERNIE-RNA).
We sincerely hope you could raise your score based on our clarification.