acceptodds
Under review as a conference paper at ICLR 2027

RfamFlow: Nested Guidance for RNA Family Generation with Discrete Flow Matching

Abstract

Designing RNA sequences that belong to a target family is central to RNA engineering. Existing generators either train a separate model per family, failing when a family has few known sequences, or condition a single autoregressive model on the family label, still degrading on the many small and medium families of the highly imbalanced Rfam database. We present RfamFlow, a conditional discrete flow-matching model that generates RNA sequences for any Rfam family from its tag alone, with no structural input. RfamFlow couples a Diffusion Transformer (DiT) with a nested classifier-free guidance scheme that follows the Rfam taxonomy, allowing a data-poor family to borrow a prior from its coarser taxonomic levels. The single trained model supports de novo generation, masked reconstruction, and zero-shot fitness prediction without task-specific fine-tuning. RfamFlow outperforms the comparable family-conditioned baseline models across large, medium, and small families, with its nested guidance helping most where family data are scarcest. Furthermore, it matches far larger RNA foundation models on reconstruction and fitness prediction.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
RNA design, discrete flow matching, family-conditioned generation, hierarchical guidance

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_ppsAOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work introduces RfamFlow, which uses discrete flow matching to conditionally generate RNA families. The core innovation is nested classifier-free guidance, which leverages Rfam’s hierarchical classification information to improve the quality of RNA family generation with small sample sizes. The experiments include RNA sequence generation, mask reconstruction, and zero-shot fitness prediction, with particularly notable improvements observed for families with small sample sizes.

Critical strengths
  1. The research problem is valuable. To address the issue of uneven data distribution across RNA families, the use of hierarchical information for conditional generation is a well-designed approach.

  2. Ablation experiments support the core method. On small-sample families, Nested CFG improved the bit score from 15.5 to 24.4 compared to Flat CFG, demonstrating the effectiveness of hierarchical guidance.

Critical weaknesses
  1. Lack of functional validation. The quality of the generated sequences relies primarily on computational metrics such as cmsearch, which do not sufficiently demonstrate that the generated sequences possess the expected biological functions.

  2. Limited generalization ability. The method relies on the known taxonomic hierarchy of Rfam and has not yet been shown to effectively handle RNA from unknown families or RNA lacking reliable taxonomic information.

Questions for authors
  1. Is Nested CFG still effective when the RNA family being tested does not appear in the training set?

  2. Is it possible to incorporate independent structural predictions or functional assessments to verify that the generated sequences not only exhibit high family similarity but also possess reasonable biological functions?

3 comments

Thank you for recognizing the value of the problem and our use of hierarchical information, and for pointing out the issues. We have uploaded a revised manuscript with new experiments. Both base models are now retrained with the same three-level label input (top, sub, and family) and differ only in how the labels are dropped during training and in the guidance, which makes the comparison between flat and nested clearer. All related tables and figures are recomputed.

1. Functional validation (weakness 1, question 2). We didn't run wet-lab experiments. We provide two assessments that do not rely on cmsearch.

(a) Biophysical features (Section 4.2, Figure 4A-B, Appendix C.3). We fold the generated sequences with RNAfold and compare the per-family GC content and MFE with the natural sequences of the family. Over all families, RfamFlow-large correlates with the natural medians at r >= 0.96 for GC and r >= 0.94 for MFE, whereas RfamGen drops to 0.34 and 0.39 on the small cohort.

(b) Structure of individual sequences (Section 4.2, Figure 4C, Appendix C.3.2). We fold each generated hit and its nearest natural sequence of the same family with RNAfold and compare the two MFE structures by tree-edit similarity. Figure 4C shows five pairs whose sequence similarity is only 0.76-0.84 and whose structure similarity is 0.93-0.97.

2. Unseen families and RNA without a reliable hierarchy (weakness 2, question 1). We added a new experiment on unseen families (Section 4.2, Figure 5, Appendix C.5). We drew 40 random families that have no non-seed sequence, so the model never saw them in training, and generated 100 sequences for each from the top and sub labels alone. For each generated sequence we find its 20 nearest natural seeds in RNA-FM and ERNIE-RNA embedding space and measure the share that have the same top and sub type. Nested guidance raises this share from 45% without guidance to 63% (RNA-FM), and from 36% to 51% (ERNIE-RNA). It rises for 40 and 39 of the 40 families. The cmsearch validity rises from 0.63 to 0.91, and the hit ratio from 11.1% to 14.0%. The flat model with guidance reaches 35% / 28%, a validity of 0.35, and a hit ratio of 8.2%. This holds at the level of RNA sub-types, and we make no claim at the level of the individual family.

Additionally, for RNA lacking a reliable sub-type or family label, we generated from the top label alone, which shows that the coarse level is informative on its own. The 20 nearest natural seeds of the generated sequences share the top type for 62% / 55% of Intron sequences, against 2% of all seeds, and for 71% / 77% of Cis-reg sequences, against 26% (RNA-FM / ERNIE-RNA).

We sincerely hope you could raise your score based on our clarification.

Thanks for your response. I will keep my positive score since the performance of these tricks is good. However, I will not raise my score since the novelty of this work is limited.

Thank you for your response and for your continued support. Your comments on functional validation and unseen families helped us improve this work.

Reviewer_RSYdOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces a method that can generate family-based RNA sequences based on discrete flow-matching model. On the benchmark proposed, RfamFlow (the proposed method) achieved SOTA on family-conditioned and SMALL-FAMILY genertaion. Ablation studies and further experiments extended more insights of the method.

Critical strengths
  1. The motivation is clear and the task is interesting.
  2. The experiemental benchmark is well-established.
Critical weaknesses
  1. The largest concern is the proposed statement "DiT with a nested classifier-free guidance scheme can bring prior from its coarser taxonomic levels" was not well-proved either from theoretical nor experimental perspective.
  2. Novelty of the method is limited. This is the biggest concern from my perspective.
  3. There should be more ablation experiments, and supportive experiment showing that 1) the pretraining model is good and generalizable; 2) more guidance methods can be applied and useful; and 3) Some exact case studies on different family-based RNA generation.
Questions for authors

What does zero-shot fitness experiment bring to the proposed method.

3 comments

Thank you for pointing out the issues. We have uploaded a revised manuscript with new experiments. Both base models are now retrained with the same three-level label input (top, sub, and family) and differ only in how the labels are dropped during training and in the guidance, which makes the comparison between flat and nested clearer. All related tables and figures are recomputed.

1. The coarse-level prior. The revised paper now provides experimental evidence for this statement (Section 4.2, Appendix C.5). For each generated sequence we take its 20 nearest natural seeds from other families in RNA-FM and ERNIE-RNA embedding space and measure the share of siblings, the seeds that share its top and sub type. We compare sequences generated without guidance and with nested guidance. Nested guidance raises it on the small cohort from 57.2% to 62.3% (RNA-FM) and from 53.0% to 61.2% (ERNIE-RNA). On medium it rises from 66.7% to 70.8% and from 68.9% to 74.7%. 2. Novelty. We agree that CFG and absorbing-state discrete flow matching are existing components, and we do not claim them. Our contributions are: (i) one non-autoregressive model conditioned on the Rfam tag, covering thousands of families without structural input. RfamFlow-base outperforms the autoregressive RFamLlama on large, medium, and small families (Table 1); (ii) hierarchy-aware conditioning, whose benefit we isolate in an ablation (Table 2); (iii) one model that also does masked reconstruction and zero-shot fitness scoring without task-specific training.

3. More ablations and supporting experiments.

1) The pretraining model is good and generalizable. On de novo generation, RfamFlow-large scores 86.7 / 82.8 / 62.7 on large / medium / small families (Table 1) and is above every baseline except RfamGen on the large cohort. Figure 3 plots the per-family validity against novelty and diversity, and RfamFlow-large lies above RfamFlow-base and RFamLlama on both, so this advantage does not come from memorization. On masked reconstruction (Section 4.4, Table 3), it recovers 76.3 / 74.9 / 73.2% of the masked nucleotides at 15 / 30 / 50% masking and is the best model at 30% and 50%, while ERNIE-RNA leads at 15% (78.3%). On zero-shot fitness (Section 4.5, Table 4), the WT-marginal Spearman correlation on tRNA / glmS / CPEB3 is 0.59 / 0.30 / 0.48 for RfamFlow-base and 0.62 / 0.28 / 0.47 for RfamFlow-large.

To test whether the coarse labels are informative on their own, we first generated from partial labels, with the finer levels left empty (Section 4.2, Figure 5). Given only a top label, 71% / 77% of the 20 nearest natural seeds of Cis-reg sequences are Cis-reg (RNA-FM / ERNIE-RNA), against 26% of all seeds. Then, to test generalization to families the model has never seen, we drew 40 random families that have no non-seed sequence, and gave only their top and sub labels. The share of the 20 nearest seeds that share the top and sub type rises from 45% without guidance to 63% with nested guidance (RNA-FM), and from 36% to 51% (ERNIE-RNA). The cmsearch validity rises from 0.63 to 0.91. The flat model with guidance reaches 35% / 28% and 0.35. This holds at the level of RNA sub-types, and we make no claim at the level of the individual family.

2) More guidance methods can be applied and useful (Section 4.3, Table 2). We compare no guidance, flat CFG, the diagonal variant (standard CFG on the nested-trained model), and the three-weight nested guidance. Each is tuned by a full sweep (Appendix C.1), and Table 2 reports the validity (cmsearch bit score) of the best setting of each. We repeat the comparison on 50 further medium and small families that were not used for tuning (Appendix C.1). 3) Exact case studies. We provide case studies on individual families at several places. Figure 4C (Appendix C.3.2) shows five generated sequences next to their nearest natural sequence of the same family, with their RNAfold MFE structures. Figure 4A-B and the per-family plots in Appendix C.3 compare the GC content and MFE of the generated and natural sequences of each family. Figure 5E-J show three unseen families (E, F) and three families each of the medium and small cohorts (G-J).

4. What the zero-shot fitness experiment brings. Ranking variants of a known sequence is a central task in RNA engineering, and fitness labels are scarce. Zero-shot variant effect prediction is a common measure of whether a sequence model has learned functional constraints, such as conserved positions and base-pairing: a model that generates functional sequences should assign higher likelihood to fitter variants. The correlation between the model's likelihood and measured fitness on three deep mutational scanning assays (Section 4.5, Table 4) therefore reflects this knowledge. We include it to show that the single trained model also serves variant effect prediction.

We sincerely hope you could raise your score based on our clarification.

Thanks for the rebuttal. It partially addresses my concerns. I will raise my score to 2.

My concern is, conditioning on a tag to generate RNA sequence is definitely an idea that can be easily proposed. So, the core thing is, how to make it generalizable, novel, and well-designed. From the content propsoed, the current manuscript does not reach this criteria.

Thank you for the follow-up and for raising your score. Your comments pushed us to test the generalization and the design of the model more thoroughly, and we will keep improving these.

Reviewer_MmtwOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

RfamFlow trains one family-conditioned masked discrete flow model across Rfam families. It combines embeddings of coarse RNA type, subtype, and family, with nested classifier-free guidance formed by separately weighting the subtype and family log-probability increments. Confidence-based iterative unmasking is used for generation. The work evaluates family membership using covariance-model scores across three family-size cohorts, and additionally evaluates masked reconstruction and mutation-fitness ranking without task-specific fine-tuning.

Critical strengths
  1. A single model addressing highly imbalanced RNA families is a useful setting, and the evaluation explicitly includes 100 large, 100 medium, and 100 small families. RfamFlow-large achieves full family coverage with substantially higher mean CM scores than the released RfamLlama baseline (Table 1); coverage is an important practical benefit over fitting separate models per family.

  2. The paper includes meaningful controls beyond a flat baseline: unguided hierarchical conditioning, diagonal guidance with shared level weights, and alternative samplers. The reconstruction results also show that the learned denoiser has useful capabilities beyond de novo generation.

Critical weaknesses
  1. The principal generation result measures CM membership without establishing the novelty/diversity trade-off under guidance. GC and MFE summaries do not determine whether a model generates diverse new family members rather than near-copies or a narrow high-scoring subset. Appendix A describes within-family 90% clustering and a seed/non-seed split, but does not clearly establish a similarity-controlled train/test separation. This limits how confidently the results support generalization to novel candidates, especially for small families.

  2. The evidence specifically attributing gains to separately weighted nested guidance is not yet decisive. In Table 3, the small-family comparison is 24.4 versus 15.5 for flat CFG, but 22.9 is already obtained by the diagonal variant with the same hierarchical embeddings. The additional nested-guidance gains over that control are 0.9, 3.9, and 1.5 bits across cohorts. Appendix C.1 selects settings by maximizing scores on the reported cohorts and gives no uncertainty for these differences. Without independent selection/evaluation and variability estimates, the strength and generality of the core incremental effect remain unclear.

Questions for authors
  1. Please report generated-to-training nearest-neighbor sequence identity and within-generated-set diversity alongside CM score for flat, diagonal, and nested guidance, particularly on the small cohort. How are seed/non-seed near-duplicates handled across the final split?

  2. Can the nested-versus-diagonal comparison be evaluated with guidance selected on a separate validation family set, followed by paired per-family uncertainty and independent training/sampling runs? Does the small-family advantage remain?

  3. For the auxiliary fitness results, Appendix C.5.2 scores the fully visible wild type at t=0.999. Given the copying behavior of absorbing denoisers discussed in Appendix B.6, what establishes this WT-marginal score as a meaningful mutation predictor? Does the conclusion hold with each scored site masked?

3 comments

Thank you for recognizing the practical benefit of full family coverage and the controls in our ablations, and for pointing out the issues. We have uploaded a revised manuscript in which both base models are retrained, so the numbers below refer to the new tables and differ from the ones the reviewer cites.

1. Novelty, diversity, and near-duplicates. We now report novelty and diversity alongside validity for flat, diagonal, and nested guidance (definitions in Section 4.1). Novelty is one minus the highest normalized edit-distance similarity between a generated hit and any non-seed sequence of its family, and diversity is the mean pairwise distance within the generated set. Appendix C.1 gives validity, novelty, and diversity of every configuration on the 100 families of each cohort (Table 10) and on 50 further medium and small families (Table 11), and Figure 3 shows the trade-off of validity against novelty and diversity on every family. RfamFlow-large lies above RfamFlow-base and RFamLlama on both axes, is second only to RfamGen on novelty, and is close to it on diversity.

On the split, our training split follows RfamGen: we train and validate on the non-seed sequences and hold out the seed sequences, after 90% MMseqs2 clustering within each family, and sequences identical to a seed sequence are removed from the training set. This split does not separate train and test by similarity, and near-duplicates of test sequences remain in the training set. Among the seed sequences of the large / medium / small cohorts, 57.7 / 55.4 / 47.4% have a training sequence at 90% identity and 80% coverage or more (MMseqs2, the thresholds of the clustering). The generated sequences of RfamFlow-base fall below this level, at 38.5 / 18.4 / 2.9%.

2. Selection and variability of the guidance weights. We search the weights with a stride of 0.1, finer than the 0.5 of the previous version, and choose them on the 100 reported families of each cohort. We evaluate them on 50 further medium and small families that were not used for the choice (Appendix C.1, Table 11), sampled in independent runs. On these families nested stays above diagonal on medium (62.6 vs 62.1), and the two are equal on small (24.3 vs 24.3). The best diagonal weights on small are (1.9, 1.9, 1.9) and the best nested weights are (2.0, 1.8, 2.0), which are close to each other, and both models are trained with the same nested dropout, since the diagonal is by definition the transition between flat and nested (Section 3.4). We also repeated the sampling at the same weights. We also note that the scores are lower on these families than on the reported ones: for nested, 62.6 against 64.2 on medium and 24.3 against 29.1 on small.

3. The WT-marginal score. WT-marginal scoring was introduced for protein language models by Meier et al. (2021) and has been used for RNA language models in RNAGym, and we score the baselines ERNIE-RNA, RiNALMo, and RNA-FM with the same scheme. Its advantage is cost: one forward pass on the wild type scores every variant of an assay, instead of one pass per variant.

Your concern is that an absorbing denoiser may simply copy the visible tokens. This is what the mutant-marginal score addresses. It masks the mutated sites of each variant and reads the model's prediction there, so the model cannot copy the observed base. We also ran it in our experiments and report it alongside the WT-marginal score (Section 4.5, Table 4, Appendix C.7). On tRNA they agree (0.59 and 0.57 for RfamFlow-base). On glmS and CPEB3 the mutant-marginal scores are lower (0.12 and 0.09, against 0.30 and 0.48 for the WT-marginal score), and the other RNA language models (ERNIE-RNA, RiNALMo, RNA-FM) show the same drop.

We sincerely hope you could raise your score based on our clarification.

Dear Reviewer, we would like to follow up on our response. It covers your three questions: the near-duplicates between seed and non-seed sequences (with their share), the guidance weights evaluated on families not used for selection (Appendix C.1, Table 11), and the mutant-marginal scoring that masks each scored site (Table 4). If anything is unclear, we would be glad to answer before this rebuttal period ends. Thank you for your time.

Thanks for the rebuttal. Compared with similar related work (such as SEMANTICIST and FineDiffusion), this paper do not methodologically improve these baseline nested cfg methods in the RNA generation field. So I think the novelty is limited and will maintain my score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.