acceptodds
Under review as a conference paper at ICLR 2027

Evolutionary Constraints as Data: Augmenting RNA Language Models with Phylogenetic Guidance

Abstract

Fine-tuning RNA language models is often limited by scarce labelled data. Data augmentation can help, but nucleotide edits may change the biological properties being predicted, making the original labels unreliable. We introduce PhyloAug, a plug-in framework that uses phylogenetic information to guide RNA sequence augmentation. PhyloAug uses relative evolutionary rates estimated from homologous sequences to select candidate editing sites, then uses a structure-conditioned RNA language model to generate replacement nucleotides. The resulting variants are used to fine-tune existing models without changing their architectures or training objectives. Across seven RNA functional prediction benchmarks and multiple model backbones, PhyloAug improves performance over unaugmented finetuning in most evaluated settings. Comparisons under matched augmentation and training budgets support the contribution of phylogenetically guided site selection. Complementary distributional diagnostics show smaller sequence-distribution shifts than less constrained editing policies. The benefits vary across tasks and label budgets, highlighting both the value and limits of evolutionary guidance for improving RNA language model adaptation with limited labelled data.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Data Augmentation, RNA, RNA Language Models, Bioinformatics, Phylogenetics

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_7JMWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

In this paper, PhyloAug, a method of using phylogenetic information to guide RNA data enhancement, is proposed. In this method, homologous sequences are retrieved and the relative evolution rate of each point is estimated, and the appropriate position is selected for Mask. Then, the structural conditional RNA language model is used to generate replacement bases and construct new fine-tuning data.

The author carried out experiments on seven RNA function prediction tasks and several models, and verified the effectiveness of the method through baseline comparison, ablation experiment, experiments with different labeled data and MMD distribution analysis. The results show that PhyloAug can improve the downstream prediction performance in most settings, but the effect depends on specific tasks and models.

Critical strengths

PhyloAug combines phylogenetic information with RNA structural condition generation, and can be applied to different RNA basic models without modifying downstream model architecture or training objectives.

The paper covers multiple tasks and models, and sets up baseline control and ablation experiments that match the training budget. It shows that the performance improvement is not only from the increase of training data, but also from the selection of phylogenetic sites.

Critical weaknesses

PhyloAug mainly combines the existing evolutionary rate estimation, data enhancement and structural condition generation techniques. EvoAug and PriFold have explored similar biological constraint enhancement ideas. In addition, the paper directly selected sites with relative evolution rate of [0.8, 1.2], but did not fully explain why these sites were more suitable for enhancement than highly conserved or rapidly evolving sites, and lacked corresponding systematic control experiments.

Methods It is assumed that the generated sequence can inherit the original function tag, but the evolution rate and structural conditions can not guarantee the function unchanged after mutation. Although the downstream performance and MMD results have improved, these evidences cannot directly prove that the generated sequence retains the original tag, thus weakening the biological rationality of the method.

Questions for authors

Q1. Why choose the evolution rate interval [0.8, 1.2]? Can we compare highly conservative, medium-speed and fast evolutionary sites under the same enhancement budget to verify the rationality of this choice?

Q2. Is there any direct evidence that the enhanced sequence can keep the original functional label? Can the existing experimental mutation effect data be used for verification?

Q3. Since the method has already retrieved homologous sequences, why not directly use the real homologous sequences screened by functional consistency for enhancement? Compared with simpler evolutionary heuristic enhancement strategies such as EvoAug, what is the additional benefit of PhyloAug?

4 comments
AuthorsOpenReview ↗

Reviewer 7JMW response Part I

We thank the reviewer for their thoughtful feedback and apologise for splitting the response to meet the character limit.

Q1. Why choose [0.8, 1.2], and how does it compare with conserved or rapidly evolving sites?

We agree that the rate-selection rule requires a direct comparison with alternative evolutionary regimes. The interval [0.8, 1.2] is a fixed heuristic around the normalised mean evolutionary rate of one; it was not selected using downstream validation performance and does not itself establish selective neutrality. To test this choice, we added a matched-budget comparison of conserved, near-mean and accelerated-site perturbations on m6A and gRNA, covering motif-driven and sequence-to-function prediction. We define conserved sites as r≤0.8r\leq0.8, near-mean sites as 0.8≤r≤1.20.8\leq r\leq1.2, and high-rate sites as r≥1.2r\geq1.2. The near-mean interval is a fixed heuristic around the normalised alignment mean of one, not a validation-selected interval. Across nine backbones and five seeds, the comparisons use matched parents, mask counts, generation settings and training protocols, with identical augmentation-budget searches.

Our matched-budget comparison shows mean m6A F1 changes over NoAug of +0.0347 for near-mean, +0.0303 for high-rate and −0.0003 for low-rate editing. The clearest separation is therefore from low-rate editing; the additional benefit over high-rate editing is smaller. Corresponding gRNA Spearman changes are +0.0055, −0.0002 and −0.0221. These results support the proposed interval as a useful default, with a modest observed advantage over accelerated-site editing and a clearer advantage over conserved-site editing. They support the choice of rate regime without establishing that the exact endpoints are uniquely optimal. We will add these experiments to the manuscript.

Experiment 1 m6A Result:

Modelacceleratednear-meanconserved
HyenaDNA+0.030$\pm$0.015+0.034±\pm0.011+0.019$\pm$0.006
SpliceBERT+0.034$\pm$0.008+0.037±\pm0.012−0.010$\pm$0.010
RNAErnie+0.018$\pm$0.042+0.026±\pm0.038−0.026$\pm$0.036
RNA-FM+0.036$\pm$0.005+0.044±\pm0.010−0.034$\pm$0.028
BiGamba+0.057$\pm$0.003+0.066±\pm0.003−0.004$\pm$0.007
RNA-MSM+0.053$\pm$0.031+0.058±\pm0.028+0.030$\pm$0.032
RNA-BERT+0.046$\pm$0.014+0.049±\pm0.013+0.029$\pm$0.013
ResNet−0.002$\pm$0.006+0.007$\pm$0.006−0.007$\pm$0.009
CNN+0.001±\pm0.007-0.009$\pm$0.007−0.000$\pm$0.002
Mean, 7 pretrained+0.0391 (7/7)+0.0449 (7/7)+0.0006 (3/7)

Experiment 1 gRNA Result:

Modelacceleratednear-meanconserved
HyenaDNA+0.0065$\pm$0.03+0.0085±\pm0.05−0.0190$\pm$0.04
SpliceBERT−0.0055±\pm0.05−0.0117$\pm$0.05−0.0183$\pm$0.07
RNAErnie−0.0038$\pm$0.05+0.0023±\pm0.03−0.0066$\pm$0.04
RNA-FM+0.0012$\pm$0.03+0.0082±\pm0.03−0.0281$\pm$0.05
BiGamba−0.0099$\pm$0.060.0000±\pm0.06−0.0509$\pm$0.06
RNA-MSM+0.0108$\pm$0.04+0.0131±\pm0.03−0.0248$\pm$0.03
RNA-BERT+0.0068$\pm$0.04+0.0090±\pm0.03−0.0274$\pm$0.04
ResNet−0.0210$\pm$0.06−0.0001±\pm0.04−0.0231$\pm$0.04
CNN+0.0130 ± 0.01+0.0201 ± 0.02−0.0008 ± 0.03
Mean, 7 pretrained+0.0009 (4/7)+0.0042 (5/7)−0.0250 (0/7)
AuthorsOpenReview ↗

Reviewer 7JMW response Part II

Q2. Is there direct evidence of functional-label preservation, including from mutation-effect datasets?

We agree that downstream improvements and MMD measure different properties from direct functional-label preservation. To assess functional-label preservation, we added two task-specific analyses of label stability. For gRNA, whilst our generated variants did not have independent experimental measurements, we instead use the external DeepSpCas9 predictor to score original and augmented sequences, measuring Lin's concordance correlation coefficient (CCC) agreement. Agreement results were 0.707 for near-mean, 0.701 for accelerated and 0.667 for conserved selection. For m6A, we measured corruption of the DRACH motif associated with the labelled site. Motif-disruption rates were 5.2%, 5.4% and 10.5%, respectively, indicating that near-mean selection retained motif compatibility in 94.8% of evaluated variants. These analyses assess predicted activity stability and preservation of a recognised biological feature, rather than directly measuring variant function.

To quantify the performance gap associated with residual motif disruption, we additionally evaluated near-mean PhyloAug with enforced DRACH preservation. Across nine models, native PhyloAug improved mean F1 over NoAug by 0.0347, compared with 0.0441 for the motif-preserving reference: an absolute gap of 0.0094 F1. Native PhyloAug therefore achieved approximately 79% of the reference’s observed augmentation gain. The higher reference performance is consistent with a cost from residual motif disruption, while the positive native gain shows that this limitation does not negate the overall augmentation benefit. These results support inherited-label augmentation as a useful practical approximation, while explicitly quantifying a remaining limitation.

ModelNo-AugPhyloAugDRACH-constrained
RNA-MSM0.6293$\pm$0.048+0.058$\pm$0.028+0.073$\pm$0.032
SpliceBERT0.6634$\pm$0.006+0.037$\pm$0.012+0.044$\pm$0.015
BiGamba0.6351$\pm$0.011+0.066$\pm$0.003+0.077$\pm$0.002
RNA-FM0.6760$\pm$0.007+0.044$\pm$0.010+0.049$\pm$0.009
RNAErnie0.6511$\pm$0.027+0.026$\pm$0.038+0.044$\pm$0.042
HyenaDNA0.6845$\pm$0.021+0.034$\pm$0.011+0.037$\pm$0.012
RNA-BERT0.5953$\pm$0.003+0.049$\pm$0.013+0.054$\pm$0.015
CNN0.519$\pm$0.004-0.009$\pm$0.007+0.006$\pm$0.006
ResNet0.529$\pm$0.006+0.007$\pm$0.006+0.013$\pm$0.001
mean, all 90.6203+0.0347+0.0441
mean, 7 pretrained0.6478+0.0449+0.0540

Q3. Why not augment with retrieved homologs, and what does PhyloAug add beyond EvoAug and PriFold?

Direct homolog augmentation is reasonable when matching task-specific labels are known; BLAST similarity alone does not establish this. PhyloAug instead uses homolog-derived rates to guide edits to labelled parents. We have not directly compared against functionally screened homologs or EvoAug on these tasks.

Table 1 isolates the added value of evolutionary selection: changing only the selector improves mean m6A F1 across 12 backbones by 0.0328 without structure conditioning and 0.0331 with it. Appendix J additionally compares PhyloAug with an offline PriFold-style operator at matched augmentation volume on three structure-prediction datasets, yielding 20 wins, 5 ties and 2 losses, with a mean advantage of 0.008 F1. This evaluates the augmentation operators, not PriFold’s complete framework or original online training schedule.

Reviewer_7JMWOpenReview ↗

Response to rebuttal

Thanks to the authors for the detailed response and the extra experiments. However my main concerns aren't really resolved: label preservation is still only backed by proxy evidence, and there's still no comparison with EvoAug. I'll keep my score at 2.

AuthorsOpenReview ↗

Response to Reviewer 7JMW

Thank you for identifying the two remaining concerns. On label preservation, we agree that our evidence is indirect. We do not have independent experimental measurements for the generated variants. Retention of a label-associated motif and agreement with an external activity predictor are useful checks, but neither verifies that a variant retains its parent’s functional label. We will state this explicitly where label inheritance is introduced: inherited labels are a training assumption, and evaluation on unaugmented held-out examples measures predictive utility.

On EvoAug, we have now completed a direct comparison of augmentation operators on m6A prediction and CRISPR on-target activity, using nine backbones and five seeds per backbone. Both policies used the same parent sequences, augmentation counts per parent, inherited labels, downstream models and seeds, with the same caps on optimisation steps and tokens. We varied how the augmented sequences were generated.

The EvoAug-based arm used six discrete-sequence operators: deletion, insertion, translocation, inversion, mutation and reverse complementation. We excluded Gaussian noise because it acts on continuous one-hot inputs, and rescaled length-dependent operator ranges for the 23-nt CRISPR inputs. We applied both policies offline. This compares how the two augmentation policies perform within the same training protocol; it does not reproduce EvoAug’s full online augmentation and subsequent fine-tuning on unaugmented data. We will report the operator settings, sampling procedure and length rescaling in the appendix.

Across all nine models, PhyloAug had a mean advantage of 1.78 F1 percentage points on m6A and 0.0232 Spearman’s ρ on CRISPR, with higher mean scores on six and seven of the nine backbones, respectively. The model-level results are below.

m6A prediction: F1

ModelNoAug, mean ±\pm SDPhyloAug, Δ\Deltamean (score SD)EvoAug-based, Δ\Deltamean (score SD)
RNA-MSM0.6293 ±\pm 0.048+0.053 (0.038)+0.068 (0.028)
SpliceBERT0.6634 ±\pm 0.006+0.041 (0.005)+0.0007 (0.009)
BiGamba0.6351 ±\pm 0.011+0.063 (0.010)+0.017 (0.005)
RNA-FM0.6760 ±\pm 0.007+0.050 (0.007)+0.015 (0.013)
RNAErnie0.6511 ±\pm 0.027+0.027 (0.022)+0.035 (0.025)
HyenaDNA0.6845 ±\pm 0.021+0.042 (0.013)+0.007 (0.019)
RNA-BERT0.5953 ±\pm 0.026+0.031 (0.013)0.000 (0.082)
CNN0.5190 ±\pm 0.004+0.001 (0.004)−0.002 (0.010)
ResNet0.5290 ±\pm 0.006−0.003 (0.006)+0.004 (0.010)
Mean, all 90.6203+0.0339+0.0161
Mean, 7 pretrained0.6478+0.0439+0.0204

CRISPR on-target activity: Spearman’s ρ

ModelNoAug, mean ±\pm SDPhyloAug, Δ\Deltamean (score SD)EvoAug-based, Δ\Deltamean (score SD)
HyenaDNA0.3184 ±\pm 0.010+0.0295 (0.010)+0.0076 (0.051)
SpliceBERT0.3198 ±\pm 0.030+0.0082 (0.030)+0.0046 (0.031)
RNAErnie0.3566 ±\pm 0.010+0.0024 (0.010)+0.0070 (0.051)
RNA-FM0.3646 ±\pm 0.020+0.0364 (0.020)−0.0169 (0.041)
BiGamba0.1779 ±\pm 0.030+0.0060 (0.020)+0.0011 (0.045)
RNA-MSM0.3252 ±\pm 0.020+0.0268 (0.020)−0.0151 (0.027)
RNA-BERT0.2322 ±\pm 0.020+0.0568 (0.020)+0.0204 (0.037)
ResNet0.2130 ±\pm 0.010+0.0290 (0.030)−0.0243 (0.070)
CNN0.1600 ±\pm 0.020−0.0260 (0.010)−0.0242 (0.030)
Mean, all 90.2742+0.0188−0.0044
Mean, 7 pretrained0.2992+0.0237+0.0012

For both tables, Δ\Deltamean is the change in mean held-out score relative to NoAug for the same backbone. Parentheses give the across-seed SD of the augmented model’s scores, not the SD of paired gains. Bold marks the higher mean of the two augmentation policies.

The EvoAug-based policy improved mean m6A F1 over NoAug and performed better than PhyloAug on some backbones. The evidence therefore supports an average predictive advantage for PhyloAug on these two tasks under the matched offline protocol. A separate m6A control helps identify one contributor to that advantage over random editing: with DRACH protected in both arms and the generator held fixed, phylogenetic site selection outperformed random site selection by 2.45 F1 percentage points on average across nine models. This shows that edit-site choice matters even after explicit motif protection. Whether individual variants retain their biological labels remains an open experimental question.

Reviewer_kr9COpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces PhyloAug, a plug-in data augmentation framework for fine-tuning RNA language models under limited labels. For each training sequence it retrieves homologs by BLAST against NCBI nt, aligns them with MAFFT, infers a tree with FastTree, and estimates site-wise relative substitution rates with PAML baseml. Positions whose normalised rate falls in [0.8, 1.2] become candidate edit sites. A structure-conditioned RNA-LM (OmniGenome) then resamples the masked positions, conditioned on the unmasked nucleotides and on a ViennaRNA structure predicted from the original sequence. Variants inherit the source label and are added to the fine-tuning pool, so the target model’s architecture and objective are untouched. Validation spans seven functional RNA benchmarks and twelve target backbones, with five matched augmentation controls (NoAug, Duplicate, Random, MSA-only, Struct-Infill), a 2×2 ablation separating site selection from structure conditioning, a label-budget sweep on m6A, an augmentation-budget sweep, a cross-generator control, a structure-conditioner substitution, a comparison against a PriFold-style operator, and kernel MMD diagnostics of augmentation-induced distribution shift. Reported gains are largest on RBP and m6A (around 4 F1 points on average) and mixed on splice type and localisation.

Critical strengths

• The problem is worth solving and the fix is easy to adopt. Labelled RNA data is expensive because it comes out of a wet lab, so anything that squeezes more training signal out of what already exists is useful to a lot of people. And because this works on the data rather than the model, you can put it in front of whatever backbone you already have. • The experimental work is very complete. Seven tasks, twelve backbones, five matched controls, and a clean 2×2 separating site selection from structure conditioning, paired by model and seed with the same mask sets reused across conditions. Including a Duplicate arm was a good decision, since it separates the effect of simply having more training rows from the effect of the augmentation itself. • The reporting is honest. The authors say outright that the benefit is not uniform across tasks, and they name the model-task combinations that get worse, including ones where both metrics drop. I would single out the phyloP comparison in the appendix. Running an experiment that risks showing your own central component is unnecessary is not something most authors do. My concerns below are about how the experiments are set up rather than whether to trust the numbers, and all of them are fixable.

Critical weaknesses

• The paper’s own appendix shows the expensive part is not needed. Working out which positions are free to mutate takes a serious pipeline here: BLAST against NCBI nt, align, build a tree, fit a substitution model in PAML. The authors cost it at roughly 16 CPU-hours per thousand queries. The appendix then replaces all of it with phyloP scores looked up by coordinate, which is a file you download, and the difference comes out at +0.0028 F1 and +0.0051 Spearman. That is nothing. The authors read the two signals as complementary, and they are right that phyloP needs genomic coordinates while their pipeline does not. But every benchmark in the paper is human and coordinate-mapped, so the one case where the pipeline is genuinely required never gets tested. As things stand a reader should use phyloP and skip the rest, which leaves the methodological contribution much smaller than the paper’s construction suggests. I would also note that the comparison was only run on two of the seven tasks. • The size of the improvement is not pinned down. Two things compound here. PhyloAug tunes how many variants it makes per sequence, trying seven settings and keeping the best, while the no-augmentation baseline has no equivalent knob and gets no equivalent search. The appendix shows this setting matters a great deal and that the best value moves around unpredictably between tasks and models, so some share of the headline gain is just having had seven attempts against one. The second problem sits in the comparison meant to show phylogenetic selection beating random selection, which has crashed runs in it. Several random-augmentation entries are recorded as 0.00 with zero variance; the authors flag them in the text as collapsed and then leave them in the averages. On miRNA localisation, taking the failures out lifts random selection from about 0.34 to 0.51, which is level with doing nothing rather than catastrophically worse. That comparison carries the central claim, so it needs rerunning. • The rule the method is built on is never tested. Only positions whose evolutionary rate falls in a narrow band around average get edited, and the band is never varied anywhere in the paper. Nothing shows that it matters. Given that any conservation signal seems to work equally well, this is the experiment that would settle whether the evolutionary story is doing real work. A smaller point in the same area: homologs more than 95% identical to the query are thrown away without explanation, and those are the sequences carrying the most information about recent constraint.

Questions for authors
  1. Where does the benefit actually come from: keeping the labels valid, or teaching the model which positions are conserved? Across the variants made from one sequence, the positions you never touch stay identical while the candidate positions change, so the model is shown a conservation pattern through the data whether or not the labels really survive editing. Two things would tell these apart. Does the upper end of the rate band matter at all, say your band against one with no upper bound at the same number of masked positions? And do the gains cluster on tasks where the signal is a conserved motif sitting in variable flanks? If it turns out to be the second, I think that is a good outcome for the paper. It would make this a data-side way of installing the same prior that GPN-MSA and PhyloGPN build into the architecture, which is a clean position against work you already cite, and it would also explain why the choice of rate estimator makes no difference.
  2. Could you report PhyloAug at one fixed M against the baseline, or give the baseline a comparable search? At the moment the augmented arms pick the best of seven settings and the baseline picks from one. This is the result that would most change my view of the effect size.
  3. What do the matched-baseline averages look like with the collapsed runs taken out or rerun? I mean the 0.00 ± 0.00 entries for Random, and RiNALMo-Giga at 0.022 under Struct-Infill. Does PhyloAug still come out ahead of Random on miRNA localisation?
  4. Did you try varying the rate band? Your band against the top or bottom rate quantile, at the same mask ratio, would show whether the band is doing the work. Given the phyloP comparison, this feels like the key remaining piece of evidence that the evolutionary signal matters.
  5. Why drop homologs above 95% identity? Those carry the most information about recent constraint, and I could not find a reason given.
4 comments
AuthorsOpenReview ↗

Reviewer kr9C response Part I

We thank the reviewer for their detailed and constructive feedback, and apologise for splitting the response to meet the character limit.

W1. Necessity of the sequence-based pipeline and the phyloP comparison

We thank the reviewer for questioning whether the computational cost of BLAST/PAML is justified. We clarify that Appendix I reports phyloP minus PAML: +0.0028 F1 on m6A and +0.0051 Spearman on CRISPR, favouring phyloP on average. PAML nevertheless determined which parents were eligible for augmentation and their per-sequence mask counts; phyloP replaced the signal used to select positions. The experiment therefore compares two site-selection signals under matched eligibility and masking budgets, rather than evaluating an entirely PAML-free pipeline. We will clarify this design and restrict its conclusions to the two evaluated tasks.

BLAST/PAML provides a sequence-based approach where reliable reference mappings or suitable precomputed conservation tracks are unavailable. Where these resources exist, Appendix I supports phyloP as an alternative site-selection signal, and we do not claim that PAML must outperform it. A fully PAML-free implementation would additionally require independently specifying augmentation eligibility and mask counts before its computational savings could be assessed.

Q1 & Q4. Origin of gains and rate-selection ablation

We agree that the rate-selection rule requires direct testing. We therefore added a matched-budget comparison of conserved r≤0.8r\leq0.8, near-mean 0.8≤r≤1.20.8\leq r\leq1.2 and accelerated r≥1.2r\geq1.2 selection on m6A and gRNA, using nine backbones and five seeds. Eligible parents, per-parent mask counts, the generator and training protocols were held fixed, with identical augmentation-budget searches. The original interval is a fixed heuristic around the normalised mean rate of one, rather than an interval selected through downstream validation.

Experiment 1 m6A Result:

Modelacceleratednear-meanconserved
HyenaDNA+0.030 ± 0.015+0.034 ± 0.011+0.019 ± 0.006
SpliceBERT+0.034 ± 0.008+0.037 ± 0.012−0.010 ± 0.010
RNAErnie+0.018 ± 0.042+0.026 ± 0.038−0.026 ± 0.036
RNA-FM+0.036 ± 0.005+0.044 ± 0.010−0.034 ± 0.028
BiGamba+0.057 ± 0.003+0.066 ± 0.003−0.004 ± 0.007
RNA-MSM+0.053 ± 0.031+0.058 ± 0.028+0.030 ± 0.032
RNA-BERT+0.046 ± 0.014+0.049 ± 0.013+0.029 ± 0.013
ResNet−0.002 ± 0.006+0.007 ± 0.006−0.007 ± 0.009
CNN+0.001 ± 0.007−0.009 ± 0.007−0.000 ± 0.002
Mean, all 9+0.0303+0.0347−0.0003
Mean, 7 pretrained+0.0391 (7/7)+0.0449 (7/7)+0.0006 (3/7)

Experiment 1 gRNA Result:

Modelacceleratednear-meanconserved
HyenaDNA+0.0065$\pm$0.03+0.0085±\pm0.05−0.0190$\pm$0.04
SpliceBERT−0.0055±\pm0.05−0.0117$\pm$0.05−0.0183$\pm$0.07
RNAErnie−0.0038$\pm$0.05+0.0023±\pm0.03−0.0066$\pm$0.04
RNA-FM+0.0012$\pm$0.03+0.0082±\pm0.03−0.0281$\pm$0.05
BiGamba−0.0099$\pm$0.060.0000±\pm0.06−0.0509$\pm$0.06
RNA-MSM+0.0108$\pm$0.04+0.0131±\pm0.03−0.0248$\pm$0.03
RNA-BERT+0.0068$\pm$0.04+0.0090±\pm0.03−0.0274$\pm$0.04
ResNet−0.0210$\pm$0.06−0.0001±\pm0.04−0.0231$\pm$0.04
CNN+0.0130 ± 0.01+0.0201 ± 0.02−0.0008 ± 0.03
Mean, 7 pretrained+0.0009 (4/7)+0.0042 (5/7)−0.0250 (0/7)

Near-mean selection achieved the highest observed mean performance in eight of nine models on each task. Across all nine models, mean m6A F1 changes relative to NoAug were +0.0347, +0.0303 and −0.0003 for near-mean, accelerated and conserved selection, respectively. Corresponding gRNA Spearman changes were +0.0055, −0.0002 and −0.0221. Thus, the editing regime affects augmentation utility, with a modest observed advantage over accelerated selection and a clearer advantage over conserved selection.

We agree that a data-side conservation prior is a plausible explanation for these benefits. The observed ordering occurs on both motif-driven m6A and sequence-to-function gRNA prediction, although these two tasks cannot establish broader task-level clustering or separate this mechanism from label preservation. Our motif and predicted-activity analyses examine preservation separately. We also distinguish the accelerated-only control from the requested no-upper-bound policy r≥0.8r\geq0.8: the present results support the proposed regime but do not isolate the necessity of the upper cutoff or establish uniquely optimal endpoints.

AuthorsOpenReview ↗

Reviewer kr9C response Part II

Q2. Augmentation-budget selection

To directly address how much gain survives without tuning the augmentation count, we report PhyloAug at a fixed M=6M=6, alongside validation-selected MM, across the same seven target backbones. NoAug, fixed M=6M=6 and selected MM are all evaluated on the same held-out test split. The table summarises mean scores and changes relative to NoAug. gRNA results are given as Spearman’s ρ×100\rho \times 100. We run the same seven models as in the paper, including OmniGenome and RiNALMo-Giga, across five random seeds.

TaskNoAug meanChange at fixed M=6M=6Change with selected MMModels improved: fixed / selected
m6A65.17+3.03+3.966/7 / 7/7
miRNA-SL51.85+2.13+3.665/7 / 6/7
gRNA39.85−2.18+1.091/7 / 6/7

Average improvements therefore persist without tuning MM on m6A and miRNA-SL, whereas fixed M=6M=6 reduces gRNA performance. Budget selection affects both the magnitude and consistency of the gains, particularly for gRNA. We will include these negative results and distinguish fixed-budget performance from validation-selected performance.

Separately, Section 5.2 applies the same search, M∈{1,2,4,6,8,10,12}M\in\{1,2,4,6,8,10,12\}, to PhyloAug, Duplicate, Random, Struct-Infill and MSA-only. Duplicate receives the same search while adding no new sequence content. Comparisons against these augmentation controls therefore address the additional question of whether PhyloAug’s advantage arises from greater search opportunities or data expansion alone.

Q3. Collapsed Augmentation Results

We additionally address whether the collapsed runs drive the reported advantages. After repeating model experiments, we found model collapse to be reproducible due to degenerate predictions, rather than execution failures. Thus, we provide a matched exclusion analysis alongside the complete results. We exclude the affected model-task pairs from every compared method, ensuring that each average uses the same retained models.

TaskModels includedRandomStruct-InfillPhyloAugPhyloAug − RandomPhyloAug − Struct
miRNA-SLAll six0.34170.50500.5550+0.2133+0.0500
miRNA-SLMatched four0.51250.48500.5575+0.0450+0.0725
CRISPRAll six0.26300.26370.3350+0.0720+0.0713
CRISPRMatched five0.29800.31200.3460+0.0480+0.0340

PhyloAug still outperforms Random on miRNA localisation. After excluding RNA-FM and RiNALMo-Giga, mean F1 is approximately 0.5575 for PhyloAug versus 0.5125 for Random, a smaller difference of 4.50 F1 percentage points, with PhyloAug ahead in all four retained models. On CRISPR, excluding RiNALMo-Giga gives mean Spearman scores of 0.3460 for PhyloAug, 0.3120 for Struct-Infill and 0.2980 for Random. The corresponding PhyloAug advantages are 0.0340 and 0.0480, respectively. Excluding the collapsed runs reduces the aggregate advantages, but positive differences remain on both matched comparisons.

Q5. Excluding homologs above 95% identity

We agree that closely related homologs can inform recent evolutionary constraint. Our >95% identity filter was intended to prevent the retained set of at most 50 homologs from being dominated by near-identical hits, leaving room for more divergent sequences. This motivation is consistent with work on representative homolog sampling from taxonomically biased databases (Zhou et al., 2014). We acknowledge the trade-off: excluding close homologs may discard useful information about recent constraint. We therefore treat 95% as a practical redundancy-control heuristic, have not established it as optimal, and will clarify this rationale and limitation in the manuscript.

[1] Chan Zhou, Fenglou Mao, Yanbin Yin, Jinling Huang, Johann Peter Gogarten, and Ying Xu. 2014. AST: An Automated Sequence-Sampling Method for Improving the Taxonomic Diversity of Gene Phylogenetic Trees. PLOS ONE, 9(6): e98844.

Reviewer_kr9COpenReview ↗

Response to the rebuttal

Thank you for the thorough rebuttal and the additional experiments on fixed augmentation budgets, collapsed runs, and evolutionary-rate selection. I have read your responses and appreciate the substantial work you put into them. After considering the new evidence, I have decided to keep my rating unchanged (2: Weak rejection).

AuthorsOpenReview ↗

Response to kr9C

Thank you for considering the additional experiments. Your questions about budget selection and collapsed runs helped us distinguish two findings that were unclear in the original submission.

  • First, at a fixed augmentation budget of M=6M=6, PhyloAug still improved mean held-out F1 over NoAug on m6A and miRNA localisation, but mean Spearman’s ρ\rho decreased on gRNA. We will report all three results together and qualify our claim accordingly: the benefit of augmentation without budget tuning depends on the task.

  • Second, you asked specifically whether PhyloAug still outperforms Random on miRNA localisation after excluding the collapsed runs. On the same four retained models, mean F1 was 0.5575 for PhyloAug versus 0.5125 for Random. The gap is smaller than in the full comparison, but PhyloAug remains ahead on all four models. We will report the matched comparison alongside the full results.

It would help us revise the paper to understand which remaining concern weighs most heavily in your assessment, particularly if there is an aspect of the evidence above that you interpret differently.

Reviewer_mK2SOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

Generating label-invariant or semantic-invariant perturbed views or contrastive views is common yet effective for data augmentation or supervised contrastive learning on images or some natural language. Authors proposed to perform perturbation on RNA sequences subject to so-called evolutionary constraints (setting up a certain range of evolutionary rate), while assigning "each generated variant the label of its source example". Experiment shows the proposed method can obtain higher prediction scores in a range of RNA prediction benchmarks and outperforms other baselines, even though authors cannot confirm if their fundmental assumptions (perturbed variants can share the original labels) hold.

I think there are some technical flaws on the core assumption of this paper and I am not sure if any perturbed variants but assigned with original labels would mislead the model training (even though it won't affect the overall prediction performance). Releasing a model trained with perturbed variants and assumed labels would not be acceptable under any ethical review for biology-related research.

Critical strengths
  1. the problem is well-motivated in general.
  2. authors proposed a simple method that can bring gains in datasets
Critical weaknesses
  1. I don't think the assumptions hold. Even though the authors have set up an evolutionary rate interval 0.8~1.2, I don't think such perturbation would not change any source labels as a method handling massive data. Authors didn't check if these perturbations are safe or semantic-invariants.

  2. the evaluation only shows gain statistically or in-general on these benchmarks. Label noises (from labelling perspectives of perturbed variants) yet larger sample can also bring implicit regularization effects and gains in classification/regression tasks.

Questions for authors

Please address the two weakness points above. Maybe for the second point, you can incorproate some case studies to show certain label noises can also bring gain: finetuning based on a dataset vs finetuning based on the same dataset + an additional dataset with noisy/randomly-reshuffled labels. If a model fine-tuned on randomly reshuffled labels manages to outperform the baseline, it would strongly suggest that the downstream evaluation pipeline or benchmark design has fundamental flaws rather than reflecting rigor biological learning.

4 comments
AuthorsOpenReview ↗

Reviewer mk2S response Part I

We thank the reviewer for their thoughtful comments and apologise for splitting the review to meet the character limit.

W1: Is there direct evidence of functional-label preservation?

We agree that, whilst our site selection methodology is carefully curated to minimise functional disruption, it cannot guarantee that every generated variant perfectly retains its source label. We clarify that our contribution is a computational augmentation framework that combines evolutionary site selection with structure-conditioned generation. Label inheritance is explicitly treated as a training assumption, with practical utility assessed through prediction on un-augmented held-out examples, with matched augmentation controls and component ablations providing further evidence. To address the reviewer’s concern about label stability directly, we included additional analyses of motif disruption and compared predicted augmented variant mutational effect with predicted non-augmented sequences. Lastly, we compare PhyloAug with a motif-preserving reference that quantifies the remaining performance gap.

We first investigate the level of label corruption in generated augmented sequences. For m6A, we refer to the DRACH motif as the motif linked directly to the positive label. We additionally compared this with PhyloAug’s near-mean selection regime (0.8≤r≤1.20.8\leq r \leq 1.2), an accelerated regime with no upper bound (r≥1.2r \geq 1.2) and a conserved regime with no lower bound (r≤0.8r \leq 0.8). We find that DRACH motif disruption, where augmentation irreparably destroyed the biological motif, occurred in 5.2% of near-mean-selection variants, compared with 5.4% for accelerated and 10.5% for conserved selection. For sequence-to-function tasks, we could not find experimentally validated scores for our generated variants, thus we calculate a proxy score using original and augmented sequences and the external DeepSpCas9 predictor. gRNA was the only eligible task with DeepSpCas9, so we limit results to gRNA. When comparing un-augmented and augmented predicted scores, we calculated Lin’s concordance correlation coefficient (CCC) agreement of 0.707, 0.701 and 0.667, respectively for near-mean, accelerated and conserved augmentation. We additionally calculate a median deviation of 0.0713 for near-mean and accelerated, and 0.0773 for conserved. These analyses provide complementary proxy evidence of predicted activity stability and motif preservation, while stopping short of experimental verification of variant function.

ModelNo-AugPhyloAugDRACH-constrained
RNA-MSM0.6293$\pm$0.048+0.058$\pm$0.028+0.073$\pm$0.032
SpliceBERT0.6634$\pm$0.006+0.037$\pm$0.012+0.044$\pm$0.015
BiGamba0.6351$\pm$0.011+0.066$\pm$0.003+0.077$\pm$0.002
RNA-FM0.6760$\pm$0.007+0.044$\pm$0.010+0.049$\pm$0.009
RNAErnie0.6511$\pm$0.027+0.026$\pm$0.038+0.044$\pm$0.042
HyenaDNA0.6845$\pm$0.021+0.034$\pm$0.011+0.037$\pm$0.012
RNA-BERT0.5953$\pm$0.003+0.049$\pm$0.013+0.054$\pm$0.015
CNN0.519$\pm$0.004-0.009$\pm$0.007+0.006$\pm$0.006
ResNet0.529$\pm$0.006+0.007$\pm$0.006+0.013$\pm$0.001
mean, all 90.6203+0.0347+0.0441
mean, 7 pretrained0.6478+0.0449+0.0540

Lastly, we compared native PhyloAug with a reference that enforces perfect DRACH preservation using sequence-based constraints. Across nine models, native PhyloAug improved mean m6A F1 over NoAug by 0.0347, compared with 0.0441 for the motif-preserving reference. Native PhyloAug therefore achieved approximately 79% of the reference’s observed augmentation gain. This quantifies the performance gap associated with stricter motif preservation and shows that substantial gains persist despite imperfect preservation. We will incorporate these findings and clarify the distinction between computational evidence supporting inherited-label augmentation and experimental validation of biological function, which remains a limitation and direction for future work.

AuthorsOpenReview ↗

Reviewer mk2S response Part II

W2: Could PhyloAug’s gains be explained by adding more training examples with noisy labels, rather than by phylogenetic guidance?

Our section 5.2 currently aims to address whether the gains arise from additional training examples, using alternative perturbation policies through Duplicate, Random and Struct-Infill controls, which were evaluated under matched augmentation-budget searches and training protocols. To explicitly answer the reviewer’s question, we additionally directly test the reviewer’s noisy-label explanation, using the same experimental setup and tasks as in 5.2. We trained 8 pre-trained backbones and additionally included 2 deep learning baselines on the same PhyloAug-generated sequences with their auxiliary labels randomly shuffled, keeping the original experimental labels and training configuration unchanged. Shuffled was ran for M∈1,2,4,6,8,10,12M \in {1,2,4,6,8,10,12}, with matched sequence and training budgets, changing only the synthetic labels.

m6A

ModelNoAugPhyloAugShuffled
HyenaDNA0.68450.71850.5458
SpliceBERT0.66340.70040.5557
RNAErnie0.65110.67710.4673
RNA-FM0.67600.72000.5337
BiGamba0.63510.70110.5341
RNA-MSM0.62930.68730.5449
RNA-BERT0.59530.64430.4254
RiNALMo-Giga0.71280.74800.4195
ResNet0.52900.53600.5030
CNN0.51900.51000.4789
Mean, 8 pretrained0.6559+0.0437−0.1526
Mean, all 100.6296+0.0347−0.1287

CRISPR

ModelNoAugPhyloAugShuffled
HyenaDNA0.3184$\pm$0.010.3479$\pm$0.010.2788$\pm$0.0270
SpliceBERT0.3198$\pm$0.030.3280$\pm$0.030.2621$\pm$0.0433
RNAErnie0.3566$\pm$0.010.3590$\pm$0.010.2829$\pm$0.0404
RNA-FM0.3646$\pm$0.020.4010$\pm$0.020.2970$\pm$0.0313
BiGamba0.1779$\pm$0.030.1839$\pm$0.020.1427$\pm$0.0298
RNA-MSM0.3252$\pm$0.020.3520$\pm$0.020.2705$\pm$0.0450
RNA-BERT0.2322$\pm$0.020.2890$\pm$0.020.1994$\pm$0.0655
RiNALMo-Giga0.2450$\pm$0.070.2760$\pm$0.060.1802$\pm$0.0553
ResNet0.213$\pm$0.010.242$\pm$0.030.1609$\pm$0.0597
CNN0.160$\pm$0.020.134$\pm$0.010.1440$\pm$0.0440
Mean, 8 pretrained0.2925+0.0246−0.0533
Mean, All models0.2713+0.0200−0.0494

Shuffling reduced mean performance below NoAug for all ten backbones on both tasks. Across the eight pretrained models, the mean m6A F1 change relative to NoAug was +0.0437 with PhyloAug versus −0.1526 with shuffled labels; corresponding CRISPR Spearman changes were +0.0246 versus −0.0533. Thus, adding the same generated sequences with shuffled supervision produces negative gains. Together with the existing matched controls, these results support the contribution of the inherited sequence-label association beyond data expansion alone. Regularisation may still contribute, and these results do not independently establish biological label preservation for every generated variant. We will add this as an additional baseline into Section 5.2 of the manuscript.

EC: How do you justify label inheritance without individual-level empirical validation, and how do you address the risks of label noise and misleading biological conclusions?

We thank the reviewer for raising this concern. We address the ICLR Code of Ethics principles of scientific excellence and honest, transparent reporting by explicitly treating label inheritance as a modelling assumption. Generated examples are used only for training; validation and test examples are unaugmented. Synthetic labels are not presented as experimental measurements, and our claims concern downstream predictive utility.

As detailed in our response to Q1, we have additionally investigated this assumption through an m6A motif-disruption audit, external-predictor assessment of gRNA activity, and comparison with a stricter motif-preserving reference. Near-mean selection produced 5.2% DRACH motif disruption, while native PhyloAug retained approximately 79% of the motif-preserving reference’s observed augmentation gain. These analyses quantify specific limitations and their associated performance gap, without claiming experimental verification of variant function.

To address the Code’s principle of avoiding harm, Appendix A explicitly discusses the lack of experimental validation and the need for empirical validation before prospective biological applications. Any release will identify each synthetic example and its parent, mark its label as inherited rather than measured, and document these limitations for users of the data and resulting models.

Reviewer_mK2SOpenReview ↗

Thanks for the new results

Many thanks for your new results. It is good that authors could be aware of "the lack of experimental validation and the need for empirical validation before prospective biological applications". It is also not surpised to see DRACH-constrained can bring gain--it demonstrating my core concern, PhyloAug, alone, as a straightforward data augmentation trick can neither ensure safty from a biological aspect nor fully explore the root-cause of performance gain. A simple prior, DRACH motif disruption, can bring biological meaningful perturbation to the sequence augmentation and also enhance the performance. I may suggest authors continue working on this line of research, til they can capture a relatively full picture of this area.

AuthorsOpenReview ↗

Further response to Reviewer mK2S

Thank you for raising the distinction between improved prediction and preservation of biological function. We agree that protecting DRACH improves mean m6A prediction in our experiment and that this result does not validate the methylation status of individual variants. To determine what phylogenetic guidance contributes once this biological prior is already in place, we conducted a new matched comparison.

Both augmentation policies preserved DRACH and used the same parent sequences, structure-conditioned generator, per-parent masking budget, training protocol and procedure for selecting the augmentation budget. Only the choice of edit sites differed: PhyloAug selected sites using evolutionary rates, whereas the control selected sites at random.

ModelNoAug (F1)Native PhyloAug (Δ\DeltaF1)PhyloAug + DRACH (Δ\DeltaF1)Random + DRACH (Δ\DeltaF1)
RNA-MSM0.6293 ±\pm 0.048+0.053 ±\pm 0.038+0.073 ±\pm 0.032+0.0364 ±\pm 0.042
SpliceBERT0.6634 ±\pm 0.006+0.041 ±\pm 0.005+0.044 ±\pm 0.015+0.0016 ±\pm 0.0136
BiGamba0.6351 ±\pm 0.011+0.063 ±\pm 0.010+0.077 ±\pm 0.002+0.0426 ±\pm 0.0115
RNA-FM0.6760 ±\pm 0.007+0.050 ±\pm 0.007+0.049 ±\pm 0.009+0.0058 ±\pm 0.0119
RNAErnie0.6511 ±\pm 0.027+0.027 ±\pm 0.022+0.044 ±\pm 0.042+0.0053 ±\pm 0.0404
HyenaDNA0.6845 ±\pm 0.021+0.042 ±\pm 0.013+0.037 ±\pm 0.012+0.0226 ±\pm 0.0180
RNA-BERT0.5953 ±\pm 0.026+0.031 ±\pm 0.013+0.054 ±\pm 0.015+0.0492 ±\pm 0.0140
CNN0.5190 ±\pm 0.004+0.001 ±\pm 0.004+0.006 ±\pm 0.006+0.0098 ±\pm 0.0059
ResNet0.5290 ±\pm 0.006−0.003 ±\pm 0.006+0.013 ±\pm 0.001+0.0031 ±\pm 0.0078
Mean, all 9 models0.6203+0.0339+0.0441+0.0196
Mean, 7 pretrained models0.6478+0.0439+0.0540+0.0234

For each model, NoAug reports mean held-out F1 ±\pm SD across five seeds. Δ\DeltaF1 is the difference in mean held-out F1 from NoAug. Boldface marks the higher mean between the two DRACH-protected policies.

With DRACH protected in both conditions, phylogenetic site selection yielded higher mean F1 in eight of nine models, including all seven pretrained backbones; CNN was the exception. Its mean advantage over random site selection was 2.45 F1 percentage points across all nine models and 3.06 points across the pretrained backbones.

The two comparisons answer different parts of the reviewer’s concern. Adding DRACH protection to native PhyloAug increased its mean gain over NoAug from 0.0339 to 0.0441, confirming the value of the task-specific prior in this setting. Holding that prior fixed and changing only the site selector reduced the mean gain from 0.0441 to 0.0196. Thus, where PhyloAug edits contributes to held-out m6A prediction even when the known motif is protected.

This identifies a contributor to predictive performance. It does not establish that the gain arises because individual variants retain their parent’s methylation status. Generated examples are used only for training, and their inherited labels remain a modelling assumption. We will include this control in the manuscript and distinguish its evidence for site selection from claims about variant function, which require independent experimental validation.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Updated Manuscript

We thank the reviewers for their constructive feedback. We have uploaded a revised manuscript incorporating the evolutionary-rate comparisons and shuffled-label control discussed in our responses. All changed areas are coloured as blue to allow for a direct comparison with the previous version. The remaining additional experiments are reported in our rebuttal responses. We will incorporate their experimental details and full numerical results into the appendix in a further manuscript revision before the discussion deadline on 18 November.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.