- Rating
- 1(−1)
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper addresses the problem of conditional RNA sequence generation, an important and active area of research that remains computationally challenging. The authors aim to generate (or inpaint) class-conditioned RNA sequences with high sequence and structural fidelity. To achieve this, they propose RNA-MDLM, a Masked Discrete Language Model integrated with a modified variant of Classifier-Free Guidance (Mod-CFG). Additionally, the authors introduce RePaint Games, a benchmark designed to evaluate a model's ability to generate sequence patterns, structural motifs, and base-pairing interactions across different RNA classes.
Critical strengths
- The targeted problem is of high practical importance, and novel methodological approaches are welcome.
- The proposed RePaint Games benchmark is well-designed and provides strong utility for the broader research community.
- The authors invest significant effort into data preprocessing and curation, which is particularly crucial given the relative scarcity of established protocols for RNA Language Models.
- The exploration of the trade-off between sequence novelty and structural fidelity highlights a fundamental challenge in generative bio-modeling.
Critical weaknesses
- The ablation studies do not clearly isolate or demonstrate the individual contributions of all specific data curation choices and training objectives.
- The claim that adding random sequences to the training set improves performance is counterintuitive and lacks sufficient empirical support or mechanistic justification.
- Sequence novelty is evaluated relative to the validation set rather than the training set, which does not accurately measure memorization or true generative novelty.
- Key experimental results discussed in the main text are offloaded to the Appendix, which violates standard page-budget guidelines.
- The manuscript omits the exact number of training steps and total tokens/sequences seen by the model across reported checkpoints.
Questions for authors
Could the authors address the weaknesses 1, 2, 3, and 5?
Q1. The ablations do not isolate the contributions of individual data curationand training choices.
Each ablation carries a clear message, and the numbers are consistent across all three levels of analysis. Removing redundancy and shrinking the data hurts. Cluster 80 and Cluster 90 against Vanilla:
Cluster 90 is close to random on 2-mer statistics for miRNA, and both cluster models fold into much weaker structures than training.
Class balancing alone did not help. The 50K model trails Vanilla on recovery in every game (G1 0.516 against 0.613, G3 0.490 against 0.597, G9 0.575 against 0.696) and on k-mer fidelity for most types (miRNA 40.7% against 71.0%, ribozyme 7.5% against 43.2%).
The 500K model did improve things, with caveats (see Q2).
Q2. The claim that random sequences improve performance is counterintuitive and under-supported.
The 500K model does recover statistics very effectively on some types. It is the closest of all six models to validation rRNA structure (MFE/nt −0.418 against −0.413; bp fraction 0.644 against 0.640), and it beats Vanilla on k-mer fidelity for tRNA (64.3% against 62.7%) and snRNA (53.9% against 44.4%).
Though the distributions hint towards memorisation, on rRNA, the 500K model is worse than random on 2-mer JS divergence (improvement over random −83.8%), its GC content is 0.623 against validation 0.526, and its normalised novelty drops to 0.053 (the lowest of any model-type pair) against 0.209 for Vanilla on the same type. It also over-structures tRNA and snoRNA (MFE/nt −0.391 against −0.365, and −0.279 against −0.220).
Q3. Novelty is measured against the validation set, not the training set.
Accepted, and we have redone it against the training set. We see no concerning similarity; the generated sequences are further from the training data than real held-out RNA is, in 34 of 35 conditions by edit distance and 32 of 35 by alignment identity, with an exact match rate of 0.0 everywhere.
Q4. The exact number of training steps and tokens/sequences seen is missing.
As requested, we are mentioning the training steps here; the details of the datasets are provided in the original text