- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces Factorised Nucleotide Supervision. It is a simple and straightforward idea to supervise causal genomic language models. It derives the marginal likelihood of each individual nucleotide from the output k-mer logits. The authors provide empirical results of GeneratorV2 on Sequence Recovery, Variant Effect Prediction, Embedding Analysis and Nucleotide Transformer Benchmark.
Critical strengths
Factorised Nucleotide Supervision provides nucleotide-level supervision on genomic language models with non-overlapping k-mers tokenisation. It is simple and effective.
Critical weaknesses
- The authors do not provide an FNS-only ablation. Several training choices described in the appendices lack controlled ablations as well.
- Section 5.4’s title is confusing. It is "SUPERVISED FINE-TUNING AND TRAINING-FREE EVALUATION" but Section 5.4 only presents supervised fine-tuning results.
Questions for authors
- Could you provide controlled ablations for FNS and the training choices you described in the appendices? What exactly changed in pretraining between GeneratorV1 and GeneratorV2?
- Could you report inference and training FLOPs under identical nucleotide input length of Evo1, Evo2 and GeneratorV2.