SpanDNA: Resolution-Aware Spectral Supervision for Genomic Pretraining

Masked language modeling (MLM) does not explicitly specify which sequence-wide statistics should be readily accessible from a pooled genomic representation. We study a training-only auxiliary objective based on the Resolution-aware Nucleotide Spectrum (RNS), a compact Fourier representation of nucleotide indicator signals that combines composition with phase-sensitive low-frequency coordinates. predicts a 36-dimensional RNS-4 target jointly with MLM and discards the auxiliary head after pretraining. Using a 117M Transformer with the same pretraining data and update budget, three pretraining seeds, and three finetuning seeds per checkpoint, mean MCC across 18 tasks increases from 0.6421 to 0.6500. The mean gain is positive for each of the three reported pretraining seeds, and 15 of 18 task-level means improve. Fixed-descriptor experiments show that RNS contains predictive signal beyond composition under linear classification. In a single-pair representation analysis, frozen linear probes recover the 32 non-DC coordinates with $R^2=0.909$ after RNS supervision, compared with $-0.176$ under MLM alone. These results show that RNS is an effective auxiliary target in the evaluated setting and that it changes the linear accessibility of supervised sequence-level statistics without modifying the deployed encoder.