Emergent Codon Frame Representations Reveal Mechanisms of Translation in Genomic Foundation Models

Interpreting scientific foundation models requires both identifying scientific variables in model representations and showing that the model uses those variables in its computation. As a test case with known ground truth, we first investigate how the triplet structure of coding RNA emerges in the representations of four genomic foundation models. Across all four models, representations of RNA coding regions exhibit stronger period-3 structure than non-coding regions, and in three of the four, coding region representations projected onto their top two principal components reorganize from nucleotide identity toward codon phase with depth. We use these findings as motivation to look more closely at the RNA-to-protein translation mechanism in MIMIC, a multimodal encoder-decoder foundation model trained jointly on RNA and protein modalities, and show that the codon phase representation is causally used for protein decoding. Rotating RNA representations from MIMIC's encoder by $120^{\circ }$ in a two-dimensional codon phase subspace shifts the model’s reading frame for protein decoding by one nucleotide. The inverse rotation induces the opposite shift and repeated rotations compose according to the expected modulo-three structure. Tracing the computation downstream, we find that amino acid identity becomes accessible through the decoder's own cross-attention channel late in encoder representations. A single cross-attention head in the decoder's final block reads the amino acid from the encoder's output representation at the codon's middle nucleotide, which it preferentially attends to, and supplying this position alone recovers the correct amino acid in the single-residue-masked assay. In a natural transcript with overlapping protein-coding reading frames, phase rotation and start-site conditioning separately control translation register and origin, together recovering the alternative protein. These results characterize mechanisms of protein translation in MIMIC and link an emergent biological coordinate to the computations that determine protein output.