- Rating
- 4(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3(+1)
Read review
Paper summary
This study looks at genomic foundation models and how they represent a nucleotide’s position within its codon (codon phase). Across four models, coding regions show stronger three-nucleotide periodicity than non-coding regions, and in three of them, deeper layers organize around codon phase. Rotating RNA models like MIMIC by 120 degrees in a fitted phase plane actually shifts the decoded reading frame by just one nucleotide while three full rotations restore everything. The authors trace one decoder attention head reading amino acid identity mostly from the very middle nucleotide of each single codon. Finally, on the POLG gene, combining rotation with start-site conditioning produces an alternative protein encoded in an overlapping frame.
Critical strengths
- The reading-frame steering result is clean and well controlled. A 120 degree rotation shifts the frame by one nucleotide, rotations compose as expected, and the effect stays local when applied to a single exon, while random-plane, scaling and UTR controls have little effect. Earlier work showed that codon phase can be decoded from genomic models; this result shows that the model actually uses it.
- I also found the readout built only from the model’s own weights useful, because it separates what a probe can decode from what the decoder actually reads. Amino acid identity is decodable early in the encoder but reaches the decoder only in the late blocks, which is a good lesson for interpreting biological models more generally.
Critical weaknesses
- The mechanism is traced only at its two ends. Steering works up to blocks 15 to 17 and amino acid identity reaches the decoder after blocks 18 to 19, but the step in between, where phase and nucleotides are combined into an amino acid, is not examined. The single-head result also seems limited to the assay where only one residue is hidden, and the Discussion itself notes that other routes compensate in less constrained settings. The POLG case is weaker than the abstract suggests: conditioning on the real POLGARF start fails, and the version that works uses a start one nucleotide off, supplies 15 protein residues, and is tested on a single transcript. Since both offsets were chosen with the answer known, I read this as a demonstration that the two controls compose, and not as recovery of the alternative protein.
- Some claims need stronger baselines. Because the phase plane is built from the three phase centroids, a 120 degree rotation roughly moves each token to the next centroid. The three plateaus and the three-fold composition would follow from the model reading phase as three categories, so they do not by themselves show a circular geometry. In the periodicity analysis, coding DNA is already period-3 at the sequence level, so the 85x enrichment needs the raw sequence as a reference, and the layer reported in Figure 2c is chosen to maximize the contrast. Finally, HyenaDNA’s weak result is attributed to its lack of attention, although it is also far smaller than the other three models.
Questions for authors
- How many protein residues are visible to the decoder in the steering experiments, and does steering still work with the entire protein hidden?
- What happens in encoder blocks 17 to 19 when a rotation is applied?
- Does combining start-site conditioning with rotation recover alternative proteins in other transcripts with overlapping frames? (without choosing the start offset by hand)
4 comments
In the periodicity analysis, coding DNA is already period-3 at the sequence level, so the 85x enrichment needs the raw sequence as a reference, and the layer reported in Figure 2c is chosen to maximize the contrast.
Coding DNA itself has a 3.25× coding/non-coding period-3 contrast when its four nucleotide tracks (A, C, G, U one-hot vectors) are scored with the same spectral pipeline as Fig 2c. Each model's input layer sits at about this level (MIMIC 3.7×, Evo 2 3.5×). MIMIC's final layer exceeds the raw sequence 26-fold, Evo 2's block 14 38-fold and NTv3 7.6-fold, so the enrichment is built by the networks. HyenaDNA (0.8×) does not exceed the sequence. To check that picking each model's strongest layer does not inflate its contrast reading, we chose the layer on one half of the molecules and measured the contrast on the other half. This reproduces every result within 9%, and for MIMIC picks the final layer. We will add the raw-sequence reference to Fig 2c.
HyenaDNA's weak result is attributed to its lack of attention, although it is also far smaller than the other three models.
We agree that we cannot separate model size and architecture effects. Across all public HyenaDNA checkpoints (0.44–6.55M parameters, 16k–1M context), the period-3 contrast stays at the level of the raw sequence (0.8–1.1× the sequence reference), with no trend in size. We will therefore no longer attribute the weak result to the absence of attention.
How many protein residues are visible to the decoder in the steering experiments, and does steering still work with the entire protein hidden?
In the rotation-only steering tests in section 5, we do not provide any visible protein residues, so all reported steering is with the entire protein hidden. Those results describe rotating the latent representation of the embedded RNA alone.
For the natural transcript experiment results presented above, we condition with 15 residues from the alternate protein (which are excluded from scoring), and while we note that protein prefix conditioning does improve compositional steering (0.36 vs 0.22 without), interestingly the specific residues given do not greatly impact the performance (0.35 with the main protein's residues / 0.31 with random residues).
We would like to thank the reviewer again for taking the time to assess our work and providing feedback to help us make it more generalizable and robust. We hope we have addressed any remaining concerns with these updated results and clarifications.
I thank the authors for addressing my questions and concerns. I think the revised abstract should also reflect the new 162 transcripts results. I raised my score to 4.
Thank you for your thoughtful comments and critiques. We agree that the abstract should emphasize the cohort-level natural transcript results and will revise the manuscript to reflect this.
We thank the reviewer for their helpful clarification questions and specific suggestions to explore the mechanism further.
Thanks to the reviewer’s comments, we applied our weight-only readout to rotated encoders. After a rotation at the output of block 17, the readout at the shifted codon's middle nucleotide gives the shifted codon's amino acid for only 0.16 of residues if the encoder is cut there, rising to 0.77 after block 18, 0.89 after block 19's attention and 0.98 after its feed-forward layer. In the same run without rotation (whole protein hidden), the original codon's amino acid is already partly readable at block 17 (0.52 → 0.86 → 0.98 → 1.00). After rotation, blocks 18–19 compute the amino acid again, now for the codon the rotated phase defines, while the rotated phase itself persists (+118° in 99.8% of tokens). Blocks 18–19 therefore compute the amino acid of whichever codon the phase variable defines, which is also why a rotation applied after these blocks no longer changes the protein.
We also examined how nucleotides are combined into an amino acid. An isolated block-19 feed-forward write selects the genetic code's amino acid for 0.97 of held-out codons, whereas an additive model of its inputs reaches only 0.64, so the conversion is a gated, nonlinear combination of nucleotide evidence computed in blocks 18–19. We will add these results to section 6.
We will update the manuscript to emphasize that in both the single-residue-masked setting and the whole-protein-masked setting, replacing this head's output with the value at the middle nucleotide of the target codon yields the correct amino acid, showing that head 19 is still a sufficient readout route when given the correct position.
We agree with the reviewer’s assessment of the POLG/POLGARF result and will update the abstract and result wording to reflect compositional multimodal conditioning and causal steering, and not unconditional generation of the alternative protein.
To further characterize this behavior, we test 162 additional natural transcripts with alternate reading frames. We perform the same rotation intervention and multimodal CDS start conditioning as in POLG, and annotate each alternative ORF separately at its natural start and at the nearest position in phase with the main start (one nucleotide after the CUG for POLG). We find that composition success varies across samples.
The rotation alone steers the output to the alternative reading frame (fraction of transcripts 0.92), but does not move to the alternate start location (0.00). Providing the alternative start site as multimodal conditioning without rotating almost never produces the alternate protein (0.01), but composing the two operations produces the desired alternate protein (0.36). This composition works most often when the alternative start lies downstream of the main start (0.67 vs 0.07 upstream) and where it is an AUG codon (0.71 vs 0.27 for other start codons, 1.00 for downstream AUGs vs 0.57 for other starts downstream). POLGARF is a CUG start upstream of POLG, so it is a rare functional example in the cohort as a whole. Conditioning instead on the in-phase position, as in POLG, also composes with rotation but slightly less often (0.30 vs 0.36), largely on the same transcripts. In POLG itself, only the in-phase position works.
Full results can be found in Table 2 in the general response. These expanded results allow us to quantitatively assess how a multimodal model honors modality conditioning based on causal interventions or on the presence of other modalities, and can inform training paradigms for future multimodal foundation model development.
We agree with the reviewer. Our interventions show a planar, direction-coded three-state variable on which ±120° rotations act as the cyclic group C₃, read out categorically. Sweeping the rotation angle snaps the output to one of the three frames. We will update our wording in the revised manuscript to reflect this description.