- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates CovFit, an ESM-2-based model for predicting viral fitness. The authors select 10 biologically relevant SARS-CoV-2 spike protein mutations that the model predicts to be beneficial and construct original, single-point-mutant, and masked sequences for each mutation. Through activation patching and component ablation, they identify similar information-flow pathways across several mutations: attention heads in later layers aggregate mutation information at specific positions, MLPs process this information, and final-layer attention heads transfer it to the <cls> representation for fitness prediction. The authors then construct and evaluate a detailed computational circuit for L455F. They also find that some highly conserved positions consistently receive substantial attention and exhibit large representation norms, suggesting that these positions may serve as attention sinks.
Critical strengths
- Interventional experiments provide evidence beyond attention visualization.
The authors combine activation patching, path patching, and MLP-output ablation to test whether specific components affect prediction differences. This provides a stronger basis for assigning computational roles than high attention weights alone. The original and mutated sequences differ only at the target position, which also reduces background differences at the input level and makes internal interventions easier to interpret. - The analysis examines both shared patterns across mutations and a detailed circuit for an individual mutation.
The paper investigates shared computational structures in later layers across multiple mutations while also examining L455F, which is not fully explained by the simpler pathway. This design reveals both common mechanisms and case-specific differences, extending the analysis beyond a single example. - The conserved-site analysis offers an observation worth further investigation.
The authors connect site conservation with attention concentration and high representation norms, examining this relationship in both CovFit and the intermediate model ESM_coronaviridae. This observation provides a starting point for investigating how the statistical properties of protein sequences shape internal Transformer computations.
Critical weaknesses
- The intervention-score formula contradicts its textual interpretation, limiting the verifiability of the core results.
According to the formula in the appendix, the score is −1 when an intervention has no effect and 0 when the mutation effect is completely removed. However, the main text and appendix state the opposite: “0 indicates no effect, and −1 indicates complete removal of the effect.” - Mutation selection limits the generalizability of the conclusions.
All 10 mutations are predicted by the model to be beneficial, and nine lie within the receptor-binding domain. Without neutral mutations, deleterious mutations, or cases in which the model makes incorrect predictions, it is difficult to determine whether the identified pathway specifically evaluates beneficial effects or instead performs general mutation encoding or information readout.
Questions for authors
- How is the intervention score actually implemented?
The formula in Appendix A.8 contradicts the interpretation that “0 indicates no effect and −1 indicates complete removal.” Please provide the formula used in the code and clarify whether the relevant figures, component rankings, or conclusions require correction. - What is the testable definition of an “MLP look-up”?
What are the keys and values, and where are they represented? Which experiments distinguish a look-up operation from general nonlinear feature processing? Can targeted modifications to the relevant representations alter the predicted mutation effect in a predictable way? - Does the identified pathway generalize to independent mutations?
Please validate it on mutations not used during mechanism discovery, covering different structural domains, beneficial/neutral/deleterious mutations, different substitutions at the same position, and cases in which the model makes incorrect predictions. Please report the proportions of cases that the pathway successfully explains and those it does not. - Does the L455F circuit contain many general-purpose components?
Please explain the criteria for including components and terminating the search, provide control circuits of comparable size, and explain the substantial deviations in Figure 3 when individual MLPs are removed. Does the circuit retain similar faithfulness on independent sequences or lineage backgrounds? - Are the path-patching conclusions robust to the intervention method?
Please compare different reference activations or ablation methods and explain how LayerNorm, the residual stream, and frozen components are handled. How does freezing intermediate components justify the claim that it “removes all confounding elements”? - Do conserved sites actually perform the proposed computational functions?
Can targeted interventions compare these sites with matched nonconserved positions? Does the enrichment persist after controlling for amino acid identity, position, and structural domain? Are the findings robust to the thresholds and definition of conservation?