- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper identifies a small set of directions within LLMs that distinguish specialized text, such as medical content, from general text. This signal becomes stronger in deeper layers of trained models but not in untrained models. Further studies show that removing these directions harms performance on domain-specific text.
Critical strengths
-
The controls are comprehensive, including random models with pretrained embeddings, matched-rank random and PCA subspaces, and arbitrary-partition and style controls. Fixed-rank comparisons help rule out dimensionality as a confound in the depth-wise trends
-
The analysis spans five model families with diverse architectures. Subspace ablations also show more selective damage to the corpora used for subspace construction than matched-rank PCA ablations, providing evidence beyond classification or visualization.
Critical weaknesses
-
The core idea of is introduced without clear motivation. It is unclear how it connects to the initial research question or why this layer-wise quantity is relevant to understanding how models “store domain identity.”
-
Some terminology in the paper is confusing. For example, “calibration” appears to refer simply to subspace estimation using what seems to be an evaluation/test set, rather than calibration as commonly understood in the LLM literature. The paper also does not clearly explain why key metrics such as Fisher S/N provide appropriate evidence for the claimed second-moment code.
-
The ablation experiments rely on in-sample evaluation, reusing text used to construct the subspace. Consequently, the selective performance degradation does not establish that the same causal effect holds on unseen documents.
-
The paper is poorly presented, making its arguments and logical progression difficult to follow. As a human reader, I found most of the logic difficult to understand despite my best efforts.
Questions for authors
Q1: The paper states that, of the 10 initial claims, 4 were withdrawn, 4 were narrowed, and 2 were retained. I am curious whether the authors randomly selected (with AI) the initial 10 claims and then removed those that were unsupported. If not, how were the initial 10 claims formulated?