- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper attempts to understand how answers emerge in layer prior to the final layer. It does so using probing to decode something (the final answer, I assume) from each layer, and aims to find the earliest readable representation. The authors define this as the earliest layer achieving readability (again, a novel metric) within epsilon of that of the final layer. The precise probes used are not clear; MLP probes appear to have been used, and linear probes as well. The authors test a variety of small (<= 1b params) models using this methodology. It's actually not clear to me what they conclude, besides that some answer-relevant information can be recovered form intermediate layers.
Critical strengths
None
Critical weaknesses
Significant presentation issues: this paper does not follow the ICLR conference format. It does not include any figures in the main text that would substantiate its claims, and includes only one table. So, to what extent it makes any coherent, falsifiable claims, all evidence for these comes from numbers cited directly in the text, or figures and tables in the appendix. The paper does not cite much relevant work, declining to cite even those papers originating the models it studies.
This paper is also, as far as I can tell, entirely AI-generated, replete with the novel and incomprehensible compound nouns so characteristic of AI-generated text (what, pray tell, is a "protocol-relative object of study" or an "in-distribution memory-compatibility ranking protocol"?). As is also typical of AI-generated text, this paper lacks clear descriptions of its methods. The authors(' coding agents) probe for something related to the final answer, but what is that something? And what kind of probes are used? I could not discern the answer to these questions. I could not find even a simple example of what was given as input to the models under study.
In short, this paper is not human-readable. I do not think it should have been sent out for review.
Questions for authors
I do not have any questions whose answers would materially affect my assessment. More precisely, no response from the authors could convince me to accept this paper.