- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The authors want to study how different trained measurement instruments (here probes) assign different values to differences between representations.
Critical strengths
- I appreciate contributions that are outside of methodological advances, such as improving our understanding of the reliability of measurements or critiquing protocols.
Critical weaknesses
- W1. For me, the motivation of the paper remains unclear. The paper spends considerable space explaining that measurements depend on how they are obtained, but I still do not understand which concrete problem the experiments address. The main problem is that the authors try to write the paper, especially the earlier parts, very vaguely, without concrete examples of what it actually means in practice. Can the authors actually give an example of a representational claim that would be wrong because of the effect studied here, and explain what their result adds to this existing concern?
- W1.b The paper is difficult to follow because the authors decided to go with an abstract formulation that comes before a clear explanation of the experiment. Even after a reader has read through the front pages and the introduction, it is still unclear what the instrument is, what its states mean, and what the authors want to infer from its output. Also the appendix then introduces many experiment identifiers, local paths, and ledgers, while essential details remain scattered across sections.
- W2. It is also unclear what the state entropy measures about the representation. As far as I can understand, it is the entropy of an internal distribution in a learned module, but the paper does not establish which property of the representation it captures. For me, it was surprising that the authors list this as a limitation despite it being essential to the paper. Especially because without that connection, I am not convinced that a difference in this statistic tells us much about the reliability of representational claims.
- W3. For me, it is also unclear why the main comparison changes the objective from segmentation to reconstruction, especially since tasks reward different information. So why should the resulting instruments assign the same magnitude to differences between representations? Keeping the architecture and input fixed does not mean that the instruments measure the same property.
- W4. I would have found an experiment with conventional probing more convincing. Especially because the main setup uses a custom module with millions of parameters and measures its internal entropy. What was the reason why the authors didn't just keep the probing task fixed and vary training choices among well-performing probes, including a standard linear classification probe?
Questions for authors
See weaknesses.