- Rating
- 3(+2)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
Summary
This is a position paper arguing that the Linear Representation Hypothesis (LRH) is actually a family of hypotheses since it has several formulations that take place in different spaces and, depending on the space, different transformations keep the object equivalent. The authors propose that the LRH be accompanied by a representation claim, which is a four-tuple including the specification of the space in which the object lives, the group that acts on the object, the procedure that produces the object, and the predicate ultimately asserted. They further define a concept of the admissibility of a representation claim, which allows for some interesting observations about the impact of the architecture on the possible group. If the practice of stating this specification were to be adopted, it would make it easier to quickly see when analyses in different papers are comparable.
Recommendation
While the paper is well-motivated and the idea of stating a specification of a representation claim is a good one, the writing and organization need more improvement than is likely possible in a short rebuttal period and therefore I recommend the paper be rejected. My supporting arguments are the weak points listed below. Briefly, I think there is too great of a burden on the reader to work out the details of Table 1, an example of a representation claim, whether the paper is concerned with multiple formulations of "linear representation" or the linear representation hypothesis, and the relationship to existing work.
Critical strengths
- The idea to give a specification along with a claim about a linear representation is a good one, and the authors provide helpful examples.
Critical weaknesses
- Is the paper concerned with the ambiguity of the linear representation hypothesis or ambiguity of "linear representation"? The authors do not state the LRH. Section 2 is titled "The linear representation hypothesis is not one hypothesis". In section 2, they list different formulations of linear representation. However, is this really supporting the claim that "The linear representation hypothesis is not one hypothesis" or is it instead supporting the claim "there are multiple interpretations of linear representation". Is linear probing really treated as equivalent to the linear representation hypothesis in the literature? Maybe if the authors defined LRH it would be easier to understand their claims.
- The related work is mixed in throughout the paper rather than in a separate section. Consequently, the answer to the following question is scattered: have other authors discussing the LRH made an effort to discuss the implicit assumptions and take care with the issues the authors bring up? For example, Park et al (2024) discuss different interpretations of linear representation. While the authors cite Park et al several times, they should add an additional citations to make the following clear: are the three common formulations the authors list in Section 2 precisely the three interpretations Park et al list in Section 1? (I don't think they are, but I think there is enough similarity to warrant a citation).
- The paper in its current state is a collection of ideas and observations, some of which are interesting and nicely explained, but I think the authors need to spend some time thinking about what they want to emphasize in the 9-page limit and how to best organize it. Some paragraphs contain redundancies or sentences that don't seem to add much content to the paper, and some parts feel rushed. To make this critique more concrete: section 5 gives Table 1 which I think has potential to be one of the more interesting and useful parts of the paper, but currently has very little exposition. Adding definitions or citations so that the reader can find the definitions (what definition/estimator are you using for intrinsic dimension?) would help. Section 7 discusses how the Platonic Representation Hypothesis seems to suffer from some of the same issues which I think should be a lower priority than explaining Table 1. I find Section 8 confusing as written. My confusion about Section 8 is detailed in Questions For Authors/Additional feedback below.
Questions for authors
Questions whose answers could materially affect my assessment
The first question in critical weaknesses above.
Additional feedback with the aim to improve the paper
Introduction
- The second and fourth paragraph of the introduction seem redundant.
Section 2
- I recommend re-reading the second paragraph and rewriting to improve clarity.
Section 3
- Giving an example of a predicate could be helpful. Even better would be an example with G, M, F, P explained.
Section 4
- What is a reading point?
- Corollary 4.2, as written, is lengthy, including a short proof/justification (lines 260-261) instead of just consisting of a statement. Is the first statement the Corollary, and the rest is the proof (providing an example of an architecture that induces different symmetries at different reading points?)
Section 8
This section seems overly verbose for the content it contains. First, it states a 'reporting principle' but could this be stated earlier? Then, the authors write "This requirement has a real cost" but the first sentence after that begins "Some conclusions become narrower once their geometric assumptions are made explicit..." One can argue having conclusions stated corrected is a benefit, so perhaps there is a stronger cost to start with? Then the final two paragraphs begin with 'A second objection is...' and 'A third objection is...' respectively, so I'm taking that to mean that we are discussing potential objections to the position. I think it could be made more clear in the section introduction and title that this is where the section was headed.
Minor comments
- In the third paragraph, the authors should state what V and \mathbb{P}(V) are (lines 41 and 43)
- In the first occurrence of 'equivariance' (line 52) the authors should define it.
- Could help to say what you mean by: "changes of description", "nuisance group" (line 124)
3 comments
Engaging with the authors
Dear reviewer,
the authors have posted a reply to your review. Please have a look and interact with the authors as needed.
Thank you, Your AC
Thank you to the authors for the thoughtful response to my review and the revisions. I apologize for the delay in my response.
I am happy with the revisions. Specifically, I appreciate the edits to the introduction, which substantially improved the clarity for me, the addition of the example in section 3.5, and the revision of Sections 7 and 8 of the original paper.
Overall, I still see room for improvement in the exposition, especially around Table 1, where more detail would help the readers. But I think the ideas in this paper are important and the paper makes a valuable contribution by identifying this issue with the ambiguity of linear representation and providing a framework to address it. I therefore raise my score to a 3.
P.S. I also found this exerpt from your response very helpful:
Park et al. (2024) describe the LRH informally as the idea that "high-level concepts are represented linearly as directions in some representation space.”[…]Our paper focuses on the second question, particularly how to define linear representation through the choice of representation object and equivalence.
I think this is nicely stated and I would encourage you to add it to introduction!
We sincerely thank the reviewer for the thorough and careful reading of our paper. We especially appreciate the detailed comments and suggestions, which reflect a level of engagement that we do not take for granted, particularly given the time constraints of academic reviewing. We have revised the manuscript accordingly, and our point-by-point replies follow below.
We thank the reviewer for raising this important question.
We did not give a single definition of the Linear Representation Hypothesis (LRH) because there is no universally agreed-upon definition. This is precisely why we wrote this paper. Park et al. (2024) describe the LRH informally as the idea that "high-level concepts are represented linearly as directions in some representation space." This description raises two important questions. The first is how we define a concept, and the second is what it means for a concept to be represented linearly. Our paper focuses on the second question, particularly how to define linear representation through the choice of representation object and equivalence.
We also do not see the ambiguity of "linear representation" and that of the LRH as two separate issues. The LRH makes a claim about the conditions a representation must satisfy to be considered linear. Different definitions of these conditions lead to different versions of the LRH. Our goal is to understand how these conditions should be specified so that the hypothesis is well-defined. This matters because a hypothesis cannot be properly tested without a clear definition. Otherwise, different studies may claim to test the LRH while actually testing different hypotheses.
Regarding linear probing, it is often used to support the LRH. For example, Park et al. (2024) themselves link linear probing to their measurement view of linear representation. In our framework, successful linear probing shows that a concept can be read out using a linear function in the dual space V∗. However, it does not show that the concept is represented by a displacement vector in the primal space V, which is relevant when using the linear probe's weights to steer. It also does not justify comparing different probe weights using cosine similarity, since this requires a choice of inner product within V∗ instead of between V and V∗. These are additional claims that require further assumptions or evidence.
We thank the reviewer for this suggestion. We have reorganized Section 8 into a Discussion section with a dedicated Related Work paragraph. We have also moved the Platonic Representation Hypothesis discussion to the appendix to make more room.
The three formulations in Section 2 are not the same as the three views discussed by Park et al. (2024), although they are related. Park et al. provide a specific definition of linear representation, while our goal is to provide a general framework for specifying and comparing different versions of the LRH. We have made this distinction clearer in the revised manuscript.
We have made the following changes to improve the clarity and presentation of the paper: