- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper uses concepts and tools from abstract algebra to formalize symbolic equivalence and observational equivalence of hidden representations, and to study when the latter implies the former.
Critical strengths
- The definitions of the observation map, symbolic assignment, etc., build a bridge from abstract algebra to the problem studied, and this framing is potentially useful for further studies.
- This is balanced by the fact that Theorem 1 and some other early results are mostly "translations" of textbook math results, e.g., Theorem 3.5 in [1]; Dummit & Foote should also contain this result. This is not a weakness in itself, though, as long as the imported math can lead to interesting results in ML — but I think it does not; see below.
[1] Blyth, Thomas Scott. Module theory: an approach to linear algebra. University of St Andrews, 1990.
Critical weaknesses
- It is not clear how Proposition 2 and the analysis of fixed probes under a single hidden representation (Theorems 2 and 3) or under multiple hidden representations (Theorems 4 and 5; termed calibration in the paper) shed light on representation learning.
-
What is the practical significance of the sign-probe as a symbolic assignment? In particular, how do we choose the state vector , and what does it mean for a problem setting? When can we further obtain those multiple calibration states? What is their practical interpretation?
- The paper admits that the results do not "assign logical semantics to probe labels." But I think you should show some examples in which the probe labels have semantics, as evidence that your math does mean something in practice.
-
For example, Theorem 2 characterizes "when the labels produced by a fixed linear probe are exactly extractable under the remaining representation transformations." But why does this question matter?
-
Similarly, Proposition 2 establishes if-and-only-if conditions between symbolic equivalence and observational equivalence, which is interesting math, but it needs examples built on real DNNs to show that it is interesting for representation learning.
-
- The exposition writing requires much more human effort than it is now. Some example issues (ordered roughly by importance):
-
Is the gauge known from the problem setting? Examples?
-
Section 5 as a whole is unclear:
- Line 270 puts requirements on the global linear symmetry group. This is necessary for your math to be well-defined. But again, how are these requirements relevant to practical NNs?
- "…upper bounds on the actual observation-preserving group rather than classifications of observational fibres." This nearly identical sentence appears in both the Introduction and Section 5, with no further explanation. What is the "upper bound" (I guess it is related to Equation 1, which appears later and is itself unclear; see below), and what is the "classification"?
- Equation 1 and the surrounding discussion are unclear; e.g., what is an "architectural datum"? When in practical NNs does it happen that it "takes values in a space with a specified left group action"? "Stabilizer" is mentioned here for the first time since the Abstract ("architectural stabilizers"), and it is undefined!
-
What are the "allowed transformations", and how do we see from the statement of Theorem 5 that they "have zero disagreement"?
-
Lemma 1 says that is a subgroup, but it appears after you have already claimed this on the line above! Neither Lemma 1 nor Corollary 2 is used later, and they indeed interrupt the flow, so they should be moved to the Appendix.
-
It is odd that the term "covector" appears in the paper; this is a highly specialized term, but the paper uses it only to distinguish row and column vectors! Possibly a sign of AI slop.
-
- The Introduction (two paragraphs) and Related Work (one short paragraph) are far too thin to convey the significance and novelty of the work.
- No contributions are listed.
- Some related work is missing: [1] the mathematical formulation seems related to Higgins et al. (2018); [2] (possibly parallel), which gives identifiability result via concrete analysis of model symmetry of a nonlinear class, is arguably more related to your paper than Godfrey et al. (2022) or Grigsby et al. (2023), since the latter two do not seem to study identifiability.
[1] Irina Higgins, David Amos, David Pfau, Sébastien Racaniere, Loic Matthey, Danilo Rezende, and Alexander Lerchner. Towards a definition of disentangled representations. In arXiv preprint arXiv:1812.02230, 2018.
[2] Pengzhou Wu. "Beyond ICA: Identifiability by Symmetry Breaking." NeurIPS 2026.
Questions for authors
- My questions are interwoven with the Weaknesses.
- Given the current LLMs, it is challenging but not impossible that the paper could be improved close to the acceptance bar, and I will take a look it the revised paper.