- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper compares the effect of interventions across five Mamba-1 models of different parameter sizes. The authors construct synthetic texts to construct two-dimensional subspaces, testing whether restoring a selected component affects the model's internal sensitivity compared to a control. They successfully discover such a component at each of the model sizes, although they are selected independently and are not compared semantically across models. Comparing them across models shows partial similarity geometrically, and gradient analyses show that the component's effect can change based on how it's evaluated.
Critical strengths
None
Critical weaknesses
- The main experiment is not sufficiently specified. The paper doesn't clearly explain what XG2/XG4 are, what the six-cell substitution design is, or how they constructed the candidate planes.
- Many parts of the paper, including figures, seem AI-generated with little human iteration, making it completely unreadable. For example, many parts of the abstract are cryptic and difficult to decipher: "prospective transport, response-independent specificity, matched necessity, and restoration establish a recurrent-state mechanism on disjoint populations", "Coordinate-insensitive comparison of matched response-blind XG2/XG4 populations yields heterogeneous, intermediate cross-scale similarity", "a scale-local interventional role can recur without shared plane identity or exact geometric invariance". The acronyms XG2/XG4 are only defined later in the paper as "disjoint response-blind generator families" (which doesn't really help much), making them confusing when they appear in the abstract. Furthermore, Figure 1, which typically summarizes the paper's main result or contribution in a visually appealing way, is completely unintelligible to me: what is the relationship and significance of the vague phrases "controlled contrasts, ..., task-margin consequence" labeled "native susceptibility mechanism" and "task-functional bridge", what P3/P5 notations mean, what the arbitrary plus/minus signs in section C mean. These presentation issues, which continue throughout the rest of the paper, make it very difficult for a human to understand and assess the contribution of the paper. Therefore, the paper's current presentation does not meet the threshold for ICLR and should receive a clear rejection.
Questions for authors
Could the authors please describe the main experiment concretely, including an example of an XG2/XG4 input pair, what the six cells in the design represent, and the steps used to construct the five candidate planes?