- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes the Answer-Basin Representation Hypothesis (ABRH) as an alternative perspective on the Linear Representation Hypothesis. It defines an answer measure over model outputs, where continuations leading to the same answer form an answer basin. ABRH suggests that shared linear directions in hidden states organize statistics of this measure, including answer concentration before generation and relative answer mass after generation. The paper further argues that concept-related probing and steering effects depend on how external concept labels align with the model's answer measure.
The authors evaluate this hypothesis across Qwen2.5 and Gemma-3 models on GSM8K, ARC-Challenge, BBQ, and SafetyBench. Their experiments show that linear probes can predict answer concentration and mass ordering, including when trained only on pairs of incorrect answers. They also demonstrate that changing the relationship between correctness labels and answer mass can reverse probing predictions and the relative effects of activation steering.
Critical strengths
-
The paper provides a relatively systematic formulation of answer basins, answer measures, collision concentration, and log-mass contrasts. Connecting pre-generation concentration and post-generation answer preferences through a common probabilistic perspective is an interesting conceptual contribution. This framework offers a potentially useful way to reconsider what linear probes and steering vectors actually capture, beyond their apparent association with external concept labels.
-
The alignment/conflict experiments are particularly interesting because correctness labels remain unchanged while their relationship with answer mass is reversed. The wrong-only experiments further show that a linear probe can recover answer-mass ordering without contrasting correct and incorrect labels during training. The steering experiments also highlight a practical concern: a vector constructed to favor correct answers can relatively increase the preference for incorrect answers when they have higher initial mass. These observations are valuable for understanding the reliability and limitations of representation-based interpretability methods.
Critical weaknesses
- The experiments do not sufficiently disentangle answer-mass information from concept-related representations, leaving the central interpretation underdetermined.
My main concern is that the experiments establish a relationship between answer mass and linear representations, but do not demonstrate that concept-related linear structure is primarily explained by answer mass rather than by external concepts. In Figure 3, Align and Conflict probes are trained using extreme positive or negative mass gaps while keeping correctness labels unchanged. However, a probe that predominantly learns answer-mass ordering would naturally produce the observed reversal when correctness and mass are positively associated in the test distribution. This result is compatible with the authors' interpretation, but it is also compatible with concept-related directions existing independently and being dominated by mass-related signals under the selected training conditions. The wrong-only experiments do not fully resolve this ambiguity. For example, in Table 2, Qwen2.5-7B achieves 78.03% Top-1 correctness with wrong-only supervision, compared with 79.36% under High-gap supervision. Yet the paper does not report the Top-1 accuracy obtained by directly selecting the highest-mass answer under the candidate measure. Since the mean-logp baseline uses a different scoring protocol (Appendix B.2), it cannot establish whether these probes provide information beyond recovering the model's own answer preferences.
A related concern applies to the pre-generation concentration results (Table 1). Answer concentration may correlate strongly with question difficulty or model confidence. Without controlling for these factors, it is difficult to determine what additional representational structure ABRH identifies beyond existing uncertainty or confidence-related signals, particularly given prior work on semantic entropy probes (Kossen et al., 2024) and question-only correctness probing (Cencerrado et al., 2025). More generally, Eqs. 5–6 formulate an idealized universal ordering, whereas the empirical results establish approximate statistical correlations (e.g., 63%–78% ordering accuracy in Table 1). Although the authors acknowledge the idealized nature of the hypothesis (L160–161), the interpretation and conclusions would benefit from more explicitly distinguishing the formal hypothesis from what is empirically supported. I consider this a central weakness because the paper's main contribution is not simply that answer mass can be linearly decoded, but that answer-measure statistics provide an alternative explanation of concept-related linear representations. The current experiments do not yet sufficiently distinguish these claims.
- The steering experiments and answer-measure constructions provide limited validation of the proposed mechanism and its generality.
The paper proposes an answer-measure transport hypothesis in Eq. 8, predicting that steering along a mass-oriented direction preferentially amplifies answers with higher initial mass. However, the main intervention in Figure 5 uses a correct-minus-wrong vector constructed from label-aligned source examples, rather than the explicitly mass-oriented direction defined in Appendix A.2. Because correctness and answer mass are already aligned in the source data, the observed reversal on conflict examples does not independently establish the proposed mass-transport mechanism. The theoretical analysis also does not fully bridge this gap. Proposition A.1 provides a change-of-measure identity, but does not establish the directional amplification predicted by Eq. 8. Thus, the central steering prediction remains largely empirical rather than theoretically derived. The experimental scope further limits the conclusions. Figure 5 evaluates only Gemma-3-4B-PT on BBQ and SafetyBench, using changes in length-normalized answer log-probability. Appendix C provides additional answer-letter logit results, but neither experiment directly measures changes in generated answers or safety-related behavior. Therefore, the results demonstrate shifts in model preference scores rather than verified changes in downstream generation behavior.
There is also an important distinction between the generation-based pushforward measure in Eq. 1 and the finite-choice candidate measures used in much of the evaluation. Both are explicitly defined by the paper, but it remains unclear how consistently the proposed representational principles transfer between these different constructions. The open-generation GSM8K experiments provide useful additional evidence, yet their alignment/conflict results are less consistent than those in the finite-choice settings (Table 3 and Appendix E, Figure 12). Finally, several representation analyses use different extraction layers (Appendices B and D), without a systematic layer-sensitivity analysis. This makes it difficult to assess how robust the proposed shared directions are to layer selection. Together, these limitations make the proposed mechanism plausible and interesting, but not yet sufficiently established to support the paper's broad interpretation of probing and steering.
Questions for authors
-
Can the authors evaluate correctness probing after controlling for answer mass, for example through mass-matched correct/incorrect comparisons or residualizing mass-related information from the representations? How much concept-related predictive signal remains? This would help distinguish whether mass-related directions replace or merely coexist with concept-related directions.
-
What is the held-out Top-1 correctness of directly selecting under the candidate measure? How does this compare with the High-gap and Wrong-only probes in Table 2? This comparison seems necessary for interpreting the reported correctness gains independently of the model's existing answer preferences.
-
How robust is the pre-generation concentration readout after controlling for question difficulty, correctness, or other uncertainty measures, such as semantic entropy? In particular, do confidently incorrect questions exhibit the same concentration-related linear structure?
-
Can the authors directly evaluate the relative answer-mass amplification predicted by Eq. 8, preferably using a mass-oriented steering direction with appropriate control vectors? Do the observed preference-score changes also translate into changes in actual answer selection or generated outputs?
-
How sensitive are the probing and steering findings to the selected layers and answer-measure protocols? In particular, which conclusions remain consistent when moving from finite-choice candidate scoring to generation-based answer measures?
5 comments
2. The highest-mass baseline and the interpretation of probe correctness (W1, Q2)
The reviewer requests a comparison between directly selecting the highest-mass answer and probe correctness; this comparison supports our interpretation. On the same test questions, directly selecting the highest-mass answer gives the correctness level associated with fully recovering the answer ordering:
| Model | Highest-mass selection | High (Table 2) | Wrong-only (Table 2) |
|---|---|---|---|
| Qwen2.5-7B | 90.39% | 79.36% | 78.03% |
| Qwen2.5-14B | 93.73% | 82.06% | 80.95% |
| Gemma-3-4B-PT | 76.91% | 73.03% | 69.70% |
| Gemma-3-12B-PT | 91.33% | 86.56% | 83.28% |
All probes fall below this reference, with no indication of performance beyond the model's own answer preferences. The probes capture linear structure organized by mass differences and use it to recover answer ordering. Wrong-only results further show that this structure can be learned without correct–incorrect supervision contrasts. Their correctness performance arises from the alignment between the recovered answer ordering and correctness labels, rather than the probes independently learning correctness information.
3. Pre-generation concentration, difficulty, and uncertainty (W1, Q3)
Operational definitions and structural explanations of difficulty and uncertainty (W1, Q3)
The reviewer suggests that concentration may correlate with difficulty or confidence, but a prior question must be answered: what exactly does confidence mean here? The probability of a single continuation, the total mass of an answer, concentration across answers, and the probability mass of the correct-answer set are different quantities. For example, a particular wording can have low probability while all wordings yielding the same answer collectively have high probability. An answer distribution can also be highly concentrated on an incorrect answer. Calling all these quantities confidence does not distinguish their properties, much less identify which structure a hidden-state direction actually corresponds to.
The answer measure provides a unified and explicit foundation for these distinctions: the mass of an individual answer, concentration of the answer distribution, and correct-answer mass obtained by incorporating external labels. ABRH further proposes how these properties organize shared linear structure and thereby predicts the concept-level effects of the resulting directions. Keeping correctness supervision unchanged while changing the source label–mass relationship can reverse probe behavior. Holding a steering vector fixed while changing the target label–mass relationship can shift relative amplification from correct to incorrect answers. These are testable structural predictions.
Controlling for difficulty or confidence requires an explicit operational definition. If confidence specifically denotes mass or concentration, it already corresponds to an object studied in this paper. Renaming these objects confidence does not provide another explanation of shared linear directions, cross-question generalization, or intervention effects. An alternative explanation must specify its quantities and structural relationships and explain how they predict these controlled changes.
Distinction from semantic-entropy and correctness-readout studies (W1, Q3)
The reviewer cites Kossen et al. (2024) and Cencerrado et al. (2025), but showing that activations predict semantic entropy or correctness is a different question from explaining what primarily organizes the linear structures supporting those predictions. Predicting correctness still leaves open why the direction exhibits correctness-related behavior, what conditions that behavior depends on, and when it fails.
ABRH connects pre-generation concentration readout, post-generation mass ordering, and relative gains under steering, and uses the label–mass relationship to make specific predictions. Its contribution advances from "a signal can be decoded" to proposing and testing an organizing principle that connects multiple forms of linear structure and explains concept-level effects, failures, and reversals.
4. Steering directions, relative gains, and random controls (W2, Q4)
The relationship between correct-minus-wrong and mass-oriented directions (W2, Q4)
The reviewer argues that the correct-minus-wrong vector in Figure 5 differs from the mass-oriented direction in Appendix A.2, but they coincide term by term on the aligned source data used in our experiments. Appendix B.4 explicitly states that the correct answer has higher mass in every source pair. Consequently, in Equation 16 is always , and its empirical average is exactly the correct-minus-wrong vector used in Figure 5.
Satisfying both the label-defined and mass-oriented definitions is central to this controlled design. We preserve the conventional correct-minus-wrong construction and the meaning of correctness labels, while using aligned sources so that the resulting vector also satisfies Equation 16. We then hold this vector fixed and compare aligned and conflict targets, testing whether its concept-level effect reverses with the target label–mass relationship. The central test is whether the same correctness-defined vector shifts its relative amplification from correct to incorrect answers as target mass ordering changes.
The roles of Proposition A.1 and Equation 8 (W2)
We explicitly introduces Equation 8 as an additional transport hypothesis. Proposition A.1 establishes the transformation relationship between answer measures before and after intervention; Equation 8 proposes a directional prediction for relative gains; and the steering experiments test that prediction. They serve the distinct roles of mathematical description, hypothesis formulation, and empirical evaluation. We do not present Equation 8 as a corollary of Proposition A.1.
Relative amplification versus answer selection and generation (W2, Q4)
We deliberately align the evaluation with the mechanistic quantity stated in Equation 8: the relative mass gains of answers under intervention. Answer flips are coarse-grained outcomes influenced by initial answer margins and the decision rule. They cannot distinguish an absence of effect from systematic amplification that does not cross the decision boundary. In particular, for deterministic binary selection, Equation 8 predicts further amplification of the initially higher-mass answer. The absence of a flip therefore need not indicate an absence of intervention effects; it may correspond precisely to relative amplification consistent with the prediction.
Continuous relative gains capture both direction and magnitude and directly test Equation 8. Even when the final choice remains unchanged, they identify which answer receives greater relative amplification. Free-generation outputs are additionally affected by subsequent token-conditional distributions, autoregressive feedback, and sampling, and thus reflect the combined outcome of the entire generation process.
Original, reversed, and norm-matched random directions (W2, Q4)
We add control-vector experiments covering two models. Consistent with the protocol in Table 1, each question retains two substantive options. We compare the original direction, its reverse, and multiple norm-matched random directions:
| Model | Intervention direction | Mean relative gain | Proportion with greater gain for the higher-mass answer (Aligned / Conflict) |
|---|---|---|---|
| Gemma-3-4B-PT | Original | 0.0108 | 95% / 84% |
| Reversed | −0.0105 | 4% / 16% | |
| Norm-matched random (mean) | 0.0006 | — | |
| Qwen2.5-7B | Original | 0.0332 | 95% / 82% |
| Reversed | −0.0344 | 5% / 17% | |
| Norm-matched random (mean) | 0.0037 | — |
The original directions produce significantly greater relative gains than the random controls, with paired confidence intervals excluding zero. Reversing the intervention reverses the gain orientation. In particular, on conflict targets, the same correctness-defined vector predominantly gives greater relative amplification to the higher-mass incorrect answer. A correct-minus-wrong construction does not guarantee a fixed correctness-enhancing effect; target mass ordering predicts which answer receives the larger relative gain.
Evidence across models and tasks (W2)
Figure 5 provides fixed-vector intervention evidence across two tasks, while the additional controls test directional effects in two models. Figure 6 further shows across multiple models and tasks that reversing the source label–mass relationship produces opposing projected directions while preserving the meaning of correctness labels. Source geometry, reversals of target concept-level effects under a fixed vector, and comparisons among original, reversed, and random directions jointly support the organizing role of the answer measure in steering directions and their concept-level effects.
We organize our response around Q1–Q5, addressing overlapping concerns from W1 and W2 together.
1. The organizing role of the answer measure and controlled correctness probing (W1, Q1)
The organizing explanation and independent correctness information (W1, Q1)
The reviewer suggests that "correctness directions may exist independently but be dominated by mass signals under the selected training conditions." This requires distinguishing two different questions. Our central contribution is to identify what primarily organizes the shared linear structures used by probing and steering, thereby explaining why these directions exhibit concept-level effects and why those effects succeed, fail, or reverse. Our main experiments test this organizing role of the answer measure, rather than proving that the model contains no independent correctness information. We additionally investigate the latter issue through correctness probing with controls on mass.
The reviewer argues that the experiments do not sufficiently establish an organizing explanation beyond linear decodability, but our evidence extends well beyond linear decodability. Figure 1 uses unsupervised PCA: without training a correctness probe, correct–incorrect geometric separation already decreases as the mass gap narrows, showing that this relationship precedes any probe fitting. Figure 3 keeps correctness-labeling rules unchanged and reverses only the training label–mass relationship, after which correctness performance reverses. The observation that "a probe learning mass ordering would naturally reverse" already adopts the answer-measure explanation emphasized by ABRH; the interpretation that "correctness supervision yields a correctness direction" alone does not predict this reversal. Wrong-only experiments further show that correct-answer selection can emerge without correct–incorrect supervision contrasts. Together, these experiments test whether concept labels or the answer measure explain the geometry, orientation, and generalization of these directions, beyond whether a statistic is linearly decodable.
Correctness readout after controlling for mass (W1, Q1)
To address whether correctness remains readable after controlling for mass, we conduct an additional experiment on ARC-C. We select correct–incorrect answer pairs with very small mass gaps, balance the numbers of pairs in which the correct or incorrect answer has higher mass, and match the gaps between these groups. This jointly limits mass differences and their association with correctness labels. We train probes using correctness supervision, maintaining question-disjoint training, validation, and test splits:
For reference, probes trained with mass-ordering supervision using the same number of training pairs under the normal gap distribution, and evaluated on the full test set, achieve ordering accuracy significantly above chance. When mass differences are restricted and their label association is balanced, the correctness probes do not show clear above-chance correctness discrimination.
The formal hypothesis and the scope of empirical support (W1)
Equations 5–6 state an idealized ordering relationship. The experiments are additionally affected by two sources of error: answer-measure estimation and approximation errors, including finite sampling and the approximation of the target answer measure through candidate scoring; and probe fitting and generalization errors. Observed ordering accuracy therefore measures how closely the practical readout follows the proposed ordering, rather than proving an exception-free relationship. We will explicitly distinguish the formal hypothesis from the scope of empirical support in the main text and conclusion.