- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes a method called Hidden States as States (HSS), which is put forward to describe the hidden representations of LLMs in terms of a sequence of discrete states. At each layer, the representations are clustered, and then the centers of the adjacent clusters are linked. In practice, every forward pass is transformed into a sequence of states across the layers in the target architecture. This approach describes the evolution of representations across layers and it is used to predict whether the model is correct and safe using a count-based classifier. Experiments are conducted using Llama and Qwen models (1B to 8B).
Critical strengths
- The simplicity and practicality of the approach, which needs no training and no labels.
- The experiments on activation editing are interestingly controlled, showing some actual evidence that the states identified are not just descriptive.
Critical weaknesses
-
I have some doubts about the evaluation protocol, which reduces the soundness of the experimental analysis on prediction. In particular, regarding the results shown in table 1: to the best of my understanding, they are achieved on a single split 40-60, most differences with the trained baselines are smaller than 3 points, and no standard deviations are reported. In addition, I could not find information on how some baseline hyperparameters (e.g., size and regularization of the MLP) were chosen. The MLP itself shows better performance than the proposed method in all the columns of table 1, and the other baselines are better on the safety columns. Based on these observations, the claim that the proposed method matches trained continuous probes on capability and safety benchmarks, given in the abstract, seems not supported by the given results.
-
A central part of the proposed method is the cross-layer alignment of states (section 3.3). However, the prediction results seem not to test this part of the method, as in fact the count-based predictor depends only on the clusters of each layer and not on the cross-layer alignment. Moreover, from what I could understand, the number of states per layer is sensitive to the number of samples, the selection tolerance and the clustering algorithm. Thereby, it risks not being really comparable across different settings.
Questions for authors
- Please report mean and std over different random splits for the results in tables 1 and 2, possibly including a statistical test against the baselines (this would support the claim in the abstract about the fact that the proposed approach matches the trained continuous probes).
- Please clarify how you chose the hyperparameters for the baselines and the settings for HSS.
- Please clarify what is adding the cross-layer alignments in terms of the results given in section 5 (as it seems that the count-based predictor is not depending on that).
- Given the sensitivity to the number samples, selection tolerance and clustering alg., please clarify how the number of states should be compared among the tested models and across the settings?
4 comments
Response to Reviewer E6Ez (2/2): W2, Q3-Q4
W2 / Q3
Question 3: Please clarify what is adding the cross-layer alignments in terms of the results given in section 5 (as it seems that the count-based predictor is not depending on that).
Response:
We thank the reviewer for pointing out this distinction. We agree that HSS-NB uses per-layer counts, so its predictions do not depend on cross-layer alignment. We did not attribute the prediction performance to alignment. Extracting additional predictive signals from the structure of the state map remains a promising next step.
At the same time, alignment gives states a shared identity across layers, which does help us understand and interpret the model. These allow us to distinguish a persistent failure-associated region from isolated low-accuracy clusters observed at individual layers. We can then examine where that region emerges, how long it persists, and which samples remain in it, enter it, or leave it. And we want to go further and understand why failures concentrate there. These are central questions in our analysis of hidden-state organization.
To further prove the necessity and legitimacy of alignment, we directly tested whether these geometric correspondences are behaviorally meaningful. For each cluster, we then measure the fraction of its samples whose final answers are correct. Across matched pairs of clusters in adjacent layers, we correlate these correctness rates on the log-odds scale, using random pairing within the same layer pairs as a control. The strong correlations show that low-accuracy regions tend to connect to low-accuracy regions, and high-accuracy regions to high-accuracy regions.
| Model–dataset | Geometric alignment: correctness log-odds correlation | Random pairing: mean |
|---|---|---|
| Qwen2–MATH | 0.93 | 0.01 |
| Llama-3.2–MATH | 0.92 | 0.02 |
| Qwen2–MMLU | 0.83 | 0.00 |
| Phi-4–MATH | 0.91 | 0.01 |
These results show that alignment links clusters in a behaviorally consistent way, providing a meaningful description of how hidden-state geometry evolves across layers.
W2 / Q4
Question 4: Given the sensitivity to the number samples, selection tolerance and clustering alg., please clarify how the number of states should be compared among the tested models and across the settings?
Response:
We thank the reviewer for carefully examining the state counts. The number of clusters naturally depends on the sample size, parameters, and clustering algorithm. These choices determine the granularity at which we describe the hidden space, but our main findings in failure prediction, interpretation, and hidden-space organization still hold across the resolutions. Finer partitions can reveal more lexical and contextual distinctions, consistent with Shafran et al.’s observations.
Our experiments provide evidence for both. In prediction, increasing the ICL tolerance substantially reduces the number of states, while much of the predictive information remains.
| Model–dataset | Mean states per layer, τ = 0% → 6% | AUROC, τ = 0% | AUROC, τ = 2% | AUROC, τ = 6% |
|---|---|---|---|---|
| Qwen2–MATH | 20.4 → 4.7 | 76.0 | 74.8 | 73.7 |
| Llama-3–MATH | 24.9 → 5.9 | 76.0 | 74.6 | 73.5 |
| Llama-3.2–MATH | 22.6 → 3.1 | 75.6 | 73.8 | 72.4 |
| Llama-2–WildJailbreak | 7.0 → 3.1 | 90.2 | 89.9 | 89.8 |
For interpretation, the behavioral consistency measured above remains strong at ICL tolerances of 2%, 3%, and 5% (, respectively). Middle-layer expansion also appears with both GMM and KMeans, despite their different partition sizes (Appendix B.4).
In general, absolute state counts should be compared using matched datasets and sample sizes, the same representation extraction procedure, and the same clustering and selection settings. Across models with different layers, we recommend comparing state-count on normalized layer depth, focusing on where expansion and compression occur and whether these patterns persist across reasonable changes in granularity. Following this approach, Figure 4(b) uses the same clustering settings and normalized depth to show that expansion and compression recur across the tested models, while their locations differ.
I thank the authors for the nice responses and for reporting the additional results. I appreciate the clarification on the hyper parameters of the baselines and the analysis about the cross-layer alignment.
The results over repeated splits, however, seem to refer to an exp. setting that is different from the one in Table 1. In facts, some models and dataset pairs seem to be different. Can you please clarify what changes wrt Table 1, and give mean / std over the splits for the same settings as in Table 1 (or, please update Table 1 in the revised manuscript)?
Moreover, in the new table, the MLP is still better in 5/7 columns (and no statistical test is given). I would suggest to reformulate the claim in the abstract regarding matching the trained continuous probes. I finally think that the manuscript should report clearly that the cross-layer alignment supports the interpretation of the results (and not the prediction).
Follow-up Response to Reviewer E6Ez: Updated Table 1 and Statistical Tests
We thank you for the quick and careful follow-up. Here we address your comments and have updated the manuscript accordingly (changes are marked in blue).
1. Repeated Splits for Table 1
For the split experiments, we reran the whole pipeline with the original model-dataset settings, from data collection to evaluation and prediction. The main conclusion holds the same: HSS-NB is above the closed-form baselines and approaches the continuous probe. Some variations are expected. One setting differs from the original submission: we now use ICL tolerance for the prediction maps, while the original version used . creates more states, and these states separate correctness better. Below are all eight Table 1 settings, as mean ± SD over ten shared stratified 40/60 train/test splits. Within a setting, all methods use the same examples, labels, and splits.
AUROC (%)
| Method | Qwen2–MATH | Qwen2–ThmQA | Qwen2–MMLU | L3–MATH | L3–ThmQA | L3–MMLU | L2–JBB | L2–HarmBench |
|---|---|---|---|---|---|---|---|---|
| HSS-NB | 76.0 ± 0.6 | 74.8 ± 1.9 | 79.1 ± 0.5 | 76.0 ± 0.8 | 75.6 ± 2.6 | 78.9 ± 0.6 | 96.6 ± 1.6 | 95.2 ± 1.5 |
| LDA | 62.9 ± 2.4 | 65.2 ± 2.2 | 69.2 ± 0.5 | 61.4 ± 0.9 | 67.3 ± 2.6 | 68.6 ± 0.5 | 93.5 ± 4.3 | 95.0 ± 4.3 |
| Gaussian NB | 70.2 ± 0.6 | 71.1 ± 2.6 | 74.5 ± 0.3 | 69.9 ± 0.9 | 70.2 ± 0.9 | 72.3 ± 0.4 | 89.1 ± 3.1 | 87.2 ± 6.8 |
| Linear SVM | 71.0 ± 1.1 | 71.0 ± 3.8 | 76.8 ± 0.9 | 68.9 ± 0.7 | 72.9 ± 2.7 | 73.1 ± 0.9 | 96.0 ± 1.8 | 94.9 ± 4.0 |
| Logistic regression | 73.8 ± 0.8 | 71.3 ± 3.7 | 73.1 ± 0.8 | 74.3 ± 0.4 | 72.8 ± 3.2 | 73.3 ± 0.8 | 97.3 ± 1.8 | 96.9 ± 2.1 |
| MLP | 76.4 ± 1.0 | 73.1 ± 4.8 | 79.6 ± 1.2 | 76.9 ± 1.2 | 75.3 ± 4.1 | 79.9 ± 1.2 | 96.3 ± 1.8 | 96.1 ± 2.2 |
Accuracy (%)
| Method | Qwen2–MATH | Qwen2–ThmQA | Qwen2–MMLU | L3–MATH | L3–ThmQA | L3–MMLU | L2–JBB | L2–HarmBench |
|---|---|---|---|---|---|---|---|---|
| HSS-NB | 69.1 ± 0.7 | 71.8 ± 2.2 | 73.5 ± 0.5 | 70.4 ± 1.2 | 77.5 ± 3.3 | 73.1 ± 0.4 | 94.3 ± 1.9 | 95.8 ± 2.3 |
| LDA | 60.0 ± 1.9 | 67.0 ± 2.9 | 67.4 ± 0.4 | 64.2 ± 1.2 | 76.5 ± 1.3 | 65.6 ± 0.5 | 94.2 ± 1.3 | 95.3 ± 1.1 |
| Gaussian NB | 68.9 ± 0.5 | 72.7 ± 2.0 | 68.0 ± 0.4 | 72.1 ± 0.5 | 77.8 ± 1.9 | 66.5 ± 0.3 | 93.9 ± 0.6 | 94.0 ± 1.8 |
| Linear SVM | 65.3 ± 0.9 | 69.6 ± 2.8 | 71.8 ± 0.3 | 71.5 ± 0.5 | 72.8 ± 2.6 | 70.7 ± 0.8 | 92.9 ± 1.6 | 89.9 ± 1.8 |
| Logistic regression | 66.9 ± 0.6 | 72.5 ± 2.3 | 70.4 ± 0.3 | 73.3 ± 0.7 | 78.8 ± 2.3 | 71.6 ± 1.0 | 94.1 ± 1.4 | 95.3 ± 1.4 |
| MLP | 70.4 ± 1.0 | 73.1 ± 1.8 | 74.2 ± 0.4 | 71.8 ± 1.0 | 79.5 ± 1.3 | 74.4 ± 0.9 | 93.7 ± 0.6 | 95.1 ± 1.3 |
HSS-NB: the number of clusters per layer is the ICL minimum () on training activations, with per-layer Laplace smoothing . MLP: architecture (five candidates) and L2 penalty (four values) selected jointly by five-fold cross-validation inside each training split, then refit on that training split. All continuous baselines use the standardized concatenation of prompt-last vectors from all layers. The full parameter list is in Appendix A.5 of the revised manuscript.
We have replaced Table 1 with the ten-split means and report the standard deviations in Appendix Tables 5 and 6, so that the manuscript is consistent with these results.
2. Statistical Test
We ran a two-sided corrected resampled t-test (Nadeau & Bengio, 2003) on the paired AUROC differences between HSS-NB and each baseline, using the same ten shared splits. The p-values below are Holm-adjusted across all 40 comparisons; bold means adjusted .
| Model–dataset | vs LDA | vs Gaussian NB | vs SVM | vs LR | vs MLP |
|---|---|---|---|---|---|
| Qwen2–MATH | 0.209 | 0.020 | 0.686 | 1.000 | 1.000 |
| Qwen2–ThmQA | 0.867 | 1.000 | 1.000 | 1.000 | 1.000 |
| Qwen2–MMLU | 0.003 | 1.000 | 0.005 | 1.000 | |
| L3–MATH | 0.035 | 0.021 | 1.000 | 1.000 | |
| L3–ThmQA | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 |
| L3–MMLU | 0.200 | 0.162 | 1.000 | ||
| L2–JBB | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 |
| L2–HarmBench | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 |
These significant differences are in favor of HSS-NB: over Gaussian NB on the four MATH and MMLU settings, over LDA on three of them (Qwen2–MMLU, L3–MATH, L3–MMLU), over SVM on L3–MATH, and over LR on Qwen2–MMLU. Against the MLP there is no significant difference in any setting.
3. Paper Presentation
For the presentation, we updated the phrasing in the abstract and now state clearly in Section 5.1 the function of cross-layer alignment.
Response to Reviewer E6Ez (1/2): W1, Q1-Q2
W1 / Q1
Question 1: Please report mean and std over different random splits for the results in tables 1 and 2, possibly including a statistical test against the baselines (this would support the claim in the abstract about the fact that the proposed approach matches the trained continuous probes).
Response:
We thank the reviewer for emphasizing statistical reliability. We repeated the prediction evaluation over ten stratified 40/60 train/test splits, refitting the state map on each training split. We also conducted a separate evaluation over five shared splits with hyperparameters selected using training data only.
Table A. Repeated-split prediction results.
For Table 2, we evaluated five response-level 40/20/40 train/validation/test splits. Each method's threshold was calibrated on validation responses to target a false-alarm rate of at most 10%.
Table 2. Early warning on Qwen2--MATH.
These statistical tests further support our paper findings. HSS-NB as a simple label-free count-based model matches continuous probes. We respectfully point out that it is still encouraging that useful predictive information survives reducing each layer's activation from thousands of continuous dimensions to a discrete state defined without labels, and can be recovered through simple class-conditional counts. What's more, this also provides a more interpretable way to track and analyze internal representations in the quantized geometric regions. For a given example, we can inspect each visited state's class counts and contribution to the score, connecting the prediction to the regions being studied. Once the map is fixed, different behavioral labels can also be examined by estimating their counts over the same partitions.
W1 / Q2
Question 2: Please clarify how you chose the hyperparameters for the baselines and the settings for HSS.
Response:
We thank the reviewer for asking us to clarify the hyperparameter choices. For the baselines, we performed a grid search over different model capacities and regularization strengths to select the best hyperparameter settings. For the MLP, we tested single hidden layers with 64, 128, 256, and 512 units, as well as a two-layer architecture with 256 and 128 units, jointly varying L2 regularization over {10−4,10−3,10−2,10−1}. We selected the configuration with the highest mean AUROC in five-fold cross-validation within each training split. For the HSS, we select the number of clusters at each layer using fixed ICL on the training activations, for the prediction, we use τ=0 for precise partition. This provides a consistent setup for assessing performance.