- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
G-SAE is an SAE with a split latent space. It consists of (a) a small supervised partition (0.4–2.5% of latents, sigmoid-weighted ReLU, no Top-K) with a BCE-trained linear head on ProcessBench step-correctness labels, along with a diversity penalty, and (b) a Top-K unsupervised partition trained only for reconstruction. Each reasoning step is represented by the last token activation of its prefix at the middle layer. Training combines 1138 labeled steps with roughly 263k unlabeled ones. Across six Qwen2.5-based models, a post-hoc linear probe on G-SAE latents improves OOD AUC over both a reconstruction-only SAE (+0.035–0.119) as well as raw activations with a small reconstruction cost. In single-feature steering on Qwen2.5-7B scored by a PRM, only the 3 tested supervised features show significant effects.
Critical strengths
-
The full-supervision ablation (A1): similar OOD AUC (within 0.030) but NMSE worsens by 0.142–0.756 vs. at most 0.037 for G-SAE. This is strong evidence for the partitioned trade-off. The method is also lightweight.
-
Careful data protocol (problem-level splits, position/difficulty balancing, held-out harder tiers) and acknowledges PRM failure cases and post-hoc example selection.
Critical weaknesses
1. The method overlaps with uncited prior work, and the improvements are not shown to be robust or specific to the SAE.
My biggest concern is that Härle et al. [1] already published a method (with the same name), G-SAE, using normalized-MSE reconstruction plus a BCE loss that ties designated latents to concept labels, with steering through the conditioned decoder directions. An earlier version of that work (arXiv 2411.07122) does the same thing with a single latent. The differences from this work, such as the multi-latent block with a linear head, dropping Top-K on the supervised units, the step-level setting, the unlabeled stream, and the diversity penalty, are incremental. Moreover, the diversity penalty doesn’t consistently help (in Fig. 4, performs worse than 0 for 2 of 6 models). At minimum, the paper needs to cite this work and explain what is genuinely new.
I also don’t think the main result is established yet. Table 1 reports single runs with no error bars, while the paper’s own sweeps move by up to approx. 0.09 AUC between neighboring settings of and , which is as large as the claimed improvement. The defaults are also odd, since 5 models use a 0.4% supervised ratio but only one uses 2.5%, which happens to be the best OOD value. No selection criterion or split is ever stated in the paper. On 2 of the 3 7B models, the gain over raw activations is only +0.016/0.026. The supervised encoder + head is itself a small classifier trained on the same 1138 labels, so beating a linear probe on raw activations might simply be from supervised capacity. Baseline results on raw activations with a tuned logistic probe and a capacity-matched MLP are missing.
2. Steering results don’t support the causal claim.
In Table 3, significance is tested against the unsteered baseline rather than random direction control, which is about +0.008 in every row. The effect of feature 6 (+0.0100) is just +0.0018 above the random one, and that contrast is never tested, so superiority over random directions has never been shown in the paper. For feature 13 with the highest AUC (0.809, which is above the OOD AUC of the full probe at 0.722), steers in the wrong direction. Moreover, the opposite steer/negate signs also show up for 5 of 6 unsupervised features. The effects are at most 0.0284, measured on one model with 3 features per partition and judged by a single PRM the authors themselves show can be gamed (A8.1). There is no final-answer accuracy and no difference-of-means or probe-direction baseline, so it’s unclear what the SAE improves over a standard steering vector.
Minor points. The FSup range in the intro/A.1 doesn’t match Table 4. Inconsistent decimals in tables.
[1] Härle, Ruben, et al. "Measuring and guiding monosemanticity." NeurIPS, 2025.
Questions for authors
-
Q1: How does your method differ from Härle et al.’s G-SAE, and do those differences matter empirically? Would their single-latent Sigmoid(TopK) reach similar OOD AUC on your data?
-
Q2: Could you document the probe and report which latents it uses, what it is trained on, and how it is regularized? What AUC does your trained head reach and a probe on the unsupervised partition alone?
-
Q3: How many seeds are behind Table 1? Please report mean ± std. On which split and by what criterion were the variables chosen? And why does differ between the two 7B models (215 for Deepseek vs 430 for Qwen)?
-
Q4: Did the authors consider a tuned logistic probe and a capacity-matched MLP on raw activations, using the same labels and splits? Similarly, why were steering results not compared against matched random directions, difference-of-means, and probe-direction baselines? Could the authors provide these comparisons with paired confidence intervals and report final answer accuracy?
-
Q5: How dense is the supervised partition in practice (mean L0, per-unit activation frequency), given that the baseline is matched only on the maximum count? Please also report dead latents, the EV definition, and spliced-in LM loss if feasible.
5 comments
Thank you for the detailed response and experiments. However, I still have these concerns:
- I could not locate the citation or discussion of Härle et al. in the revised manuscript. While the label differs, the method is nearly identical, with NMSE + BCE on a designated latent block and decoder-column steering. The modifications are , a linear head, and no Top-K on the supervised block. A run with your protocol would show whether these matter. If the gap is small, the paper is an application of guided SAEs to step correctness, which should be stated explicitly.
- I am still missing an explanation for why differs between the two 7B models.
- Could you report paired CIs comparing steer/negate effects directly against DoM and probe-direction controls? A single latent outperforming a DoM direction would be an important result, but this comparison is not yet statistically established. Also, Unit 63 still steers in the opposite direction from its predicted sign.
I appreciate the additional work, but the methodological novelty and the interpretation of the steering advantage remain my main concerns.
Thank you for the follow-up. We address the three points in turn.
1. Relation to Härle et al. (2025)
You are right that the citation was missing; we apologize. It is now discussed in Related Work, where we state explicitly that the core idea (NMSE + BCE on a designated latent block, steering along decoder columns) comes from Härle et al.'s Guided SAE, and that we extend this paradigm to step-level reasoning correctness. Our design differs in using several gated units (sigmoid · ReLU, outside Top-K) read by a linear head, a diversity loss, and step-level activations with an unlabelled reconstruction stream.
We ran the closest variant: G-SAE with a single supervised unit (k_sup = 1). With one unit, the head only rescales that unit, so, as in Härle et al., the correctness loss acts on one latent. Unsup and G-SAE: mean over 5 seeds (Table 1); k_sup = 1: single seed. Δ = G-SAE − (k_sup = 1).
| LLM | k_sup | AUC Unsup | AUC k_sup = 1 | AUC G-SAE | ΔAUC |
|---|---|---|---|---|---|
| Qwen2.5-0.5B | 22 | 0.687 | 0.721 | 0.713 | −0.008 |
| Qwen2.5-Math-1.5B | 37 | 0.669 | 0.667 | 0.677 | +0.010 |
| DS-R1-Qwen-1.5B | 230 | 0.684 | 0.701 | 0.704 | +0.003 |
| DS-R1-Qwen-7B | 86 | 0.689 | 0.663 | 0.667 | +0.004 |
| Qwen2.5-7B | 86 | 0.707 | 0.686 | 0.723 | +0.037 |
| Qwen2.5-Math-7B | 86 | 0.705 | 0.721 | 0.735 | +0.014 |
With one unit, AUC is mixed relative to Unsup (higher on 3 models, lower on 3). The full block is higher on 5 of 6 models at the same reconstruction, but clearly beyond seed variability (std 0.008–0.038) only on Qwen2.5-7B. Together with Recon-G-SAE (no labels, similar AUC to G-SAE), this suggests that the small gain comes from having several gated units rather than from the labels. The remaining difference (a sigmoid latent inside Top-K) is untested. The revised Related Work now positions G-SAE as a step-level extension of the guided SAE approach.
2. Why K_s differs between the 7B models
K_s was selected per model on the GSM8K+MATH validation split, from the same grid for all models. For DeepSeek-R1-Distill-Qwen-7B the validation AUC favoured K_s = 215 (1.0% of the Top-K latents active), while the two Qwen2.5-7B models favoured K_s = 430 (2.0%).
3. Steering against DoM and probe directions, and unit 63
We ran the comparison you asked for on the existing generations. Every unit, DoM and probe condition uses the same problems and sampling seeds, so we report Δ_unit − Δ_control with a paired problem bootstrap (10,000 draws) and Holm correction over the conditions, per metric and control (two new tables in Appendix A.12).
On Qwen2.5-7B, against the probe, 4 of the 15 units are stronger, 3 weaker and 8 not different. Against DoM, 6 are stronger and 1 weaker, but this mostly reflects DoM being a weak direction: against random, neither control changes pass@1 significantly. Unit 67 exceeds both on pass@1 (+4.8 pp vs. the probe; +6.2 / −6.5 pp vs. DoM). Single units therefore steer about as well as a supervised probe direction, and the paper now states exactly this, with no general superiority claim.
Unit 63 is reported as steering opposite to its head-weight sign, and it is also weaker than both controls on the PRM. Across the five G-SAEs on Qwen2.5-7B, 2 of the 10 significant unit conditions go against the sign. This is why the revision describes the sign as predictive only: "+" is the direction the correctness head associates with correct steps, and whether steering along it helps is tested, not assumed.
We thank you again for the careful reading, which has made the paper's scope and claims more precise.
Response to Reviewer 4iVY
We thank the reviewer; see also the general response.
1. Härle et al. (W1, Q1). We now cite it.
Difference from Härle et al. (NeurIPS 2025). Although both methods introduce supervision into a partitioned SAE latent space, they differ in the supervision target, architectural mechanism, and intended feature organization.
Härle et al. associate designated latent dimensions with predefined semantic concepts (e.g., toxicity, style, and privacy attributes), using concept-specific objectives on designated latents. Their objective is to localize known concepts into designated features and evaluate their monosemanticity. In contrast, our method does not prescribe the semantic identity of individual latents. Instead, it uses a shared behavioral signal, step-level correctness, to guide a multi-dimensional supervised subspace through a linear prediction head. The internal features are learned from the activation data rather than assigned to predefined concepts. Thus, supervision acts as an inductive bias on a learned representation rather than as a direct concept-to-neuron assignment.
This distinction is particularly relevant to reasoning evaluation. Step correctness is a single binary outcome, but incorrect reasoning can arise from heterogeneous failure modes, including algebraic errors, invalid deductions, and incomplete reasoning. Our approach does not assume a one-to-one correspondence between the correctness label and a latent dimension. Instead, it allows multiple learned features to contribute to a shared behavioral prediction objective.
Architectural distinction. Härle et al. incorporate supervised latents into a Top-K-constrained representation, whereas our method decouples the supervised partition from Top-K competition. The supervised partition uses ReLU-gated activations and is trained through a linear prediction head, while the reconstruction-oriented partition retains Top-K sparsity. This separation allows behavioral supervision to guide the supervised subspace without requiring its features to compete directly with reconstruction-oriented features for a global Top-K activation budget.
Therefore, our contribution is not simply the introduction of supervision into a partitioned SAE, but a different formulation of supervised feature learning: using a shared behavioral objective to guide a learned latent subspace, and it targets step-level correctness in chain-of-thought reasoning specifically.
2. Seeds and selection (W2, Q3). Table 1 now reports mean ± std over 5 SAE seeds, with per-seed results in App. A.10 (Table 9). G-SAE has the higher mean AUC on 5 of 6 models and wins on 21 of 30 model–seed pairs; its AUC also varies less across seeds than the Top-K SAE's on 5 of 6 models. The probe is fixed and identical for all SAEs (Sec. 4.2), trained on GSM8K+MATH and tested on held-out OOD steps; the advantage holds on the same five models for every C from 10−3 to 10 (App. A.10, Table 10).
3. Tuned probe and MLP (W3, Q4, Q2). Both are added (Sec. 4.3; App. A.10, Table 11): a logistic probe on standardized raw activations with C selected on validation, and a capacity-matched MLP (hidden width ksup, 5 seeds), on the same labels and splits. They reach comparable or higher AUC than G-SAE (e.g. Qwen2.5-7B: probe 0.740, MLP 0.735, G-SAE 0.723; Qwen2.5-Math-7B: probe 0.741, MLP 0.725, G-SAE 0.735). Unlike these classifiers, G-SAE keeps the correctness information inside a dictionary that reconstructs the activations, and supervision makes its supervised units up to 2–3× more selective than in the label-free Recon-G-SAE (on the 7B models) at no AUC cost (App. A.9). The probe is specified in Sec. 4.2.
4. Density, dead latents, EV (Q5). These are reported in App. A.9 (Table 8): supervised units are active on 28–49% of steps vs. 52–82% without labels. EV is defined in Sec. 4.2; spliced-in LM loss is left to future work.
5. Steering (W4, Q4). Redone with random-direction, DoM and probe controls, a problem-level bootstrap, Holm correction and pass@1 (Sec. 4.6, Table 3; App. A.12; details in our reply to D9Qn). Supervised units shift the PRM beyond random directions with larger effects than the baselines, two with both signs, and one unit changes final-answer accuracy in both directions; neither baseline does either.
6. Minor (W5). We corrected the FSup range and unified decimals.