- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper studies how to identify intervention targets before reasoning generation begins. It contrasts content-matched Chat and Raw inputs at their final prompt positions, then uses SAE feature screening, prompt-component controls, and layer localization to select 23 features in Qwen3-8B. A joint intervention applied throughout generation reduces sample-weighted accuracy from 86.1% to 41.2% across GSM8K, CruxEval-O Adv, and HumanEvalPlus. The evaluation includes four random-feature control sets, visible-CoT controls, intervention-strength sweeps, and failure-profile analyses.
Critical strengths
1. A clear procedure for finding intervention targets. The content-matched Chat–Raw contrast, prompt-component controls, and layer localization provide a concrete way to select a compact intervention target. This is a useful contribution because it connects differences observed before generation to effects that can be tested across mathematical and code-related tasks.
2. Behavioral analyses distinguish different failure modes. On HumanEvalPlus, the targeted intervention largely preserves code extractability while reducing functional correctness. The visible-CoT controls also distinguish entering a reasoning trace from successfully completing the task. These analyses help clarify what the intervention disrupts.
Critical weaknesses
1. The role of the selected features remains difficult to isolate. In Section 4.1 and Appendix A, the intervention sets the target SAE activations to zero, replaces the original residual state with an SAE reconstruction, and applies an additional update along nonorthogonal decoder directions. The paper does not separately evaluate these operations, leaving their contributions to the accuracy loss unclear. The random controls use the same operator, but are not matched for activation magnitude or frequency and may produce different perturbation magnitudes. The current comparison therefore leaves open whether similarly active non-target features would cause comparable damage under perturbations of the same magnitude.
2. The stability of the discovery strategy is not established. Sections 3.2 and 3.3 report feature selection using a particular set of discovery examples. The paper does not show whether repeating this procedure with different examples would identify similarly effective targets. Since the discovery strategy is a main contribution, this sensitivity matters: the current results do not establish how reliably the procedure finds effective intervention targets.
Questions for authors
1. Clarification. Were the discovery examples, screening thresholds, and layer choices fixed before inspecting downstream benchmark performance? Did any examples used for feature selection or layer localization overlap with the evaluation sets?
2. Additional validation. A comparison using control features with comparable activation levels and perturbation magnitudes would help address the attribution concern. Does the target–random accuracy gap on GSM8K and CruxEval-O Adv persist under such controls?