- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes Activation-Interval Neuronal Gating (AING), a training-free method for steering Big Five personality in LLMs. The motivation is that personality-related neurons are polysemantic, but different traits and general reasoning tasks occupy partially distinct activation ranges within the same neuron (Fig. 2, KDE on Llama-3.1-8B-Instruct). AING has three steps. (1) It selects the top 0.10% neurons by the mean activation difference between paired high/low BIG5-CHAT responses and filters out neurons that are highly active on a general corpus. (2) For each selected neuron, it fits Gaussians to target and non-target activations and picks an interval that maximizes target coverage minus times non-target overlap, subject to coverage (Eq. 6). (3) At inference, it scales the activation by only when the runtime activation falls inside that interval (Eq. 7). Experiments on Llama-3.1-8B-Instruct and Qwen2.5-7B-Instruct compare against prompting, SFT/DPO-style alignment, and activation-steering baselines. AING reports Avg.P of 4.63/4.69 with Avg.G of 69.52/72.77 (base: 69.83/72.90) and the lowest average non-target drift (Avg.Dnon 0.068/0.095). Ablations remove each component.
Critical strengths
- The idea is simple and well matched to the problem. Restricting a neuron-level intervention to an activation interval is a natural way to reduce collateral effects from polysemantic neurons. The ablation supports this: removing the interval gate increases Avg.Dnon from 0.068 to 0.123 (Llama) and from 0.095 to 0.158 (Qwen), with a drop in Avg.G (Table 4).
- The evaluation includes cross-trait drift, not only target strength. Reporting Dnon and Dmax with the full five-trait profile for every trait-pole setting (Table A1) is useful, and many steering papers skip it. The baseline set is also broad, covering prompting, fine-tuning, and steering methods on two model families.
Critical weaknesses
-
The empirical gains over strong baselines are small and not tested for significance. On Llama, AING's Avg.P (4.63) is below DPO (4.64) and close to POGP-DPO (4.58) and PERSONA-FLOW (4.59). Its Avg.G (69.52) is within 0.2 of PERSONA-FLOW (69.46), Spherical Steering (69.39), and CAA (69.34). On Qwen, the Avg.G gap to PERSONA-FLOW is 0.03. On cross-trait drift, NPTI on Llama (Avg.Dnon 0.073) is essentially tied with AING (0.068), and in Table A1 NPTI has lower Dnon than AING in several settings (e.g., O-Low, C-High, N-Low). All results appear to be single-run with greedy decoding, with no variance or confidence intervals. With differences this small on a 1–5 scale, it is hard to say AING is better than the strongest baselines. In addition, Table 2 compares capability changes only against DPN-LE, which is the weakest capability-preserving baseline in Table 1. The comparison that matters is against CAA or PERSONA-FLOW. The text also uses wording such as "catastrophic forgetting" for SFT, whose Avg.G drop is about 3 points; this overstates the evidence.
-
The method and evaluation are underspecified in ways that affect soundness.
- Personality metric. App. A.1 says the score is on a 1–5 Likert scale but does not say who or what produces it (LLM judge, questionnaire, or classifier), which prompts are used, or how high- and low-pole scores are combined into Avg.P. BIG5-CHAT is used for both calibration and evaluation, and no split is described, so overlap between calibration and test data cannot be ruled out.
- Gating logic. The interval is estimated from activations on responses that already express the target pole, and Eq. 7 only intervenes when the runtime activation is inside that interval. This seems to amplify the neuron only when it is already in the target state. It is unclear how this moves a neutral model toward the target pole, and in particular how it supports low-pole steering. Eq. 7 also combines an explicit with a signed , so the effective direction is ambiguous.
- Formulation details. Eq. 6 uses , but Eq. 5 defines . The value of and the procedure for solving Eq. 6 are not given. Sec. 4.2 says varies across backbones and traits, while App. A.2 says all hyperparameters are identical across traits and models. The "neuron" (MLP intermediate vs. residual dimension, which layers) and the filter quantity (described as both an activation value and an activation frequency) are not clearly defined.
- Motivating evidence. The separability claim rests on a single representative neuron (Fig. 2). Large-scale MLP activations are often heavy-tailed rather than Gaussian. A population-level statistic, such as the fraction of selected neurons with a well-separated interval or the distribution of overlap, would be needed to support the Gaussian model and the general claim.
Given these gaps, I could not fully reconstruct or verify the method from the paper. Novelty is also incremental. Range-restricted neuron intervention was introduced by NeuronLens (Haider et al., 2026), and candidate selection follows NPTI. The contribution is mainly applying the two together to personality steering.
Questions for authors
Items 1–3 could change my assessment. Items 4–6 are clarifications.
- How is the personality score computed (judge model, prompt, scoring rubric)? How are BIG5-CHAT calibration and evaluation data separated? How are low-pole results mapped into Avg.P?
- Please report variance over seeds or prompts for Avg.P, Avg.G, and Avg.Dnon, plus a significance test against PERSONA-FLOW, CAA, and NPTI. Please also add NPTI, C4, and C5 to Table 1, and CAA or PERSONA-FLOW to Table 2.
- At inference on a neutral prompt, how often does a selected neuron's activation fall inside the target interval? How does gating on the target interval steer the model toward a pole it does not yet express? Please clarify the sign convention in Eq. 7.
- What is , how is Eq. 6 solved, and is fixed or per-trait?
- Which layers and activations are the "neurons"? Can you provide population-level evidence that activations are approximately Gaussian and that target and non-target intervals are separable for most selected neurons?
- Do the personality changes hold in open-ended multi-turn dialogue, and is fluency affected?