- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper studies why changing user persona information can change a model’s answer to the same question. The authors use patching in both directions, from correct to wrong runs and vice versa, to identify relevant attention heads. They separate these into heads shared across persona axes K and the remaining heads R. Experiments suggest that K also supports ordinary task computation, and that persona information interacts with these heads differently across tasks. The authors support this with ablations, probing, logit attribution, and evaluations without persona prompts. They also show that targeted activation interventions can fix some persona-sensitive errors while preserving most originally correct answers.
Critical strengths
- The research question is interesting. The definition of problem is clear and easy to follow. The paper measures how persona information affects answer correctness, investigates the underlying mechanisms, and uses steering to fix some of the errors. This makes the findings useful beyond just observing that models are sensitive to persona prompts.
- The authors use several experiments to check what the identified heads actually do. The experiments are comprehensive and easy to follow. In particular, showing that these heads still support the task without persona information gives stronger support to the claim that persona effects involve ordinary task computation.
Critical weaknesses
- The experiments are limited to relatively simple classification and multiple-choice tasks. It is unclear whether the findings also apply to complex reasoning or open-ended generation, which limits what we can conclude about their usefulness for improving model capabilities.
- There is no controlled comparison across model sizes. The overlap between task and persona information could be more pronounced in smaller models. Testing different sizes from the same family, such as Qwen, would help clarify whether the findings hold as models get larger.
Questions for authors
-
How much do the K sets overlap across tasks? Have you tested whether K discovered for one task also supports another? This would help clarify whether these heads perform task-specific computation or more general answer selection and label prediction.
-
Since the keep/knockout experiments only intervene at the decision position, how do you distinguish heads involved in solving the task from heads that mainly read out information computed elsewhere? What evidence supports a more specific functional interpretation of K and R?
-
Do the existing results show any difference between the 3B and 8B models in how separated task and persona information are? Could the observed overlap partly reflect limited model capacity? A controlled comparison across sizes within the same family, such as Qwen, would be a useful additional experiment.
-
Have the intervention coefficients been evaluated on data separate from the data used to select them? Also, do they transfer to unseen persona descriptions or other tasks? This would help assess how much the repair results depend on setting-specific tuning.
6 comments
Response to Reviewer jzBn (Part 2)
Q1. Cross-task overlap and functional transfer
Thank you for raising this question. Although we have not directly evaluated cross-task functional transfer, we examined cross-task structural overlap and the functionality of globally shared heads.
Cross-task structural overlap. For each task , we define the bidirectional shared intersection as . We then identify the global core , containing heads shared across all five tasks and both discovery directions.
| Model | Original K Jaccard | Reverse K Jaccard | Global core heads |
|---|---|---|---|
| Llama-3B | 0.407 | 0.331 | 2 |
| Llama-8B | 0.317 | 0.265 | 2 |
| Qwen-7B | 0.265 | 0.288 | 1 |
Jaccard values are averaged across the 10 task pairs per model.
These results reveal partial cross-task recurrence, but only 1–2 heads per model belong to the global intersection. Thus, is not an identical universal circuit across tasks.
Functional comparison. We additionally compare task-specific with the global core using normalized probability-margin faithfulness and knockout effects across all 40 matched settings.
| Head group | Faithfulness ↑ | Normalized knockout effect ↑ |
|---|---|---|
| Task-specific KK | 0.987 | 0.792 |
| Global core | 0.086 | 0.042 |
Because the global core contains substantially fewer heads than task-specific , this comparison establishes the limited functionality of the small universal intersection but does not isolate task specificity from circuit size.
Task-specific exhibits substantially stronger functional effects than the global core. This indicates that the small universal intersection alone is insufficient to reproduce the functionality of task-specific , suggesting that additional task-dependent components contribute to performance.
Cross-task transfer. While these analyses establish partial structural recurrence and the functional importance of task-specific , directly testing whether discovered for task supports another task requires cross-task interventions with appropriate size- and layer-matched controls.
Overall, our findings show that is not a universal set of identical heads across tasks, and task-specific has substantially stronger functional contributions than the small globally shared core. These results support task-dependent functional organization while leaving open the possibility that some identified heads also participate in general answer selection or label prediction. We will clarify this distinction in the revised manuscript.
Response to Reviewer jzBn (Part 3)
Q2. Task computation versus decision-position readout
Thank you for raising this distinction. Our faithfulness, knockout, DLA, and independent no-persona validation support 's task functionality. We conducted additional causal experiments to investigate its role in persona-conditioned behavioral changes.
-
Causal cross-patching of K and R.
We performed bidirectional cross-patching between matched correct and wrong persona-conditioned runs, measuring the fraction of the gold-answer margin gap recovered by each component.
Head group Median recovered margin gap ↑ Original K 95.5% Reverse K 95.5% KK intersection 91.6% Original R 16.8% Reverse R 19.5% Evaluation pairs are disjoint from discovery pairs. recovers substantially more of the margin gap than , despite generally containing fewer heads. This provides direct causal evidence of 's involvement in persona-conditioned prediction changes, beyond structural overlap or decodability.
-
Persona-reading-head analysis.
We independently identify candidate persona-reading heads using persona-token attention () and persona-conditioned activation responsiveness (), then evaluate their overlap with and through cross-patching.
Head group (Original discovery) Settings Median heads Repair ↑ Degradation ↑ K excluding P 58 13 0.943 0.932 K excluding P′ 58 12 0.938 0.933 P overlapping K 13 1 0.064 0.103 P′ overlapping K 23 2 0.095 0.081 R excluding P 58 25 0.165 0.166 P overlapping R 50 2 0.002 0.003 Values are median normalized causal effects across available settings; empty intersections are excluded.
retains approximately 94% of its cross-patching effect after excluding candidate persona readers, whereas their intersections with show much smaller effects. Thus, 's causal contribution is not confined to these identified readers, although the intersections are smaller and not size-controlled.
-
Upstream persona-reader → K intervention.
We mean-ablate upstream persona readers preceding the earliest layer, comparing reductions in 's cross-patching effect against 10 matched random controls per setting.
Using an upstream-ablated receiver baseline, the additional reduction is negligible across 53 settings (median +0.000310; stronger than random in 30/53). Alternative normalization yields a similar conclusion (−0.000145; 26/53).
These results provide little evidence that the selected upstream readers form a consistently necessary pathway into , although other pathways remain possible.
-
Direct persona-reader contribution removal.
We selectively remove complete persona-token contributions, their persona-specific and persona-invariant components, and persona-dependent task-source contributions from identified reader heads.
Across 60 settings, all four interventions produce zero median change in accuracy and instability, with negligible persona-margin changes (−0.00024 to −0.00049). Random-head controls were also evaluated.
Together with upstream ablation, these results provide little evidence that the selected direct persona-reading contributions are consistently necessary for persona-conditioned behavioral changes.
-
Activation-direction nullification.
We nullify activation directions distinguishing correct and wrong persona-conditioned states within and , comparing their effects against random-direction controls.
Nullified component ΔAccuracy (pp) ΔInstability (pp) K +17.30 +3.60 R +0.15 +0.28 K random directions ≈0 ≈0 Nullifying directions produces substantially larger effects than or random controls, supporting their causal contribution to task predictions. However, the increased instability indicates that this is not necessarily a robustness improvement.
-
Functional interpretation.
Together, these experiments support interpreting as shared task-functional circuitry involved in task predictions and persona-conditioned behavioral changes. Its contribution is not confined to the identified persona readers, and the tested upstream pathways show no consistent necessity.
However, decision-position interventions cannot distinguish upstream reasoning from evidence integration or answer readout. Thus, may support multiple task-related functions rather than independently performing the complete reasoning process. Similarly, is axis-dependent but not necessarily dedicated to persona processing.
We will incorporate these findings and clarify the functional interpretation of and in the revised manuscript.
Response to Reviewer jzBn (Part 1)
We thank the reviewer for their thoughtful evaluation and constructive questions. We address the scope, methodology, and implications of our findings below.
W1. Generalization beyond restricted-output tasks
Thank you for raising this question. Our use of restricted-output tasks is a deliberate experimental choice that enables controlled causal comparisons across persona conditions, with a consistent evaluation metric and a well-defined decision position. Although our experiments cover scientific reasoning, ethical judgment, safety classification, and sentiment analysis, our conclusions are limited to the evaluated tasks and do not establish generalization to complex, open-ended generation.
Extending mechanistic circuit analysis beyond narrowly defined tasks remains an important open research challenge (Sharkey et al., 2025). Recent work has made progress in this direction; for example, Lindsey et al. (2025) demonstrate circuit-tracing approaches for investigating multi-step reasoning.
Open-ended generation introduces additional challenges in constructing controlled counterfactuals, aligning generated sequences, and attributing causal effects across output positions. While path patching is not inherently restricted to single-token outputs, extending our bidirectional discovery and intervention framework to these settings requires additional methodological development.
Our findings therefore demonstrate selective improvements in persona-conditioned task accuracy, rather than general improvements in model reasoning capabilities. We will clarify this scope in Section 7 (Limitations) and identify complex reasoning and open-ended generation as directions for future investigation.
References
W2/Q3. Model scale and the capacity hypothesis
We compare Llama-3.2-3B-Instruct and Llama-3.1-8B-Instruct. Although they share a lineage, differences in training and architecture prevent a fully controlled scaling comparison.
Behavioral sensitivity. Llama-8B exhibits lower mean persona-conditioned instability than Llama-3B (2.95% vs. 4.71%).
Task–persona organization. We compare task and persona decodability using balanced-accuracy differences (BAK−BAR), alongside K/R ratios of persona-token attention, persona-dependent value-content changes, and attention-routing changes. Positive probe differences favor K; ratios above 1 indicate stronger effects in K.
Model comparison (Original discovery)
Results are aggregated across four persona axes. Probe differences measure relative decodability; K/R ratios measure relative persona-related attention, content, and routing effects.
Findings. Both models exhibit similar task-dependent organization: task-answer information is more decodable from K, while persona information is relatively more decodable from R on ARC and from K on ETHICS, Safety, and SST-2. Attention, content, and routing measurements also vary by task.
However, Llama-8B does not consistently exhibit greater separation than Llama-3B. Its content and routing K/R ratios are lower on Safety and SST-2 but higher on ARC and ETHICS. Reverse discovery also shows mixed differences.
Overall, Llama-8B is behaviorally more stable, but mechanistic measurements do not show uniformly greater task–persona segregation. Thus, our findings do not support a simple capacity-only explanation, although capacity effects remain possible. Controlled comparisons across model sizes, such as within the Qwen series, are needed to assess scaling.