- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies how political personas affect the relationships among concepts in LLM representations. The authors analyze 9,493 words and 33 semantic axes across eight models. They find consistent persona dependent changes, agreement with embeddings trained on partisan texts, and corresponding changes in steering effects and fixed probe outputs. The main contribution is a systematic analysis of how adopting a viewpoint changes relationships among multiple semantic features.
Critical strengths
-
The main observation is supported by several complementary checks, including baseline prompts, analyses across layers, replication across model families, and comparison with political text corpora.
-
The steering experiments connect the measured geometry to changes in model behavior. Gives the geometric analysis practical relevance.
-
The findings motivate useful future work on evaluating probes across contexts and separating factual truth from a role’s current judgment.
Critical weaknesses
-
The controversial statements are labeled by expected partisan endorsement rather than ground truth. This makes it unclear whether changes in probe outputs indicate monitoring errors or simply reflect changes in the persona’s judgment.
-
The reported agreement across models and with word2vec may be driven by a few strongly affected axes, such as political identity. The paper does not show whether the agreement remains strong after removing these axes.
Questions for authors
-
What is the intended target of the truth probes?
-
How much agreement across models and with word2vec remains after removing the political axis and the other most affected axes?
3 comments
The axis removal analysis addresses my main concern about the agreement across LLMs. The weaker agreement with word2vec after removing the political axis makes me view the corpus comparison as supporting evidence rather than a central result.
The new probe experiment also shows that training across personas can reduce the reported shift. Since the labels come from the neutral persona, it remains unclear whether this improves factual accuracy. Could you clarify how the reduction of over 60% was measured and whether it was evaluated on held out claims or topics? I am keeping my rating for now.
Our controversial items are grouped into 12 topics for truth and 30 topics for morality. When we train probes on controversial items using the neutral persona's labels, we split the controversial items in half between topics, so half the topics were in one set and half the topics in the other. Each probe was trained on all the uncontroversial items and half the controversial items.
Each probe's result is in fact the aggregation of two classifiers, one trained on set A and tested on set B, and the other trained on set B and tested on set A. The number of "flips" in classification is the sum between these two.
"it remains unclear whether this improves factual accuracy"
All probes already do very well on uncontroversial factual items that have a clear ground truth. Below we present results from truth and morality probes on a held out set of 1840 uncontroversial factual items and 400 uncontroversial moral items. We show results from our original design (trained only on uncontroversial items, only on neutral persona), on a "mixed" set of uncontroversial and controversial items (called "mixed neutral"), and on a mixed set of uncontroversial and controversial items across all three personas ("Mixed all personas"). Classification of factual truth is almost perfect across probes.
| Model | Truth: original | Mixed neutral | Mixed all personas | Morality: original | Mixed neutral | Mixed all personas |
|---|---|---|---|---|---|---|
| Llama 8B | 98.26 | 98.26 | 98.04 | 99.50 | 99.50 | 99.50 |
| Llama 70B | 99.57 | 99.54 | 99.51 | 100.00 | 100.00 | 100.00 |
| Ministral 8B | 98.32 | 98.37 | 98.04 | 99.50 | 99.50 | 99.50 |
| Ministral 14B | 97.83 | 97.99 | 98.02 | 99.50 | 99.38 | 99.50 |
| Qwen 8B | 97.77 | 97.80 | 97.64 | 99.50 | 99.25 | 99.38 |
| Qwen 32B | 99.13 | 99.13 | 98.99 | 99.75 | 99.75 | 99.75 |
| Gemma 4B | 95.92 | 95.95 | 94.29 | 99.00 | 98.62 | 98.62 |
| Gemma 27B | 99.29 | 99.27 | 99.27 | 100.00 | 99.88 | 99.50 |
Difficulty emerges on topics where factual truth is contested. On these topics, I do not make a distinction between which ones are "really actually true" and which are not. They include statements like "Financial deregulation in the decade preceding 2008 was the primary cause of the global financial crisis." Such statements are politically freighted and remain matters for debate.
No one, including LLMs, has access to absolute "ground truth." In training the "neutral persona," labs attempt to select a perspective that follows scientific consensus, but even scientific consensus can be wrong. So on such highly contested items, the question we ask is not "what is the model's factual accuracy?" We ask "what perspective's representation of truth is the model adopting?"
Thank you for these thoughtful remarks. We respond below:
"The controversial statements are labeled by expected partisan endorsement rather than ground truth. This makes it unclear whether changes in probe outputs indicate monitoring errors or simply reflect changes in the persona’s judgment … What is the intended target of the truth probes?”"
We followed the existing literature’s standard procedure for training truth probes. These probes purportedly serve as classifiers that discriminate true from untrue statements. Our contribution, however, is to suggest that truth probes trained in this way actually discriminate statements the current persona treats as true versus untrue. This means that important statements like “COVID-19 vaccines significantly reduce the risk of severe illness, hospitalization, and death” may or may not be classified as true, depending on the perspective being adopted. The purpose of this demonstration is not so much to train the perfect classifier as to show that training classifiers by standard methods can produce unexpected and potentially undesirable behavior that has not been previously described in the literature.
However, following another reviewer’s recommendation, we run a test in which we train the probe across personas while maintaining the neutral persona’s labels for positive and negative cases. For example, if the neutral persona rates the claim “sanctuary cities lead to higher crime rates” as false, it is used as a “false” example even when training the probe under the conservative persona, which would rate it as true. In a sense, the resulting probe no longer tracks whether the LLM is currently treating the claim as true, it tracks whether the neutral persona would treat the claim as true. The resulting probe still shows politically-aligned deviations in classifications corresponding to the political personas, but the effect is attenuated by over 60%. These results are presented in pages 9-10 in the revised, uploaded manuscript in the section entitled “Training probes across personas mitigates discrepancies.”
"How much agreement across models and with word2vec remains after removing the political axis and the other most affected axes?"
Thank you for suggesting this robustness test. We now include a figure in Appendix E on page 24 of the revised, uploaded manuscript displaying correlations after excluding each of the top three most affected axes (including liberal-conservative) individually and together. Mean correlation coefficients across models are presented below:
Removing the political axis does indeed attenuate the association between LLM personas and word2vec models. But their correlation remains positive and statistically significant after its removal. Individually removing each of the next two most affected axes produces smaller changes, and removing the top three most affected axes reduces the correlation to 0.261 (but still statistically significant at p < 0.05). This suggests that, while much of the agreement between the LLM personas and the word2vec representations comes from the top most affected axes, the general association is not completely confined to a few axes.
Moreover, the LLM-LLM associations remain quite strong even after removing the top three axes, suggesting that there may be consistent persona differences across other semantic axes that are captured by LLMs that are not captured by word2vec.