- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper find evidence for causal representations of 'granularity' in the residual stream of LLMs. The paper finds both geometric and causal evidence that representations derived from the diffmean of role playing datatsets can mediate granularity of a given role.
Critical strengths
- This paper uses clear evidence and an understandable argument. Namely, I appreciate the use of random controls, causal evidence, human validation/rating of steered outputs. This is more thorough validation than some papers.
- The use of PCA and diffmean are well motivated.
- I also appreciate the use of qualitative examples. These are meaningful and helpful for understanding the argument.
- Replication across multiple models is helpful as well
- The teacher-forced continuation experiment and the matched-domain controls help the construct validity and reasoning-level concerns
Critical weaknesses
- My main concern is that the notion of "granularity" doesnt seem to be well defined. That is that granularity as you use in this paper seems to be only related to levels of policy intervention, as opposed to a more general notion of granularity which would transfer to other tasks and situations. This is fine. But that notion of granularity is very limited in its applications and interpretation. This unclear definition also brings up questions about construct validity as well.
- I also notice that there is limited external validation of the effect on reasoning. In your motivation you cite “individual reasoning where systemic analysis is needed, or institutional abstraction where local judgment is appropriate”, while we see LLM/human judgements and qualitative examples, to me, this seems to enough to support a notion of stylistic or surface level change (indeed the qualitative examples seem to only vary in small ways), but not necessarily that the reasoning itself is changing. I would like to see evaluations which substantively test the reasoning itself. For example, if we are motivated by representational validity, survey bias, social simulation, why not demonstrate that this intervention meaningfully addresses these concerns?
- I also worry about the claim that this is a “representational primitive, beyond a stylistic surface variable”. I dont know that we see enough evidence to support this claim, namely there has been multiple works which supports the idea that behaviors that coherent on the surface level and can be steered together are not necessarily “primitives” (1, 2).
- The concerns above come together make me question the overall usefulness of this study. If the papers notion of granularity is very narrow and it doesnt convincingly show that the narrow notion of granularity it studies is at a deeper reasoning level then it doesnt demonstrate that this granularity axis would actually solve representational validity, survey bias, social simulation. It remains open whether this result is practically useful. Then my other concern would be novelty. At this point it is well understood that essentially any coherent phenomenon/behavior is represented accessibly in the residual stream of these models. And especially since the “representational primitive” claim is insufficiently tested, I worry that this paper does not meaningfully advance our understanding in this regard.
References
- Vennemeyer et al. (2026), Sycophancy Is Not One Thing: Causal Separation of Sycophantic Behaviors in LLMs. arXiv:2509.21305
- Zhao et al. (2026), LLMs Encode Harmfulness and Refusal Separately. arXiv:2507.11878
Questions for authors
Please define granularity more specifically