- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper asks whether LLMs can develop patterns that resemble how symptoms of mental disorders reinforce one another—for example, guilt leading to low self-esteem, which then increases guilt. It does not claim that LLMs actually feel these emotions. Its main contributions are:
- A framework for studying these reinforcing patterns inside LLMs.
- A method called S3AE to identify and manipulate twelve internal features associated with depression- and mania-related language.
- Experiments testing whether activating these features changes other features and the model’s behavior. The authors test twelve LLMs across three model families. They activate one feature, repeatedly ask questions while feeding previous answers back into the model, and check whether related features become active and remain active after the intervention stops. They also test whether the models make more problematic decisions in simulated interactions and resist instructions to respond neutrally. They report persistent, mutually reinforcing patterns, with several effects becoming stronger in larger models. However, the experiments do not clearly separate the proposed internal mechanism from the model simply continuing the narrative established by its previous answers.
Critical strengths
- The paper connects internal representations to behavioral effects through direct interventions. Rather than relying only on symptom-like responses to questionnaires, the authors identify symptom-related features and manipulate model activations to test their effects on generated responses and simulated decisions. This provides stronger evidence that these representations influence behavior and makes the work a substantive step toward mechanistic investigation.
- The study tests an explicit theoretical framework across multiple models and complementary experiments. The authors translate network theory into testable predictions about feature propagation, persistence, and resistance to correction, then investigate them across twelve LLMs from three families. This strengthens the empirical contribution and provides a concrete framework that subsequent research can test and refine.
Critical weaknesses
Each reconstruction round receives the previous round’s answers as context, so symptom-related patterns could persist because the model continues a consistent narrative. Excluding the previous answer to the same question does not eliminate this explanation, since related answers remain available. Moreover, causal discovery assumes that the twelve measured features capture all relevant common causes, despite the richer textual context driving responses. This limits support for the central claim of an internal, self-reinforcing symptom network.
Questions for authors
- The model reads its earlier responses each round. How do you know the patterns persist because of an internal reinforcing loop, rather than because the model keeps following the story in those responses?
- You created the “one active feature” and “multiple active features” conditions differently. Could differences in wording or repetition explain why one group followed corrective instructions less well?
We thank the reviewer for the review. We provide our brief responses below.
We clarified our position in the shared response above.
Thank you for pointing this out. We believe the key variables (the number of active features and their magnitudes) are controlled for between the two conditions. Given that the input context involves many factors (e.g., different grammar, tone, words, etc.), we think it is generally infeasible to fix all variables but the number of active features. However, we will consider further control in a future version of this paper.