acceptodds
Withdrawn from ICLR 2027

Emergence of Psychopathological Computations in Large Language Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Can large language models (LLMs) instantiate computations of psychopathology? In this work, we establish a computational-theoretical framework to provide an account of psychopathology applicable to LLMs. Based on the framework, we conduct experiments supporting two key claims: first, that network-theoretic computational structures of psychopathology exist in LLMs; and second, that executing these computational structures results in psychopathological functions. We further observe that as LLM size increases, the computational structure of psychopathology becomes denser and the functions more effective. Taken together, the results suggest that network-theoretic computations of psychopathology may have emerged in LLMs. We discuss alternative explanations, including pattern matching, persona modeling, and semantic coherence, and argue that they are either complementary to our interpretation or less consistent with the data.

Primary area:
interpretability and explainable AI
Keywords:
LLMs, Mechanistic interpretability, Activation steering, Causal inference, Computation, Computational psychiatry

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_r4VVOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper asks whether LLMs can develop patterns that resemble how symptoms of mental disorders reinforce one another—for example, guilt leading to low self-esteem, which then increases guilt. It does not claim that LLMs actually feel these emotions. Its main contributions are:

  • A framework for studying these reinforcing patterns inside LLMs.
  • A method called S3AE to identify and manipulate twelve internal features associated with depression- and mania-related language.
  • Experiments testing whether activating these features changes other features and the model’s behavior. The authors test twelve LLMs across three model families. They activate one feature, repeatedly ask questions while feeding previous answers back into the model, and check whether related features become active and remain active after the intervention stops. They also test whether the models make more problematic decisions in simulated interactions and resist instructions to respond neutrally. They report persistent, mutually reinforcing patterns, with several effects becoming stronger in larger models. However, the experiments do not clearly separate the proposed internal mechanism from the model simply continuing the narrative established by its previous answers.
Critical strengths
  1. The paper connects internal representations to behavioral effects through direct interventions. Rather than relying only on symptom-like responses to questionnaires, the authors identify symptom-related features and manipulate model activations to test their effects on generated responses and simulated decisions. This provides stronger evidence that these representations influence behavior and makes the work a substantive step toward mechanistic investigation.
  2. The study tests an explicit theoretical framework across multiple models and complementary experiments. The authors translate network theory into testable predictions about feature propagation, persistence, and resistance to correction, then investigate them across twelve LLMs from three families. This strengthens the empirical contribution and provides a concrete framework that subsequent research can test and refine.
Critical weaknesses

Each reconstruction round receives the previous round’s answers as context, so symptom-related patterns could persist because the model continues a consistent narrative. Excluding the previous answer to the same question does not eliminate this explanation, since related answers remain available. Moreover, causal discovery assumes that the twelve measured features capture all relevant common causes, despite the richer textual context driving responses. This limits support for the central claim of an internal, self-reinforcing symptom network.

Questions for authors
  1. The model reads its earlier responses each round. How do you know the patterns persist because of an internal reinforcing loop, rather than because the model keeps following the story in those responses?
  2. You created the “one active feature” and “multiple active features” conditions differently. Could differences in wording or repetition explain why one group followed corrective instructions less well?
1 comment

We thank the reviewer for the review. We provide our brief responses below.


W1. Each reconstruction round receives the previous round’s answers as context, so symptom-related patterns could persist because the model continues a consistent narrative.

W2. Causal discovery assumes that the twelve measured features capture all relevant common causes, despite the richer textual context driving responses.

Q1. The model reads its earlier responses each round. How do you know the patterns persist because of an internal reinforcing loop, rather than because the model keeps following the story in those responses?

We clarified our position in the shared response above.


Q2. You created the “one active feature” and “multiple active features” conditions differently. Could differences in wording or repetition explain why one group followed corrective instructions less well?

Thank you for pointing this out. We believe the key variables (the number of active features and their magnitudes) are controlled for between the two conditions. Given that the input context involves many factors (e.g., different grammar, tone, words, etc.), we think it is generally infeasible to fix all variables but the number of active features. However, we will consider further control in a future version of this paper.

Reviewer_UcyvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates whether LLMs exhibit symptom-related internal representations and dynamics consistent with the network theory of psychopathology. It introduces a network-based perspective for studying psychopathology-related representations in LLMs. The paper validates the identified units through symptom–activation correlations, graded interventions, and prompt generalization.

Critical strengths
  1. The paper combines interpretable features, dynamic causal modeling, and the network theory of psychopathology to study symptom-related representations in LLMs. This offers a new perspective on the organization and interaction of these representations.
  2. The paper examines internal representations, dynamic propagation, behavioral effects, and model scaling. This provides a broader evaluation of the proposed framework across multiple aspects of model behavior.
Critical weaknesses
  1. The experiments do not fully distinguish the proposed feedback mechanisms from alternative explanations, such as contextual memory, semantic coherence, and broader effects of joint interventions. This limits the reliability of the interpretation.
  2. The experiments do not sufficiently establish their specificity to psychopathology or rule out more general explanations based on feature interactions in LLMs.
Questions for authors
  1. What motivates the use of sentence-level mean-pooled activations and strong multi-label supervision, rather than standard token-level SAEs? Could the supervision force the learned units to match predefined symptom categories, rather than reveal features naturally learned by the model?
  2. Could responses from earlier steps influence activations many steps later, even if they are not directly connected to the previous iteration? If so, could the observed symptom persistence be explained by earlier context rather than self-reinforcing cycles?
  3. Do interventions on individual units affect other behaviors or general model capabilities? How sensitive are the results to the choice of model layer? Furthermore, do different computational structures rely on distinct activation patterns, or do they share underlying features?
  4. Could the cyclic relationships between units arise from overlapping features or shared underlying factors, rather than direct causal interactions between units?
1 comment

We thank the reviewer for the review. We provide our brief responses below.


W1/W2/Q2.

We clarified our position in the shared response above.


Q1. What motivates the use of sentence-level mean-pooled activations and strong multi-label supervision, rather than standard token-level SAEs? Could the supervision force the learned units to match predefined symptom categories, rather than reveal features naturally learned by the model?

Supervision is used to conduct a targeted discovery of the symptom features. With an unsupervised approach, there is little guarantee that the model will discover the features that we look for. As clarified in the footnote on page 3, our criteria do not require the identified features to be natural, minimal, or mono-semantic units in which LLMs process information. The original network theory endorses the same view that a symptom node can consist of multiple sub-processes or representations.


Q3. Do interventions on individual units affect other behaviors or general model capabilities? How sensitive are the results to the choice of model layer? Furthermore, do different computational structures rely on distinct activation patterns, or do they share underlying features?

Steering with activation addition (i.e., our intervention) in general can deteriorate model capability. It provides a bias toward certain concepts or thoughts during its token generation, so its responses can become biased or brittle. We provided other behavioral consequences in Fig 3A. We used four different layers for both measurement and intervention of feature activations. Since we did not optimize the layer choice (we used ones at depths 1/5, 2/5, 3/5, and 4/5), we did not perform a sensitivity analysis.


Q4. Could the cyclic relationships between units arise from overlapping features or shared underlying factors, rather than direct causal interactions between units?

The hypothesis that 'a shared underlying factor is triggering our observed cycle-like patterns' is plausible. However, we argue that this is not likely. In Figure 5 of Appendix B, we show how the obtained features are highly distinct and even orthogonal to each other across all four layers. If the features had overlaps substantial enough to corrupt our interventions, we would expect greater similarities across the features. Finally, the symptom classification based on the feature activations shows that the activations are sensitive and precise about corresponding symptoms (Table 1 of Appendix B).

Reviewer_QxL3OpenReview ↗
Rating
2
Confidence
1
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates whether LLMs exhibit computational mechanisms analogous to human psychopathology. It proposes a framework based on the network theory of psychopathology and uses S3AE to identify and intervene on symptom-related features. Experiments suggest that these features form dynamic causal structures, influence model behavior, and exhibit stronger self-sustaining and resistance properties in larger models.

Critical strengths

The paper investigates an interesting research question regarding whether LLMs exhibit computational mechanisms analogous to human psychopathology. It provides a theory-grounded framework to study the interactions, propagation, and self-sustenance of symptom-related representations in LLMs. The experiments are relatively comprehensive, and the paper is clearly written and easy to follow.

Critical weaknesses
  1. The paper lacks sufficient comparison with recent studies published in 2025–2026. For example, Kambeitz et al. (2025) demonstrated that LLM representations can capture empirical relationships among psychopathological symptoms. Although this paper further investigates dynamic causal relationships and their behavioral consequences, a more explicit comparison with recent work would help clarify the additional insights and contributions of the proposed framework.

  2. The paper attributes persistent feature activations and stronger resistance under joint activation to cyclic causal relationships. However, the experiments do not sufficiently distinguish the contribution of these cycles from the effects of repeated or accumulated symptom-related information in the input context.

[1] Kambeitz, J., Schiffman, J., Kambeitz-Ilankovic, L. et al. The empirical structure of psychopathology is represented in large language models. Nat. Mental Health 3, 1482–1492 (2025). https://doi.org/10.1038/s44220-025-00527-y

Questions for authors
  1. Could the authors investigate whether disrupting the inferred causal cycles affects the propagation and persistence of symptom-related activations? For example, would suppressing a feature involved in a feedback cycle reduce self-sustaining activation compared with suppressing a feature outside the cycle?

  2. Could the authors introduce an additional control condition in which a single symptom-related feature is repeatedly reinforced across the conversation history, while matching the input length and activation magnitude of the joint-activation condition? Comparing the degree of activation suppression under the same corrective instructions would help determine whether the stronger resistance arises from repeated symptom-related information or from interactions among different features.

1 comment

We thank the reviewer for the review. We provide our brief responses below.


W1. The paper lacks sufficient comparison with recent studies published in 2025–2026.

  • We thank the reviewer for pointing to the related works. We clarify that our paper uniquely integrates (i) computational theory, (2) mechanistic interpretability, and (3) behavioral simulations to evidence potential psychopathological computations in LLMs. We will incorporate these studies in our future version.

W2. However, the experiments do not sufficiently distinguish the contribution of these cycles from the effects of repeated or accumulated symptom-related information in the input context.

  • We clarified our position in the shared response above.

Q1. Could the authors investigate whether disrupting the inferred causal cycles affects the propagation and persistence of symptom-related activations?

  • We thank the reviewer for a constructive idea. We will consider this ablation study in our future manuscript.

Q2. Could the authors introduce an additional control condition in which a single symptom-related feature is repeatedly reinforced across the conversation history, while matching the input length and activation magnitude of the joint-activation condition?

  • We kindly note that the existing control condition IS exactly the one the reviewer described: "single symptom-related feature is repeatedly reinforced across the conversation history", with "matching the input length and activation magnitude" (Fig. 3C&D)

Comments on the paper

1 comment

We thank the reviewers for their comments. We chose to withdraw our paper. However, even with withdrawal, we believe it is important to address the reviewers' questions and concerns, so we respond to the shared concerns here. Broadly, we believe that many key criticisms may stem from misunderstandings of our position or theoretical background.

First shared concern: The authors’ interpretation of the observed phenomenon is not distinguished from alternative ones, such as LLMs’ tendency to maintain contextual coherence. (QxL3 W2, Ucyv W1, r4VV W)

  • The two interpretations do not collide. Ours reside at the mechanistic level, and the alternative ones reside on the surface (see Discussion section).
  • The contribution of the “proposed internal mechanism” and the “input context“ to our observed phenomenon does not need to be distinguished. The input context can have its effect only because certain mechanisms support the effect (i.e., they are mathematically in an input-to-function relationship), and our analysis showed the mechanism may satisfy the requirements of network-theoretic computations of psychopathology.
  • Unlike many other mechanistic interpretability studies, our proposed mechanisms (psychopathological computations) are defined to operate ‘across’ multi-round contexts (See Theory section), so a separation of the mechanisms from context is theoretically infeasible.

Second shared concern: The experiments do not sufficiently establish their specificity to psychopathology or rule out more general explanations based on feature interactions in LLMs (QxL3 W2, Ucyv W2, r4VV W)

We argue that, to a certain meaningful extent, we have established specificity to psychopathology.

  • Our specificity requires two criteria: mechanisms and behavior (see Theory section).
  • Mechanisms alone do not establish specificity. Even in human psychology research, theorists began to view various psychological states (other than psychopathology) as sustained by similar cyclic mechanisms, but composed of different nodes and behavioral consequences. For example, studies proposed “psychological resilience” as a consequence of cyclic feedback among positive psychological states [1]. Likewise, we also believe our proposed mechanism (cyclic feedback) is likely a general mechanism inside LLMs.
    • [1] Bouncing Back From Life’s Perturbations: Formalizing Psychological Resilience From a Complex Systems Perspective, Psychological Review 2025
  • What establishes the specificity is the functions/behaviors that the mechanisms support. Going back to human research, both psychopathology and resilience may have the same underlying mechanisms of cyclic feedback. What distinguishes them is (1) the nodes that they are composed of and (2) the functional-behavioral consequences of the mechanisms. We showed that the proposed cyclic feedback mechanisms may cause those behavioral signatures of psychopathology.
  • We acknowledge that we did not establish specificity of the observed LLM behaviors to the proposed mechanisms. We provided causal evidence that the proposed mechanisms support the behaviors, but they may not need to be specific consequences of the proposed mechanisms.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.