acceptodds
Under review as a conference paper at ICLR 2027

Towards Effective Theory of LLMs: A Representation Learning Approach

13 views

Abstract

We propose Representational Effective Theory (RET), a framework for describing large language model computation in terms of learned macrostates rather than microscopic details. RET learns these macrostates from hidden-state trajectories using a BYOL/JEPA-style self-supervised objective, coarse-graining activations into macrovariables that preserve higher-level structure relevant for prediction and interpretation. We evaluate whether these macrovariables are practically relevant for interpretability: RET yields temporally consistent states that reveal “mental-state” trajectories of reasoning, capture high-level semantic structure, support early prediction of behavioral outcomes such as sycophancy and alignment faking, and provide causal handles for steering generations toward interpretable computational phases. Together, these results suggest that LLM computation admits useful effective descriptions via RET: high-level, dynamically meaningful variables that support interpretation, prediction, and intervention.

Primary area:
interpretability and explainable AI
Keywords:
Mechanistic Interpretability, Science of LLMs, Effective Theory, Steering

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_DcnjOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a Representational Effective Theory (RET) to abstract large language model computations into macrostates learned from hidden activations using a BYOL/JEPA-style self-supervised objective. These macrostates are used to represent long reasoning traces as sequences of higher-level states, which the authors interpret as distinct phases of the model's reasoning process. The authors demonstrate that these representations yield coherent and interpretable trajectories, enable early prediction of downstream behaviors such as sycophancy and alignment faking, and provide causal handles for steering model generation.

Critical strengths
  1. The paper demonstrates that RET can identify coherent reasoning trajectories from a model's internal computations, which have multiple interesting applications, such as the demonstrated prediction of sycophancy and alignment faking, and interventions that influence model generation. The general capturing of model behavior in these macro states seems very useful for understanding and evaluating the complex computations of large language models.

  2. The evaluations indicate that RET captures the semantic context of tokens rather than primarily their lexical identity. For example, occurrences of the same token (such as "and" in Figure 3) are represented according to their surrounding context, unlike the evaluated SAE baselines. This suggests that RET captures higher-level contextual and temporal structure, which is also supported by its improved next-macrostate prediction performance.

Critical weaknesses
  1. The paper claims that the macro-state trajectories are interpretable but does not evaluate this sufficiently. It provides a lot of qualitative examples, but leaves them for the reader to judge. The LLM-based MIS score provides some insights; however, this is based on LLM-generated descriptions for LLM-annotated states, so it remains unclear how much of this is LLMs reinforcing themselves and how much is useful insight for humans. Since interpretability is one of the central claims of the paper, the evaluations in this regard are not fully convincing to me.
  2. The paper focuses mostly on the NuminaMath domain. I agree that mathematical reasoning is interesting, but it would be good to validate the method on other domains and types of reasoning as well, e.g., coding, planning, or common-sense reasoning, which might follow different patterns. It remains open whether RET would capture similar meaningful states in these domains.
Questions for authors
  • Q1: The macro-states shown in figures 1, 7, etc. do not appear one after the other, but there is some jumping around, oscillating between G10 and G7, for example, in figure 1. While this generally makes sense, how do you handle it for next-state prediction? It seems this could be better captured as a stochastic process, or even as an explicit automaton with multiple possible next states.
  • Q2: Why do you choose the 50-token window for calculating the MIS score? Isn't this abstracting such oscillations away, or maybe even losing macro-states completely if they appear only a few times?
  • Q3: Have you ablated the choice of K and G? Would be interesting to see if there are some macro-state types that stay consistent despite the choice of K and G.
  • Q4: Since it is my main weakness point: how do you define interpretability for RET? The macro-states are a simplification of the thinking traces, for sure, yes, but it still takes a long time to understand the mapping of states to the tokens and the general trend of which state follows which. It feels like a human still needs to do a lot of work, and then the states are not even guaranteed to be correct interpretations. Have you considered applying process mining to macro-state sequences to discover more concise representations of the underlying reasoning processes, such as recurring transition patterns or higher-level state machines?
Reviewer_G8BjOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Representational Effective Theory (RET), a framework for understanding LLM computation through learned macrostates. Inspired by effective theories in physics, RET learns compressed representations of hidden-state trajectories using a BYOL/JEPA-style predictive objective. The learned macrostates are evaluated in terms of their temporal consistency, approximate dynamical closure, interpretability, behavioral prediction, and ability to support causal interventions.

Critical strengths

I find the central idea of this paper very interesting and conceptually compelling. In particular, I appreciate the perspective that understanding LLM computation may require identifying higher-level dynamical variables rather than focusing exclusively on individual activation features.

Critical weaknesses
  1. Although Appendix N provides extensive robustness analyses, it remains unclear whether independently trained RET models consistently recover semantically equivalent macrostates and whether their steering effects remain stable across different seeds and target clusters. Further evaluation of semantic consistency and steering reproducibility would strengthen the claim that RET discovers meaningful and reliable computational states.
  2. Appendix D mainly evaluates one-step self-prediction, which may favor slowly varying representations without necessarily capturing the underlying computational dynamics. Multi-step prediction and cross-distribution evaluations would provide stronger evidence for the proposed approximately closed macro-dynamics.
Questions for authors

See Critical Weaknesses.

2 comments
Reviewer_G8BjOpenReview ↗

Response to Authors

Thank you for the detailed response and additional experiments. I appreciate the authors' efforts to address my concerns. In particular, I find the copy-forward baseline and longer-horizon prediction results convincing.

Overall, I appreciate the rebuttal and maintain my positive assessment of the paper.

Thank you for the positive assessment and for finding the additional results convincing. Since these experiments were intended to directly address the concerns raised in your review, we would be grateful if you could let us know whether any substantive concern remains. If not, we respectfully ask you to consider whether an increased score would better reflect your updated assessment.

Reviewer_NEVaOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper investigates whether LLM computation can be described through compact, interpretable macrostates that capture higher-level computational structure. It proposes Representational Effective Theory (RET), which uses a causal encoder to compress hidden-state prefixes into 128-dimensional representations and a BYOL/JEPA-style objective to predict the next macrostate.

Critical strengths
  1. The paper motivates studying computation at a level between individual activation features and complete model behavior.

  2. RET produces more coherent trajectories than the evaluated raw-state, PCA, SAE, and temporal-SAE baselines. The controlled scene-boundary experiments assess both within-context consistency and sensitivity to context changes, making the evidence more informative.

Critical weaknesses
  1. High next-state prediction accuracy does not sufficiently establish meaningful autonomous dynamics beyond temporal smoothness. A simple baseline that carries the current state forward, together with multi-step prediction results, would help clarify whether RET learns meaningful dynamics beyond local continuity.

  2. Interpretability and steering evaluations rely heavily on generated narratives and linguistic markers, providing limited evidence of faithful computational explanations or substantive reasoning changes.

  3. Behavioral prediction gains depend strongly on supervision, leaving the specific contribution of the predictive objective unclear.

Questions for authors

Please see weaknesses.

Comments on the paper

1 comment
Area_Chair_aYZnOpenReview ↗

rebuttal ddl

Dear reviewers, the ICLR deadline is only 4 days away. The authors have already rebuttaled the issues raised. If you still have questions, please participate in the rebuttal.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.