- Rating
- 2(+1)
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The paper studies whether transformer trained on time series of LLM on real text follow the pattern where earlier layers model shorter autocorrelations, and later layers model longer lags.
Critical strengths
Understanding what LLMs encode in their internal representations is important. The paper's central methodology is the autocorrelation function of equation (1), where dot product for a layer's embedding on token at time T and token at time T - tau is computed. Since such dot product on normalized activations is also used as a similarity measure in other fields such as RAG, it is to be a reasonable measurement. The main results such as Figure 1 supports the finding regarding earlier vs later layer in what time lapse tau they encode.
Critical weaknesses
A central weakness is takeaway or contribution from the paper. Intellectually: it is understood, prior to this paper, that generally speaking later layers tend to encode more abstract concepts, and earlier layers encode more local and lexical aspects. The paper does point out related works that share this finding. Therefore, the paper's contribution can be viewed as adding more evidence towards this known aspect. Application-wise: the paper examines transformer trained on time series and LLM applied to general text data like WikiText. However, it is not discussed whether this interpretability angle sheds new light on how LLMs encode concepts (except earlier vs later layer distinction in autocorrelation lag), how we should view or use LLMs, or how deep neural nets can be used to model time series more effectively.
A second weakness is organization of the paper. The paper is has 3 sections: Intro, Results, Discussion and Perspectives. Its methodology is written embedded within the Results, making the paper sometimes difficult to follow. A roadmap and motivation for what experiments and why these experiments would be helpful.
Questions for authors
There're no additional questions that would materially affect my assessments. Main thoughts are in Strengths and Weaknesses.
3 comments
I thank the authors for the clarification. It is helpful to clarify that the paper is not framed as further evidence for later layers encoding more abstract concepts, but more about quantifying that aspect with autocorrelation. However, it is still not clear what problem the paper solves or what's surprising about LLMs that the paper revealed. Overall, I raise my score to 2.
Our work does not solve a specific problem: it is aimed at the scientific understanding an empirical property of transformer activations. We appreciate that the reviewer doesn't regard this as a meaningful contribution, but we believe that works like ours, combining insights from toy models with real-world experiments, significantly expand our theoretical understanding of how LLMs process information.
There are many examples of works with a similar goal in the machine-learning literature. For instance, https://arxiv.org/abs/1611.00740 (among others) characterised the advantage of deep over shallow networks from an approximation theory perspective. By then, the community had already realised that 'Deep is better than shallow' for vision models. Yet, this paper improved our theoretical understanding of the phenomenon and spurred further fundamental work.
Scaling laws are another example: they have been driving Industry developments for a few years already, but their origin is still debated (e.g., https://arxiv.org/pdf/2602.07488 ; https://arxiv.org/pdf/2606.25008) and generating valuable theoretical research.
In the same spirit, our work quantifies how feature persistence and autocorrelations increase with transformer depth, and uses the RHM to explain how this layer dependence can arise from a hierarchical structure of data.
The takeaway/contribution is not "later layers encode more abstract concepts"
We thank the reviewer for the detailed critique. We agree that the submitted version did not state the contribution sharply enough. The revision addresses this as follows. A clarification on scope: the paper does not train models on time series: this term appeared only as an analogy in the old abstract. The synthetic data are sentences generated by a hierarchical grammar (the RHM), and all the other experiments use LLMs pretrained on text.
1. The takeaway/contribution is not "later layers encode more abstract concepts"
We take this statement as a starting point, not our conclusion. We argue that the literature lacks a way to quantify abstraction in a model-native way: current evidence relies on labelled probes, paired data such as translations; or SAE features, whose abstraction is judged by the subjective interpretation of features. The last route is especially fragile: what an SAE feature represents is not well defined, and its meaning is usually assigned by asking another LLM to summarise its activating examples. As a result, "abstract" has no operational definition, and no theory can predict the outcome of its measurement.
Instead of defining abstraction in trained LLMs, we use an observable that quantifies how long the information they encode persists across tokens: the autocorrelation cℓ(τ).
For the theory, we use the RHM, where all latent variables are known and cℓ(τ) decomposes into contributions of named latents. The persistence of these latents predicts
The test pretrained LLMs use SAE features only to validate the relationship between cℓ(τ) and persistence (Fig. 4c), through when they are active and not through what they mean. We reframed the abstract of the revised manuscript to clarify the scope.
2. New light on how LLMs encode concepts.
It does, as it suggests that LLMs hold the code of a variable fixed over the span of the input governed by that variable. Therefore, deeper layers show longer-range autocorrelations because the variables they encode govern longer spans, not because of a generic smoothing with depth.
3. How should we view, use or train LLMs?
Since cℓ(τ) requires only raw activations, it applies where labels, paired data or SAEs are unavailable. We are running two experiments to test possible applications, we will report on the outcomes when completed.
Locating language-independent content without paired data. Neighbour-based measures locate the layers that encode language-independent content by comparing the representations of translated sentences (Acevedo et al., 2026). In preliminary experiments, the depth profile of the integrated autocorrelation χℓ matches that of this representation similarity almost perfectly (r=0.97, averaged over four LLMs). This is promising, as autocorrelations are measured on a single language and are much cheaper to compute.
Signal available to latent-prediction objectives. A recent theory (Korchinski, Favero & Wyart, 2026) shows that, on hierarchical data, networks trained to predict their own latent representations (as in data2vec or JEPA) need exponentially fewer samples than networks trained to predict tokens, because correlations decay much faster with distance between tokens than between latents. A modified autocorrelation, cswap(τ), measures this signal without knowing the latents: we replace all the tokens after position T−τ with those of an independent sentence, and subtract the resulting autocorrelation from c(τ). The difference isolates the dependence between the text before and after position T−τ. In preliminary experiments on the RHM with 220 sentences, this signal is within the sampling noise at the token level at all lags, whereas layer 2 resolves it up to lag 3 and layer 3 up to lag 6, with a signal-to-noise ratio of 10–30 (below 2 at initialisation). This implies that a next-latent prediction method could leverage a higher learning signal than a next-token prediction method. Whether this is feasible in practice also depends on when the signal becomes available during training, and requires tests on real data, for instance by tracking how c(τ) and cswap(τ) develop over the publicly available checkpoints of pretrained LLMs.
4. Organisation of the paper
The revision has a dedicated methods section (Sec. 2) separate from the results. The contribution section now includes a roadmap linking each experiment to the claim it supports. Results on synthetic and real data are split across different sections. We added new schemes to simplify the presentation of our results in Fig. 1d and Fig 2a, and significantly revised Section 3 to improve readability.