- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
In this paper, the author investigated the underlying mechanism that enables the large language models to make prediction for hidden Markov models. First, the authors conducted empirical studies to identify three classical models, namely linear n-gram, Non-linear n-gram, and Spectral methods to be those that can approximate LLM better than others. Then, the authors show that indeed the LLM can implement them, finally, they conducted detailed layer-wise probing-and-intervention analysis aimed at identify causal relationship between low -dimensional algorithmic models and LLM's prediction. The overall conclusion suggest that LLM utilizes different models dynamically, although gradient-descent-like ones seem to play more important roles.
Critical strengths
The authors conducted extensive empirical analysis on different aspects of the problem. Especially the layer-wise probing-and-intervention analysis is quite insightful and of considerable impact.
The theoretical analysis, while not surprising, is also new and valuable to the community.
Critical weaknesses
My main concerns of the paper is that it is not conclusive. While many of the discoveries that have been discussed in the paper are interesting and beneficial to our understanding of LLM's predictive capability with respect to HHM, they do not provide any affirmative answer to the problem the authors raised at the beginning of the paper. It felt that while the paper has considerable potential, some work towards conclusions is still required.
Questions for authors
Q1: In the first stage of the studies, there seems to be no details on how exactly are generated, could you provide related information?
Q2: the argument for utilizing the coefficient of determination is rather weak, it would be great that references or some theoretical background can be provided.
Q3: The paragraph between L350-354, does not provide enough information on why it should be considered as a contribution. It now suggests that each of the individual mechanism is standard, and the contribution is on how they are applied.
2 comments
Thank you for your response, I will maintain my score.
Thank you for your thoughtful review. We address your concerns and questions below.
Thank you for pointing this out. The generation code is included in the supplementary material (
hmm.py). We fix M=4 hidden states and vary E(A),E(B)∈{0,0.25,0.5,0.75,1} (entropies normalized by M and N) and N∈2,4,8, giving 75 configurations.We will add these details to Appendix A.
Here is some background on the coefficient of determination. It is attributed to Wright [1], as the proportion of variance in one variable explained by others. In regression, it is defined as R2=1−SSres/SStot, which in our notation is 1−MSE/Var(rt). This is the form Kvalseth [2] identifies as the most common, and it can be negative on held-out data. It is also a standard metric for evaluating linear regression probes of neural representations [3].
We use it because normalizing by the target variance makes it comparable across candidate representations with different scales: 0 corresponds to predicting the mean and 1 to perfect recovery. Under label shuffling, held-out R2 drops to ≈0 (Appendix G.1), so high values reflect real alignment between activations and the target.
Our wording undersold the contribution. The individual tools are standard, but we combine them to address an open problem: discovering which algorithm a model implements when that algorithm is not known in advance.
Prior mechanistic work studies tasks where the optimal algorithm is known a priori, such as induction heads [5], modular addition [6], permutation tracking [7], and Markov-chain ICL [8]. There, the goal is to verify that the model implements a known algorithm. HMM prediction is different. The Bayes-optimal forward algorithm requires the unknown parameters, and multiple learning algorithms (spectral methods, finite-window regression, parameter estimation) all converge toward it with different inductive biases. The model is also pretrained rather than trained on the task, so there is no reason to expect it to implement any one of these exactly. Identifying which candidate the model implements therefore requires combining evidence: behavioral comparison to narrow the candidates, constructions to establish that each is implementable, and causal interventions to discriminate between them.
[1] Wright, Sewall. "Correlation and causation." Journal of Agricultural Research 20 (1921): 557–585.
[2] Kvalseth, Tarald O. "Cautionary note about R2." The American Statistician 39.4 (1985): 279–285.
[3] Gurnee, Wes, and Max Tegmark. "Language models represent space and time." International Conference on Learning Representations (2024).
[4] Belinkov, Yonatan. "Probing classifiers: Promises, shortcomings, and advances." Computational Linguistics 48.1 (2022): 207–219.
[5] Elhage, Nelson, et al. "A mathematical framework for transformer circuits." Transformer Circuits Thread (2021).
[6] Nanda, Neel, et al. "Progress measures for grokking via mechanistic interpretability." International Conference on Learning Representations (2023).
[7] Li, Belinda Z., Zifan Carl Guo, and Jacob Andreas. "(How) do language models track state?" International Conference on Machine Learning (2025).
[8] Edelman, Ezra, et al. "The evolution of statistical induction heads: In-context learning Markov chains." Advances in Neural Information Processing Systems 37 (2024).