- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper investigates how the internal representations of the Llama family of models allow such models to perform a concept learning task (the Number Game) and how this Llama behavior matches and differs from Bayesian and human learning. The main paper is in the form of a case study of a particular Llama model in this context. The results provide an interesting direction for research but the generality of the result is debatable given that only a single model family and a single type of learning task is studied in the paper.
Critical strengths
-
the general problem, of understanding how language models come up with internal hypotheses for concept classes, is important and interesting
-
the paper is generally clearly written
-
the Appendices provide extensive details about the methods and experiments in the paper
Critical weaknesses
-
Focusing the case study only on Llama models and on a very specific task (the Numbers Game) significantly limits the potential impact of the paper and the generality of the conclusions drawn. I am aware for example that prior related work also focused largely on Llama models: but this still seems like quite a limiting factor (even if other studies made the same choice). It would be quite a bit more interesting and useful to the AI research community if a study of this nature could analyze results across multiple SOA open-weight models and assess if they all have similar characteristics or not. It would also be helpful to validate these results on a different type of number-related learning problem. I would encourage the authors to pursue these extensions in future work on this topic. As a sidenote, the results on the larger 70B model in Appendix ) seem to suggest that although there are similarities with the model in the main paper, there are also some differences, providing further motivation for analyzing these behaviors acros a broader range of model sizes and families.
-
Bayesian modeling of LM Behavior (lines 130 to 146): its interesting that the proposed Bayesian model fits well, but I would like to see here some discussion of alternative potential models, e.g., is this particular model giving significantly better predictions than other candidate models we might consider, or are there multiple plausible models for Llamas's outputs. This seems like a crucial question.
-
Similar comments apply to the model in Section 6 and Appendix N (even more so, given that this is one of the main proposed contributions of the paper). The authors propose a more complex Bayesian model (than the original) that they argue more closely reflects the mechanisms underlying how Llama reasons about the Numbers Game. But its not clear what this is really telling us. Are there other simpler (or equivalent complexity) models that would also achieve such Kl scores? How sensitive are the results to the choices for the priors or the specific choices for the 6 families? (Appendix N).
Questions for authors
No specific questions beyond Weaknesses above