- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper focuses on the phenomenon of grokking, that is when a neural networks starts generalizing on fresh data many epochs later interpolation on the training set. The authors aim to understand what fails when a model never generalizes compared to when it groks. The authors claim that this can be due to the fact that some heads focuses on portion of the inputs that are not relevant to solve the task and, therefore, training get stuck. Moreover, they show that, when transferring the embeddings and the readout layers of models capable of generalizing, the models that were failing to generalize now exhibit grokking. They also show that, when a model get stuck because of an head focusing on useless information, this typically happens in the first stage of the training. They thus introduce SoftBind, a technique preventing the head to specialize on a specific part of the input, before having understood the general task to accomplish.
Critical strengths
The paper addresses an important question, that is trying to understand why certain models trained on math tasks fail to generalize while some others exhibits grokking.
The paper shows some potentially interesting phenomenology and empirical findings, such as that transferring embeddings or specific layers of a model capable to generalize on a given task save some seeds of another model that are failing to generalize, to instead start grokking. They also propose a mechanics to avoid early specialization of heads on useless parts of the input, before the model has a general understanding of the task.
Critical weaknesses
I think the writing of the paper as well as the plots should be consistently improved. It is especially hard to understand the novelty of this works since a "related works" section is missing. Moreover, the level of details to describe the experiments and the eccessive terminology and keywords most of the time obscures the logic behind the experiments themselves.
I think that the main weakness of the paper is that it seems more a report of a series of experiments, with many phenomenon detected but not a clear explanation and identification of the underlying mechanism generating them.
Questions for authors
-
How standard is the grokking criterion established by the authors? Because in Fig. 2 it seems that if we wait a little bit longer, also the validation loss of those seeds the authors say they never grok would instead grok.
-
The math operations "det", "dot" and "poly" do not contain "traps", so why they do not grok?
-
Why H > K has just been investigated for some tasks and not all of them? Same for H <= K.