- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This submission studies how the initialization of the network and the training affect the task-related neuron organization in large language models (LLMs). The authors first study the networks with initial weights, while manipulating the words in the input frame or the candidate answers. It was reported that both input context and prediction targets influence the task-related neuron organization. The authors also studied these factors affect neuron organization early in the training.
Overall, the paper studies a meaningful question and some of the observations reported seemed promising. However, the results are too preliminary and the conclusions are premature. The writing is messing at the moment. The study is not as well motivated, and not well grounded on prior work. Some heavy-lifting work would be needed for turning this work into publishable quality.
Critical strengths
None.
Critical weaknesses
-
While I find that the topic is be relevant and some of the analyses are interesting, the results are too preliminary and the results are not carefully interpreted. Additional control analyses would be needed to better understand these results. The study only concerns the initial state of the network and early in training, what happens towards later phases of the training and how that relates to the organization in early training wasn’t clear.
-
The writing needs major improvements. Right now, the submission reads like a preliminary technical report.
Questions for authors
Can the authors provide a formal argument for the similarity of the patch attribution and loss gradient shown in Fig. 1?
What was the interpretation for plots in the Fig. 4? It appeared that the traces were basically flat.