- Rating
- 1(−1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1(−1)
- Soundness
- 2
Read review
Paper summary
This paper proposes the idea of “rung lock,” where a higher-order capability can remain locked until lower-order capabilities sharing its constituent parts have formed. The authors give a theoretical argument for why this happens and test it using exact-enumeration Boolean experiments and injected tasks on Pythia-70m. In the Pythia experiments, removing a degree-2 precursor keeps a degree-3 target skill at chance for up to 45x the usual emergence budget, while an unrelated helper of matched difficulty provides no measurable benefit.
Critical strengths
-
The three-arm design in Figure 1(a) (true precursor, unrelated helper trained to accuracy 1.000, no helper) is a good control. It rules out warm-up and general optimization effects as the explanation, which plain observation of sequential emergence cannot do. This is the main reason I find the causal claim credible.
-
The paper is unusually open about what is proved and what is assumed. Table 1 lists the status of each claim, and failed preregistered predictions are reported rather than dropped.
Critical weaknesses
-
The injected tasks are not described in enough detail. Appendix L only says “marked two-token mappings written into natural-text windows,” and I could not find what the degree-2 and degree-3 tasks actually are or what they share. Since Findings 2 and 3 depend on this, it should be in the paper.
-
The causal evidence is narrow. Findings 2 and 3 come from one injected task pair on Pythia-70m at a single checkpoint, with only 3 runs for Finding 3, while Finding 1 is observational. The paper itself explains the null effect in Finding 3 by checkpoint 1000 having “already paid” the readout factor (Section 5), and in the lab an unrelated skill gives a 3.9x speed-up (Section 3.2), so the result may depend on the checkpoint. The title and conclusion read more broadly than this. I would like the authors to say how far they expect the Finding 3 result to hold.
Questions for authors
-
What exactly are the degree-2 and degree-3 tasks injected into Pythia-70m, and what do they share?
-
Do you expect Finding 3 (an unrelated skill buys nothing) to hold at earlier checkpoints, given that the paper attributes the null result to checkpoint 1000 having already paid the readout factor?