acceptodds
Under review as a conference paper at ICLR 2027

Rung Lock: Skills Only Grow on the Right Skills

Abstract

During training, a capability that is about to emerge can look exactly like one that will never emerge. Both remain at chance, with no visible sign of what will happen next. We find that some capabilities are locked in this state until several pieces are in place at the same time. They earn no partial credit: if any one piece is missing, the training signal for the capability is exactly zero. The pieces therefore cannot be learned independently and assembled later. Each must help bring the others into place. We tested this by removing from the training data a simpler skill built from the same pieces, while changing nothing else. The target capability did not merely emerge more slowly. It never started. Across nine runs (thirteen with a preregistered extension), its score remained at chance; in two runs, it stayed there for the training normally required for emergence. We then tried to replace the missing skill with an unrelated skill of the same difficulty, training it until it reached perfect accuracy. The target capability remained exactly where it had been without the helper. When we could control the underlying pieces directly, the result became even simpler: a helper's value depended only on how many pieces it shared with the target, not on which pieces they were. A missing rung is not a delay. It is a lock—and the wrong key does nothing at all.

Primary area:
interpretability and explainable AI
Keywords:
emergence, skill acquisition, training dynamics, capability formation, feature learning, SGD noise, curriculum, interaction spectrum, phase diagram, interpretability

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_MmGZOpenReview ↗
Rating
1(−1)
Confidence
2
Informativeness
2
Presentation
1(−1)
Soundness
2
Read review
Paper summary

This paper proposes the idea of “rung lock,” where a higher-order capability can remain locked until lower-order capabilities sharing its constituent parts have formed. The authors give a theoretical argument for why this happens and test it using exact-enumeration Boolean experiments and injected tasks on Pythia-70m. In the Pythia experiments, removing a degree-2 precursor keeps a degree-3 target skill at chance for up to 45x the usual emergence budget, while an unrelated helper of matched difficulty provides no measurable benefit.

Critical strengths
  1. The three-arm design in Figure 1(a) (true precursor, unrelated helper trained to accuracy 1.000, no helper) is a good control. It rules out warm-up and general optimization effects as the explanation, which plain observation of sequential emergence cannot do. This is the main reason I find the causal claim credible.

  2. The paper is unusually open about what is proved and what is assumed. Table 1 lists the status of each claim, and failed preregistered predictions are reported rather than dropped.

Critical weaknesses
  1. The injected tasks are not described in enough detail. Appendix L only says “marked two-token mappings written into natural-text windows,” and I could not find what the degree-2 and degree-3 tasks actually are or what they share. Since Findings 2 and 3 depend on this, it should be in the paper.

  2. The causal evidence is narrow. Findings 2 and 3 come from one injected task pair on Pythia-70m at a single checkpoint, with only 3 runs for Finding 3, while Finding 1 is observational. The paper itself explains the null effect in Finding 3 by checkpoint 1000 having “already paid” the readout factor (Section 5), and in the lab an unrelated skill gives a 3.9x speed-up (Section 3.2), so the result may depend on the checkpoint. The title and conclusion read more broadly than this. I would like the authors to say how far they expect the Finding 3 result to hold.

Questions for authors
  1. What exactly are the degree-2 and degree-3 tasks injected into Pythia-70m, and what do they share?

  2. Do you expect Finding 3 (an unrelated skill buys nothing) to hold at earlier checkpoints, given that the paper attributes the null result to checkpoint 1000 having already paid the readout factor?

Reviewer_3BP9OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper is about the emergence of skills during LLM training. However, the writing is completely illegible, and I am confident that it was entirely or nearly entirely AI generated. Despite being familiar with the domain of LLM skills, I could not understand a single sentence in it. Being mindful of my time as well as everyone else's, I will not review this work any further and strongly recommend rejection.

Critical strengths

See summary.

Critical weaknesses

See summary.

Questions for authors

See summary.

Reviewer_9Jn3OpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates why learning a composite skill can depend on having the right precursor skills. It uses a Fourier-based analysis to connect skill formation with available features and training noise, derives exact identities for a one-hidden-layer model, and tests the account in Boolean tasks and resumed Pythia models. In the transformer experiments, a related precursor enables the target skill, while removing it or replacing it with a fully learned unrelated skill leaves performance near chance within the tested budgets.

Critical strengths
  1. The interventions provide useful evidence that transfer depends on the particular helper skill. The paper reports 13/13 no-precursor runs near chance across batch sizes 2–64, including two extended runs. A separate unrelated skill reaches perfect accuracy without enabling the target (pages 7–8 and 23). These controls are more informative than merely observing the order in which skills emerge.

  2. The paper distinguishes exact results from assumptions and reports predictions that did not work, including the self-fit and clean batch-scaling tests (pages 5–6, 19, and 24). This transparency helps readers judge the limits of the proposed explanation.

Critical weaknesses
  1. The central noise explanation needs a stronger justification. Similar noise variance in individual channels does not establish independence between channels, and a diagonal kernel can still scale their updates differently. Thus diagonality alone does not establish the equally strong, direction-independent update noise claimed in Proposition 1(a2) (page 16). The single-channel measurements in Figure 5 do not resolve this gap. Although the paper states relevant assumptions, this particular implication remains unsupported and weakens the explanation of formation times.

  2. The Pythia experiments show that the right helper skill improves learning, but do not yet show why. Appendix L (page 33) does not fully specify the degree-2/degree-3 tasks, their precursor relationship, or which features and gradient channels are reused. Without this information, it is difficult to connect the observed transfer effect to the proposed gradient-blindness mechanism.

Questions for authors
  1. What additional conditions ensure independent and equally strong update noise across the relevant channels, and do the experiments satisfy them? How should the claim in Proposition 1(a2) on page 16 be corrected?

  2. Can you provide the exact injected tasks and precursor relationship, and identify the features or gradient channels reused in Pythia? Does the related-helper advantage persist when target-symbol exposure and non-target losses are matched to the unrelated-helper control?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.