acceptodds
Withdrawn from ICLR 2027

What Should Data Teach? Moving Bottlenecks Across Circuit, Store, and Use

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

What should data teach a language model at a particular point in training? A circuit view reveals three distinct bottlenecks: forming a computation, making its required content available, and selecting among available routes. A shared diagnosis-to-data principle connects them: localize the missing operation, preserve its causal relation, vary shortcut-bearing context, and re-audit the residual. Formation-sensitive selection and prerequisite ordering accelerate a binding–matching–transport path; a brief early prefix from the same training multiset retains a validation advantage through 100B tokens. Availability counterfactuals then distinguish writing content from invoking available memory, while paired supervision and context-opportunity ranking improve matched route decisions and long-context answer likelihood. A continuous 350M-model experiment connects the three interventions on the same facts: early circuit training improves subsequent learning, and the complete sequence outperforms stage-replacement controls on facts withheld from Use teaching. Independent query surfaces and opposed-source decisions expose conditional arbitration as the remaining frontier. Together, these results show why a change in the limiting operation calls for a change in supervision, not merely a new ranking of difficult examples.

Primary area:
interpretability and explainable AI
Keywords:
mechanistic interpretability, induction heads, transformer circuits, data selection, language model pretraining

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_q7bKOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a mechanistically-motivated framework for pretraining data selection organized around three sequential bottlenecks: Circuit (the required computation, e.g., a binding-matching-transport read path, has not formed), Store (content is unavailable in parametric memory, split into M-Write and M-Use), and Use (content is available but the query-authorized route fails to control the output). A shared "diagnosis-to-data" principle - localize the missing operation, preserve its defining relation, vary shortcut-bearing context, re-audit the residual - derives stage-specific supervision.

Empirically: (i) a 1.485B / 100B-token matched-multiset run where a 1.2B-token Curriculum prefix (formation-time MDA + joint-to-matching order) crosses Ordinary's final validation PPL at 82B and retains the advantage through 100B; (ii) auxiliary 160M order studies showing domain-selective gains (Overall/Math/Code improve, Web does not); (iii) a Store 2x2 factorial (Write x Use) on 512 circuit-dragged facts showing Use contributes more than Write; (iv) paired-Use supervision improving strict-pair accuracy by 4.4pp over balanced-unpaired on familiar queries, and CCUR beating LongCE and a score-shuffled control for long-context window ranking; (v) a continuous 350M chain on 640 common facts where the complete Circuit-Store-Use sequence outperforms four stage-replacement controls. The paper reports negative results honestly: independent-query transfer fails, opposed-source arbitration remains open, and Web does not benefit from Circuit training.

Critical strengths
  1. Experimental controls are substantially more rigorous than typical curriculum-learning papers. The matched-multiset design (positions change, data does not), the order study (joint-to-matching vs interleaved vs reverse), the exit sweep, three independent 3B initializations, and score-shuffled / document-disjoint replications for CCUR make the positive claims more credible. This matters because the headline effects are small (~0.43% relative PPL at 100B), and without matched comparisons they would be hard to distinguish from noise.

  2. The Write/Use distinction in Store is a genuinely useful conceptual contribution. Separating "content not written" from "content written but suppressed by a natural carrier," and showing via a 2x2 factorial that Use training contributes more than Write for dragged facts (Table 24), has practical implications for knowledge editing and retrieval-augmented settings. The paper is also commendably transparent about where the framework fails (Web, independent-query transfer, CCUR's memory-authorized regression).

Critical weaknesses
  1. The Use stage - the framework's terminal and most important stage - does not deliver on its promise. By the paper's own audits, paired supervision underperforms Ordinary on independently phrased queries across every configuration (Table 28), CCUR improves context-authorized decisions while worsening memory-authorized ones with no joint gain (Table 32), and "conditional arbitration" is explicitly left open. The positive Use result reduces to "matched-route engagement under familiar queries," a much narrower claim than the three-stage framing implies. Since the framework's value proposition is that diagnosing the current bottleneck tells you what to teach next, an unsolved terminal bottleneck leaves the chain incomplete and is the primary reason I lean toward rejection.

  2. Effect sizes are small and statistical support is thin. The headline 100B result is a ~0.43% relative validation PPL difference from a single trajectory per arm; seed variance exists only for three 3B runs (1.00-1.58%), and its transfer to 100B is unclear. The Store factorial reports NLL gains without confidence intervals, and the 350M chain's absolute pair accuracy is 5.26% with no ceiling analysis. The novelty is also incremental: MDA supplies the Circuit signal and LongCE the long-context baseline, and the formation-vs-endpoint difference (Table 13: 3.79557 vs 3.82161 at 1.2B) is modest with no significance test.

  3. Presentation density makes the paper hard to evaluate. The abstract and introduction are heavy with terminology defined only later (or never), the notation is extensive and mostly single-use, and nine pages of main text carry seven experimental threads so no result is fully digested. For a paper whose contribution is a unifying principle, the writing barrier substantially limits its usefulness to the community.

Questions for authors
  1. Scale dependence. Have you observed the Circuit-Store-Use ordering at 7B+, even in a small probe? If induction-head formation timing shifts with scale, does the Curriculum prefix length need rescaling, and by what rule?

  2. Statistical confidence at 100B. The 3B runs give 1.00-1.58% gain while 100B gives 0.43% from a single trajectory per arm. Is this shrinkage expected, and can you provide any seed-variance estimate at the 100B scale (e.g., from the dense late-training window in Table 17)?

  3. CCUR's source prior. Table 32 shows CCUR improving context override while degrading memory reject. Is this inherent to the full-local response signal, or could balancing authority-mode exposure during selection mitigate it? This bears directly on whether the Use stage can be made to work.

Reviewer_8PBxOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes an operational distinction among three LM training bottlenecks: Circuit, Store, and Use. It investigates corresponding interventions through mechanism-sensitive curricula, fact-writing and invocation supervision, paired source-authority training, and long-context data selection. An early curriculum retains a validation advantage through 100B tokens, while a separate 350M continuous-training experiment combines the three stages and evaluates their contributions on shared facts.

Critical strengths
  1. The distinction between computation, knowledge availability, and source selection offers a useful perspective on why superficially similar prediction errors may require different supervision. The paper appropriately frames these as operational failure categories rather than separate physical modules.
  2. Several experiments include thoughtful controls. These include fixed-multiset ordering comparisons for Circuit, Write×Use comparisons for available but suppressed facts, and difficulty-matched and score-shuffled controls for long-context selection.
  3. The persistence of an early curriculum’s validation advantage through 100B tokens is an interesting empirical finding. The paper also transparently reports limitations, particularly failed transfer to independently phrased queries and the absence of improved joint arbitration.
Critical weaknesses
  1. The framework’s diagnostic value remains under-tested. While Table 3 establishes the benefits of combining interventions, it does not directly show whether diagnosed bottlenecks predict their relative effectiveness.
  2. As the authors acknowledge, Use gains do not reliably transfer across query formulations, and CCUR does not improve joint arbitration, limiting the scope of the positive results.
  3. Table 3 reports only 5.26% pair accuracy, measured at the first divergent token, without uncertainty estimates; complete-answer performance and the reliability of the gains remain unclear.
Questions for authors
  1. Does each diagnosed failure category benefit more from its corresponding intervention than from the alternatives?
  2. Can the authors provide complete-answer accuracy and fact-level confidence intervals for Table 3?
Reviewer_8NzbOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

In this paper, the authors study what data should teach a language model at a particular point in training. The whole paper is quite hard to follow. Terms and numbers appear suddenly without sufficient explanations. References are cited merely in the related work section and almost never used in the main text. The experimental results are poorly organized. Overall, I recommend reject.

Critical strengths

S1. The authors study what data should teach a language model at a particular point in training, and propose that circuit formation, content availability, and route control require different supervision.

Critical weaknesses

W1. The authors cite a lot of papers in the related work section. However, these cited papers are merely mentioned and almost not cited in the main body.

W2. The whole paper is hard to follow. The abstract is confusing. A lot of terms appear suddenly such as binding–matching–transport path. Two mentioned numbers are confusing as well: through 100B tokens and A continuous 350M-model experiment.

W3. Experimental results are presented everywhere in the paper like presentation slides. In the main text there is no experiment section in which the authors should provide basic experimental setup, introduce competitors and present a consolidated result table.

Questions for authors

N/A

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.