acceptodds
Under review as a conference paper at ICLR 2027

A Theoretical Analysis on the Emergence of Implicit Curriculum in Transformers

Abstract

Empirical studies of language-model pretraining suggest an implicit curriculum: localization and retrieval capabilities often emerge before capabilities that compose retrieved information. Existing theory, however, typically analyzes these ingredients separately, studying attention-based token selection with simple readouts or nonlinear composition on fixed representations, leaving open whether joint training can itself induce a structured acquisition order. We study this question in a stylized shared-attention multi-task Transformer, in which a softmax attention layer, shared across tasks, feeds task-specific ReLU heads. The model is trained by stochastic gradient descent on a fixed mixture of two tasks whose input sequences contain informative tokens among distractors: a token-selection task, labeled by the sign of a single informative token, and an XOR task, labeled by the XOR of the signs of two informative tokens. Both tasks are present throughout training, and the shared attention-weight parameter and both heads are updated in parallel, without an externally imposed curriculum. Our main result proves that the joint trajectory nevertheless exhibits a definite learning order: token selection reaches small test error while the XOR error is still bounded away from zero, and the XOR error becomes small only at a later iteration. The mechanism is that token-selection learning amplifies attention to a positional signal shared by both tasks, while parity cancellation and cumulative stability control keep XOR updates from disrupting this amplification. Experiments on synthetic data from the model and MNIST-based token sequences support the predicted learning order. As supporting empirical context, preliminary measurements on open intermediate large language model (LLM) checkpoints show a compatible pattern in which perception-like skills emerge earlier than composition-like skills.

Primary area:
learning theory
Keywords:
Large Language Models, Feature Learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_DX5SOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper theoretically characterizes the emergence of an implicit curriculum in a stylized shared-attention Transformer, showing that token selection is learned before XOR composition despite simultaneous training on both tasks.

The authors claimed that in a simplified model with shared attention and task-specific ReLU heads, the token selection would reach lower test errors before the XOR composition.

Then, they also gives a lower bound for XOR-only training under same training budgets and explains the ordering.

Finally, the three experiments (Synthetic data, MNIST sequences, LLM checkpoints) support the main prediction, but the evidence becomes less mechanistically conclusive as the setting becomes more realistic.

Critical strengths
  1. The theoretical analysis provides a learning-order guarantee under simultaneous training and the theoretical derivations are clearly presented and carefully structured.

  2. The experiments complement the theoretical analysis by demonstrating the predicted learning order on synthetic and MNIST-based tasks and the results partially support the assumption.

Overall, this is an interesting paper. Although the theoretical analysis relies on relatively strong assumptions, the experiments show that the proposed method is effective to some extent. If the authors can further improve the completeness of the paper and extend the method to widely used llm, the paper could become a promising line of work worth following.

Critical weaknesses
  1. The key weakness is the gap of theory and experiment(especially contemporary large language models).

The theory establishes a concrete mechanism for implicit curriculum, but relies on restrictive structural and scaling assumptions, particularly an explicitly shared relevance signal and a reduced multi-task architecture. Consequently, its applicability to modern LLM pretraining remains uncertain. Pythia-6.9B, Amber-7B, OLMo-2-7B the experiments used is totally outside the theoretical setting and the author also do not provide theoretical anaslyation for the LLM checkpoints.

  1. The experimental section is brief with insufficient analysis without ablation study.

The experiments mainly show the predicted learning order, but they do not clearly test the reason behind it. In particular, the LLM experiments are quite different from the theoretical setting. Similar learning curves alone cannot prove that the proposed mechanism works in real LLMs. More controlled experiments and ablation studies would help connect the theory to the experimental results.

For example, have the authors considered controlled comparisons between ReLU and SwiGLU to test the robustness? More broadly, how sensitive is this behavior to architectural choices.

I understand that space is limited. The authors could reorganize the paper, for example, by moving some less important equations to the appendix.

Questions for authors

As noted in the Weaknesses section.

The paper appears to be primarily theoretical, as the experimental section is limited in both scope and depth of analysis, however, two of the four contributions listed at the end of the intro are about experiments and the authors' main insight comes from the gap between the experimental observations and the theory, therefore, I believe the authors need to explain this key weakness more clearly.

5 comments
AuthorsOpenReview ↗

Response to Reviewer DX5S (Part 1/2)

We appreciate the reviewer’s thoughtful comments and provide our responses below.

W1: The key weakness is the gap of theory and experiment(especially contemporary large language models).

Thanks for raising this! We acknowledge that our theorem does not directly characterize the training dynamics of modern LLMs, and we do not claim it does. Our goal is to keep the key structure that matters for an implicit curriculum and drop the rest, so we proposed the stylized problem to capture the training dynamic of a shared-attention multi-task Transformer. What we keep: both tasks are trained together from the first step under a fixed mixture, attention and the ReLU heads are updated at every iteration, and the two tasks share the need to locate the relevant tokens (the shared relevance signal). Prior theory on this question freezes one part, trains in stages, or imposes an explicit curriculum [4,5,6], so on this point our setting is closer to the actual empirical setting.

A simple model also lets us pin down the cause. Theorem 4.2 shows that without the selection data, XOR stays near chance under the same budget. Such a statement cannot be proved, or even tested, on a 7B model, since the end-to-end training dynamics of deep Transformers are still open problems. This is also common practice in closely related theory work [1,2,3], which pairs simplified theory with experiments on Pythia, BERT and Llama.

W2.1: The experimental section is brief with insufficient analysis without ablation study. Similar learning curves alone cannot prove that the proposed mechanism works in real LLMs.

Thank you. We would first like to clarify that our experiments are intended as an extension of our theoretical analysis, exploring whether the proposed mechanism and phenomenon may generalize to a broader range of scenarios beyond those covered by our theory. For the experiment conducted under the theoretical settings, we refer the reviewer to the section Parameters That Match Theoretical Settings in our global rebuttal.

To test the reason behind the ordering, and not only the ordering itself, we added two controlled experiments (Global Rebuttal).

(1) Results for weak selection. With data, initialization, learning rate, attention cap, and update budget all matched, joint training reaches 95.08 ± 1.32% XOR accuracy, composition-only training reaches 60.40 ± 1.01%, and a frozen uniform query reaches 57.74 ± 1.51%. Removing the selection data removes most of the XOR learning.

(2) Ablation on loss component ratio. On binary MNIST, with the composition objective and data fixed, composition accuracy stays at chance for selection-loss weight β≤0.01\beta\le 0.01 and reaches about 90% for β≥0.05\beta\ge 0.05 within the same 6,000 updates.

In addition, freezing the attention on MNIST keeps composition at chance (51.39 ± 0.79%, Figure 5).

AuthorsOpenReview ↗

Response to Reviewer DX5S (Part 2/2)

W2.2: Have the authors considered controlled comparisons between ReLU and SwiGLU to test the robustness? More broadly, how sensitive is this behavior to architectural choices.

Thanks for the suggestion on architectural choices experiments! We expanded the comparison to 12 configurations across three tasks, with five seeds per setting. Across all 180 runs, perception reached sustained 90% balanced accuracy before parity. This ordering persists under changes to activation functions, attention, depth, normalization, pooling, and the training objective. These results suggest that the perception-before-parity learning order we study extends to a broader range of architectures and training objectives.

VariantSynthetic XORMNIST: 2 signalsMNIST: 3 signals
ReLU baseline8.8 ± 2.790.0 ± 23.5462.0 ± 108.0
SwiGLU FFN11.2 ± 2.370.0 ± 21.2399.0 ± 124.9
GELU FFN + GELU head17.6 ± 7.9106.0 ± 33.6410.0 ± 80.1
1 layer20.0 ± 2.4120.0 ± 31.6873.0 ± 153.1
4 layers9.6 ± 3.892.0 ± 25.9429.0 ± 76.2
Linear-attention blocks8.4 ± 3.3176.0 ± 25.11568.0 ± 461.5
Mean pooling8.8 ± 2.790.0 ± 23.5474.0 ± 153.6
Squared loss8.0 ± 2.884.0 ± 20.7437.0 ± 110.8
Post-LayerNorm25.2 ± 15.840.0 ± 7.1134.0 ± 8.9
RMSNorm10.4 ± 1.790.0 ± 15.8402.0 ± 53.2
Linear attention + mean pooling4.0 ± 1.4184.0 ± 27.01568.0 ± 333.3
SwiGLU + SiLU head + squared loss14.8 ± 3.070.0 ± 12.2419.0 ± 73.1

We also appreciate the reviewer’s suggestion to move some less important derivations and technical details to appendix and save some space for the experiment results. We add these results in Table 1 in the revised manuscript.

References

[1] Chen et al. Distributional Associations vs In-Context Reasoning: A Study of Feed-forward and Attention Layers. ICLR, 2025.

[2] Panigrahi et al. Progressive distillation induces an implicit curriculum. ICLR, 2025.

[3] Shi et al. Why Larger Language Models Do In-context Learning Differently? ICML, 2024.

[4] Abbe et al. Provable advantage of curriculum learning on parity targets with mixed inputs. NeurIPS, 2023.

[5] Wang et al. Learning Compositional Functions with Transformers from Easy-to-Hard Data. COLT, 2025.

[6] Kim et al. Transformers Provably Solve Parity Efficiently with Chain of Thought. ICLR, 2025.

Thank you for the responses and additional experiments. The new ablation study strengthen the evidence and the architectureal comparision support the robustness of the observation of learning order.

However, my main concern remains the connection between the machanism and the mordern LLM experiments. The additional results do not establish that the same machanism explain explains the observaion in modern LLMs

I appreciate the clarification of the theory’s scope, but the role of the LLM experiments and the scope of the empirical contributions still need clearer justification. Therefore, I will maintain my original score.

Reviewer_Hs3KOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

Prior studies have found that retrieval capabilities frequently emerge during pre-training before compositional capabilities. This paper provides a theoretical analysis of this phenomenon in the context of a shared-attention multi-task Transformer, in particular one in which the model is performing a token-selection task and an XOR task. The authors find that the token selection task is learned first, and then later the XOR task is learned. The authors then perform experiments on synthetic data on the model as well as MNIST-based sequences, where they again see the same order of learning.

Critical strengths

I thought that the paper was well-written, but the theoretical analysis of course requires substantial effort. Given that this is the papers primary contribution, this is of course necessary and I don't believe that it can be made any easier. I also appreciated the authors' empirical exploration, and if anything I felt that the paper could have been made more exciting and accessible by expanding on the studies of training on synthetic data (this is a stylistic choice)

Critical weaknesses

I don't believe that there are any critical weaknesses in the manuscript. The primary weakness is that the basic phenomenon is neither new nor surprising, but the authors are providing a theoretical analysis of the ordered learning so I don't think that this is a critical weakness. However, a more clear exposition of the advance relative to Wang et al "Learning Compositional Function..." and Kim and Suzuki would be appreciated. In some parts of the manuscript it seemed that the authors described the order simply as "harder tasks are learned later", but in fact I believe that the authors have perhaps demonstrated that the retrieval task has to be learned first (the experiments certainly seem to indicate this, given that the mixed training allowed for learning the XOR in a timeframe that the XOR only training didn't). As an empiricist, I was very curious about this aspect (which didn't seem to get much discussion in the paper). In particular, I think that a very interesting question is whether an actual curriculum (ratio of examples over time?) would allow for more rapid acquisition of the ability to perform the XOR task. I understand that this is not the authors' interest as they are primarily attempting to experimentally validate the theoretically proposed ordering. On this point, however, the examples that the authors explore (even the synthetic training?) is done outside of the regime that the authors' have actually proved, so the examples are primarily there to show the generality of the effect. From this standpoint, the examples are not quite relevant to the regime that was explored in the proof (but I like the examples, so don't remove them!).

Questions for authors

If I understand the theory correctly, the authors predict some surprising effects on the delay at large width. Is this something that can be observed experimentally?

More generally, I think that the experiments with synthetic data could be leveraged considerably more, as this is an explicit opportunity to both confirm the validity of the theory in the regime that was proven but also to test how far outside the regime the ordered learning persists. Of course, if retrieval has to be learned first, then perhaps the ordered will always be present?

5 comments
AuthorsOpenReview ↗

Response to Reviewer Hs3K (Part 1/3)

Thank you for the helpful feedback; we address your comments below.

W1: A more clear exposition of the advance relative to Wang et al "Learning Compositional Function..." and Kim and Suzuki would be appreciated.

Sure! We’d love to elaborate the difference.

Wang et al. [1] establish an implicit curriculum under mixed-data training. Their theoretical algorithm takes few gradient steps on the attention matrices while keeping the readouts fixed, followed by a single update for readout. In our setting, attention and the hidden weights of the ReLU prediction heads are updated together, at every iteration. Our analysis therefore captures their interaction throughout training: changes in attention affect the features learned by the heads, while changes in the heads affect how attention learns. We establish the learning transition under these simultaneous updates, which is not addressed in [1].

Kim and Suzuki [2] analyze attention learning with a fixed feedforward function explicitly designed to implement local parity operations. In our setting, the nonlinear features needed for XOR must also be learned through randomly initialized ReLU hidden weights, which is a more common choice in empirical settings.

W2 && Q2.2: In some parts of the manuscript it seemed that the authors described the order simply as "harder tasks are learned later", but in fact I believe that the authors have perhaps demonstrated that the retrieval task has to be learned first (the experiments certainly seem to indicate this, given that the mixed training allowed for learning the XOR in a timeframe that the XOR only training didn't). As an empiricist, I was very curious about this aspect (which didn't seem to get much discussion in the paper). && If retrieval has to be learned first, then perhaps the order will always be present?

Yes! We think retrieval task is an instance of simple task (mostly focus on pattern matching), while xor is an instance of harder task (mostly focus on logical operation). We are happy to discuss more about the retrieval task itself in our setting.

Our lower-bound proof identifies a dependence on representation quality. While attention remains in the weak regime specified in Lemma K.5, even an optimal classifier of the pooled representation has XOR error close to chance (Section 5.3). XOR-only SGD stays in this regime throughout the matched training budget (Lemma K.3), whereas mixed training learns XOR within that budget (Theorem 4.1). Thus, token-selection training helps develop the informative shared representation needed for XOR. We add more elaboration of this in Section 6 of our revised manuscript. So back to the question, it depends on the data setting. In our theoretical setting, retrieval has to be learned first and the order will always be present. However, if we extend to more empirical scenarios, for example, MNIST, the attention weight is still learnable, so the composition-only can still be learned, but much slower than joint training (Figure 5 in revised manuscript).

We also provide experiments for this. (i) Under the same attention cap and budget, joint training reaches 95.08% XOR accuracy, while composition-only training reaches 60.40%, close to a frozen uniform-attention control at 57.74% (Global Rebuttal, Results for weak selection). (ii) On binary MNIST, composition accuracy stays at chance when the selection-loss weight is β≤0.01\beta\le 0.01 and reaches about 90% for β≥0.05\beta\ge 0.05 (Global Rebuttal, Ablation on loss component ratio).

AuthorsOpenReview ↗

Response to Reviewer Hs3K (Part 2/3)

W3: Whether an actual curriculum (ratio of examples over time?) would allow for more rapid acquisition of the ability to perform the XOR task.

We can check this both theoretically and empirically.

Theoretically, an explicit curriculum could shorten the initial representation-learning phase. In our setting, both tasks share the same marker identifying informative tokens. A selection-only warm-up could therefore teach attention where to look before the model learns how to combine the two signals. If the resulting attention preserves both informative tokens and is then held fixed, XOR learning becomes a separate prediction problem on the learned representation. Prior work establishes few-step attention learning under suitable conditions [3,4], motivating this approach. Yes, our current theorem instead studies how the learning order emerges under a fixed task mixture, which is generally harder than a theorem that uses multi stage learning and explicit curriculum of data.

Empirically, we compared joint training on a 50:50 mixture with selection-only pretraining followed by composition-only training. The composition learning rate and minibatch size are identical. Increasing only the selection-pretraining learning rate by 16× substantially accelerates synthetic XOR learning:

WidthMixed updatesSelection pretrainingComposition updatesTotal updates saved
100594.0 ± 263.539.0 ± 2.15.0 ± 0.092.6%
200472.0 ± 171.537.5 ± 2.65.0 ± 0.091.0%

Values are mean ± SD over 10 paired seeds. Selection training (as the first step of curriculum) ends after validation accuracy reaches 99% at two consecutive checks. Every synthetic run reached the 95% XOR target and finished at 100% accuracy. High level intuition is that relatively easy tasks can update with a relatively large learning rate, which can reach a “strong selection” regime quickly. After that, the learning of XOR can receive more significant gradient updates with fewer noise.

A static version of the ratio question is in the Global Rebuttal (Ablation on loss component ratio): composition learning within a fixed budget switches on once the selection-loss weight exceeds a small threshold (β\beta between 0.02 and 0.05).

W4 && Q2.1: The examples are done outside of the regime that the authors' have actually proved, so the examples are primarily there to show the generality of the effect. && The experiments with synthetic data could be leveraged considerably more, as this is an explicit opportunity to both confirm the validity of the theory in the regime that was proven but also to test how far outside the regime the ordered learning persists.

Thanks for indicating this!! Yes, most of the results presented in paper are used to show the generality of the effort. We provided more detailed experiments that show in regime effort of our result, which is provided in The parameters that match theoretical settings in our global rebuttal.

AuthorsOpenReview ↗

Response to Reviewer Hs3K (Part 3/3)

Q1: If I understand the theory correctly, the authors predict some surprising effects on the delay at large width. Is this something that can be observed experimentally?

In Section 3 (“Network Model”), we initialize the hidden weights with standard deviation 1/md1/\sqrt{md} and fix the output weights at random signs scaled by 1/m1/\sqrt{m}. This is equivalent to a common mean-field parameterization used to study feature learning [5,6]. The idea is to average the contributions of the hidden neurons, keeping the overall scale controlled as width changes. Together with the corresponding learning-rate scaling, this allows features to continue evolving even in very wide networks, making it suitable for studying how token selection and XOR learning develop during training.

Under this initialization, random fluctuations decrease as m−1/2m^{-1/2}, while the learned XOR signal has no leading-order width dependence at comparable attention and feature-learning states (Section 5.1). The signal-to-fluctuation ratio therefore grows as m\sqrt{m}, which can allow XOR learning to catch up with selection sooner. The width upper bound in our analysis preserves the early separation by keeping initialization fluctuations dominant at the selection checkpoint.

To examine this experimentally, we swept m=25m=25–800800 with ten seeds per width. We define tA,95t_{A,95} and tX,95t_{X,95} as the first of three consecutive evaluations with accuracy at least 95%, and compute the paired delay Δt95=tX,95−tA,95\Delta t_{95}=t_{X,95}-t_{A,95}. For widths where all ten seeds reach this threshold, the results are:

Width mmSelection tA,95t_{A,95}XOR tX,95t_{X,95}Paired delay Δt95\Delta t_{95}
100439 ± 39.0638 ± 277.1199 ± 251.1
200417 ± 39.5546 ± 185.7129 ± 150.4
400402 ± 36.1496 ± 191.394 ± 158.4
800399 ± 32.5420 ± 46.221 ± 31.4

Values are optimizer updates, reported as mean ± sample SD across ten seeds. Selection learning time changes only modestly, whereas XOR learning accelerates substantially, reducing the mean delay from 199 to 21 updates. At m=25m=25 and 5050, only 4/10 and 9/10 seeds, respectively, reach the XOR threshold within the 1,500-update budget. The corresponding learning curves and initialization diagnostic are shown in Appendix M.4, Figure 6.

References

[1] Wang et al. Learning Compositional Functions with Transformers from Easy-to-Hard Data. COLT, 2025.

[2] Juno Kim and Taiji Suzuki. Transformers Provably Solve Parity Efficiently with Chain of Thought. ICLR, 2025.

[3] Oymak et al., On the Role of Attention in Prompt-tuning, ICML 2023.

[4] Magen et al., Benign Overfitting in Single-Head Attention.

[5] Zhengdao Chen, Eric Vanden-Eijnden, and Joan Bruna. On Feature Learning in Neural Networks with Global Convergence Guarantees. ICLR 2022.

[6] Jimmy Ba, Murat A. Erdogdu, Taiji Suzuki, Zhichao Wang, Denny Wu, and Greg Yang. High-dimensional Asymptotics of Feature Learning: How One Gradient Step Improves the Representation. NeurIPS 2022.

Reviewer_3H4ZOpenReview ↗
Rating
4
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper investigates the theoretical mechanisms behind the emergence of an implicit curriculum during Transformer pretraining, where perception-like localization skills are acquired before composition-like reasoning skills without an externally imposed data schedule. The authors formulate a stylized multi-task Transformer featuring a shared single-head softmax attention layer coupled with task-specific two-layer ReLU MLP heads, trained simultaneously via stochastic gradient descent (SGD) on a fixed mixture of a token-selection task and a two-token XOR task among long-context distractors. They prove a non-asymptotic timescale separation showing that mixed training achieves low test error on token selection while XOR error remains near chance, before subsequently learning the XOR task as amplified positional attention accelerates product-feature formation (Theorem 4.1), whereas training on XOR alone fails within the same sample and iteration budget (Theorem 4.2). The theoretical predictions are supported by experiments on synthetic data, single- and multi-layer Transformers on MNIST-derived visual token sequences, and capability emergence evaluations across open intermediate pretraining checkpoints of 7B-scale large language models.

Critical strengths
  1. Novel End-to-End Simultaneous Multi-Task Analysis: Unlike prior theoretical studies that analyze attention-based token selection with linear readouts or two-layer ReLU parity learning on static inputs in isolation, or rely on explicit data curricula, layer freezing, or alternating updates, this work rigorously tracks the coupled, simultaneous SGD trajectory of shared softmax attention and non-linear ReLU heads under a fixed task mixture.

  2. Rigorous Timescale Separation and Information-Theoretic Lower Bound: The paper establishes both a two-phase convergence guarantee in the weak-selection regime (via a uniform curvature envelope that bounds error amplification to d1/128d^{1/128} without requiring attention saturation) and a sharp Total Variation lower bound (Theorem 4.2) proving that parity cancellation traps XOR-only training near initialization so that even an optimal classifier on the pooled representation fails within the same budget.

  3. Multi-Scale Empirical-Theoretical Bridge: The authors complement their mathematical proofs with a cohesive three-tiered empirical validation—spanning synthetic parameter sweeps, single- and multi-layer Transformers on multi-signal MNIST sequences with attention-entropy diagnostics, and 7B-scale open LLM pretraining checkpoints—demonstrating that the perception-before-reasoning ordering generalizes beyond the stylized theoretical setup.

Critical weaknesses
  1. Unexplained Noise Projection Asymmetry and Partitioned Parameter Constraints: In Section 3, the XOR distribution P2\mathcal{P}_2 projects out the noise component along the positional direction p\mathbf{p} solely for the two informative tokens while keeping isotropic noise on distractors and on all tokens in P1\mathcal{P}1, and Assumption 3.1 splits the two-sided window on network width mm (which is essential to prevent initialization fluctuations from being overwhelmed at τA\tau_A) and the tight noise scaling σϵ2≍d−1\sigma\epsilon^2 \asymp d^{-1} across separate clauses; the authors might consider explicitly motivating the noise projection as a technical convenience to equalize axial attention weights (s1=s2s_1 = s_2) and consolidating the parameter couplings for transparency.

  2. Architectural Decoupling of Task Heads vs. Fully Shared Pretraining: The theoretical model relies on disjoint task-specific prediction heads (W1,W2\mathbf{W}_1, \mathbf{W}_2) so that the strong first-order positional and semantic drift from P1\mathcal{P}_1 does not corrupt the parity-cancellation mechanism in the XOR head (Lemma 5.2); it would be helpful to discuss how standard Transformers with fully shared MLPs avoid such cross-task gradient interference during pretraining (e.g., via task-identifying context tokens or subspace orthogonality).

  3. Quantitative Gap Between Theoretical Regimes and Empirical Protocols: The theoretical guarantees require tiny learning rates (d−6≤η≤d−3d^{-6} \le \eta \le d^{-3}), long contexts (T≥d(log⁡d)17/16T \ge d(\log d)^{17/16}), and weak attention selection (o(1)o(1) signal mass), whereas the synthetic and MNIST experiments operate in a strong-selection regime with rescaled initialization/signal constants and omit a composition-only baseline, multi-seed variance reporting, and several architectural/evaluation details on MNIST and the LLM checkpoints.

Questions for authors

Please see the weaknesses.

2 comments
AuthorsOpenReview ↗

Response to Reviewer 3H4Z (Part 1/2)

We thank the reviewer for the valuable comments and present our responses below.

W1: Unexplained Noise Projection Asymmetry and Partitioned Parameter Constraints.

The projection is more than a technical simplification: it equalizes the attention weights of the two informative tokens when the query aligns with the positional direction. In the revised manuscript, we add explicit motivation for this theoretical setting in Section 3.

Empirically, we also tested whether XOR learning depends on removing noise along the positional direction of the two informative tokens. We varied this noise’s standard deviation from zero (None) to half (Partial) and the same as in other directions (Full). Noise in all other directions, distractor tokens, and selection-task inputs were unchanged.

We used the m=100m=100 synthetic mixed-training baseline from our width sweep: shared attention with two task-specific ReLU heads, d=T=200d=T=200, signal magnitude 3, positional magnitude 1.8, and noise standard deviation 0.2. All conditions used the same initialization scheme (Wij(0)∼N(0,1/(md))W_{ij}(0)\sim\mathcal N(0,1/(md)), fixed aj=±1/ma_j=\pm 1/\sqrt{m}, and q(0)=0q(0)=\mathbf{0}), SGD learning rate 1, and 1,500 updates with 4,000 fresh examples per update, equally split between the two tasks.

The table reports final accuracy on 5,000 held-out XOR examples for the same five seeds:

SeedNonePartial: 50% noise SDFull
0100.00%100.00%99.38%
1100.00%100.00%99.28%
2100.00%76.04%75.78%
3100.00%99.98%98.24%
4100.00%99.78%74.34%
Mean ± SD100.00% ± 0.00%95.16% ± 10.69%89.40% ± 13.11%
Seeds reaching 95%5/54/53/5

Reaching 95% requires three consecutive evaluations, performed every 10 updates. With the projection, all five seeds reach zero test error. Without it, two seeds still have ~ 25% test error after the same 1,500 updates. These results suggest that the projection also improves empirical reliability: removing it can leave some seeds at substantially higher XOR error under the same training budget.

For parameter couplings, we have reorganized Assumption 3.1 into five explicit two-sided windows without changing the admissible parameter region. We also clarify the noise scale, context-length regime, and distinct roles of the width bounds. The confidence-dependent compatibility condition is discussed in Appendix A.

AuthorsOpenReview ↗

Response to Reviewer 3H4Z (Part 2/2)

W2: It would be helpful to discuss how standard Transformers with fully shared MLPs avoid such cross-task gradient interference during pretraining (e.g., via task-identifying context tokens or subspace orthogonality).

Sure! We would be happy to discuss how fully shared MLPs can mitigate cross-task gradient interference. Subspace orthogonality offers one possibility: replace zμcz\boldsymbol\mu_c in P1\mathcal P_1 with zrz\mathbf r, where r\mathbf r is a unit vector orthogonal to p,μ1,μ2\mathbf p,\boldsymbol\mu_1,\boldsymbol\mu_2. This separates the semantic directions of the two tasks while retaining their coupling through the shared positional vector p\mathbf p.

Set ϵi=0\boldsymbol\epsilon_i=\mathbf0 and replace the two heads with a shared f=a⊤ϕ(Wx~)f=\mathbf a^\top\phi(\mathbf W\widetilde{\mathbf x}). For each neuron and i∈1,2i\in\\{1,2\\}, the mini-batch gradients satisfy

\left\langle\nabla_{\mathbf w_j}\widehat{\mathcal L}_{1,\tau}, \boldsymbol\mu_i\right\rangle &=\frac{a_j}{b_1}\sum_{(\mathbf X,y)\in D_{1,\tau}} \ell'(yf)\\,y\\,\phi'(\mathbf w_j^\top\widetilde{\mathbf x}) \left\langle\widetilde{\mathbf x},\boldsymbol\mu_i\right\rangle =0,\\\\ \left\langle\nabla_{\mathbf w_j}\widehat{\mathcal L}_{2,\tau}, \mathbf r\right\rangle &=\frac{a_j}{b_2}\sum_{(\mathbf X,y)\in D_{2,\tau}} \ell'(yf)\\,y\\,\phi'(\mathbf w_j^\top\widetilde{\mathbf x}) \left\langle\widetilde{\mathbf x},\mathbf r\right\rangle =0,\\\\ \left\langle\mathbf w_j(\tau+1)-\mathbf w_j(\tau), \boldsymbol\mu_i\right\rangle &=-\eta\lambda \left\langle\nabla_{\mathbf w_j}\widehat{\mathcal L}_{2,\tau}, \boldsymbol\mu_i\right\rangle,\\\\ \left\langle\mathbf w_j(\tau+1)-\mathbf w_j(\tau), \mathbf r\right\rangle &=-\eta(1-\lambda) \left\langle\nabla_{\mathbf w_j}\widehat{\mathcal L}_{1,\tau}, \mathbf r\right\rangle. \end{aligned}$$ Thus, each task updates its own semantic coordinates within the same MLP. A convergence guarantee would additionally require controlling the shared positional component and ReLU activation patterns. However, our original setting retains semantic overlap: both tasks use $\boldsymbol\mu_1,\boldsymbol\mu_2$. Selection promotes a signed linear response, which can reduce the XOR margin on examples with two negative semantic signs. A shared MLP can represent both tasks, but its gradients must coordinate the growth of linear and nonlinear features while preserving the symmetry underlying XOR learning. Our separate heads isolate this interaction while retaining transfer through shared attention. Extending the analysis to task-conditioned specialization with shared semantic directions is a promising next step. ### W3:Quantitative Gap Between Theoretical Regimes and Empirical Protocols. We thank the reviewer for pointing out this gap. The tiny learning rate $d^{-6}\le\eta\le d^{-3}$, the long context, and the weak-selection regime are sufficient conditions used to control the coupled SGD dynamics over a long horizon; they are not claimed to be necessary for the phenomenon. To address the concern directly, we added: 1. **An experiment in the exact theoretical regime** ($\eta=d^{-3}$, $T\ge d(\log d)^{17/16}$, unit signals), with five seeds and mean ± SD. Selection precedes XOR in every seed (Global Rebuttal, **Experiments in the exact theoretical regime**). 2. **Weak-selection experiments.** At the first sustained selection-95% event, attention on the relevant token is only 2–3.5% for $T\in[100,500]$, so learning proceeds without attention saturation, consistent with the $o(1)$ signal-mass regime in our analysis (Global Rebuttal, **Results for weak selection**). 3. **A composition-only baseline** is already exist in Figure 1(a), which we name it as xor only. For MNIST data, Because of the discrepancy of real world data and synthetic data, the composition only baseline of MNIST data can still be learned but much slower than joint training, see Figure 5. 4. **Multi-seed reporting** Our synthetic experiments already include multiple seeds. For MNIST, we repeated the original experiments with five seeds using the same architectures, hyperparameters, and training budgets. Figures 2 and 3 now replace the single-seed plots with mean curves and bands showing one sample standard deviation across seeds. 5. **Arch details.** We report all architectural/evaluation details on MNIST and the LLM checkpoints in Appendix M in the revised manuscript.

Comments on the paper

3 comments
AuthorsOpenReview ↗

Global Rebuttal (Part 1/2)

We thank reviewers 3H4Z, Hs3K, and DX5S for their thoughtful feedback and constructive suggestions. We appreciate 3H4Z’s recognition of the novelty of our joint analysis of attention and nonlinear feature learning, including the timescale separation and information-theoretic lower bound. We also thank DX5S for highlighting our learning-order guarantee under simultaneous training and the clearly structured theoretical derivations, and Hs3K for appreciating the writing and empirical exploration. Below, we address the shared concerns and present additional controlled experiments and ablations.

1. The parameters that match theoretical settings

Our original experiments use larger learning rates and stronger signals than those required by our theorems. We agree, and we now separate two roles of the experiments: (i) verifying the theorem in its own regime, and (ii) testing how far the phenomenon persists outside it, which is already presented in our manuscript.

We follow the explicit scalings in Assumption 3.1: d=16d=16, T=48≥d(log⁡d)17/16T=48 \ge d(\log d)^{17/16}, width m=128m=128, unit semantic signals, noise standard deviation 0.250.25, total batch size 22, and learning rate η=d−3=1/4096\eta=d^{-3}=1/4096, which lies inside the admissible range d−6≤η≤d−3d^{-6}\le\eta\le d^{-3}. We define each threshold-crossing time as the first of three consecutive evaluations at or above that threshold, computed separately for each of five seeds.

QuantityMean ± SD
Selection reaches 95%: training steps1,093,248 ± 48,012
XOR reaches 95% under mixed training: training steps1,148,160 ± 17,649
Paired delay: XOR95 − selection9554,912 ± 32,544
XOR accuracy at the selection80 event59.42% ± 4.58%
XOR accuracy at the selection90 event73.09% ± 12.42%
XOR accuracy at the selection95 event78.53% ± 15.50%
Mixed-training XOR accuracy at its XOR95 event96.89% ± 1.19%
XOR-only accuracy at the same seed-specific training steps57.11% ± 1.79%

Selection reaches 95% before XOR in all five seeds. At the earlier selection80 event, XOR accuracy is only 59.42%±4.58%59.42\% \pm 4.58\%; by the time mixed training reaches sustained 95% XOR accuracy, XOR-only training remains at 57.11%±1.79%57.11\% \pm 1.79\% under the same update and sample budget. These results support the qualitative learning order and matched-budget advantage of mixed training described by our theory. We summarize these results in Section 7 and provide the paired comparisons and experimental settings in Appendix M.4 (Table 2) of the revised manuscript.

2. Results for weak selection

Explicitly limiting selection. We use a uniform–softmax mixture, s~i=(1−α)/T+αsi\tilde{s}_i=(1-\alpha)/T+\alpha s_i, with α=0.15\alpha=0.15. The total effective attention on the two informative tokens is then bounded by 2(1−α)/T+α=18.54%2(1-\alpha)/T+\alpha=18.54\% throughout training. We compare joint training against composition-only training under the same cap, data, initialization, learning rate, total batch size, and update budget. We also include a frozen uniform-query control.

Joint, capped attentionComposition only, same capFrozen uniform query
95.08 ± 1.32%60.40 ± 1.01%57.74 ± 1.51%

Attention at selection95 with different sequence lengths. We next retain the original softmax and compare sequence length TT. All other settings, initialization seeds, and evaluation samples are identical. The table reports attention at each seed’s first sustained selection95 event.

TTAttention score on relevant token when Selection task reach 95% accuracy
1003.44% ± 0.60%
1503.13% ± 0.97%
2002.81% ± 0.97%
2502.62% ± 1.02%
3002.47% ± 1.19%
3502.45% ± 1.33%
4002.44% ± 1.51%
4502.55% ± 1.92%
5002.32% ± 1.63%

These experiments imply that while one doesn't need to reach a strong selection ratio, e.g., attention concentrates on the informative tokens, the learning can still be performed.

AuthorsOpenReview ↗

Global Rebuttal (Part 2/2)

3. Ablation on loss component ratio

We use a binary-MNIST construction with real images of digits 0 and 1. The selection task predicts one digit’s parity, while the composition task predicts the product of two parity labels. Images are L2-normalized; context length is 64, distractor-noise standard deviation is 0.2, and head width is 256. We vary the auxiliary coefficient β\beta in L=βLselection+Lcomposition\mathcal L=\beta\mathcal L_{\mathrm{selection}}+\mathcal L_{\mathrm{composition}}.

All six conditions use the same five initializations, SGD learning rate 0.01, 6,000 updates, and 128 examples per task per update. The composition objective and target-example count remain fixed, including at β=0\beta=0. We evaluate balanced composition accuracy on 8,192 held-out-image contexts per seed.

Selection-loss weight β\betaComposition accuracy, mean ± SD
049.85 ± 0.63%
0.00549.90 ± 0.63%
0.0150.04 ± 0.57%
0.0252.99 ± 0.93%
0.0589.77 ± 0.13%
0.190.44 ± 0.27%

This controlled sweep shows that sufficiently strong selection-task supervision can accelerate composition learning in this binary-MNIST setting.

AuthorsOpenReview ↗

Summary of Discussion and Revisions

As the reviewer–author discussion period concludes, we would like to provide a brief summary of the main revisions and new experiments added in response to the reviewers' feedback.

Overall Consensus & Score Update All three reviewers recognized the theoretical contribution of our end-to-end simultaneous training analysis and the clarity of our derivations (Reviewers 3H4Z [Rating: 4] and Hs3K [raised to Rating: 3] support acceptance; Reviewer DX5S [Rating: 2] acknowledged that our new ablations and architectural comparisons strengthened the evidence, while raising a follow-up concern regarding the LLM experiments).

Key Revisions & New Evidence Added

  1. Bridging Theory and Experiments (3H4Z, Hs3K, DX5S):
  • Exact theoretical regime: Added 5-seed experiments strictly matching Assumption 3.1 (T=2000,η=10−4T=2000, \eta=10^{-4}), confirming that token selection consistently precedes XOR and mixed training substantially outperforms XOR-only under a matched budget (Section 7, Appendix M.4).
  • Weak-selection & mechanism checks: Demonstrated that learning succeeds even under capped attention (95.08% joint vs. 60.40% composition-only) and low attention mass (2–3.5% on relevant tokens), alongside loss-weight ablations on binary MNIST and a noise-projection ablation (Section 7 and Global Rebuttal).
  • Width scaling & explicit curriculum: Verified the predicted width-dependent delay reduction (m=100→800m=100 \to 800) and compared implicit vs. explicit curricula (Appendix M.4).
  • Architectural robustness: Expanded comparisons across 12 configurations (ReLU, SwiGLU, GELU, 1–4 layers, linear attention, RMSNorm/Post-LN, squared loss; 180 runs total), confirming that the learning order holds across all settings (Table 1).
  • Clarity & rigor: Reorganized Assumption 3.1 into explicit two-sided windows, added multi-seed error bars for MNIST (Figures 2–3), expanded comparisons to Wang et al. and Kim & Suzuki, and added a discussion on fully shared MLPs.
  1. Addressing Reviewer DX5S’s Remaining Concern (LLM Mechanism & Scope): In response to Reviewer DX5S’s follow-up comment, we made two targeted updates in our latest revision (marked in red):
  • Causal attention interventions on OLMo 2 7B (Appendix M.5): On 240 TabMWP problems across 8 intermediate checkpoints, we confirmed that operand retrieval emerges well before direct compositional solving (e.g., at 504B tokens: 74.6% retrieval vs. 21.7% direct, rising to 71.7% when operands are supplied). Crucially, matched-budget attention interventions show that suppressing attention to relevant table cells drops answer accuracy from 38.9% to 7.1% at 504B (with ≤1%\le 1\% effect when operands are given), while boosting it raises accuracy from 26.5% to 55.5%, providing preliminary causal evidence for an attention-mediated retrieve-then-compose mechanism at inference time.
  • Clarified scope (Introduction & Conclusion): We explicitly reframed our three-tiered empirical validation so that the LLM evaluations and interventions are positioned as supporting real-world context rather than a claimed theoretical characterization of 7B pretraining dynamics.

We thank the Area Chair and all reviewers for their time and constructive feedback throughout the review process.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.