- Rating
- 2(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper studies how RNN layers and Attention layers with softmax acquire retrieval functions in hybrid language models. They propose synthetic copying experiments with counterfactual activation patching, and freeze/unfreeze interventions to argue that the learning in early recurrent layers suppresses the attention layers from learning retrieval. The authors propose Better Layer Division (BLD) which freezes recurrent blocks during the early stage of pre-training, then trains all parameters. Experiments across six KDA model sizes and GDN/GLA variants show consistent aggregate improvements in retrieval, alongside increased reliance on full-context attention.
Critical strengths
Retrieval improvements are persistent across model sizes and recurrent architectures, under matched training budgets.
- Tables 1 and 2
- Appendix C specifies matched data views, token budgets, and optimizers to rule out extra training data as a confounder.
The interventions seem to provide evidence that training history affects the functionalities the models are learning.
- changes in attention attribution (Figure 4, Appendix A.4), or retrieval performance (Figure 6, Appendix H)
Critical weaknesses
- The attribution score does not seem to distinguish a layer that retrieves information, from a layer which passes along the information that has been retrieved earlier.
- Equation (6) seems to measure, how replacing a mixer output changes the final answer's logits.
- Correct me if I am wrong, but this can contain information retrieved by earlier layers.
- Figures 2 to 4 show changes in the predictions, but I do not think these results establish the claim that recurrent learning reduces attention's retrieval specific learning signal.
- Does freezing blocks isolate competition between recurrent and attention mixers?
- The observed changes in Table 5 and 9 could arise from freezing other parts such as MLPs/normalization which could affect the retrieval learning.
- I feel the paper demonstrates a useful training recipe, but does not establish that it works by correcting competition over retrieval learning.
Questions for authors
What are the held-out synthetic retrieval accuracy and cross-entropy trajectories around freezing and unfreezing? The attribution plots do not establish whether the observed redistribution accompanies improved task performance or occurs at comparable performance. This distinction would affect how I interpret the intervention's benefit.
3 comments
Response to abWS 2/2
Weakness 2: Does freezing whole blocks confound mixer competition with MLP/normalization effects?
Response W2:
We conducted component-wise freezing controls on the 240M Chinchilla configuration, using identical training settings and 16K context in both stages. Mixer-only freezes only the KDA mixers, whereas MLP-only freezes only the MLPs in the corresponding KDA blocks. Normalization remains trainable in both controls, and all parameters are trainable in the second stage.
| Initial-stage freezing | Recall ↑ | RULER ↑ |
|---|---|---|
| Whole KDA blocks | 31.27 | 26.89 |
| KDA mixers only | 33.17 | 25.47 |
| KDA-block MLPs only | 24.13 | 16.16 |
RULER averages the manuscript’s 57 task–length combinations at 1K–16K (§4.2; Appendix E);
Mixer-only freezing achieves retrieval performance comparable to whole-block freezing while keeping MLPs and normalization trainable. MLP-only freezing falls substantially behind both. These controls directly support the effectiveness of the mixer-targeted intervention motivated by Competing Learning: freezing MLPs or normalization is unnecessary to obtain strong retrieval performance, and MLP-only freezing cannot reproduce it.
Together with the mixer-only gradient experiment, these controls support the effectiveness of intervening on recurrent mixer learning, without requiring MLP or normalization freezing.
Question 1: Held-out synthetic retrieval accuracy and cross-entropy around freezing and unfreezing.
Response Q1:
Although recurrence-dominated and attention-dominated solutions both ultimately achieve nearly perfect performance, Spaced Retrieval serves as a controlled tool for observing how the division of labor forms. The synthetic experiments demonstrate two distinct paths through which retrieval capability can develop, with either recurrence or attention assuming the dominant role. The practical evidence that BLD improves the division of labor in hybrid models comes from its clear advantages in the subsequent LLM pretraining and downstream evaluations (Tables 1–2 in the paper), together with the attention-head analyses that support the intended specialization (§4.5; Figs. 8 and 10).
We compare the same three freezing times used in the layer-resolved analysis: steps 50, 150, and 400. All three conditions reach 100% held-out exact-match accuracy at step 1,500 in every one of the three training seeds, while developing different layer contributions.
The table 1 places retrieval performance and layer contributions side by side. The table 2 then shows how each solution develops before and after unfreezing.
Table 1: Final performance and layer contributions at step 1,500
| Freeze step | Final exact accuracy (%) | Final answer CE | L3 KDA contribution C | L4 Attention contribution C |
|---|---|---|---|---|
| 50 | 100.00 | 0.000991 | 0.000 | 18.684 |
| 150 | 100.00 | 0.002333 | 15.173 | 1.317 |
| 400 | 100.00 | 0.000160 | 22.350 | 0.012 |
Table 2: Held-out accuracy and answer CE during training
| Training step | freeze@50 accuracy (%) | freeze@50 CE | freeze@150 accuracy (%) | freeze@150 CE | freeze@400 accuracy (%) | freeze@400 CE |
|---|---|---|---|---|---|---|
| 50 | 0.00 | 2.204266 | 0.00 | 2.204266 | 0.00 | 2.204266 |
| 150 | 0.85 | 1.208567 | 12.23 | 0.870078 | 12.23 | 0.870078 |
| 400 | 84.00 | 0.125638 | 54.08 | 0.314016 | 99.47 | 0.006071 |
| 500 | 90.48 | 0.072701 | 60.67 | 0.259134 | 99.75 | 0.002461 |
| 501 | 82.25 | 0.117765 | 64.40 | 0.240101 | 99.73 | 0.002830 |
| 510 | 79.77 | 0.139830 | 29.40 | 0.631124 | 99.22 | 0.006686 |
| 700 | 99.77 | 0.006469 | 95.27 | 0.043743 | 99.93 | 0.001037 |
| 1,000 | 99.98 | 0.001590 | 99.88 | 0.004146 | 100.00 | 0.000210 |
| 1,500 | 100.00 | 0.000991 | 100.00 | 0.002333 | 100.00 | 0.000160 |
All three conditions reach 100% accuracy through different paths: attention dominates with freeze@50, KDA dominates with freeze@400, and freeze@150 shows a transition ending in KDA dominance. BLD’s LLM and downstream gains (Tables 1–2 in the paper) demonstrate the practical value of steering this specialization.
If you have any remaining concerns, we would be happy to address them. If these clarifications and additional results resolve your concerns, we hope you will consider updating your score to more accurately reflect the paper’s scope and contributions.
Thank you for the response and additional experimental results the authors have brought up. They address my some of concerns.
For the first weakness, I still feel the skip-connection argument does not rule out the relay by itself. The output which has been substituted can bring forward information about the answer. I think overall, the empirical result in Figure 5 is more informative but are from a small parameter model in a synthetic setting.
For weakness 2, I have one doubt about the parameter size being 240M, which does not seem to match Table 1. BLD evaluated in Table1-2 still freezes entire blocks and I do not think these results directly test the mixer adequately.
Looking at the extensive rebuttal response from other reviewers' comments as well, I raise my score from 1 to 2. I still think this is a borderline paper and would like to see what how other reviewers assess the paper after the rebuttal.
Response to abWS 1/2
Dear reviewer abWS,
Than you for distinguishing functional attribution, training signals, and task performance. We address these separately below.
We have updated the manuscript and highlighted the revisions in blue.
Weakness 1.a: Attribution versus information relayed from earlier layers.
Response W1.a:
You have raised a potential relay confound: a downstream layer could receive information retrieved upstream and obtain a high score when its output is replaced. Our intervention replaces only the mixer output f(xA) with f(xB), while keeping the recipient layer’s input xA unchanged. The addition therefore becomes xA+f(xB). Information already present in xA remains available through the unchanged skip connection. Information already available upstream therefore retains an unchanged path to subsequent computation; merely passing through the layer does not automatically receive attribution.
We additionally examine this concern empirically using the new layer-resolved figure (Fig. 5). In the KDA-dominant regime, L3 has a large contribution while the downstream L4 attention mixer remains negligible, despite receiving the preceding computation. In the attention-dominant regime, only L4 has a substantial contribution. Contributions coexist near the transition, where restoring recurrent updates produces seed-dependent takeovers relative to the permanent-freeze controls. The detailed result is added in our updated paper with blue fonts, Section 3.3.2.
The following table reports final mixer-output contribution scores C at training step 1,500, averaged over three seeds on Spaced Retrieval (gap 8,192). KDA blocks are frozen at the indicated step and unfrozen after step 500 in every condition. Bold marks the largest contribution in each row.
Early freezing at step 50 yields attention dominance (L4: 18.684; L3: approximately zero), whereas late freezing at step 400 yields KDA dominance (L3: 22.350; L4: 0.012). At step 150, both retain measurable contributions (L3: 15.173; L4: 1.317), with KDA dominant at the endpoint. Because all conditions resume KDA updates at the same step, this reversal supports Competing Learning: the eventual allocation depends on the components’ preceding learning history.
The results also support the metric’s practical selectivity. In the KDA-dominant regime, downstream L4 remains near zero despite receiving L3’s computation; it does not automatically receive a large score merely because retrieval information is available upstream. Conversely, early freezing produces a large L4 contribution with negligible contributions from the other layers.
Across almost all settings, a single layer has a markedly higher score than all other layers, although its identity changes with the freezing schedule. This consistent separation provides strong empirical support for the score’s effectiveness in identifying the dominant layer contributing to retrieval (Fig. 5; §3.3.2).
Weakness 1.b: Evidence about retrieval-learning signals.
Response W1.b:
We assess retrieval through end-to-end predictions, including the representations and readout needed to produce the answer. In our synthetic task, supervision comes exclusively from retrieved answer tokens (Appendix A.2). Evidence for competition comes from the systematic redistribution of retrieval contributions: KDA dominates under standard training (Fig. 2), whereas sufficiently early KDA freezing enables attention to take over (Fig. 4, left). After unfreezing, attention either reinforces or loses its advantage depending on its preceding learning history (Fig. 4, right; Appendix A.4).
Our evidence is not limited to prediction changes. The controlled gradient experiment (Fig. 6; §3.3.3; Appendix I.1; Table 24) freezes only the KDA mixers and measures unclipped attention gradients on first-answer-token retrieval cross-entropy. It preserves full backward passes and uses the same step-specific baseline clipping coefficient before masking frozen updates. Across ten seeds, freezing KDA progressively amplifies attention gradients relative to the never-frozen baseline, while resuming KDA updates attenuates this amplification. Together with the history-dependent freeze–unfreeze results, this directly supports competition during retrieval learning. We do not interpret attribution curves alone as direct measurements of retrieval-specific gradients.