acceptodds
Under review as a conference paper at ICLR 2027

When Routing Must Be Right: Irreversible Lock-In in Attention Routing

Abstract

Hybrid sequence models pair a cheap recurrent path with a small budget of attention, and something has to decide which tokens get it. On multi-query associative recall that decision is worth three orders of magnitude: spending a fixed 6.25% budget on the right tokens gives 0.997 accuracy, a static schedule tuned to the same budget gives 0.0002. Learned routers are notoriously hard to train, and the usual explanation, that top- selection is discontinuous, so unselected candidates get no gradient, is incomplete. REINFORCE supplies unbiased gradient to unselected tokens and still lands at chance, and one router that had been correct and stable for 25,000 steps produced no improvement at all. We argue the variable that matters is timing, and measure it. Holding the number of correctly-routed steps fixed and varying only their position, misrouting the first 8,000 steps leaves accuracy at chance while delaying the identical window by fifty steps recovers almost everything; the failing arm is the one with more time to recover afterwards. Ten seeds at each of nine onsets put the half-way point at 23 steps, and a sweep over the deficit's length puts its threshold at 48, two measurements agreeing on the scale from opposite directions, and settling the outcome inside the first 0.06% of training. The obstruction is that the signal a router needs does not exist when it is needed: the marginal utility of attention is under misrouting and 2.72 under correct routing. We probe four ways for a usable substitute and none survives, because each returns the model's existing reliance on attention, which is itself a product of the routing it was trained under. Two explanations that would make this an artifact are ruled out by measurement: learned routing stays at chance with four times the budget, and a four-block model in which attention feeds later positions locks in more readily than one block, not less. We also give a mechanism, and a pre-registered prediction it makes about learning-rate scaling that we test and falsify.

Primary area:
foundation or frontier models, including LLMs
Keywords:
conditional computation, token-level routing, hybrid state-space models, training dynamics, associative recall

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_TmvrOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies learned routing failures in hybrid Mamba–attention models on associative recall. Its main experiment holds the number of correctly routed training steps fixed while varying when misrouting occurs. Early misrouting can cause lasting failure, whereas delaying the same deficit by about 50 steps largely preserves performance. The authors also measure attention-pathway suppression and test several routing interventions. A proposed mechanism offers a qualitative explanation, although its learning-rate scaling prediction is falsified.

Critical strengths
  1. The timing experiment is a useful and well-controlled contribution. Matching the amount of correct routing and including an equal-recovery control provide strong evidence that early routing history matters. This goes beyond observing that learned routers are difficult to train.
  2. The paper reports important negative results and limitations openly. Multi-seed experiments, failed controls, the falsified scaling prediction, and the corrected architectural artifact make the empirical investigation more informative.
Critical weaknesses
  1. The role of the optimization setup remains unresolved. The effect depends strongly on learning rate and occurs within the first few dozen updates, but the warmup schedule is not clearly documented. Lower learning rates substantially reduce or eliminate failure in the reported deficit experiments. These results leave open whether the phenomenon persists under a less aggressive training schedule. They also weaken the conclusion that permanent external routing information is necessary.
  2. The evidence does not yet establish that early lock-in explains the learned-router failures. The main timing experiments bypass the router. Similar attention-suppression trajectories are suggestive, but different routing methods may fail for different reasons. Testing learned routers under settings where the imposed deficit is harmless would help distinguish these explanations. Furthermore, Table 3 starts at step 500, after the reported sensitive interval, so it does not directly establish that useful routing signals are absent when they are needed.
Questions for authors
  1. Is learning-rate warmup used? If not, does adding warmup preserve the early lock-in effect?
  2. Do the learned routers in Table 1 still fail at the lower learning rate where the imposed deficit is harmless? If so, what evidence connects their failure to early lock-in?
  3. Are marginal-utility measurements available within the first 50 updates? What do they show before the attention pathway has been suppressed?
4 comments
AuthorsOpenReview ↗

Response to Reviewer Tmvr

We thank the reviewer for the precise and constructive questions, which have helped sharpen the paper's central claim.

Q1, W1 (warmup and optimization setup). We thank the reviewer for pointing out that the schedule was not documented. The main protocol (Secs. 3–6) uses a constant learning rate of 3e-3 without warmup, and the depth experiments use a 2000-step linear warmup; the revision includes a per-experiment schedule table. To answer the question directly, we repeated the Sec. 6 protocol with a 2000-step linear warmup to 3e-3 (10 seeds per condition). Warmup does not remove lock-in; it moves the sensitive window to the end of warmup:

  • A 250-step deficit at steps 2000–2250 (the end of warmup) survives in 10/10 runs without warmup but in only 3/10 with warmup (Fisher's exact test, p = 0.003).
  • The same 250-step deficit at steps 0–250 (inside the warmup ramp) survives in 7/10 runs with warmup, against 3/10 without warmup.
  • An 8000-step deficit from step 0 fails in all runs with warmup (0/10).

We agree that the effect depends on the learning rate, and we now state the conclusion more precisely. Under misrouting, a router trained on the task loss has only a few early steps of signal (Q3 below), which is why we argue that routing information must come from outside the task loss. The requirement that this information be permanent applies to the high-learning-rate regime in which lock-in occurs.

Q3, W2 (signal within the first 50 updates). Table 3 did not cover these early steps, and the new measurements address them directly. We measured δ at 16 points between steps 1 and 250 (5 seeds; learning rates 3e-3 and 1e-3).

  • At 3e-3, the learning rate of the main protocol: under misrouting, δ at query positions is positive only between steps 7 and 10 (22–42% of the oracle value). From step 15 it is no longer distinguishable from zero (means between −0.021 and +0.022, standard deviations 0.019–0.030), while the attention output is still large (attention/SSM RMS ratio 5.9 at step 35). The pathway is suppressed afterwards (ratio 0.35 at step 50, 0.03 at step 250).
  • At 1e-3, δ is positive for longer (steps 10–25, 19–27% of the oracle value), and its loss coincides with the suppression of the pathway (steps 25–35).
  • Under oracle routing, δ remains positive throughout (≥ 0.06 from step 15 at both learning rates).

Under misrouting, the signal therefore exists only within the sensitive window identified by the timing experiment, and it is lost there: at 3e-3 before the attention pathway is suppressed, at 1e-3 together with it. A label-free ranking score (the KL divergence between outputs with and without attention) shows no query signal in this period under any routing, including oracle. Table 3 and Fig. 1 now include these measurements.

Q2, W2 (learned routers at the lower learning rate). We appreciate this suggestion, which directly tests our account. We have run all four Table 1 routers at 1e-3 (5 seeds each), with an oracle-routing control that trains at this rate (5/5). Interpreting them requires knowing whether an imposed deficit is harmless at the same constant learning rate. Our earlier evidence for this used warmup, and the warmup results above show that the schedule matters. That control finishes tomorrow, and we will post the complete answer by Nov 13.

The revised PDF, with all changes marked in blue, will be uploaded by Nov 16.

AuthorsOpenReview ↗

Round-2 update for Reviewer Tmvr

Update: learned routers at the lower learning rate (Q2, W2). As promised, we report the remaining results. All runs use a constant learning rate of 1e-3.

  • An imposed deficit is nearly harmless at this rate. With misrouting on steps 0–250 or 0–8000 and correct routing afterwards, 9/10 runs survive in each case (10/10 without a deficit).
  • Oracle routing trains (5/5; accuracy 0.999).
  • The four Table 1 routers (straight-through, marginal-utility, MoD, REINFORCE) all remain at chance (0/20; 5 seeds each), with query recall at the random level (≈0.06).
  • A router trained to raise its selection probability at the labeled positions, i.e. one given an external routing signal, succeeds in 5/5 runs (accuracy 0.74; query recall 0.73, range 0.67–0.84), against 0/20 for the Table 1 routers (Fisher's exact test, p = 2×10⁻⁵). These are the runs of the label-position control described in our update to Reviewer 1i7q.

These results sharpen the paper's claim into two complementary findings. First, the task loss does not supply a usable routing signal: the Table 1 routers fail even at a learning rate at which the backbone recovers from misrouting. Second, at higher learning rates, and more readily at greater depth, misrouting additionally locks in the backbone (Sec. 6, including under warmup, and at depth 4 even at 1.5e-3). At 1e-3, a router given an external routing signal succeeds in every run, which indicates what a working router needs. At 3e-3 the same router fails (see our update to Reviewer 1i7q). We agree with the reviewer that different failures can have different causes; the abstract, TL;DR and introduction now present the two findings separately.

Thank you for the detailed rebuttal and substantial additional experiments. The new results address my main concerns and significantly strengthen the paper. In particular, the lower-learning-rate experiments are convincing. The finding that imposed misrouting is largely recoverable (9/10), while all four learned routers still fail (0/20), demonstrates that learned-router failure cannot be explained solely by early backbone lock-in. The success of the externally supervised router (5/5) further supports the distinction between insufficient routing supervision and backbone lock-in. The authors have adequately addressed my main concerns, and the revised empirical findings make a meaningful contribution. I have therefore raised my rating from Weak Reject to Weak Accept.

Thank you for reading the new results so carefully and for raising your rating. Your questions on the learning rate led to the experiments that separate the two obstructions and the paper is clearer for it.

Reviewer_YmUsOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper looks at why models fail to learn which tokens should get attention, making a case that early misrouting is responsible for lasting failures since the information needed to learn correct routing was not yet available at that stage. It addresses an important problem and aims to conduct controlled experiments to single out the effects of routing decisions.

Critical strengths

The paper's biggest strength in my opinion is that it sets out to do some precise experiments to answer an interesting question.

Critical weaknesses

The paper also contains many irregularities.

In Prop 1, the paper claims ∇waL=waE[α2]−E[yα]\nabla_{w_a}L=w_a\mathbb{E}[\alpha^2]-\mathbb{E}[y\alpha], which is false, since y^=ws⊤s+wa⊤α\hat y=w_s^\top s+w_a^\top\alpha implies ∇waL=E[αα⊤]wa+E[αs⊤]ws−E[αy]\nabla_{w_a}L=\mathbb{E}[\alpha\alpha^\top]w_a+\mathbb{E}[\alpha s^\top]w_s-\mathbb{E}[\alpha y]. Omission of E[αs⊤]ws\mathbb{E}[\alpha s^\top]w_s needs justification. Similarly, assuming E[αy]=0\mathbb{E}[\alpha y]=0 is not justified: mm being independent of (a,y)(a,y) does not mean the expectation in question is zero. So one cannot straightforwardly claim that the attention readout decays to zero and that zero is a stationary point, which forms a basis for further predictions. It is therefore not surprising that, for instance, the Ncrit∝1/ηN_{\mathrm{crit}}\propto1/\eta prediction fails, which the paper records empirically in Section 7.

Figure 1's measurement points are too coarse: in particular, they do not cover the first 50 or so steps that Section 6 identifies as important, weakening the claims based on Figure 1.

There are other confusing statements. In section 6, applying Fisher's exact test to [27 62; 0 55][27 \ 62; \ 0 \ 55] should give the pp of 4.13×10−74.13 \times 10^{-7} and not 3×10−143 \times 10^{-14}. In Appendix O, the statement that every 30,000 step run is worse than the 6,000 step analog contradicts Table 5. In Section 6, it is claimed that onsets later than 50 are 'indistinguishable' from the no-deficit reference, which contradicts the data from Table 4 for both accuracy and survival, although it is true that they are higher.

The paper makes strong conclusions that are at times not supported at the level of mathematical derivations and empirical evidence.

Questions for authors

No further questions other than addressing the above irregularities and confusions in the weaknesses.

2 comments
AuthorsOpenReview ↗

Response to Reviewer YmUs

We thank the reviewer for the careful and detailed reading of our derivation and numbers. We agree with each point, and addressing them has strengthened the paper.

Proposition 1. We thank the reviewer for this correction. With y^=ws⊤s+wa⊤α\hat y = w_s^\top s + w_a^\top\alpha, the gradient is ∇waL=E[αα⊤]wa+E[αs⊤]ws−E[αy]\nabla_{w_a}L = \mathbb{E}[\alpha\alpha^\top]w_a + \mathbb{E}[\alpha s^\top]w_s - \mathbb{E}[\alpha y]. The cross term must be retained, and independence of the mask does not make E[αy]\mathbb{E}[\alpha y] zero. In the revised sketch, gradient descent relaxes waw_a toward the partial-regression coefficient E[αα⊤]−1(E[αy]−E[αs⊤]ws)\mathbb{E}[\alpha\alpha^\top]^{-1}(\mathbb{E}[\alpha y]-\mathbb{E}[\alpha s^\top]w_s); the mask density sets the rate of this relaxation but cancels in the fixed point. This coefficient is small while the attention feature carries little information about yy beyond ss. Under misrouting, the attention parameters receive supervised gradient only at the few query positions that happen to be selected, and that gradient is scaled by waw_a. The sketch therefore explains why the attention pathway is suppressed, without implying decay to exactly zero or a stationary point. Persistence is now presented as an empirical result (Sec. 6), and the sketch's assumptions are stated explicitly. We agree that the simplified gradient likely contributes to the failed Ncrit∝1/ηN_{\text{crit}}\propto 1/\eta prediction. Consistent with this, the new measurements below show the attention pathway collapsing at a similar point (between steps 25 and 50) at learning rates 3e-3 and 1e-3.

Figure 1. We thank the reviewer for this suggestion; the early steps are indeed where the outcome is decided. We measured δ at 16 points between steps 1 and 250 (5 seeds; learning rates 3e-3 and 1e-3).

  • At 3e-3, the learning rate of the main protocol: under misrouting, δ at query positions is positive only between steps 7 and 10 (22–42% of the oracle value). From step 15 it is no longer distinguishable from zero (means between −0.021 and +0.022, standard deviations 0.019–0.030), while the attention output is still large (attention/SSM RMS ratio 5.9 at step 35). The pathway is suppressed only afterwards (ratio 0.35 at step 50).
  • At 1e-3, δ is positive for longer (steps 10–25), and its loss coincides with the suppression of the pathway (steps 25–35).
  • Under oracle routing, δ remains positive throughout.

At the higher learning rate, the signal is thus lost before the pathway is suppressed. This ordering is consistent with the revised sketch: the readout is driven toward zero once the attention feature has become uninformative. The statement "δ ≈ 0 under misrouting" holds after this early interval, and Fig. 1, Table 3, the abstract and Sec. 4 now give the time-resolved result.

Fisher's exact test. We confirm the reviewer's value, p=4.13×10−7p = 4.13\times10^{-7}; the conclusion is unchanged. This is the only hypothesis test in the paper. The logistic fits and confidence intervals in Sec. 6 are computed by script from the per-run results included in the supplement.

Appendix O. The sentence is revised to match Table 5: at 30k steps, SSM-only models degrade at depth ≥ 2 and dense-attention models only at depth 4, while 1- and 2-block dense models improve. The same revision is made in App. J.

"Indistinguishable." We agree that this word overstated the result and have replaced it with the data. All runs with onsets ≥ 50 survive except one at t0=100t_0 = 100 (9/10), with mean accuracy 0.86–0.97 against 0.997 for the no-deficit reference.

Strength of conclusions. We appreciate this point. We have revised the wording throughout so that each claim matches its evidence; for example, "established" and "ruled out" are replaced by the measured quantities. Further revisions, including two to the abstract, are listed in our general comment.

The revised PDF, with all changes marked in blue, will be uploaded by Nov 16.

AuthorsOpenReview ↗

Revised PDF

Thank you again for the careful reading of our derivation and numbers. We have uploaded the revised PDF (summary in our top-level comment). In response to your points, the mechanism is presented as a qualitative sketch with explicit assumptions (Sec. 7, App. B), the time-resolved signal is in Fig. 1 and App. S, and the 30k-step depth results are stated consistently in App. J, App. O and Table 6. We would be glad to answer any remaining questions before the discussion closes.

Reviewer_1i7qOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies why learned token routers in hybrid SSM/attention models fail to train, using multi-query associative recall (MQAR) with a single-block Mamba + attention model at d_model = 128 and a fixed 6.25% attention budget. It argues that the standard “top-k gives no gradient to unselected tokens” explanation is incomplete. Four router-training methods all stay at chance, including REINFORCE, which does deliver gradients to unselected tokens. The authors attribute the failure to timing.

Critical strengths
  1. The core experimental design is strong. Holding total steps fixed makes the recovery confound favor late-onset failure, so the observed early-onset failure is conservative. Ten seeds per onset, a logistic fit with bootstrap CIs, and individual-run reporting for bimodal outcomes make the onset result convincing.
  2. The paper anticipates and tests many natural objections: budget headroom (1×–4×), evaluation-batch noise (12 independent batches), single-block readout (depth 4), and whether a learned router reaches the same state as an imposed deficit.
Critical weaknesses
  1. Limited scope and unclear practical relevance. All positive results come from one synthetic task, a tiny model, and mostly a single block. The real-text experiments (WikiText-103) find no reliable attention headroom in any configuration, so the phenomenon is not shown anywhere attention is load-bearing on natural data. The MoE early-routing literature is offered as motivation, but the authors acknowledge they run no MoE experiments. Without at least one larger or more realistic setting, it is hard to judge whether practitioners should change anything.
  2. Baselines for Section 3 are thin. Each method has 3 seeds, and tuning effort is not described. The paper omits Gumbel-top-k and other continuous relaxations, expert-choice routing, auxiliary or load-balancing losses, and variance-reduced or per-token rewards for REINFORCE. The REINFORCE failure is attributed to reward variance from an 8192-way cross-entropy, which is a separate, known problem. So it does not cleanly refute the gradient-flow account as the paper suggests.
  3. The task is close to a worst case by construction. The budget exactly equals the number of query positions, and supervision exists only at those positions. As the paper itself notes, δ is undefined at 94% of positions. The finding that the routing signal is absent is therefore partly a consequence of sparse supervision. The introduction states that the necessity result “holds on real text with dense supervision,” but I could not find a real-text δ measurement supporting this in Section 4. Please clarify.
Questions for authors
  1. Does the lock-in effect survive a standard linear warmup, for example, over 500–2000 steps, at 3e-3? If the warmup removes it, how do you interpret the finding?
  2. Can you identify a learning rate and setup where both the 1-block and 4-block models train with a flat schedule, and the deficit still bites, avoiding the in-window learning-rate switch?
  3. What tuning was done for the four router baselines? Did you try Gumbel-top-k, expert-choice routing, or a per-position reward for REINFORCE?
5 comments
AuthorsOpenReview ↗

Response to Reviewer 1i7q

We thank the reviewer for the positive assessment and the thoughtful, concrete questions.

W1 (scope). We appreciate this point. We now present the paper as a controlled mechanistic study and add a paragraph on when the effect should be expected: attention must be load-bearing, the early learning rate high, and routing learned from the task loss. We replace the introduction's sentence about real text (see W3) and shorten the MoE discussion to motivation. A larger-scale study is beyond what the discussion period allows; we discuss it explicitly as a limitation and a direction for future work.

W2, Q3 (baselines). We thank the reviewer for these suggestions. The Table 1 routers were run at the default learning rate (3e-3) without a sweep. The revision states this, extends the 3e-3 rows to 5 seeds (all four routers remain at chance, 0/5 each), and adds a second learning rate (1e-3).

  • Expert-choice routing. At evaluation, every learned router in Table 1 selects a fixed budget by top-k over the sequence. This is expert-choice selection with the attention pathway as the single expert. During training, the MoD gate and the marginal-utility router also select top-k over each sequence. The straight-through router selects the top 6.25% of positions across the batch, and REINFORCE samples each position independently, with a penalty holding the expected budget at 6.25%. Sec. 3 now describes this.
  • Load-balancing losses. With a single attention pathway and a hard budget of exactly 16 tokens, load is fixed by construction, so a load-balancing loss is degenerate here. The closest mechanism we tested is a contribution floor that keeps the attention output above a fixed fraction of the SSM output's scale (Sec. 4); it also remains at chance.
  • REINFORCE. Our REINFORCE already uses a per-token reward (the negative loss at each position) with a baseline, which the revision now makes explicit. We agree that its failure alone does not settle the gradient-flow question. We therefore added two methods: a variance-free counterfactual policy gradient, which gives each token the exact effect of attending, δₜ; and Gumbel-top-k with a straight-through relaxation. Both have been run at 1e-3 and 3e-3. One of them requires an additional control before it can be interpreted, and we will report both by Nov 13.

W3 (task construction). On budget headroom: with budgets of 1–4× the number of query positions (App. E), the straight-through router remains at chance at every budget (0/3 each), and its query recall rises only as random selection would (0.064 at 1× to 0.247 at 4×, against 0.25 for random selection at 4×). The marginal-utility router succeeds in 1 of 3 runs, only at 4× (accuracy 0.593; we count a run as successful when its final accuracy exceeds 0.5, since outcomes are bimodal). On sparse supervision, our new measurements indicate that the absence of signal is not a consequence of sparsity alone. Under oracle routing, the same sparse supervision yields a positive δ throughout the first 250 steps. Under misrouting, δ is positive only briefly (steps 7–10 at 3e-3, steps 10–25 at 1e-3) and is gone by step 50, by which point the attention pathway is suppressed (details in our reply to Reviewer Tmvr). On real text with dense supervision, we had measured δ but had not included it in the submission: from step 500 onward, |δ| < 0.007 under random routing against 0.19–0.40 under oracle routing (WikiText-103, 3 seeds). The revision reports this in an appendix, with the caveat that it measures reliance on attention rather than benefit, and it replaces the introduction's sentence.

Q1 (warmup). With a 2000-step linear warmup to 3e-3 (10 seeds per condition), lock-in persists, and the sensitive window moves to the end of warmup. A 250-step deficit at steps 2000–2250 survives in 10/10 runs without warmup but in only 3/10 with warmup (p = 0.003), and an 8000-step deficit from step 0 fails in all runs with warmup (0/10). Details are in our reply to Reviewer Tmvr.

Q2 (constant schedule at depth). Following the reviewer's suggestion, we are running depths 1 and 4 with a 2000-step warmup followed by a constant 1.5e-3. The 8000-step deficit starts after warmup, so the learning rate is constant throughout the deficit window (10 seeds per arm). In earlier runs, depth 4 trains in this setting without a deficit (3/3). Results by Nov 13.

The revised PDF, with all changes marked in blue, will be uploaded by Nov 16.

AuthorsOpenReview ↗

Round-2 update for Reviewer 1i7q

Update: new baselines (W2, Q3) and depth (Q2). As promised, we report the remaining results.

Baselines (5 seeds per method and learning rate, except the counterfactual gradient at 1e-3, which has 10):

  • Table 1 routers at 1e-3. The four Table 1 routers also remain at chance at 1e-3 (0/20), while oracle routing trains (5/5); see our update to Reviewer Tmvr.
  • Gumbel-top-k (straight-through relaxation, τ annealed from 1.0 to 0.1) fails at both learning rates (0/10). Its query recall stays at the random level (0.06–0.07), and the backbone trained alongside it cannot use correct routing either (accuracy ≈ 0 even with oracle routing at test time).
  • Variance-free counterfactual policy gradient (advantage = the exact per-token effect of attending, δ_t) succeeds in 3/10 runs at 1e-3 and 0/5 at 3e-3. To identify the source of these successes, we ran a control that raises the router's selection probability at every labeled position with a constant advantage and no δ. The control succeeds in 5/5 runs at 1e-3 (query recall 0.67–0.84, against 0.10–0.77 for the counterfactual gradient; 5/5 vs 3/10, Fisher's exact test, p = 0.026) and in 0/5 at 3e-3. On this task the labeled positions are the routing targets, so the counterfactual gradient's successes reflect where the labels are rather than δ. Removing all reward variance therefore does not turn δ into a usable routing signal, consistent with Sec. 4. At 3e-3, even the control's router does not learn to select the queries (query recall at chance).
  • Evaluation. In evaluating these routers, we found that top-k over sigmoid probabilities produces ties once more than 16 probabilities saturate at 1.0 in floating point. The counts and recalls above use this original evaluation. Where ties occur, it chooses among the positions the router scores highest, so a success still requires the router to rank the queries among them. In nine instrumented reruns, evaluating on raw router scores gave equal or higher query recall in every run and did not change the outcome of any router that fails. The revision describes this and reports both evaluations for the reruns.

Depth (2000-step warmup, then a constant 1.5e-3; an 8000-step misrouting deficit on steps 2000–10000, so the learning rate is constant throughout the deficit; 10 seeds per arm):

No deficitDeficit
Depth 110/10 (0.998)10/10 (0.998)
Depth 410/10 (0.999)0/10 (0.005)

Both depths train without a deficit. At this learning rate the deficit is harmless at depth 1 but causes lasting failure in every depth-4 run (Fisher's exact test, p = 1.1×10⁻⁵, both against the depth-4 reference and against the depth-1 deficit arm). The depth result therefore does not depend on the in-window learning-rate switch: the four-block model locks in more readily than the single block. App. D now reports this experiment alongside the original protocol.

AuthorsOpenReview ↗

Correction to our round-2 update

Correction to our update of Nov 13. Two of the three counterfactual-gradient runs we counted as successes at 1e-3 reach an accuracy above their query recall (0.947 vs 0.325, and 0.534 vs 0.342). Routing therefore cannot fully explain their accuracy, and our sentence that a success requires the router to rank the queries does not hold for these two runs. The pattern did not recur in nine instrumented reruns, where accuracy on unrouted queries never exceeded 0.03, so we report the two runs as unexplained. Counting only the success consistent with routing, the counterfactual gradient succeeds in 1/10 runs against 5/5 for the label-position control (Fisher's exact test, p = 0.002); counting all three, as in our update, gives p = 0.026, so the conclusion holds under either count. The revision reports both counts and the individual runs in App. U.

Comments on the paper

3 comments
AuthorsOpenReview ↗

Summary of responses and revisions

We thank the three reviewers for their careful and constructive reviews. The reviewers note the value of the paper’s controlled experiments, in particular the timing experiment (a fixed number of correctly routed steps, with only their position varied), and their reviews have helped us make the surrounding claims precise. We reply to each reviewer separately and summarize the changes here.

Corrections.

  • Prop. 1 is rederived with the full gradient and now supports only qualitative claims (we thank Reviewer YmUs).
  • We correct the Fisher p-value, the statement in App. J/O about 30k-step runs, and the word "indistinguishable" in Sec. 6.
  • Our new early-training measurements correct a claim in the abstract and TL;DR. Under misrouting, the marginal utility of attention is 0.0000 ± 0.0000 at step 4000, but it is briefly positive within the first ~25 steps. The abstract and TL;DR now state that under misrouting the signal is briefly present and is removed within the sensitive window.
  • The abstract's statement on budget headroom now also reports the one marginal-utility run that succeeds at 4× the budget (App. E).
  • While revising, we found that chance accuracy is 1/4096, not 1/8192, because values are drawn from half the vocabulary. The failed arms sit at this chance level.
  • The five-intervention table missing from Sec. 4 is restored, and App. D now separates runs with and without warmup.

Clarifications: We add a per-experiment learning-rate and warmup table. The main protocol uses a constant learning rate without warmup; the depth experiments use a 2000-step linear warmup. "Irreversible" is now defined operationally: not recovered within 32k correctly routed steps.

New results (details in the individual replies):

  • δ measured at 16 points over steps 1–250 (5 seeds, two learning rates). Under misrouting, the routing signal is present only briefly (steps 7–10 at 3e-3, steps 10–25 at 1e-3) and is gone by the time the attention pathway is suppressed (steps 25–50).
  • Lock-in under a standard 2000-step linear warmup to 3e-3 (10 seeds per condition). Warmup does not remove lock-in; it moves the sensitive window to the end of warmup.
  • Still running, with results by Nov 13: learned routers at a lower learning rate; Gumbel-top-k; a variance-free counterfactual policy gradient; and the depth-4 experiment with a constant learning rate inside the deficit window.

A revised PDF with changes marked in blue will be uploaded by Nov 16.

AuthorsOpenReview ↗

Round-2 summary of new results

Summary of the new results. The additional experiments identify two complementary obstructions to learned routing, and what resolves them.

  1. Under misrouting, the task loss carries a routing signal only during the first ~25 steps, and a variance-free policy gradient that uses it does no better than a control that ignores it. Learned routers fail even at a learning rate at which the backbone recovers from misrouting (0/20), while a router given an external routing signal succeeds at that rate (5/5).
  2. At higher learning rates, and more readily at greater depth, misrouting locks in the backbone. This persists under a standard warmup, which moves the sensitive window to the end of warmup. At depth 4 it also persists with a constant learning rate throughout the deficit window (0/10, against 10/10 at depth 1).

Details are in our updates to Reviewers Tmvr and 1i7q. The revised PDF will follow by Nov 16.

AuthorsOpenReview ↗

Revised PDF: summary of changes

We have uploaded a revised PDF that incorporates the experiments and clarifications described in our responses; changes are marked in blue. The main changes are:

  • Framing. The abstract and introduction present the two findings separately: lock-in of the backbone under early misrouting, and loss of the routing signal in the task loss. "Irreversible" is defined operationally: not recovered within 32k correctly routed steps.
  • Sec. 3, Table 1 and App. U (learned routers). Table 1 reports the four routers at two learning rates (3e-3 and 1e-3, 5 seeds each), together with Gumbel-top-k, the counterfactual policy gradient and the label-position control. App. U gives the details and individual runs.
  • App. P and App. U (evaluation). App. P describes the top-k tie artifact. App. U reports both evaluations for the nine instrumented reruns.
  • Sec. 4, Fig. 1 and App. S (time-resolved signal). Fig. 1 and App. S add the early-training measurements: the routing signal is lost within a few dozen steps under misrouting.
  • Sec. 4 (intervention table). Restored the table with all runs (0/72; the text previously counted only K ∈ {100, 2000}), and corrected the temporary-signal row to 0/24.
  • Sec. 6, App. T and App. D (lock-in). Sec. 6 and App. T report lock-in under warmup, and the Fisher test is corrected. Sec. 6 and App. D add the depth experiment at a constant learning rate (depth 4: 0/10; depth 1: 10/10), and App. D separates warmup from no-warmup runs.
  • Sec. 7 and App. B. The mechanism is presented as a qualitative sketch, with its assumptions stated explicitly.
  • Sec. 9. Adds a "When this applies" paragraph that states the conditions under which the effect appears, with scale as the main open direction.
  • App. R and App. V. App. R adds the learning-rate schedule table, and App. V adds the marginal utility of attention on real text.
  • Corrections. App. J and the Table 6 caption specify the 30k-step depth results by arm and depth, and chance accuracy is stated as 1/4096.
  • Correction to Reviewer 1i7q. App. U reports the runs discussed in our correction to Reviewer 1i7q.

We thank the reviewers for their comments, which improved the paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.