acceptodds
Under review as a conference paper at ICLR 2027

Routing the Past: How Retrieval Circuits Control Apparent Memory Capacity

Abstract

Ask a language model for the current value of a variable that has been reassigned times and it answers at chance. This is usually read as working memory running out. We show that much of this apparent capacity is set by which query-time circuit retrieves the answer, not by what the model stores. The evidence is causal. An answer-agnostic scalar gain on a few copy heads, applied only at the query token so that it cannot alter the context, flips genuinely binding-limited base models from failing to passing (Llama-3.1-8B , Pythia-6.9B ; positive in of models at one fixed setting; budget-matched random heads reach only against ), and ablating the same heads runs the switch in reverse. The lift survives free generation and interleaved distractors that remove the “last line” cue. The value was in the residual stream all along: under adversarial controls, a linear probe reads it above the model's own output in of models across seven families. The mechanism transfers to code, dialogue, and documents (code retrieval with no code-specific labels), and it reframes evaluation: under a raw-completion harness the capacity ranking of chat models dissolves and the scale trend vanishes. Across models (70M–32B) and three benchmarks (PI-LLM, bAbI, MultiWOZ), we characterize the occurrence-order code that crowds with load, prove that position-only routes can reach only endpoints, and identify a query-conditioned pointer-head signature behind the rare interior access of gemma-4-31B. Apparent working-memory capacity is, in large part, retrieval routing.

Primary area:
interpretability and explainable AI
Keywords:
Working Memory, Retrieval Routing, Mechanistic Interpretability, Causal Interventions, Copy Heads, In-Context Memory, Language Models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_vDaWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies whether failures to retrieve updated values reflect storage limitations or retrieval routing. It amplifies selected attention heads at the query position, improving latest-value retrieval, and combines these interventions with probes and representation analyses to explain apparent memory capacity.

Critical strengths
  1. Query-only head amplification improves latest-value retrieval while preserving context representations. Random-head, wrong-position, and budget-matched controls strengthen the causal evidence.

  2. Cross-model, cross-language, and surface-form transfer experiments show that the effect extends beyond a single model or prompt template.

Critical weaknesses
  1. The causal interpretation is overstated. Amplification shows that selected heads can improve retrieval, but does not identify the original failure mechanism. The code-crowding explanation lacks positive intervention evidence, and all-position reverse ablation cannot establish storage-preserving bidirectional control.

  2. Limited practical validation. Oracle deletion of stale entries simplifies the task using privileged information; its benefits cannot validate head amplification. The non-oracle intervention primarily strengthens recency bias, harms non-latest queries, and lacks consistent natural-task gains.

Questions for authors
  1. Does Figure 4 support a broader memory-capacity claim, or only improved latest-value retrieval at the expense of access to earlier bindings?

  2. Would the reverse transition persist under query-only ablation? Figure 38 uses all-position ablation, whereas §5 claims that both directions preserve the context.

  3. Is the claim that interior access “must pay the crowding cost” justified, given that Appendix E.6 treats the geometry as descriptive and does not establish its causal role?

5 comments
AuthorsOpenReview ↗

Response to Reviewer vDaW 1/3

Thank you very much for your clear and constructive review. We are glad that you found the query-only amplification and its controls convincing, and that you saw the transfer across models, languages, and surface forms as a strength. Each of your concerns led us to run new experiments, and together they make the causal story both stronger and more precise. All new and changed text in the paper is shown in blue, and the new experiments are collected in Appendix N.

  • Both directions of the switch between regimes now hold when we edit only the final query token. Removing a few heads there moves GPT-2-large and two chat models into the binding-limited regime and sharply lowers accuracy at high load in two more, while the context stays exactly the same.
  • We now test the original failure directly. At the query token, heads that favor the latest value compete with heads that favor older values, and removing four of the latter doubles accuracy at u = 16 in three of the five models we tested.
  • Crowding is now presented as a description only. We removed the claim that reading a middle value "must pay the crowding cost", and Figure 4 is now limited to retrieve-latest capacity.
  • The gain is presented as a causal probe of one retrieval route, and eviction as an upper bound on what can be recovered.

Weakness 1 and Question 2, the reverse transition with an edit at the query token only

The reverse transition holds when the heads are removed only at the query token. We repeated the GPT-2-large experiment with the same six copy heads removed only at the final query token, and we checked that every activation in the context stays exactly the same at every layer. Accuracy at u = 32 falls from 0.16 to 0.08 (chance is 0.03) and the curve meets the binding-limited criterion, while all 15 sets of random heads leave the model in the recency-copy regime. On 1,000 new prompts per load the result holds again, with accuracy falling from 0.12 to 0.07 (95 percent interval 0.06 to 0.09). On four more base models the query-only edit removes 42 to 91 percent of the endurance lost under the original edit (Appendix N.2).

Chat models show the same effect under their own chat templates (Appendix N.8). We selected heads on a separate set of prompts and removed them only at the final prompt token. In gemma-2-2B-it, which is far from the regime boundary, removing four heads lowers accuracy at u = 32 from 0.82 to 0.30, and in Qwen3-4B it falls from 0.58 to 0.09. Nine sets of four to sixteen random heads keep these two models at 0.76 or above and at 0.49 or above. When we instead remove the heads that hurt short streams the most, gemma-2-2B-it stays at 0.78 for u = 32, so the selected heads matter for long streams in particular. In gemma-2-2B-it and GPT-2-large the removed heads also leave accuracy on keys that appear only once almost unchanged, so in these models they carry the route that reads the latest of several updates. Qwen3-1.7B under its chat template is an exception to this. Qwen3-1.7B and gemma-3-4B-it, which are closer to the boundary under their templates, become binding-limited when two and four heads are removed.

AuthorsOpenReview ↗

Response to Reviewer vDaW 2/3

Weakness 1, the original failure

We agree that making a route stronger does not by itself show why it was weak, so we tested the failure side directly (Appendix N.6). We first measured how strongly each head pushes the answer toward each value at the query token. In Qwen3-1.7B the copy heads favor the latest value by a small margin, while four other heads favor older values at every load. The margin of the latest value over its strongest older competitor is negative and grows more negative with load, from -0.46 logits at u = 4 to -2.15 at u = 32. We then edited both sides of this competition at the query token only, with heads chosen on one set of prompts and every edit tested on new prompts.

Accuracy at u = 8 (chance 0.13)Qwen3-0.6BQwen3-1.7BPythia-2.8BQwen3-4BLlama-3.1-8B
No edit0.220.240.420.140.18
Remove four heads that favor older values0.410.510.610.150.22
Remove four random heads (5 sets)0.220.240.420.130.19
Remove four heads that favor the latest value0.070.180.280.100.14
Amplify the heads that favor the latest value0.420.740.550.460.32
Both edits0.370.880.760.450.35

In all five models, removing the heads that favor the latest value lowers accuracy or leaves it at the floor, and amplifying them raises it, while random heads stay within 0.02 of baseline on average. In Qwen3-0.6B, Qwen3-1.7B, and Pythia-2.8B, removing four heads that favor older values doubles accuracy at u = 16, and this gain holds with interleaved distractors and under free generation. In Llama-3.1-8B this effect is small, and in Qwen3-4B it is not reliable, so in these two models the pull toward older values is not concentrated in a few heads. The original failure is therefore a lost competition between two groups of heads at the query token, and in three of the five models the pull toward older values comes from a few heads that we can identify. We leave the earlier circuit that sets this balance to future work.

Weakness 1 and Question 3, the crowding explanation

We agree, and we have removed the claim that reading a middle value "must pay the crowding cost". We ran two new tests that could have given crowding a causal role. At a fixed load, the geometry of each prompt's stored order code does not predict whether the model succeeds on new prompts, with AUROC between 0.47 and 0.51. Spreading the order codes apart by a factor of 2 or 4, at one of five layers at a time, leaves accuracy at baseline (Appendix N.4). The paper now presents crowding as a description of the stored code throughout.

Proposition 1 keeps a narrower role in the revised paper. It limits what a read through the order code can deliver, and it predicts that heads which read by position alone reach only the first or the latest update. A new measurement finds that 68 to 93 percent of heads peak at one of these two ends (Appendix N.5). A new experiment also shows the way around this limit that the proposition allows (Appendix N.3). When each update carries its number, seven chat models read the middle value at 0.84 to 1.00 at u = 32, while six of them are at chance without numbers. The middle value is stored, and what these models lack is a way to find the n-th of several unlabeled mentions.

AuthorsOpenReview ↗

Response to Reviewer vDaW 3/3

Weakness 2, practical validation

We agree with you on both points, and we have changed how the paper presents eviction and the gain. Eviction shows that the latest value can still be recovered from the context, so we now present it as an upper bound and not as a validation of the gain. The gain is a causal probe of one retrieval route and not a ready-to-use fix. It has no effect on PI-LLM, where the baseline is already near zero, or on MultiWOZ, where interference is mild. When the query is a full sentence after the stream, the gain lowers latest accuracy from 0.24 to 0.14 at u = 8 and moves answers toward the first value instead. The gain is therefore tied to the query form that its heads were selected on.

We also ran the transfer test that a practical claim needs, with the heads, the gain, the task forms, the query wordings, and the controls all fixed in advance (Appendix N.7). The frozen gain lifts the code form under all three query wordings we tried, for example from 0.11 to 0.76 and from 0.30 to 0.56 at u = 8, and by more than both controls. It does not transfer to the dialogue and memo forms, where it lowers accuracy. Section 4.8 and the limitations now state these roles and limits. Within this scope we added two further results.

  • The cost on other queries can be avoided. The prompt states the type of query, so a fixed text rule can apply the gain to latest queries only. By design this keeps every other query at baseline, and on Qwen3-1.7B at u = 8 it gives the best average over four query types (0.27, against 0.14 at baseline and 0.22 when the gain is always on).
  • Two input formats that need no annotation show that the stored values can be told apart once the model has an address to match. On interleaved streams at u = 32, filtering the stream by the queried key raises Qwen3-1.7B from 0.04 to 0.42, although it does not help Qwen3-4B at that load. A version label on each update lets the same model read a middle version at 0.95, and Qwen3-4B at 0.99, although it helps much less for the latest version (0.24). Smaller models such as GPT-2-large do not use the label, so we report these formats as evidence about the mechanism rather than as a general fix.

Question 1, what Figure 4 supports

Figure 4 supports better retrieval of the latest value at the expense of access to earlier values, and not a broader claim about memory capacity. Section 4.7 now says this explicitly, and our capacity claims now concern retrieve-latest capacity only. We think this trade-off is the most informative part of the figure. A general improvement in memory would lift every query, while the gain lifts exactly one query and pulls every other query toward the latest value, which shows that it acts on a recency route and not on storage. The earlier values also remain stored, since an explicit number lets the same models read them.

Summary

Thank you again for these thoughtful and challenging points, which made the paper more careful and more convincing. The revision adds query-only tests of both directions of the switch, in GPT-2-large and in chat models, and a causal test of the original failure that holds strongly in three of five models. It also adds a transfer test fixed in advance that shows where the gain helps and where it does not. Finally, it adds direct tests of the crowding account, a numbered-update experiment, a rule that applies the gain only to latest queries, and input formats that need no annotation. We hope these results address your concerns, and we would be grateful if you would consider the new evidence. We are happy to answer any further questions.

Reviewer_CfrhOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies whether failures to retrieve updated values reflect limited storage or problems with retrieval. It finds that amplifying selected attention-head outputs at the query token can substantially improve latest-value retrieval without changing context representations. The paper also examines occurrence-order geometry, interior-slot retrieval, and sensitivity to the elicitation interface across a broad set of models. Additional experiments include linear probes, head ablations, distractor controls, free generation, and transfer to naturalistic task formats.

Critical strengths
  1. The query-token amplification result is compelling. The intervention improves retrieval while leaving context representations unchanged, and the authors include random-head, wrong-position, norm-matched, and matched-search-budget controls. Improvements also survive free generation on several base models. This provides useful evidence that poor retrieval accuracy can underestimate the information a model can access.

  2. The interface analysis has a clear implication for evaluation. Testing the same chat-model weights under completion and chat interfaces substantially changes both accuracy and model rankings. I also appreciate that the paper reports failed geometry interventions and the activation-sink artifact, which help distinguish the supported findings from the proposed explanation.

Critical weaknesses
  1. The bidirectional causal claim needs clarification. Figure 5 and the conclusion describe query-token interventions in both directions with context unchanged. However, Figure 38 explicitly states that the GPT-2 ablation acts at all positions. That intervention can change context representations, so it does not isolate retrieval from storage in the reverse direction. The paper needs to clarify which results use each intervention.

  2. The random-access interpretation is not fully isolated. Using the same ordinal template controls wording, but it does not separate constructing the requested index from retrieving its value. Gemma-4-31B succeeding on that template does not rule out counting or address-construction failures in other models. The endpoint theorem also assumes convex positional kernels, which Appendix C acknowledges are not verified. I think the evidence supports a narrower claim about performance under the tested cues.

Questions for authors

Do the interior-slot failures persist when assignments are explicitly numbered or when alternative cues reduce the need to construct an ordinal address?

2 comments
AuthorsOpenReview ↗

Response to Reviewer Cfrh 1/2

Thank you very much for your careful and balanced review. We are glad that you found the query-token amplification and its controls convincing, and that you valued the interface analysis and our honest reporting of failed interventions. Both of your concerns pointed to real gaps in the paper, and each one led us to run new experiments. All new and changed text in the paper is shown in blue, and the new experiments are collected in Appendix N. We believe the paper is now more precise and better supported, and we summarize the main changes below.

  • Both directions of the switch between regimes now hold when we edit only the final query token. Removing a few heads there moves GPT-2-large and two chat models into the binding-limited regime and sharply lowers accuracy at high load in two more. A new table also lists where every causal edit is applied.
  • Numbering the updates separates finding the right position from reading the value. At u = 32, seven chat models read the middle value at 0.84 or higher once each update carries its number, so the failure lies in finding the position, as you suggested. Table 1 now shows this in a new column.
  • We measured the convexity assumption behind the endpoint result in five models, and we report all of these measurements.

Weakness 1, which results use which intervention

You are right that the GPT-2-large ablation in the original figure removed the heads at every position. We repeated it with the same six heads removed only at the final query token. Under causal attention no earlier token can see that position, and we also checked that every activation in the context stays exactly the same at every layer.

With this query-only edit, GPT-2-large drops from 0.16 to 0.08 at u = 32 (chance is 0.03). Its interference endurance, which is the area under the accuracy curve above chance, falls by half from 0.28 to 0.14. The curve now meets the binding-limited criterion, while all 15 sets of random heads removed in the same way leave the model in the recency-copy regime. Because this result sits close to the regime boundary, we repeated it on 1,000 new prompts per load. The edit again gives a binding-limited curve, with accuracy falling from 0.12 to 0.07 at u = 32 (95 percent interval 0.06 to 0.09), and all five random sets stay recency-copy. On four more base models the query-only edit removes between 42 and 91 percent of the endurance lost under the original edit, while six or ten random heads never lower endurance by more than 0.03 (Appendix N.2).

Chat models show the same effect under their own chat templates (Appendix N.8). We selected heads on a separate set of prompts and removed them only at the final prompt token. In gemma-2-2B-it, which is far from the regime boundary, removing four heads lowers accuracy at u = 32 from 0.82 to 0.30, and in Qwen3-4B it falls from 0.58 to 0.09. Nine sets of four to sixteen random heads keep these two models at 0.76 or above and at 0.49 or above. When we instead remove the heads that hurt short streams the most, gemma-2-2B-it stays at 0.78 for u = 32, so the selected heads matter for long streams in particular. Qwen3-1.7B and gemma-3-4B-it, which are closer to the boundary under their templates, become binding-limited when two and four heads are removed.

In binding-limited models the same heads also move one model in both directions. Under completion, the four copy heads of Qwen3-1.7B raise accuracy at u = 4 from 0.34 to 0.80 when we amplify them at the final query token, and lower it to 0.18 when we remove them there.

Finally, a new table in Appendix N.1 lists every causal result together with where the edit is applied, its direction, and whether the context stays unchanged. The original cumulative ablation is the only one that acts at every position, and the text, Table 1, and the figure captions that mention it now say so clearly.

AuthorsOpenReview ↗

Response to Reviewer Cfrh 2/2

Question 2, crowding as a cause or as something that only goes along with load

We ran two direct tests of the crowding account. The first test holds the load fixed, so that load cannot explain why one prompt succeeds and another fails. In Qwen3-1.7B, the geometry of each prompt's stored order code does not predict whether the model succeeds on new prompts, with AUROC of 0.48, 0.47, and 0.51 at u = 4, 8, and 16. The second test changes the geometry of the stored code directly. Spreading the order codes apart by a factor of 2 or 4, at one of five layers at a time, leaves accuracy at baseline (Appendix N.4). Together with the compression result already in the paper, this does not support a causal role for crowding, so the paper now presents crowding as a description of the stored code throughout.

The revision also shows where models with the same stored code differ. Once attention sinks are controlled for, every model writes the same kind of order code, so the geometry cannot explain why models fall into different regimes, and the difference lies in retrieval (Appendix N.6). We measured how strongly each head pushes the answer toward each value at the query token. In Qwen3-1.7B the copy heads favor the latest value by a small margin, while four other heads favor older values at every load. Removing these four heads at the query token raises accuracy at every load, from 0.24 to 0.51 at u = 8 and from 0.05 to 0.18 at u = 32. Random heads removed in the same way leave accuracy within 0.02 of baseline on average. We then automated this test for five models from the Qwen, Pythia, and Llama families, selecting heads on one set of prompts and testing them on new prompts. In Qwen3-0.6B, Qwen3-1.7B, and Pythia-2.8B, removing four of these heads doubles accuracy at u = 16, and the gain holds with interleaved distractors and under free generation. In Llama-3.1-8B the effect is small, and in Qwen3-4B it is not reliable. We also checked which older value wins when the model is wrong. The errors are spread almost evenly across the older values, so this competition sets how far above chance the curve sits, while the decline with load comes from a growing number of similar competitors.

Question 3, middle values with explicitly numbered updates

Your suspicion was right, and the experiment shows it clearly (Appendix N.3). We kept the keys, the values, and their order the same and changed only how the position is given. In the numbered condition each line carries its number after the key, and the query keeps the original wording, so the model no longer needs to count. At u = 32 and without numbers, the middle value is at chance in six of seven chat models (0.01 to 0.04, with chance at 0.03) and reaches 0.28 in gemma-4-12B. With a number on each line, all seven read it at 0.84 to 1.00. Two other wordings of the question give the same picture in four models, and Table 1 now shows the numbered condition in a new column. Because the numbered condition keeps the exact wording of the instruction, a misreading of the instruction cannot explain the original failure. The failure lies in finding the n-th of several unlabeled mentions, and Section 4.5 and Appendix D now say so.

The middle value can also be read without adding any tokens. On the original stream without numbers, a question that names the value just before the target retrieves the middle value above chance in every model (0.17 to 1.00 at u = 32). A question that names the value just after the target works less well and stays near chance in four models. This is what we would expect if the model copies forward, from a token to whatever followed it earlier in the stream. It shows on one and the same stream that the value is stored, while the way to find it by position is missing. gemma-4-31B is the one model in our set that finds it by position, and it is also the model with the query-dependent pointer head of Section 4.6.

Summary

Thank you again for these excellent questions, which made the paper clearer and more convincing. The revision adds query-only reverse edits in GPT-2-large and in chat models, a causal test of the failure that holds strongly in three of five models, two direct tests of the crowding account, and a numbered-update experiment. We hope these results address your concerns, and we would be grateful if you would take them into account in your final assessment. We are happy to answer any further questions.

Reviewer_8y5AOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introdues whether failures to retrieve updated values comes from limited storage or from ineffective retrieval. It shows that amplifying selected attention heads at the query token improves latest-value retrieval, while the earlier context representations stay unchanged. The evaluation combines a 47-model behavioral survey, controlled interventions, representation probing, interface comparisons, and both synthetic and published benchmarks.

Critical strengths

Query-token amplification raises accuracy from 0.03 to 0.60 on Llama-3.1-8B and from 0.12 to 0.71 on Pythia-6.9B at load 16. This shows the information is recoverable even though baseline retrieval is poor

The budget-matched random-head, wrong-position and norm-matched controls make the intervention results stronger. The improvements also hold up under free generation on three base models, which reduces the concern about constrained-choice scoring

Across 19 chat models, median accuracy goes from 0.09 under raw completion to 0.79 under chat elicitation, and the rankings only weakly agree. This points to a important confound in memory evaluation

Critical weaknesses

Figure 38 says the GPT-2-large ablation is applied at all positions, but the conclusion describes a bidirectional conversion where context representations are left untouched. Since this ablation can affect context processing, it dont isolate retrieval as cleanly as the query-only amplification does.

The explanation based on order-code crowding lacks causal confirmation. Sink-controlled compression changes accuracy by at most around 0.05, and the slot-code swaps and order-head ablations are null. Therefore as shown in Appendices D and E.6 these results limit the claim that the measured geometry determines retrieval capacity

Also in sec 4.5 and Appendix D, Gemma-4-31B’s success on the ordinal template does not rule out counting or instruction-interpretation failures in the other models. Since they perform worse on ordinal than on semantic latest-value queries, the cause of the interior-slot failures is still unresolved

Questions for authors

If the GPT-2-large ablation is restricted to the query token only, does it reproduce the reverse conversion reported in Figure 38?

Given the null geometry interventions, what evidence distinguish a causal crowding bottleneck from a representation pattern that is just correlated with increasing load?

Do the interior-slot failures persist when occurrences are explicitly numbered, so that ordinal-address construction is separated from value retrieval?

2 comments
AuthorsOpenReview ↗

Response to Reviewer 8y5A 1/2

Thank you very much for your careful and helpful review. We are glad that you found the query-token amplification and its controls convincing, and that you saw the interface result across 19 chat models as an important point for memory evaluation. Each of your three questions pointed to an experiment we had not yet run, so we ran all three, along with several follow-up studies, and revised the paper. All new and changed text is shown in blue, and the new experiments are collected in Appendix N. We first summarize the main results and then answer each question in turn.

  • The reverse direction now holds when we remove heads only at the final query token. This moves GPT-2-large and two chat models into the binding-limited regime and sharply lowers accuracy at high load in two more, while everything in the context stays exactly the same.
  • We can now point to the failure itself. At the query token, heads that favor the latest value compete with heads that favor older values, and removing four of the latter doubles accuracy at u = 16 in three of the five models we tested.
  • When each update carries its number, seven chat models read the middle value at 0.84 or higher at u = 32. The value is stored, and what these models lack is a way to find the n-th mention, exactly as you suspected.
  • The crowding of the stored order code is now presented as a description only, because two new direct tests do not support a causal role.

Question 1, the reverse conversion with an edit at the query token only

Yes, the reverse conversion holds with an edit at the query token alone. We repeated the GPT-2-large experiment and removed the same six copy heads only at the final query token. Under causal attention no earlier token can see that position, and we also checked that the largest change in any context activation, at any layer, is exactly zero.

GPT-2-large, 250 prompts per loadRegimeAccuracy at u = 32Interference endurance
Intact modelrecency-copy0.160.28
Six heads removed at all positions (original figure)binding-limited0.080.07
Same six heads removed at the query token onlybinding-limited0.080.14
Random heads removed at the query token only (15 sets)recency-copy in all 150.14 to 0.180.26 to 0.30

Interference endurance is the area under the accuracy curve above chance. The query-only edit cuts it in half, which is about two-thirds of the drop from the original edit, as we would expect when the context is left untouched. Because this result sits close to the regime boundary, we repeated it on 1,000 new prompts per load. The edit again halves endurance and gives 0.07 at u = 32 (95 percent interval 0.06 to 0.09), while all five random sets stay recency-copy. The same six heads also leave accuracy on keys that appear only once unchanged, so they carry the route that reads the latest of several updates. On four more base models the query-only edit removes 42 to 91 percent of the endurance lost under the original edit (Appendix N.2).

Chat models show the same effect under their own chat templates (Appendix N.8). We selected heads on a separate set of prompts and removed them only at the final prompt token. In gemma-2-2B-it, which is far from the regime boundary, removing four heads lowers accuracy at u = 32 from 0.82 to 0.30, and in Qwen3-4B it falls from 0.58 to 0.09. Nine sets of four to sixteen random heads keep these two models at 0.76 or above and at 0.49 or above. When we instead remove the heads that hurt short streams the most, gemma-2-2B-it stays at 0.78 for u = 32, so the selected heads matter for long streams in particular. Qwen3-1.7B and gemma-3-4B-it, which are closer to the boundary under their templates, become binding-limited when two and four heads are removed.

In binding-limited models the same heads also move one model in both directions. Under completion, the four copy heads of Qwen3-1.7B raise accuracy at u = 4 from 0.34 to 0.80 when we amplify them at the final query token, and lower it to 0.18 when we remove them there. The abstract, the contributions, Table 1, Section 4.7, the figure captions, and a new table in Appendix N.1 now state clearly which result uses which edit.

AuthorsOpenReview ↗

Response to Reviewer 8y5A 2/2

Question 2, crowding as a cause or as something that only goes along with load

We ran two direct tests of the crowding account. The first test holds the load fixed, so that load cannot explain why one prompt succeeds and another fails. In Qwen3-1.7B, the geometry of each prompt's stored order code does not predict whether the model succeeds on new prompts, with AUROC of 0.48, 0.47, and 0.51 at u = 4, 8, and 16. The second test changes the geometry of the stored code directly. Spreading the order codes apart by a factor of 2 or 4, at one of five layers at a time, leaves accuracy at baseline (Appendix N.4). Together with the compression result already in the paper, this does not support a causal role for crowding, so the paper now presents crowding as a description of the stored code throughout.

The revision also shows where models with the same stored code differ. Once attention sinks are controlled for, every model writes the same kind of order code, so the geometry cannot explain why models fall into different regimes, and the difference lies in retrieval (Appendix N.6). We measured how strongly each head pushes the answer toward each value at the query token. In Qwen3-1.7B the copy heads favor the latest value by a small margin, while four other heads favor older values at every load. Removing these four heads at the query token raises accuracy at every load, from 0.24 to 0.51 at u = 8 and from 0.05 to 0.18 at u = 32. Random heads removed in the same way leave accuracy within 0.02 of baseline on average. We then automated this test for five models from the Qwen, Pythia, and Llama families, selecting heads on one set of prompts and testing them on new prompts. In Qwen3-0.6B, Qwen3-1.7B, and Pythia-2.8B, removing four of these heads doubles accuracy at u = 16, and the gain holds with interleaved distractors and under free generation. In Llama-3.1-8B the effect is small, and in Qwen3-4B it is not reliable. We also checked which older value wins when the model is wrong. The errors are spread almost evenly across the older values, so this competition sets how far above chance the curve sits, while the decline with load comes from a growing number of similar competitors.

Question 3, middle values with explicitly numbered updates

Your suspicion was right, and the experiment shows it clearly (Appendix N.3). We kept the keys, the values, and their order the same and changed only how the position is given. In the numbered condition each line carries its number after the key, and the query keeps the original wording, so the model no longer needs to count. At u = 32 and without numbers, the middle value is at chance in six of seven chat models (0.01 to 0.04, with chance at 0.03) and reaches 0.28 in gemma-4-12B. With a number on each line, all seven read it at 0.84 to 1.00. Two other wordings of the question give the same picture in four models, and Table 1 now shows the numbered condition in a new column. Because the numbered condition keeps the exact wording of the instruction, a misreading of the instruction cannot explain the original failure. The failure lies in finding the n-th of several unlabeled mentions, and Section 4.5 and Appendix D now say so.

The middle value can also be read without adding any tokens. On the original stream without numbers, a question that names the value just before the target retrieves the middle value above chance in every model (0.17 to 1.00 at u = 32). A question that names the value just after the target works less well and stays near chance in four models. This is what we would expect if the model copies forward, from a token to whatever followed it earlier in the stream. It shows on one and the same stream that the value is stored, while the way to find it by position is missing. gemma-4-31B is the one model in our set that finds it by position, and it is also the model with the query-dependent pointer head of Section 4.6.

Summary

Thank you again for these excellent questions, which made the paper clearer and more convincing. The revision adds query-only reverse edits in GPT-2-large and in chat models, a causal test of the failure that holds strongly in three of five models, two direct tests of the crowding account, and a numbered-update experiment. We hope these results address your concerns, and we would be grateful if you would take them into account in your final assessment. We are happy to answer any further questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.