acceptodds
Under review as a conference paper at ICLR 2027

Stable MLP Pools with Task-Dependent Communication Geometry in Context-Memory Arbitration

Abstract

How language models arbitrate between parametric memory and prompt context remains unclear, since ordinary factual question answering reveals the answer but not its source. We use controlled context-memory conflicts, in which the prompt and the model's memory support different answers, to test whether this choice is associated with low-dimensional, reusable MLP communication resources. In the primary Mistral experiments, we find that conflict-related MLP writes are captured by low-dimensional source–target paths that map to stable, reusable neuron pools. We further find that the same high-loading neurons recur under memory-required and context-required demands, while their low-dimensional write geometry changes with the task. Matched-control interventions reveal a path-dependent functional pattern, with both task-defined pools showing larger effects under context-required prompts on the first two retained Mistral paths, whereas the final path favors memory-required prompts. Applying the same analysis to Llama, Qwen, as well as to model-specific PopQA-derived subsets, we recover model-specific low-dimensional paths and overlapping neuron pools. On each model's earliest retained path, both pool definitions again show larger effects under context-required prompts, while later paths vary by model. Together, these results identify a population-level organization in which largely shared MLP resources are reused across task requirements, but their communication geometry and local functional effects depend on where and how they are engaged. This suggests that knowledge-source arbitration is implemented through the flexible reconfiguration of a shared population rather than a dedicated conflict circuit, offering a mechanistic handle for interpreting and designing efficient, interpretable models.

Primary area:
interpretability and explainable AI
Keywords:
mechanistic interpretability, large language models, knowledge conflicts, parametric memory, MLP neurons, communication subspaces, causal interventions

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_rnxQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates the MLP circuits that are important for the model to decide whether to use in-weight or in-context information for factual question answering. The paper finds shared MLP neurons and shows that their representational geometry is different when the task requirement is different.

Critical strengths
  • The paper is relatively clearly written
  • Most of the figures are high-quality
Critical weaknesses
  • Some terms are not well defined. For example, what is a two-task eligibility test? What is the validation rule? What is validation write-energy?

  • The author may have over-claimed or over-interpreted the results. The author concludes that the model does not require a separate population for each knowledge source. However, the authors state that “only 65-68% of each task specific pool belongs to the shared set”. If only half of the MLP neurons are not shared, then the non-shared neurons could be dedicated to each knowledge source. Also, the paper only examines MLP neurons, which does not rule out the possibility that there could exist dedicated attention heads for each knowledge source.

  • The paper does not show the described geometry has causal effect.

Questions for authors
  • Can the authors define all the terms (or point to their definitions) introduced in the paper?
  • Can the authors commend on the possibility of over-interpretation mentioned in the weakness?
1 comment

Thank you for identifying three issues: terms that are not defined, a risk of over-interpretation, and no causal evidence for the described geometry. We accept all three. We will add a consolidated definitions table and have already narrowed the exclusivity claim and tested the geometry directly. Your third point in particular moved us from description to intervention.

1. Definitions (W1, Reviewer Q1)

These terms already have scattered definitions in §3.2 and Appendix A.1, in the revision we will add a consolidated 11-term table in Appendix A.1 for easier reference. Here are the three you named.

The two-task eligibility test requires both tasks to exceed the 95th percentile of R²/cosine against 50 random source bases, and the 95th percentile of coordinate correlation against 50 paired shuffles.

The validation rule keeps, for each source layer, the target layer with the highest mean Validation R² across both tasks, no Test causal result is used.

Validation write energy is the squared norm of the centered Validation shared write. The reconstruction rate is the fraction captured by the basis. The remaining terms (conflict effect, U, communication load, loading-defined pool, shared/context-only/memory-only, reconstruction drop, w/τ/s, path-dependent) will go into the same table; non-standard usages such as path-dependent will get a one-sentence definition.

2. Shared overlap, non-shared members, and attention (W2, Reviewer Q2)

Sharing 65–68% of each pool does not mean there is no specialization. We took the exact difference sets of the two Top-512 pools and ablated them under both tasks, against same-layer, same-size random pools.

Table 1 | Difference-set ablations. L19, 109 Test facts; s = own-task τ − other-task τ; shared-core context τ is shown for reference.

Sub-poolSelectivity s [95% CI]
context-only (181 neurons)+0.565 [0.508, 0.627]
memory-only (181 neurons)+0.426 [0.367, 0.491]
shared-core (331 neurons)context τ = 1.000

Non-shared members do carry weak but reliable selectivity, far smaller than shared-core, at L20 the values are +0.142 / +0.288, and by L24 they approach zero. The accurate statement is therefore "shared population first, weak specialization alongside," and we no longer make the claim that no dedicated components exist.

You also noted that we examined only MLP neurons. We added a head screen across all 32 layers and an L19 ablation: top-8 heads overlap 6.84/8 on average between tasks (minimum 4/8), and top-16 overlap 14.56/16 (minimum 12/16); at L19 the top-8 sets are identical, and top-16 overlap 13/16. Ablating either set hurts the context task more (context-top16 1.374, memory-top16 3.243), with no task-specific dissociation; generation switches are 3.7% and 11.6%, against 0.3% for random. We therefore find no evidence for dedicated heads.

3. Causal role of the geometry (W3)

You say the paper does not show that the described geometry has causal effect. We agree: the discussion already states that we do not establish that the geometry itself causes the intervention effects, if the framing suggested a causal claim, we will clarify it in the revision. To answer directly, we removed G by projection and by a norm-preserving swap, and we removed the 16-dimensional U.

Table 2 | Causal checks for G and U. Test 109, Validation 70, 3 templates each; swap = mirror reflection across the two task bases (norm preserved); τ>0 = stronger weakening than random.

CheckResult
G Test0.171 / 0.342
G Validation−0.049 / −0.020
G swap+0.346 / −0.206 (both tasks toward memory)
G prediction vs randomAUC 0.922 vs 0.914
U projection1.685 (matched 1.725) / 0.069

G is weak on Test and does not replicate on Validation. Under swap it is asymmetric, and it is no better than a random projection at predicting choice. It flips no answers. G is therefore an organization pattern, not a task-specific causal mechanism. Removing the 16-dimensional U, by contrast, has a strong context-specific effect. Details of the direct interventions are in our response to ci2m (Sections 1–2). We will stop calling the geometry a causal mechanism and keep only the shared population and its task-related organization, the functional role of U is reported separately.

Reviewer_bWn8OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates how a model internally decides between information provided in a prompt and knowledge stored in its own memory when the two conflict. It finds that largely the same set of neurons is involved in both types of tasks, but the "writing direction" (in a geometric sense) of these neurons shifts depending on the task. Furthermore, it reveals that the intervention effect is path-dependent: earlier paths tend to favor "responding based on context," while later paths (at least in the case of the Mistral model) tend to favor "responding based on internal memory."

Critical strengths

this paper is interesting for LLM research, especially for the time in the agentic era. If we have a better understanding of the Context–Memory Conflict, we can have better solution to design the trajectory to teach the LLM agent when to search and when not to search.

Critical weaknesses

The study is confined to pre-agentic models and a single-turn short-answer format, which limits the relevance of its conclusions to current LLMs. All three models (Mistral-7B-Instruct-v0.3, Llama-3-8B-Instruct, and Qwen2.5-7B-Instruct) are 7–8B instruction-tuned models from a generation trained before the current emphasis on agentic, tool-using behavior. Recent agentic models are trained extensively to acquire information through tool calls (search, file reads, code execution) and to revise their beliefs from the returned evidence during a trajectory. Arbitration between parametric memory and context is central to how such models work, and their training may have changed it substantially. For example, they may defer to in-context evidence more readily, or condition this choice more on the source of the evidence (e.g., a tool output versus a user assertion). The neuron pools, path geometry, and especially the direction of the path-dependent effects reported here (early paths favoring context-required prompts, later paths varying by model) may therefore not transfer to these models. Notably, the later-path effects are already inconsistent across the three closely related models studied.

The framework assumes dense MLPs, and its conclusions may not hold for Mixture-of-Experts models. The analysis treats each layer’s MLP neurons as a fixed population that is computed for every token. The conflict effect in Eq. (1), the loading score in Eq. (2), and the Jaccard membership comparison all rely on this property. In Mixture-of-Experts (MoE) FFNs, which many current open-weight and agentic LLMs use (e.g., DeepSeek-V3, Mixtral, the MoE variants of Qwen3), each token activates only a few experts. A neuron’s conflict effect may then reflect a routing change between conditions rather than a change in what it writes, and the additive-contrast assumption no longer holds. The loading score would also need to account for router gate weights and the separation between shared and routed experts. In addition, pool interventions may change routing in later layers. More importantly, the central conclusion that arbitration is implemented by “the flexible reconfiguration of a shared population rather than a dedicated conflict circuit” may not transfer: an MoE model could implement arbitration partly through routing, sending memory-required and context-required prompts to different experts, which would be closer to the dedicated-circuit account the paper argues against. The paper does not discuss this. I suggest that the authors explicitly restrict their claims to dense-FFN Transformers and discuss how the framework would extend to MoE. A natural test case is Mixtral-8x7B, the MoE counterpart of the paper’s primary Mistral model.

Questions for authors

please refer to the weakness section

1 comment

Thank you for recognizing the point of this work: understanding context–memory conflict well enough to help an agent decide when to search is also our long-term goal. You point to two limits: the models are pre-agentic and single-turn, and the framework assumes dense MLPs. We accept both and will state the verified scope explicitly in the revision. In response to your Weakness section, we ran three small-scale checks: a tool-trained dense model (Qwen3-8B), a small MoE (OLMoE), and Mixtral-8x7B, the model you named. Your comments also point to where we should go next.

1. Tool training and source adoption (W1)

You raised the concern that tool training could change how models adopt evidence. On Qwen3-8B, we compared three presentations of the same evidence: a user message, a tool return, and a label-only condition that keeps the tool channel but changes only the declared source ("User statement" vs "Tool output"). We used 109 Test facts × 3 tasks × 3 templates, with context−memory delta and 32-token generation as readouts. The user and tool conditions differ in both channel and format; label-only holds the channel fixed, so the two contrasts separate presentation from declared source.

Table 1 | Source presentation in Qwen3-8B. 109 Test facts × 3 tasks × 3 templates; delta = context−memory (logits); flips = fraction of generated answer categories that changed; label-only uses the same channel as tool and changes only the declared source.

ComparisonContext taskMemory taskGeneration flips
tool − user−10.94 [−12.62, −9.20]−3.74 [−4.64, −2.93]12.8%
label-only − tool−10.00 [−10.90, −9.06]+5.58 [+4.88, +6.35]66.1%

Generation adoption on the context task falls from 0.991 under user to 0.917 under tool and 0.578 under label-only. So the direct answer is yes: in this tool-trained model, source adoption changes—both presentation and declared source matter. Transferring the mechanism would require a separate analysis, which we did not run here; we therefore tighten the claim to dense models and do not claim transfer of pools, paths, or geometry.

2. MoE and routing (W2)

We agree that the framework assumes dense FFNs, and that MoE breaks three assumptions: the write is a gated sum, neuron identity becomes (layer, expert, neuron), and pool ablation may change downstream routing. We ran a routing diagnostic on two MoEs. We first verified that summing expert contributions reconstructs the actual output (R² > 0.99), then decomposed the conflict effect into a routing term and a within-expert term.

Table 2 | Routing diagnostics for two MoEs. 60 Test facts × 6 conditions × 3 templates; Jaccard = overlap of the task-specific expert sets; routing share from the conflict-effect decomposition; Mixtral is bf16 on one 96GB GPU, peak about 94.5GB.

MetricOLMoE-1B-7BMixtral-8x7B
Structure16 layers / 64 experts / top-832 layers / 8 experts / top-2
Expert Jaccard0.9400.857
Routing share24.1%23.8%
Late-layer task Jaccard— (overall 0.940)0.723
Reconstruction R²0.999970.99984

Both MoEs are write-dominant: about 76% of the condition difference comes from within-expert writing—mostly what the same expert writes, not which expert is selected. Mixtral shows task-specific routing in late layers (18–31; Jaccard 0.723, routing 24.9%), while early layers are at 0.961 / 10.9%; OLMoE is more shared overall (0.940), with collapsed routing (11.5/64). MoE is therefore heterogeneous: Mixtral is write-dominant with late routing, OLMoE is collapsed routing with write dominance. We restrict the core claim and its verified scope to dense-FFN models; for MoE we report only this diagnostic and do not claim mechanism transfer. We also did not run pool ablation or a gate-weighted loading score in MoE, so whether pool interventions change downstream routing remains open.

One possible reading is that arbitration looks more like writing than switching; Mixtral's late routing makes it write-dominant rather than write-only.

Reviewer_mHgVOpenReview ↗
Rating
1
Confidence
3
Informativeness
1(−1)
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies how language models, mechanistically, resolve conflicts between their parametric knowledge and in-context knowledge. Motivated by empirical results from the field of neuroscience, the authors hypothesize that communication subspaces between neuron populations (specifically low-rank subspaces of a source MLP layer's output that predict a later target MLP layer's input) can provide useful insights into this behavior. Their main experiments are on Mistral-7B-Instruct, using QA prompts about country facts where each prompt contains an in-context reference that agrees with, contradicts, or is neutral to the model's own memory, along with an instruction to either follow the reference or rely on its own memory. For a given activation, they define a “conflict effect” as its value under the contradicting reference minus its average under the other two. For a pair of MLP layers, they use PCA and SVD on the source's output and target's input conflict effects to obtain a d=16d=16 dimensional communication subspace, and select layer pairs where a ridge regression from the projected source conflict effects predicts the target's with higher held-out R2R^2 than a random subspace control. Each retained subspace is mapped back to the source's neurons, and the top-512 neurons under the two instruction settings form a “pool." They zero-ablate these pools on held-out facts to measure how much the model's preference for the instructed answer drops relative to random pools of the same size. They find that the two pools overlap substantially and that ablating either reduces the instructed-answer preference more than the control, but that the pools are not specific to the setting they were identified for: ablating either pool hurts reference-following more at earlier layer pairs and memory-following more at the last. The authors frame this as a “path-dependent” effect, and report that the earliest-layer-pair result replicates on Llama-3-8B, Qwen2.5-7B, and PopQA-derived data. They conclude that conflict resolution works by reconfiguring a shared neuron population rather than through a dedicated conflict circuit.

Critical strengths

The main strength of this paper is the idea of communication subspaces between different MLP layers, which as far as I know has not been studied much in the field of interpretability. This idea seems fairly general, and could also be used for studying mechanisms beyond knowledge-conflict behavior, which is useful. Of course, more evidence for the existence of such a low-rank subspace would need to be established first. I also appreciated the grounding of this framework in prior neuroscience work, although I felt that the authors did not clearly explain why this would be a useful lens to understand knowledge conflicts in LLMs (beyond simply stating that it would be).

Critical weaknesses

The paper's headline claim, that "knowledge-source arbitration is implemented through the flexible reconfiguration of a shared population rather than a dedicated conflict circuit," is not supported by the experiments. At a high level, it is unclear whether the paper is studying arbitration at all, since in both settings the model is explicitly instructed which source to use. It could be that the mechanism being interpreted is closer to instruction following.

The main confounder is that the score used to select neurons conflates two things, namely how strongly a neuron's activation changes under conflict and how strongly it writes into the communication subspace. It is not surprising or significant that ablating the most conflict-responsive neurons affects the answer more than ablating random neurons, even if the random neurons do not write into the communication subspace, so the ablation results do not show that the subspace is important. Similarly, since both instruction settings use the same references and questions, the most conflict-responsive neurons are likely to be the same under both settings, so the overlap between the two pools could come entirely from the conflict-effect term of the score rather than from the subspace. Comparing against pools selected by conflict-effect magnitude alone would address both these concerns.

Additionally, since the two pools share most of their neurons, it is unsurprising that they behave identically under ablation. The "path-dependent" effect the authors describe, i.e. that ablating either pool hurts context-following more at earlier layers and memory-following more at the last layer, could simply be because overriding the model's memory is harder than following it, since all facts are filtered so that the model's memory agrees with the ground truth. Since the shared-pool and ablation results are the paper's main evidence, without stronger controls they do not tell us anything specific about communication subspaces or knowledge conflicts.

A more faithful characterization of the results would be that some neurons activate selectively under knowledge conflicts and matter for the model's answer (i.e. ablating them shifts the model's response), which is already known from prior work. The only evidence for a communication subspace is that it predicts the later layer's conflict effects better than a random subspace does. This is purely correlational and does not meet the bar for the mechanistic claim that conflicts are resolved through a shared neuron population communicating via low-dimensional subspaces.

As a more minor (but salient) point, this paper was very difficult to follow. I had to re-read certain sections multiple times, skip ahead to other sections, and consult with an LLM in order to fully understand what was going on. For example, the key idea of a source-target MLP communication subspace was never defined, and many non-standard terms were used without context ("additive contrast assumption", "path-dependent functional pattern", "behavioral readout", "task-specific loadings", "loading-defined pools", etc...). While I listed this as a minor point, I think it actually contributes quite a bit to my recommendation. I would have no reason to doubt, a priori, the epistemic quality of the paper, but the lack of care given to the writing makes me question if the authors actually vetted their experimental results.

Questions for authors

Clarifications

  1. How were the candidate MLP layers (listed in Appendix A.3) selected?
  2. What are the baseline task-aligned margins under each instruction setting, and what fraction of test facts flip answers under targeted vs. random ablation?
  3. Table 9 lists controlled-dataset experiments for Llama and Qwen (440 facts) as cross-model replication, but Section 4.4 and Appendix only report PopQA-derived results. Were these run?
  4. Several targeted-minus-random effects are negative (e.g. both pools under memory-required prompts on L20→L24). How should these be interpreted?

Suggestions

  1. Please consider doing selective ablations using the context-only vs. memory-only portions of the neuron pools (i.e. excluding the shared neurons). It would be useful to know if these neurons are 1) significant compared to control pools and 2) more selective to their instruction setting.
  2. Would control pools matched on conflict-effect magnitude, or pools selected on conflict-effect magnitude alone, give similar results? This would help separate the effect of the subspace from the effect of selecting conflict-responsive neurons.
  3. Do the identified pools or subspaces predict, or causally affect, the model's answer under the free-resolution prompts, where the model is not told which source to use? This would strengthen the paper's claim about context-memory arbitration considerably.
  4. It would be interesting to intervene on the communication subspace directly (e.g. projecting the source MLP's output out of the 16 directions) rather than only zero-ablating neuron pools.
2 comments

Thank you for recognizing the core idea: linking MLP communication subspaces to interpretability, inspired by neuroscience, extends beyond knowledge conflict. We examined your Weakness and Question sections and ran an amplitude-matched 2×2, a diff-set ablation, a free-resolution test, and a direct subspace intervention.

1. Amplitude and alignment in the loading score (W2, S2)

The pool was not selected on activation alone. Communication load is conflict amplitude multiplied by write strength into U, and the top-512 pool uses that product (§3.2). To isolate alignment, we ran the amplitude-only control you suggested. At full amplitude, the amplitude-only pool reaches 0.913 on context, against 1.462 for λ; on memory, 0.187 against 0.439. With amplitude fixed, alignment adds +0.525 on context and +0.316 on memory, and replicates at L20 (+0.342 [0.215, 0.473]). λ is therefore not a single mechanism measure: amplitude recovers about 62%, alignment contributes the rest.

Table 1 | Amplitude vs. alignment. Test 109 × 3 templates; τ>0 = stronger weakening than random; a-only = amplitude alone; alignment increment = high- vs low-alignment at matched amplitude.

CheckContextMemory
λ pool1.4620.439
a-only pool0.9130.187
Alignment increment+0.525 [0.383, 0.664]+0.316 [0.203, 0.427]

2. Pool effects without a source instruction (W1, S3)

We removed the instruction but kept the reference, so the model had to resolve the conflict on its own. Removing the context pool shifted choice toward memory (−0.522 [−0.670, −0.348]). Removing the memory pool shifted choice toward context (+0.186 [0.041, 0.345]). Generation showed the same pattern. Answers switched in 8.9% of cases, against 0.6% for random, and 16.7% of context-leaning samples changed their answer. Changing only the source label from "authoritative" to "blog" also flipped the effect (interaction +1.18 [0.961, 1.398]). The pools weight the source itself, they do not merely follow an instruction.

3. Shared overlap, non-shared members, and task difficulty (W3, S1)

You are right that the pools share most members, and that the late reversal could reflect that overriding memory is harder. Difficulty explains part of it: memory baseline is 18.13 logits vs 12.28 for context, and effects concentrate on low-baseline facts; these facts were filtered so memory agrees with the ground truth, making memory harder to override. Yet the exact difference sets carry weak, reliable selectivity. At L19, context-only (181 neurons) has s=+0.565 [0.508, 0.627], memory-only (181) s=+0.426 [0.367, 0.491], and shared-core (331) reaches 1.000 on context. Both sets also exceed same-size random pools on their own tasks (τ 0.460 and 0.167); by L24 the effect is near zero. Sharing therefore dominates, with weak specialization alongside—path-dependent and concentrated on earlier paths.

4. Causal role of G and functional role of U (W4, S4)

You are right that predictive evidence alone does not meet the mechanistic bar. We intervened on the geometry directly. G is weak on Test (0.171 / 0.342), does not replicate on Validation (−0.049 / −0.020), pushes both tasks toward memory after swap (+0.346 / −0.206), and flips no answers. G coordinates reach AUC 0.922, no better than a random 32-dimensional projection at 0.914. Removing the 16-dimensional U, by contrast, has a strong context-specific effect (1.685; 1.725 after matching; memory 0.069). G is an organization pattern, not a causal mechanism; U is the strongest new causal evidence here. Details: ci2m response (Sections 1–2).

5. Candidate layers (Reviewer Q1)

Layers were not chosen by Test performance. We localized layers on Discovery (16–19 emerging, 20–24 integrating, 28–31 approaching the decision), fixed 19/20/24/31 and six forward pairs, then used Validation R² to select each source layer's target; no Test result entered. The full table is in Appendix A.3.

6. Llama and Qwen controlled experiments (Reviewer Q3)

Both were run. Controlled Llama retains 130 facts (101/29) with 43 Test; Qwen2.5 retains 166 (117/35) with 14 Test. They completed structural checks plus a light subpool test, not the full two-pool protocol used for Mistral; Section 4.4 and the appendix therefore report the complete two-pool results on PopQA. Table 9's "440 controlled facts" is the candidate pool, not the per-model N; we will replace it with the actual N and stage.

7. Baselines, flips, and negative effects (Reviewer Q2, Q4)

Baselines are 12.28 / 18.13 logits and no fact is near a tie; instructed-task flips are 0.2% for both target and random pools. Negative targeted-minus-random values occur mainly in low-alignment and diff-set pools. In your L20→L24 memory-required example, both pools weaken near zero or below, random more so, hence the negative difference. This is not reverse causality; we report absolute magnitudes and intervals.

I’ve read your response. Based on your new results, it seems my concerns haven't been resolved. For example, the amplitude-only pool accounts for most of the ablation effect, and targeted ablations are not any more significant than random ablations. You've also walked back some main claims, e.g. that the geometry is causal. Even in the response, there are still overclaims, e.g. "the pools weight the source; they do not merely follow an instruction" is based on pools that were selected using instructed prompts, with no amplitude-only control. I still think the paper would need a major rewrite, and the response did not give me confidence this would happen (e.g. your response appears almost entirely LLM-generated) so I'm keeping my score.

Reviewer_ci2mOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates how LLMs arbitrate between parametric memory (internal knowledge) and prompt context (external evidence). Using controlled context-memory conflicts—where a model's prompt and memory support conflicting answers—the authors examine whether knowledge-source selection relies on low-dimensional communication subspaces and reusable MLP neuron populations. Rather than requiring separate dedicated circuits for each source, the study reveals that models reuse a largely shared population of neurons whose communication geometry and functional roles dynamically reconfigure based on task requirements and layer depth.

Critical strengths
  1. This paper introduces principled, neuroscience-inspired concepts (low-dimensional communication subspaces and shared population coding) to demystify how LLMs arbitrate knowledge sources.
  2. Employs strict, multi-layered baselines—including random-source bases, pairing shuffles, and matched random-pool ablation controls—to rigorously separate true structural findings from artifacts.
  3. Replicates findings across diverse model architectures (Mistral, Llama, and Qwen)
Critical weaknesses
  1. The authors demonstrate that task demands alter the 16-dimensional write geometry of shared neuron pools, yet the causal interventions ablate entire 512-neuron pools. Can the authors clarify whether they attempted targeted interventions within the specific communication subspaces (e.g., steering or clamping along the task-specific bases)?
  2. The paper successfully demonstrates that shared MLP pools exhibit task-dependent write geometries, but the causal interventions fail to target this geometry directly. By ablating entire 512-neuron pools rather than manipulating specific geometric subspaces, the experiments prove that these neurons are functionally important, but they do not prove that the geometric shift itself is the causal mechanism driving knowledge-source arbitration.
Questions for authors

On the final retained path (L24$\rightarrow$L31) in the primary Mistral setting, the matched-control causal effects unexpectedly reverse, favoring memory-required computation for both pool definitions. Does this late-stage reversal indicate that deeper layers abandon the early-path context bias, or could it be an artifact of how answer-score margins are read out from the final residual stream?

1 comment

Thank you for your review. We ran experiments for Weakness 1, Weakness 2, and the L24→L31 reversal question. Your comments helped us separate causal objects from organization patterns and narrow our claims to a clearer, more reasonable scope.

Our experimental logic is as follows: We fitted a 16-dimensional communication subspace U and then selected a pool of the top 512 neurons ranked by the strength with which each neuron writes into U (331 neurons overlap between the two tasks). Finally, we fitted task-specific 16-dimensional write directions G on this pool. Our claim is not that G is a causal mechanism. As stated in the Discussion, we do not claim to establish that this geometry causes the intervention effects, and we apologize if our wording suggested otherwise. The three sections correspond to W1, W2, and the reviewer's Q1, with each section heading indicating the corresponding item.

1. Targeted interventions on the shared-pool geometry (G) and the communication subspace (U) (W1)

Table 1 | Targeted interventions on G and U. Test set: 109 facts × 3 templates; intervention = removing 16 dimensions from the write/output; control = same-layer random 16 dimensions (U additionally matched on removed energy); τ>0 means stronger weakening of the task-required answer than the random control; full definitions in §3.2.

ObjectInterventionTest τAnswer flips
G (context basis)remove 16 dims0.1710
G (memory basis)remove 16 dims0.3420
U (context)remove 16 dims1.685 (matched 1.725)0.9%
U (memory)remove 16 dims0.0690%

The two G interventions are weak (0.171 / 0.342) and produce no flips. Among the additional U interventions, only the context one is strong (1.685; 1.725 after matching), the memory one is weak (0.069). G is therefore an organization pattern, whereas U is a functional direction.

2. Causal role of the geometry G (W2)

You are right that whole-pool ablation cannot establish that the geometry is the mechanism. The weak Test effect is reported in Section 1, here we report only Validation, swap, and prediction.

Table 2 | Causal checks for G. Validation: 70 facts, 3 templates (Test in Section 1); swap = mirror reflection of the shared write across the two task bases (norm preserved); τ>0 means stronger weakening of the task-required answer than the random control; flips = fraction of changed answer categories.

CheckResult
Cross-data replicationValidation −0.049 / −0.020
Direction (swap)+0.346 / −0.206
Behavior (flips)0

The three results are consistent. The same intervention does not replicate on Validation. If G were a task-symmetric causal code, swap should push the context task toward memory and the memory task toward context; instead, both tasks are pushed toward memory (+0.346 / −0.206), so the direction is asymmetric. The flip rate is 0, so G does not change answers. Thus G is an organization pattern, not a causal mechanism. Your concern directly motivated these interventions, and we have downgraded G from "mechanism" to "organization."

3. Late-layer reversal: functional handoff or readout scale? (Reviewer Q1)

To answer your two questions, we ran three checks. First, the reversal is below 1% of the baseline (12.28 / 18.13 logits). Second, answer flips are 0.2% for both the target and random pools, so there is no behavioral difference. Third, the readout and intervention checks do not change this picture. The full-answer readout is nearly identical to the first-token readout (1.465 vs 1.423). Zeroing all prompt positions raises the margin effect by 20–67%, but the flip rate remains 0–0.3%.

The three results point consistently to a graded, readout-scale effect rather than a "deep abandonment of the early context bias." For the first question, we find no evidence of a functional handoff. For the second, your "readout artifact" hypothesis is, in our view, the explanation more consistent with the data, but with final-residual readouts alone we cannot fully exclude a real late-layer contribution. A layer-wise readout would require a substantial and relatively independent mechanistic analysis, which we will discuss in detail in future work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.