Stable MLP Pools with Task-Dependent Communication Geometry in Context-Memory Arbitration

How language models arbitrate between parametric memory and prompt context remains unclear, since ordinary factual question answering reveals the answer but not its source. We use controlled context-memory conflicts, in which the prompt and the model's memory support different answers, to test whether this choice is associated with low-dimensional, reusable MLP communication resources. In the primary Mistral experiments, we find that conflict-related MLP writes are captured by low-dimensional source–target paths that map to stable, reusable neuron pools. We further find that the same high-loading neurons recur under memory-required and context-required demands, while their low-dimensional write geometry changes with the task. Matched-control interventions reveal a path-dependent functional pattern, with both task-defined pools showing larger effects under context-required prompts on the first two retained Mistral paths, whereas the final path favors memory-required prompts. Applying the same analysis to Llama, Qwen, as well as to model-specific PopQA-derived subsets, we recover model-specific low-dimensional paths and overlapping neuron pools. On each model's earliest retained path, both pool definitions again show larger effects under context-required prompts, while later paths vary by model. Together, these results identify a population-level organization in which largely shared MLP resources are reused across task requirements, but their communication geometry and local functional effects depend on where and how they are engaged. This suggests that knowledge-source arbitration is implemented through the flexible reconfiguration of a shared population rather than a dedicated conflict circuit, offering a mechanistic handle for interpreting and designing efficient, interpretable models.