Multi-Query Associative Correction: Enriching the Readout Interface for History-Conditioned Low-Rank Adaptation

Standard LoRA learns a fixed low-rank correction map, while results from explicitly partitioned adapters suggest that how a fixed adaptation budget is organized can matter. We ask whether such differentiation can instead be learned from a common training stream, without predefined subset identities. We study this through a readout-centric view of low-rank adaptation and identify an allocation–expression gap: allocating multiple or wider readouts does not by itself produce differentiated corrections. Such differentiation requires data-dependent variation to occupy the exposed coordinates and remain distinct after decoding. This analysis highlights the joint importance of data-dependent evidence exposure and differentiated decoding. Under a compact recurrent-state design, these motivate multiple conditioned reads from a shared state and group-specific decoding, while previous-state retrieval additionally motivates a direct path for current evidence. MQ-ACM instantiates this design with one shared associative state, while joint task-gradient initialization provides a coordinated starting geometry for the paired input–output coordinates. On our main Llama-3.1-8B generation setting, the default $r=8$, $K=4$ configuration improves rank-8 LoRA by $0.50$ on MT-Bench, $5.96$ points on GSM8K, and $6.30$ points on HumanEval. A complementary commonsense evaluation under a distinct adaptation setting provides additional validation. Effective-rank and alignment diagnostics characterize realized correction structure; interventions support functional use of associative and query-specific decoded terms. These results support readout organization as a distinct design dimension for history-conditioned low-rank adaptation, with additional computation despite fixed state storage.