Knowledge-Aware Attention Steering and Redistribution

Large language models often fail to faithfully utilize the knowledge provided in context, producing answers that contradict the given evidence. To bridge this gap, we systematically investigate the attention allocation across different functional input regions and its causal contribution to information propagation and final predictions via information-flow analysis and extensive attention interventions. We trace this unfaithfulness to a striking mechanistic root: attention heavily goes where information is not. Through large-scale analysis, we reveal three core mechanistic insights: (1) Attention–Saliency Imbalance. The system region absorbs over 60% of attention (up to 80% in LLaMA) yet contributes marginally to prediction, whereas the knowledge region drives information fusion and prediction in mid-to-late layers with merely 16% of attention. (2) Redundant Attention Bias. A substantial portion of system attention functions as a structural bias rather than informative computation, and scaling it down is harmless and even yields performance gains. (3) Attentional Alignment. Both causal interventions and supervised fine-tuning trajectories confirm that elevating the knowledge attention in mid-to-late layers causally improves knowledge utilization and accuracy. Motivated by these findings, we propose Knowledge-Aware Attention Steering and Redistribution (KASR). During inference, KASR dynamically intercepts the redundant attention from the system region at critical layers and redistributes it to knowledge tokens, thereby amplifying the latent factual signals. KASR is training-free and serves as a plug-and-play inference approach. Extensive experiments on 10 models across 5 knowledge-intensive benchmarks demonstrate that KASR yields substantial and consistent improvements, raising accuracy by 4.18% on average with negligible overhead.