Enhancing Multi-Agent Communication through Attention Steering with Context Relevance

LLM-based multi-agent systems often accumulate long communication histories. As a result, relevant information is increasingly diluted by irrelevant context, leading to degraded performance. Existing context management methods compress or prune the history, which may discard subtle but critical information. Attention steering preserves the full history, but identifying relevant context remains challenging as multi-agent communication evolves at every reasoning step. To address this, we introduce Agent-Radar, an attention steering method that leverages the inherent structure of multi-agent communication to dynamically guide agents’ attention toward relevant context. Across five benchmarks, Agent-Radar outperforms state-of-the-art methods, yielding average gains of 4.17 absolute points over the strongest baseline. Furthermore, our analysis shows that Agent-Radar remains effective and robust as the number of agents and interaction rounds increases. Finally, the ablations show that the core components in Agent-Radar are crucial to performance.