IF-MAP: Influence-Function-Inspired Agent Prioritization in Multi-Agent Systems

Improving every agent in an LLM-based multi-agent system can be prohibitively costly, making it important to identify which agent should be prioritized to achieve the greatest system-level gain with limited resources. Failure attribution can in- form this decision, but an agent’s responsibility for failures does not directly re- veal the system-level gains achievable by improving it. We therefore introduce IF-MAP, an influence-function-inspired method for identifying agents with high potential for system-level improvement. For each agent, IF-MAP measures end- to-end loss changes under controlled semantic prompt perturbations, fits a regular- ized local response model in a low-dimensional prompt-embedding subspace, and uses the estimated response norm as a priority score. This black-box procedure requires no access to model parameters or gradients and produces agent priorities that can guide subsequent improvement effort. We evaluate IF-MAP across six multi-agent workflows spanning code generation, question answering, and math- ematical reasoning, using the same priorities for prompt optimization and model upgrades. Optimizing the selected agents’ prompts achieves a macro-average gain of 3.81 percentage points, compared with 1.85 for uniform agent selection. Across five target models, upgrading the selected agents yields macro-average gains of 7.62–16.42 percentage points, compared with 3.81–8.62 for uniform agent selec- tion. Despite its probing overhead, IF-MAP reduces Qwen3-8B and DeepSeek- V3.2 token use by 74.2% and 35.2%, respectively, relative to optimizing every agent, and reduces target-model tokens by 63.6–64.9% relative to upgrading ev- ery agent. These results support the use of IF-MAP to identify promising agents and direct improvement effort toward greater end-to-end performance gains.