Safe to Omit, Worth Omitting? Keeping the Answer and Cutting Latency When an LLM Agent Skips Another's Message
Abstract
In collaborative multi-agent systems, a receiver agent often waits for a message from a helper agent, which can be either natural-language text or its latent KV cache. While this communication can improve performance, it also adds latency. We investigate when a receiver agent can skip the helper agent's message safely (the final answer rarely changes) and beneficially (the time saved exceeds the cost of deciding). We propose a dynamic skipping mechanism that reads the receiver agent's confidence without the message and applies a finite-sample statistical test. This test certifies a confidence threshold at which, with high probability, at most 5% of skipped answers change; if no threshold passes, the system keeps communicating. We evaluate our mechanism on five benchmarks: three multiple-choice benchmarks with three model pairs each, plus an extractive (SQuAD) and a math (GSM8K) benchmark. Three main insights emerge. First, oracle headroom, a common metric for routing among paths, is weak evidence that messages help. Control inputs without answer-specific content, matched to change answers as often as real messages, give comparable headroom and reproduce a median 76% of the oracle's gain over the receiver agent. Whether text messages beat these controls with statistical significance depends on how they are matched. Second, whether a message can be skipped depends on how often and how predictably it alters the receiver agent's answer, not on its overall accuracy. Third, safety does not determine efficiency: at batch size one, skipping text messages saves hundreds of milliseconds per query, whereas skipping latent KV cache saves far less. Certification extends to SQuAD, which a label-free rule chose from eight candidate settings, while on GSM8K no threshold passes. Finally, in eleven tests on unseen questions, every certified policy we could test kept its change rate's 95% upper bound below 5%, though only four tests had a pass criterion set in advance, and the sealed KV-cache result holds only under our answer parser. The code is available at: https://anonymous.4open.science/r/skip-message-cert.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.