What Do Agents Miss When They Decide? Exploring Epistemic Gaps in LLM-based Multi-Agent Systems through Sibling Probing
Abstract
LLM-based multi-agent systems distribute decision making across agents, such as planning and execution. Recent research has shown that failures can arise when a decision lacks necessary information support despite correct execution, a phenomenon known as an epistemic gap. Existing approaches mainly use retrospective feedback from past interactions to calibrate subsequent decisions. However, such feedback offers limited insight into which decision contains an epistemic gap, as decisions are made top-down with obtained information, while later information emerges bottom-up and outcomes entangle multiple decisions. We formulate decision-level epistemic exploration in MAS to acquire information that resolves such gaps. Two challenges arise: epistemic gaps can accumulate across hierarchical decision levels, making it unclear which decision level to explore; explored trajectories entangle information across decision levels, making decision-specific information requirements difficult to extract. We design HIVE, which starts from an observed failure and probes alternatives in a bottom-up order to mitigate the effects of epistemic gap accumulation, while constructing explored trajectories to disentangle role-specific information requirements through trajectory comparisons. Based on these comparisons, HIVE identifies why current knowledge fails to support the decision and updates role-specific knowledge online within the current task. Across six benchmarks, HIVE improves task success by 4.46% over the compared baselines while reducing average trajectory length by 23.67%.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.