LatentHarness: Adaptive Latent Context Management for Multi-Agent Systems

Latent communication enables LLM agents in multi-agent systems (MAS) to collaborate by exchanging continuous representations. Methods for organizing and delivering these representations fall into two categories: (1) Training-based methods learn message compression or information allocation but require additional training data and parameter optimization. (2) Training-free methods often deliver complete messages or select representations from predefined positions, limiting adaptation to receiver needs. To address this limitation, we introduce LatentHarness, a training-free framework for adaptive latent context management with two components: i) state selection, which adaptively selects hidden-state positions across upstream responses by balancing relevance to the question and receiver role against redundancy; and ii) context construction, which assembles short spans of consecutive hidden states ending at these positions in upstream response and token order. Experiments on four scientific benchmarks with three LLMs show accuracy gains in most settings. On GPQA Diamond, LatentHarness improves Qwen3.5-4B accuracy from 63.77% to 71.74% while reducing textual token volume by 75.57% relative to passing all preceding responses as text. The gains also extend to hierarchical MAS.