acceptodds
Under review as a conference paper at ICLR 2027

Compositional Coordination Confidence: Anytime-Valid Monitoring for Multi-Agent LLM Trajectories

Abstract

Multi-agent LLM systems can carry an erroneous intermediate step many turns before a final output is inspected. We study online monitoring under an operational requirement: bound, uniformly over time, the probability of halting a trajectory before any coordination fault occurs. Compositional Coordination Confidence (CCC) calibrates per-turn evidence rather than a decision threshold: an LLM judge's rubric score becomes a role-conditional factor with null mean at most one, so products and prespecified mixtures of restarted or tempered products are test supermartingales, and one factor calibration, fit on fault-free or pre-fault turns, yields anytime-valid false-alarm control for each prespecified member of the family without refitting. Calibrating at the factor makes the guarantee's premise, a history-conditional moment restriction, a per-turn quantity that can be tested directly. On AFTraj (math, coding, agentic) and on Who&When, the primary restarted mixture has observed false-alarm rates at or below 3.5% on all five evaluation subsets at nominal 5%, with restricted post-onset detection of 25.9% on Who&When HC and 6.9% and 2.9% on AFTraj math and agentic. The premise test does not reject on coding or agentic; on math it finds a conditional factor of 1.35 under one judge and 1.24 under a second, with cluster-bootstrap intervals excluding 1 in both cases, an excess the forward product exposes on 15.5% of outcome-safe-labelled traces while the primary statistic stays below nominal. An exploratory state-conditioned recalibration reduces that excess while held-out math detection falls from 4/58 to 0/58. CCC therefore provides a reusable anytime-valid construction together with an audit that shows, per deployment, when the theorem's certificate does not apply.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.