$\pi$-Bench: Benchmarking Proactive Speech Interaction with Full-Duplex Conversational AI

Full-duplex conversational AI enables users and models to speak simultaneously, supporting more natural and temporally continuous interaction than traditional turn-based systems. However, existing evaluations remain largely passive: they mainly test whether models can tolerate user behaviors such as interruptions, backchannels, topic shifts, or side conversations. In most cases, the model itself still behaves as a passive listener that waits for turn completion before responding. As a result, a fundamental capability of human conversation remains largely unexplored: producing cooperative conversational actions during an ongoing user utterance. In this paper, we study proactive interaction, defined as the ability to respond before turn completion when sufficient conversational evidence has already emerged. To evaluate this capability, we introduce $\pi$-Bench (proactive interaction benchmark), the first benchmark for proactive speech interaction in full-duplex conversational AI. $\pi$-Bench contains 1,723 human-verified examples spanning 4 tasks: contextual and non-contextual proactive buzz-in, clarification requests, and backchanneling. These tasks evaluate whether models can produce temporally appropriate and behaviorally coherent mid-turn responses, including content-bearing feedback, non-lexical listener signals, and clarification-seeking behaviors under ambiguity. Experiments on $\pi$-Bench show that current open-source models struggle to engage proactively during ongoing user speech, especially when timely clarification or cooperative backchanneling is required. Although supervised fine-tuning with curated proactive interaction data improves performance, human-level proactive interaction remains far from solved in current speech conversational AI systems.