TimeActBench: Benchmarking Time-Aware Actions in Real-Time Interactive Models

Temporal precision is fundamental to human-level intelligence, allowing actions to be coordinated with events as they unfold. Recent real-time interactive models enable continuous perception and timely responses during ongoing interactions, opening the way to evaluating this capability beyond traditional turn-based settings. However, existing temporal benchmarks primarily emphasize retrospective temporal understanding or specialized forms of timing-sensitive interaction, leaving the ability to determine whether and when to execute task-specific actions under causal streaming observations insufficiently characterized. To address this gap, we introduce TimeActBench, a benchmark for systematically evaluating time-aware actions in real-time interactive models. TimeActBench comprises seven task families and 196 evaluation instances spanning audio, video, and Audio-Visual settings, and organizes time-aware action into two complementary dimensions: Perception and Detection (PD), which evaluates whether models can detect task-relevant events and emit the prescribed detection action when those events become observable, and Response and Interaction (RI), which evaluates whether models can execute appropriate actions according to temporal constraints imposed by an evolving interaction. Our main comparison evaluates 15 proprietary and open-source real-time interactive models, and we conduct a human study under the same evaluation protocol. Our results reveal substantial limitations in current models, with a pronounced overall gap between model and human performance across action correctness and temporal precision. These findings establish time-aware action as a challenging and underexplored capability of real-time interactive models. We expect TimeActBench to provide a systematic testbed for studying this capability and to advance research on temporal intelligence in real-time interactive models, and it will be fully open-sourced.