Investigate on Demand: Evidence-Qualified Safety Auditing of Agent Trajectories

Trajectory auditing is essential for assessing the safety of LLM agents, as actions judged harmless or suspicious in isolation during execution may have different safety implications in the full trajectory context. One-pass auditing can miss cross-step violations in long trajectories due to context loss, while indiscriminate deep investigation increases computational cost and can introduce false alerts. We introduce IoD-Auditor, a hierarchical on-demand framework for offline trajectory safety auditing. Across trajectories, a lightweight escalation policy uses trajectory length and direct-audit confidence to select between one-pass auditing and multi-agent investigation. Within each investigated trajectory, an evidence-driven multi-agent procedure adaptively examines unresolved evidence gaps. A shared behavioral-evidence record connects authorization, behavioral stage, counterevidence, and observable consequences, guiding further inspection and evidence-qualified adjudication. Across four heterogeneous benchmarks, IoD-Auditor achieves a macro-average F1 of 82.4, outperforming evaluted SOTA one-pass, sequential, and multi-agent investigation baselines. It reduces average model calls by 24% relative to investigating every trajectory, while budget-matched comparisons and controlled ablations demonstrate the benefits of selective escalation and evidence qualification.