acceptodds
Under review as a conference paper at ICLR 2027

AgentMarkArena: A Unified Platform for Evaluating Agent Behavior Watermarking

Abstract

LLM agents increasingly execute real-world tasks autonomously, making it important to identify which agent produced an observed sequence of actions. Their execution trajectories also contain reusable workflows, so attribution helps protect agent intellectual property. Behavioral watermarking addresses these needs by embedding keyed signals into agent decisions. However, comparison is difficult because watermarks alter tool choices and task outcomes, detectors require different evidence, and record edits differ from behavioral changes. We introduce AgentMarkArena, a unified and extensible platform for developing and evaluating agent behavioral watermarks under matched conditions. It combines 249 graded tasks across diverse domains and integrates methods into a shared agent runtime through common selection and verification interfaces. The platform distinguishes watermark carriers from verifier evidence, separating mechanism improvements from advantages of richer verification access. Its automated pipeline handles execution, grading, attacks, and aggregation, producing access-specific leaderboards covering detectability, task utility, robustness, behavioral distortion, and overhead. Across six watermark mechanisms and three agent backbones, we compare methods at a target 1% false-positive rate and stated verifier access, and evaluate robustness under post-execution record edits. With minimal verifier evidence, SeqWM reaches a 77.1% true-positive rate while retaining 92.0% utility. In contrast, TraceWM-T reaches 100% only with a trusted run identifier, and removing duplicate observations reduces it to 1%. Our results motivate future designs that exploit semantic redundancy, accommodate realistic verifier access, and explore equivalent argument choices as additional carriers. Our framework is available at https://anonymous.4open.science/r/AgentMarkArena-336D/.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.