SaysWho: Evaluating Authorization Without Authentication in Tool-Using Agents

Tool-using agents increasingly act as data custodians: they may legitimately retrieve private records, but must decide whether the person asking should receive them. That decision requires more than recognizing a plausible role. An agent may rely on a requester's self-description, on contextual evidence that a named person is eligible, or on protected evidence of who actually sent the request; these forms of evidence answer different authorization questions. Existing benchmarks study privacy, access, and disclosure in tool-using agents, but do not isolate the distinction between evidence that a claimed identity is eligible and evidence establishing who the requester actually is. To address this gap, we introduce SaysWho, a benchmark of 275 live recipient-authorization tasks across Gmail, Notion, Slack, and Discord, to test whether agents disclose on the basis of self-asserted authority and what changes when more information is available. Across 12 models, we evaluate three stages. First, without independent requester identity, blind-role forged claims increase disclosure over the original source requests; for example, DeepSeek V4 Flash rises from 8.4% to 70.8%. Second, we supply a directory: a record-specific list of people's names and relationships. This reduces disclosure to unsupported claims in all 12 models, but increases disclosure to unverified claims of eligible identities in 11 of 12; Sonnet 5 rises from 8.4% to 83.6%. Checking that a claimed person is eligible does not establish that this person sent the request. Third, we supply protected system metadata identifying the requester independently of the message. Even then, 6 of 12 models disclose on more than half of records when the identified requester is ineligible but claims an eligible identity. Our results show that, under this evaluation setup, supplying independent requester identity does not reliably make release follow that requester's permissions. This gap makes recipient authorization a central safety problem for tool-using agents and motivates evaluation and system design that bind release decisions to an independently established requester rather than to a plausible requester-written claim.