Capability Calibration in Role-Playing Language Models: A Benchmark and Metacognitive Agentic Framework

Research on role-playing large language models (LLMs) has largely emphasized fidelity to the identities, personalities, and behaviors of characters and personas. For professional roles across career stages, however, a key challenge remains underexplored: providing useful assistance within the expertise and decision authority supported by a persona's career history—a requirement we term career-grounded capability calibration. LLMs may draw on capabilities beyond those supported by the assigned profile, causing junior or unrelated personas to respond like experts. We introduce CapRoleBench, which holds 355 workplace tasks fixed while varying career evidence across Senior, Mid, Junior, and Unrelated profiles to evaluate response fidelity and capability ordering. We further propose CapRoleAgent (CRA), a training-free, model-agnostic metacognitive framework that aligns career-supported capabilities with task requirements, enforces behavioral contracts, and audits overreach and omission. Across 14 open and proprietary LLMs, we observe systematic capability leakage under direct career-profile prompting. On a matched 100-task subset, CRA raises macro-average Ordering Consistency from 65.9% to 81.6% and lowers macro-average Overreach Rate from 32.0% to 10.9%, with both metrics improving for all 14 models. Profile-aware evaluation further shows higher Knowledge Fidelity and Behavior Coherence with fewer omissions, indicating improved role fidelity while preserving useful assistance. Evaluations by three LLM judges and persona-blind human raters corroborate the gains in capability ordering. The code is available at https://anonymous.4open.science/r/CapRoleAgent.