Measuring and Repaying Skill Debt in Lifelong Language Agents

Persistent skill libraries allow large language model agents to reuse procedural knowledge across long task streams. However, continued accumulation can reduce performance when irrelevant, conflicting, stale, or redundant skills compete under fixed retrieval and context budgets. We define this phenomenon as skill debt: budget-matched performance regret caused by library contents or organization that are suboptimal for the current evaluation window. We introduce Counterfactual Skill Stewardship (CSS), an auditable lifecycle policy for measuring and mitigating skill debt by estimating the incremental utility of making each skill available. CSS performs paired shadow executions that differ only in the availability of an audited skill and applies time-block cross-fitted estimation to infer skill-level marginal utility from sparse reward differences. An uncertainty-aware policy keeps supported skills active, freezes uncertain or historically useful skills to exclude them from routine retrieval while preserving recoverability, and retires repeatedly harmful skills subject to dependency constraints. We also develop SkillDebtBench, a trajectory-aware evaluation protocol that controls task, model, retrieval, context, and inference budgets while recording complete library evolution and maintenance costs. Across controlled program tasks and interactive agent environments, the protocol varies library scale and resource constraints, injects distinct sources of negative transfer, and evaluates recurring task distributions. This framework supports falsifiable tests of when skill accumulation creates debt, which mechanisms cause performance degradation, and whether lifecycle interventions improve long-term reward without irreversible forgetting.