Can Verification Sustain Skill?
Abstract
A common way to make AI delegation reliable is to have AI generate an output that a human verifies and corrects. A static view of this workflow treats verification competence as fixed. Over time, however, practice and non-use can change this competence. We model a worker whose skill determines manual and verification reliabilities and thereby shapes workflow-specific performance losses and completion costs. We study how skill evolves when the worker chooses among manual work, verified delegation, and pure delegation to maximize current utility by trading off performance loss against completion cost. Current skill determines workflow choice, which shapes future skill. Under verified delegation, reviewing provides verification practice, whereas generative practice occurs only when the worker detects and repairs an AI error. We characterize the global dynamics and identify three verification-related regimes: collapse to zero skill, an interior skill level sustained by verified delegation, and competence sustained at a manual–verified switching boundary. Crucially, verified delegation can remain optimal over a skill range while no positive skill level is sustainable. Adding verified delegation can improve current utility yet eliminate an otherwise high-skill basin. Holding learning dynamics fixed, workflow incentives alone can produce all three regimes. Better AI can further reduce retained human skill by reducing manual practice and the errors that trigger repair-based learning. The regimes persist over nontrivial learning-parameter ranges and under additional learning from correct AI outputs, while task-level randomness can drive trajectories near a stable verified equilibrium into pure delegation. Thus, keeping a human in the loop need not preserve the capability required for effective future oversight.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.