HIVE: Counterfactual Human Oversight Allocation for Long-Horizon Agent Workflows

As AI agents support a growing share of operational workflows, organizations gain parallel throughput while human reviewers supervise a growing stream of actions. We study how to allocate a limited review budget to workflow nodes where intervention can prevent downstream failures and preserve reviewer capability for future cases. We introduce HIVE, a zero-preserving estimator trained on paired replays. HIVE encodes the factual execution and a label-free hypothetical review, then propagates their semantic difference through a typed dependency graph. We combine it with a practice-dependent competence state and the finite-horizon HIVE-DP planner, which jointly accounts for immediate protection, remaining capacity, and competence transitions. On a reproducible enterprise-workflow benchmark with paired potential outcomes, HIVE improves capacity-constrained allocation over outcome and confidence baselines. Across 16-period episodes, HIVE-DP reduces regret over one-step allocation by 48.4-51.3% at capacity four and 32.0-46.3% at capacity six, with gains that persist under faster decay, weaker retention, and an unmodeled nonlinear retention transition. A held-out transition grid and validation-selected risk controller assess sensitivity to dynamics and estimator disagreement. Together, HIVE's intervention-attributable graph score and HIVE-DP's explicit reviewer state form an auditable estimation-planning interface for scarce, dynamically changing oversight.