What to Learn and Who to Ask: Adaptive Curriculum and Expert Allocation for Interactive Imitation

Interactive Imitation Learning (IIL) enables agents to master complex behaviors through real-time human interventions. However, its scalability is hindered by high supervision costs. Prior work has studied adaptive curricula and teacher selection, but not their coupled non-stationarity in intervention-based online IIL, where scenario value and supervision needs evolve with the learner. We propose COACH (Cost-Optimized Adaptive Curriculum & Human-supervision), which formulates this setting as a coupled non-stationary decision problem. COACH tracks scenario-specific learning dynamics to determine what to train next, while routing episode-level supervision according to the selected scenario, learner proficiency, and expert cost. COACH adopts a factorized but context-coupled architecture and uses Lazy Re-evaluation to selectively refresh non-stationary curriculum estimates. Extensive experiments on MetaDrive and MetaWorld demonstrate that COACH outperforms state-of-the-art baselines in task performance and cost reduction, results further confirmed through validation with human participants. Finally, we show COACH generalizes to skill-to-task matching, autonomously identifying human domain specialists without cost incentives.