Is Dense Supervision Worth Its Cost? A Budget-Controlled Study of Supervision Allocation in MAML

Does supervising more adaptation steps produce a better meta-learner under a fixed training budget? We benchmark dense, randomly sampled and fixed-horizon supervision in MAML with exact and approximate differentiation, pairing task streams and evaluation episodes, measuring complete update cost, and evaluating the full adaptation path. Across five training seeds on CIFAR-FS and MiniImageNet, sampled and intermediate fixed-horizon supervision achieve lower mean full-path negative log-likelihood than dense supervision at a 1,800-second budget under exact Conv4 full-model adaptation. Fixed-5 achieves the lowest mean on both datasets. Both intermediate fixed horizons outperform full-budget dense supervision in mean loss at half the budget. Allocation and differentiation interact: larger relative gains can accompany worse absolute performance. Early and late prediction depths can favor different fixed targets. Later adaptation can improve accuracy while worsening NLL; temperature adjustment reduces this penalty while preserving fixed-5’s mean advantage over sampled supervision. Our analysis separates training-target bias from differentiation error and gives a local quadratic cost–progress condition for a biased direction to outperform a same-target reference. These findings motivate joint selection of allocation and differentiation at the intended budget using depth-weighted probability scores alongside accuracy.