Rethinking Nonparametric Teaching via Geometry

Nonparametric teaching studies how example selection can accelerate learning in reproducing kernel Hilbert spaces (RKHS). Existing formulations adopt a first-order perspective, selecting examples with large functional gradients while retaining the learner's native RKHS geometry. Through Mercer spectral analysis, we explain how this fixed geometry links the RKHS cost of prediction errors to the kernel-scaled response of first-order updates. To address the resulting spectral imbalance, we propose Geometry-Aware Teaching (GAT), a second-order framework that integrates curvature-aware batch selection with preconditioned functional updates. For realizable squared loss on finite pools, we justify its teaching score through a common lower bound on prediction-risk decrease and establish convergence guarantees for GAT and first-order teaching baselines. These guarantees distinguish first-order dependence on kernel conditioning from GAT's dependence on batch geometry. With exact batch selection and preconditioning, this spectral independence extends to GAT's prediction trajectory, which remains invariant to positive kernel-eigenvalue rescaling under a fixed teaching protocol. To identify the contribution of example selection, we further establish finite-horizon teaching optimality for a fixed preconditioned learner in a complete pool space. Empirical results confirm the practical benefits of GAT.