Gradient-Momentum Coupling: A Parameter-Space Proxy for Learning Progress

Measuring learning progress is at the core of curiosity-driven exploration, which rewards an agent for going where its model is still learning. However, the abstract notion of learning progress is not directly measurable, and existing methods often derive it from the prediction error in the output space. This paper proposes Gradient-Momentum Coupling (GMC), which measures how strongly a sample drives change in the parameter space, given by the normalized absolute product of its gradient with the momentum of previous gradients. Directions of change that persist across samples accumulate in momentum, while noise cancels out. In controlled experiments GMC allocates near uniform priority across tasks with varying levels of noise, where prediction error chases the noisiest, and orders learnable tasks by improvement speed rather than difficulty. On four MiniGrid MultiRoom tasks, substituting GMC for prediction error inside the Intrinsic Curiosity Module (ICM) recovers exploration ICM loses to unpredictable observations.