Heterotelic Learning: One Training Loss, Many Learning Ends

Machine learning usually ties what a model learns to the loss optimized during training. Yet learning systems may need to adapt from simple and widely available objectives while serving purposes that are not directly expressed by those objectives, which is what current learning models cannot easily do. To alleviate this limitation, we propose Heterotelic Learning, a paradigm in which a mechanism learned in advance shapes updates from a shared online objective to serve different intended learning ends. We instantiate this idea with Learned Objective Transformation (LOT), which learns a temporary compiler from supervision specifying desired behaviors or target gradients. During adaptation, the compiler shapes the updates induced by a shared online objective and is then removed, leaving only the resulting parameter changes. Using the next-token prediction (NTP) as the shared objective in language modeling, we provide theoretical and empirical evidence that the same NTP objective can support substantially different adaptation effects, including changes in update direction, regularization, answer-oriented adaptation, and learning over longer streams. Evaluations on unseen contexts and multiple language-model backbones demonstrate that the proposed LOT can effectively serve different learning ends via a shared objective.