acceptodds
Under review as a conference paper at ICLR 2027

Heterotelic Learning: One Training Loss, Many Learning Ends

Abstract

Machine learning usually ties what a model learns to the loss optimized during training. Yet learning systems may need to adapt from simple and widely available objectives while serving purposes that are not directly expressed by those objectives, which is what current learning models cannot easily do. To alleviate this limitation, we propose Heterotelic Learning, a paradigm in which a mechanism learned in advance shapes updates from a shared online objective to serve different intended learning ends. We instantiate this idea with Learned Objective Transformation (LOT), which learns a temporary compiler from supervision specifying desired behaviors or target gradients. During adaptation, the compiler shapes the updates induced by a shared online objective and is then removed, leaving only the resulting parameter changes. Using the next-token prediction (NTP) as the shared objective in language modeling, we provide theoretical and empirical evidence that the same NTP objective can support substantially different adaptation effects, including changes in update direction, regularization, answer-oriented adaptation, and learning over longer streams. Evaluations on unseen contexts and multiple language-model backbones demonstrate that the proposed LOT can effectively serve different learning ends via a shared objective.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Meta-learning, gradient shaping, transfer learning, continual learning

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_mqMbOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes Heterotelic Learning, which is implemented as Learned Objective Transformation (LOT). In specific, LOT train a temporary compiler to shape the gradients produced by next-token-prediction loss, and online adaptation can serve an entirely separate goal, such as improving QA performance. Comprehensive experiments demonstrate the effectiveness of the method.

Critical strengths

The authors includes massive experiments to validate the effectiveness of the proposed HL method.

Critical weaknesses

The distinction between LOT and prior meta-learned update rules is not sufficiently established. For the related methods, Meta-SGD and Meta-Curvature learn update directions or gradient transformations, while CaMeLS uses a downstream QA objective to shape NTP-based online adaptation. To my understanding, C-only BCA explicitly transforms the LoRA-writer gradient, making its core operation closely related to a learned gradient map. However, the paper does not include a matched learned-optimizer or preconditioner baseline to show that the temporary compiler provides a distinct capability.

Questions for authors
  1. How is LOT conceptually distinct from meta-learned gradient transformations or preconditioners, as described above?
  2. Does “the same online objective” mean only that the NTP targets and loss form are unchanged?
Reviewer_U9wBOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Heterotelic Learning, a framework that separates the objective used during online adaptation from the intended learning outcome. The authors introduce Learned Objective Transformation (LOT), which meta-learns a temporary compiler to reshape gradients induced by a shared next-token prediction (NTP) objective. The compiler is removed after adaptation, leaving only the updated model parameters. The paper provides theoretical analyses of gradient transformation and expressivity, along with experiments on gradient control, regularization, question answering, and continual adaptation. Results demonstrate that LOT can induce different learning behaviors under the same NTP objective and improve downstream adaptation performance over direct NTP baselines.

Critical strengths
  1. The paper offers a novel perspective on separating online learning objectives from desired adaptation outcomes.

  2. The paper provides theoretical analyses explaining how LOT transforms gradients and influences learning.

  3. The experiments cover diverse adaptation goals and multiple model backbones, demonstrating the effectiveness of LOT.

Critical weaknesses
  1. Insufficient novelty discussion. The paper does not sufficiently distinguish LOT from recent test-time learning and gradient transformation methods. More detailed comparisons are needed to clarify its novelty.

    References:

    • Chen et al. (2026). PERK: Long-Context Reasoning as Parameter-Efficient Test-Time Learning. ICLR.
    • Sun et al. (2025). Learning to (Learn at Test Time): RNNs with Expressive Hidden States. ICML.
    • Hennigen et al. (2025). On the Duality between Gradient Transformations and Adapters. ICML.
  2. Gap between theory and practice. The theoretical analysis mainly considers idealized settings, such as zero-initialized writers and single-step updates. Its applicability to practical multi-step adaptation remains unclear.

  3. Limited experimental evaluation. Experiments focus on relatively small models (0.6B–1B) and limited test contexts. Many evaluations rely on short answer-prefix likelihoods rather than complete answer generation.

Questions for authors
  1. Could the authors further clarify the key novelty of LOT compared with recent TTT methods, PERK, and existing gradient transformation approaches?

  2. LOT improves QA performance but introduces substantial changes to unrelated predictions. Does LOT still outperform existing methods when compared under matched KL divergence or forgetting constraints?

Reviewer_S4KBOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors introduce Learned Objective Transformation - a form of meta-learning - that adjusts how a model's parameters respond to a fixed training objective. The application as far as I can understand is to enable LLMs to be retrained online with new information via parameter updates without requiring explicit supervision from a downstream task.

The authors provide a theoretical analysis and empirical results demonstrating that parameter updates via a next-token prediction learning objective can be adapted. SLMs show improvements on downstream QA likelihood compared with conventional adaptation.

Critical strengths
  • An interesting, and potentially important research direction as far as I can understand its setting. The idea to enable models to dynamically update their parameters to acquire new information is one important paradigm. The authors separate the training objective from the intended learning outcome which feels novel.
  • The paper leans in to both the theory and an empirical investigation giving the study a solid foundation.
Critical weaknesses
  • The premise and assumptions behind the method are opaque to the reader. I can accept that there is a community to whom the problem setting is very clear but unfortunately this isn't me. I would have liked to see the assumptions made explicit early in the paper to excite me and get me thinking. I'm assuming that the basis for the method is that parameter updates have longevity, potential for personalization, and don't suffer from context window limits compared to say RAG. It would have been nice to see the method critiqued against inference time memory management methods which seem like a natural competitor.
  • It's not quite clear why unrestricted NTP is the best setting to test this idea. Surely not every piece of new information needs to be learnt?
  • Whilst I understand that learning from experience is the objective, it's not clear to me unfortunately from the review of related work (section 2) why LOT is the natural method to try in this setting. Where is the gap?
Questions for authors
  • I think the key question here will be, how can LOT determinate what should be learnt as opposed to how it should be learnt?
Reviewer_eBRfOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a Learned Objective Transformation approach to solve the the objective changing problem between the training period and the applied scenarios.

Critical strengths

The main strengths are listed as follows:

S1: The manuscript is well structured and clearly formulated. S2: The authors conduct extensive experiments using different LOT settings. S3: The figures are informative.

Critical weaknesses

The main weaknesses are listed as follows:

W1. Research Motivation and Significance The broader significance and potential applications of this work could be better articulated. The paper moves relatively quickly toward technical details, before sufficiently establishing the importance of the underlying research problem. A clearer discussion of the broader motivation and potential impact would help better communicate the contributions of this work.

W2. Experimental Evaluation and Comparisons Although the paper provides extensive experimental results to demonstrate the effectiveness of Learned Objective Transformation (LOT), most of the comparisons are conducted among different implementations proposed by the authors. Comparisons with relevant existing methods discussed in the Section 2 are not provided. Including more comparisons with these self-defined settings would help better assess the effectiveness of the proposed method and clarify its advantages over existing alternatives.

W3. Presentation and Organization The paper is generally clear and well illustrated, although some aspects of the presentation could be further improved. For example, in Section 2, the discussion of related work could go beyond describing individual existing methods and their respective mechanisms. It would be helpful to provide a more structured analysis of their strengths and limitations, and then explain how these observations motivate the proposed approach and distinguish it from existing methods. Additionally, Section 4.1 introduces three backbone models, but the main text presents detailed experimental results primarily for one of them, while the results for the other two models are mainly reported in the appendix. It would be helpful to include the results for all three backbones in the main paper to facilitate direct comparison and provide a clearer picture of the method's generalizability.

Questions for authors

Are none of the methods discussed in the Related Work section suitable as baselines? If so, could the authors clarify why these methods were not considered for comparison?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.