acceptodds
Desk rejected at ICLR 2027

Understanding and Steering Supervised Fine-Tuning through Entropy Flow

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Supervised fine-tuning increases the likelihood of externally provided demon- strations, but likelihood improvement alone does not explain how supervision reshapes a model’s predictive distribution. We investigate this process through entropy flow: the local entropy responses to target-token supervision and their evolution throughout training. This perspective connects model-state-dependent selection and weighting with the allocation of supervision across tokens and ex- amples, while distinguishing local logit-space responses from their effects under shared model parameters. Our aim is to understand and steer fine-tuning through these evolving responses rather than to assess learning solely by the likelihood of observed tokens.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Models, Supervised Fine-Tuning, Entropy Flow, Token-Level Reweighting, Training Dynamics

Related papers

Discussion (0)

Sign in to comment.