Understanding and Steering Supervised Fine-Tuning through Entropy Flow
Desk rejected at ICLR 2027, so left out of the paper lists and the map.
Abstract
Supervised fine-tuning increases the likelihood of externally provided demon- strations, but likelihood improvement alone does not explain how supervision reshapes a model’s predictive distribution. We investigate this process through entropy flow: the local entropy responses to target-token supervision and their evolution throughout training. This perspective connects model-state-dependent selection and weighting with the allocation of supervision across tokens and ex- amples, while distinguishing local logit-space responses from their effects under shared model parameters. Our aim is to understand and steer fine-tuning through these evolving responses rather than to assess learning solely by the likelihood of observed tokens.
- Primary area:
- foundation or frontier models, including LLMs