Understanding and Steering Supervised Fine-Tuning through Entropy Flow

Supervised fine-tuning increases the likelihood of externally provided demon- strations, but likelihood improvement alone does not explain how supervision reshapes a model’s predictive distribution. We investigate this process through entropy flow: the local entropy responses to target-token supervision and their evolution throughout training. This perspective connects model-state-dependent selection and weighting with the allocation of supervision across tokens and ex- amples, while distinguishing local logit-space responses from their effects under shared model parameters. Our aim is to understand and steer fine-tuning through these evolving responses rather than to assess learning solely by the likelihood of observed tokens.