acceptodds
Under review as a conference paper at ICLR 2027

Bidirectional Transfer, Asymmetric Expression: Math Fine-Tuning Conceals the Code Gains It Carries

Abstract

Transfer between mathematics and code is usually scored as net change on a target benchmark against the base model. Reports disagree even on its sign, yet all read that change as what the update contains. We find instead that an update carries two things, the capability it has learned and the answering mode it adopts at its first token, and the latter can mask the former across domains. We call this masking concealed transfer and measure it against a format-matched control, the base plus the update rows that carry format. Against this control, transfer on Qwen3-4B runs both ways but asymmetrically: code fine-tuning raises mathematics directly, whereas mathematics fine-tuning shows no detectable difference on code. This apparent absence is not an absence of transfer. An edit fixed before evaluation and blind to target scores lifts the code score above both the fine-tuned model and the control. The same concealment recurs at three checkpoints across two architectures, each with its own releasing edit. On Qwen3-4B the concealment traces to one direction in weight space carrying a vanishing share of the update's energy. Removing it restores code openings and releases the gain with no measurable source-domain cost. Random directions of matched rank and energy do neither, and this direction alone does not shift the opening. These findings recast cross-domain transfer as separating what a model can solve from how it begins to answer, and an undetected net difference does not establish zero transfer.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.