acceptodds
Under review as a conference paper at ICLR 2027

Transferable Rankings, Shifted Action Boundaries: The Value of BatchNorm Memory

Abstract

BatchNorm (BN) running statistics retain information from earlier domains, but when should a model carry that state into a new domain? We measure its value through paired carry/reset interventions with frozen weights and identical target batches. Across 12 CIFAR-C receiver checkpoints and all 210 directed corruption transitions per checkpoint, an external effect map correlates with receiver effects (, ), yet its zero-threshold rule loses 1.44 percentage points relative to resetting and is harmful at every checkpoint. On a locked 150/60 edge split, a single threshold learned from the other 11 receivers gains 0.83 points on held receivers and held edges, close to the 0.86 points from a receiver-specific affine fit; the zero threshold loses 1.02 points on the same edges. The transferred threshold also gains 0.93 points over resetting on 120 closed-loop streams with known domain identities and boundaries, without using held-receiver labels. State and layer interventions show that predictive utility is nonlinear even though BN-buffer differences decay exponentially under matched updates. These results identify a shared shift in the useful carry/reset threshold across the tested receivers: much of the source ranking remains useful once the threshold is calibrated.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.