Normalization as Control: Batch Normalization Is the Deadbeat Corner of a Kalman-Tuned Normalizer Family

Batch normalization (BN) remains the standard normalizer in convolutional networks, and many variants replace its per‑batch statistics with running estimates. Yet it is unclear where BN sits among such tracking normalizers, what should set a tracker’s rate, and why principled trackers end up no better than BN. We answer in three steps. First, BN solves a small convex problem exactly on every batch; tracking that solution instead yields a family of normalizers that contains BN as its one‑step, or deadbeat, member. Second, a Kalman‑filter view sets the tracker’s gain from the noise it assumes, and we prove that, once the mean is tracked, the tracker stays at a distance from BN that grows with that assumed noise: the noise model decides how far a tracker may leave BN. Third, experiments on ResNet‑20 test this prediction. The standard noise model treats every activation as an independent sample, but the true independent units are the images, so it understates the noise 17‑ to 31‑fold. The tracker then collapses onto BN and even finishes training below it, because its stored inference state is as noisy as a single batch; a one‑pass recalibration removes the deficit. Changing only the noise model moves the same tracker away from BN in the predicted order, adding 6–11 points over plain test‑batch statistics on corrupted data at batches of 8 or fewer.