EqWD: Bounded Weight Decay from Standardized Ratio Deviations

Weight decay controls parameter scale in modern optimizers, and decoupled implementations expose it as a separate update. Most recipes keep its coefficient fixed while loss-driven update size changes during training. The gradient-to-weight norm ratio tracks that size, but absolute values vary across tensors and mix persistent scale with temporal change. We measure each tensor's deviation from its moving ratio mean and normalize it by a moving residual scale. We use this standardized deviation to drive adaptive decoupled decay, which we call Equilibrium-Driven Weight Decay (EqWD). EqWD increases the base coefficient in proportion to the deviation, changes only radial decay, and leaves the base optimizer's loss update unchanged. Including the current residual in the scale bounds the per-step coefficient; the exponential moving-average (EMA) dynamics also bound accumulated squared modulation over a finite horizon and characterize when the signal vanishes. Across three paired 90-epoch ImageNet-1K ResNet-50 runs, EqWD improves best-epoch Top-1 by $1.049\pm 0.238$ points over the reference fixed-decay recipe. The exposure-matched fixed-decay control reduces this margin to 0.255 points, showing that effective decay strength explains much of the reference gain. On 300-epoch ViT-Tiny/ImageNet-1K, EqWD reaches $73.739\pm 0.060$ final Top-1 versus $73.718\pm 0.113$ for fixed decay. On COCO2017 DINO (DETR with Improved DeNoising Anchor Boxes) detection, EqWD raises average precision (AP) by 0.6 points with ResNet-50 and 0.1 points with Swin-T.