When and How Often to Defend? On the Overlooked Role of Learning Rate Schedules in Poisoning

Data-poisoning defenses are typically applied throughout training, even though they can substantially reduce model accuracy. In this work, we ask: when and how frequently should a defense be applied? We highlight the overlooked role of the learning-rate schedule and prove that the effect of a defense at a given step scales with the squared learning rate ${\eta }^2$. Defending therefore matters most within the window containing the largest cumulative ${\eta }^2$ mass. We further relate defense decisions to the current gradient-state information and classify training-time defenses into three regimes: state independent, state dependent, and mediator. We evaluate these regimes against the seminal Witches' Brew attack on CIFAR-10, CIFAR-100, and Tiny ImageNet, using a DP-SGD-style defense applied to the aggregated gradient. Our experiments show that attack success rises primarily around the first learning-rate drop despite a stable poisoning frequency. When the defense is restricted to the window carrying most of the ${\eta }^2$ mass, gradient-state information becomes valuable: a state-dependent policy matches or outperforms a random policy at a lower activation rate. Most importantly, activating the defense at only a small fraction of training steps, well distributed across the schedule, fully prevents the attack while only slightly reducing clean accuracy, yielding a better security/accuracy trade-off. Effective defense therefore requires far fewer interventions than commonly assumed, and where theses interventions fall on the learning-rate schedule matters as much as their frequency.