Replay-Gap Thresholds: Sharp Burst Guarantees and Coupled Retention
Abstract
Sparse replay can retain training examples even when their update frequency vanishes, but the placement of updates matters. We prove a sharp worst-case replay-gap threshold for constant-step online logistic regression on any separable interfering pair. For a fixed burst of rare updates, let be its total scalar logistic response from margin , let in coordinates , and set . Under an explicit sufficient step-size bound, every replay-time ratio sequence confined eventually to a fixed interval guarantees eventual all-time retention from every finite initialization. Geometric schedules above the threshold establish worst-case sharpness; equality remains open. The proof uses a common invariant region controlled by the derivative of the full burst response. We also extend the centered-map method to any finite number of orthogonal common examples coupled through one interfering rare example, obtaining an exact geometric sign criterion in the stated exposure regime and a computable local guarantee for arbitrary persistent ratio variation. In this family, two schedules whose rare-update counts differ by at most one at every prefix can have opposite retention outcomes, robustly under small bounded shared log-time jitter. The results concern logarithmically sparse replay and raw training margins, not population accuracy.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.