Renewal–Persistence Replay for Returning Concepts in Timestamped Multimodal Streams
Abstract
An image–text pair may become useful again after its concept disappears and returns, but a finite replay buffer can evict the pair before its renewed value is visible. Renewal–Persistence Replay (RPR) addresses both decisions: a frozen harmful-staleness ranker updates priority from teacher disagreement, embedding migration, and stable-anchor support, while cluster sampling mass and eviction floors keep low-frequency history reachable. Recurring, low-displacement concepts account for 28.5–35.8% of future queries in our three timestamped streams, making this return regime substantial rather than exceptional. With the model, contrastive objective, buffer, update budget, and checkpoint rule held fixed, five-seed SciCapStream comparisons reduce forgetting by 0.70.5 points versus MIR-style replay and 0.90.5 versus GSS; RPR also lowers forgetting from 14.10.6 to 9.40.4 relative to strict timestamp ordering without reducing future retrieval. The ordering repeats on PMC-Quarterly and RedCaps-Monthly and transfers to ViT-L/14. Its mechanism is localized: the SciCap RPR–Recency effect expands to -6.60.9 points for high recurrence but contracts to -0.20.5 for weak recurrence; corrupting visible time erases RPR's margin over a shared timestamp-agnostic MIR trajectory; and oscillating concepts in ConceptDrift-5K gain +14.71.8 R@1. These results identify returning-concept streams as a consequential replay regime and show that coupling current utility to continued reachability improves replay there.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.