Revisability Buys Geometry-Independence: Competitive Analysis of Batched Memory Consolidation
Abstract
Moving memory items into network weights, known as consolidation, incurs costs that depend heavily on batching: items consolidated together share a fixed setup price and influence each other's utility based on batch composition. This paper establishes a pricing framework for this consolidation process, yielding a central negative result. The most straightforward consolidation strategy, where each batch fixes its internal alignment, lacks a bounded competitive ratio because an adversary can strategically flood any fixed configuration. Escaping this impossibility requires revisability: the ability to adjust a prior batch's alignment. We analyze two mechanisms for revisability (a lightweight approach of replaying past data during new consolidation steps and a costly joint refitting procedure) and derive their respective prices. Pre-registered experiments on real feature geometry show that while our core theoretical guarantees (the ledger, dominance arithmetic, scheduling reductions, and worst-case bounds) and the separation construction hold on natural data, the exact closed-form quantitative levels do not transfer directly. All pre-registered evaluation criteria are fully detailed in the appendix.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.