What to Write: Structured Sparse Writing for Test-Time Training

Test-time training turns an input stream into fast weights through successive memory writes. Each write carries a decision that is usually left implicit: when only a fraction of the available contributions can enter memory, which ones should, and which ones should enter together? The choice is separate from the update rule and the chunk schedule a backend already fixes, and it is not about volume alone, because an update combines its contributions before any read sees them. We introduce Structured Sparse Writing (SSW), a model-independent operator that makes the decision explicit: it scores contiguous blocks of contributions with signals the update path already produces, selects the highest-scoring blocks under a budget, and gathers their contributors in temporal order before invoking the original write. SSW trains no selector and preserves the full read stream, target construction and commit schedule, so one interface precedes both optimizer-based and additive memories while exposing the selection unit, the ranking signal and the budget as separate design choices. Analysing the objective this induces delimits what it explains: it admits a tight worst-case omission guarantee for norm-scored additive contributions, yet the same argument proves it must prefer token-wise selection to any grouping — while grouping is what measurably helps. That advantage therefore lies in the interactions between retained contributions, which no scalar score encodes. Experiments on two backends agree: at a matched budget the retained identities govern quality, sparse writes reach the dense update's quality using a fraction of its contributions, and accuracy and answer likelihood prefer different budgets on identical inputs. Write allocation is thus a design axis of streaming memory, defined by the selection unit, the ranking signal and the readout it serves.