MARSEA: MULTIVARIATE RELATIONS BY SELECTIVE EXCLUSION IN ATTENTION

Attention represents relations pairwise: Aij is a statement about one query and one key. Under multivariate competitive exclusion a set of candidates jointly bears on a target and competes for a finite influence budget, settled by comparison within the set: the winner takes the bulk, close rivals less, everything past a content- decided cutoff exactly none. We prove four things about this class. Separation: no query-local scheme — rows computed independently per query, as in softmax with any positional bias or element-wise gate — can enforce a non-product-form constraint on a key’s degree. Conservation: no row-normalized scheme can bound every key’s fan-out below nq /nk , and doubly stochastic attention meets that bound only by giving every key the same content-free fan-out. A precision identity: with m gold items among n candidates, every full-support attention has evidence precision exactly m/n, whatever the scores. Sharp recovery: a sparsemax program over a relation slice returns the gold support exactly iff its exclusivity τ lies in [1/(W + mδ), 1/W ), for score margin δ and within-set spread W ; it degrades weakest-first above the interval, and the interval only widens as the slice shrinks. MarSea is the mechanism these results leave room for. On the query or the key margin it places a bounded-size relation and, at each constrained element, one closed-form program that inherits the base normalizer’s mass on the relation and re-shapes it by sparsemax, so members can receive exactly zero, under a shape control τ and a mass control λ. The query-local row form can learn a partition but not enforce one; the query-coupled column form can, and under causal de- coding its exclusions are permanent. We deploy the row form on a softmax-one base and ablate the column form. On multi-value retrieval with a 1.5B decoder (three seeds), MarSea matches or exceeds softmax, softmax-one, causal learned- marginal transport and row-wise sparsemax on exact-set accuracy, precision and re- call at every list length, with the fewest repeated and omitted values on 32-value lists; it beats softmax’s exact-set accuracy by 6 and 13 points at 16 and 32 values, its margin over softmax-one alone is within seed variance, and row-wise sparsity alone hurts.