acceptodds
Under review as a conference paper at ICLR 2027

Exact Undo: Inverse-Bound Writes Make Recurrent States Provably Revisable

Abstract

Sequence models can write, but they cannot unwrite. Attention appends keys and values that are never removed; delta-rule networks overwrite by decay. Revision–undo a write exactly, in constant-size state, at unbounded nesting depth–has no architectural primitive. We supply one: inverse-bound writes, where a token and its inverse share materialized floating-point parameters at the bit level, so executing the inverse composes the exact inverse of the forward write into the state. Exactness is constructive and parameter-free: one round returns the state to O(u) for every parameter value (Theorem 1); K rounds accumulate at most K*O(u), while a repeated learned mismatch accumulates at first order (Theorem 3); the construction ports into DeltaNet at a proved price (Theorem 6), and a corollary proves exact cancellation unreachable in that family's open parameter domain; measured, its depth-128 undo stacks collapse (0.207) where the orthogonal carrier stays exact. Closure under inverses further makes the state an exact O(1)-state solver for group word problems, with zero-error 8x length extrapolation. Measured on a group-program benchmark, the taxonomy orders itself as the theory demands: append-only transformers plateau at 0.320; approximate-inverse DeltaProduct shares capability but not cancellation; inverse-closed-but-unbound OS reaches exactness with 5-10x more samples; inverse-bound OS is exact at the matched schedule, its state exact from initialization, through depth-128 undo stacks. What compute buys is the readout channel–a linear probe decodes depth-128 answers from the unbound model's contracted signal at 1.000 while its first-layer operator is off by 0.8, an error the trained readout's margin tolerates; what the binding guarantees, in the vocabulary-bound layer, is the whole operator, every direction, any probe. A dose-response boundary chart closes the paper: each inverse-bound layer monotonically degrades byte-level language modeling–composition, not compression. The scope is the strictly nested (LIFO) core of revision.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.