Order Stability in Masked Generation: Swap Depth and Selective Repair
Abstract
We develop a quantitative theory of order dependence for positive masked conditional tables. Two structural effects govern reliability. First, along a route with D layers of disjoint adjacent swaps and total source-weighted local Hellinger energy E, the endpoint laws satisfy TV^2 <= 15DE. A matching product-distribution construction makes the depth dependence necessary, although the upper bound allows arbitrary statistical dependence. Second, every fixed order can be exponentially close to an anchor while a content-adaptive order remains a constant distance away, even with all token probabilities in [1/4,3/4]. The same construction gives an exact exponential coverage penalty for independent-mask auditing. We connect local swap audits to anchor-conditional errors and selective repair. For a fixed deterministic policy, replacing selected rows has exactly additive forward-KL benefits. Under a known probability floor, independent anchor replay learns a near-optimal repair budget and certifies its residual error; bounded unresolved-prefix width makes the required conditional evaluations tractable. Bounded-score Chernoff certificates expose the finite audit cost. Exhaustive checks and sample-selected repair experiments demonstrate the structural scalings and nonvacuous certificates. The guarantees measure reliability relative to an anchor; external accuracy requires a separate anchor-quality bound.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.