acceptodds
Under review as a conference paper at ICLR 2027

When Can a Zero Be Deleted? Exact State Compression for Gated Low-Rank Training

Abstract

A low-rank component can leave the predictor while still determining the optimizer's next accepted update. We solve the resulting state-elimination problem for anchored gated low-rank training with one shared Euclidean ball. A current-checkpoint certificate keeps selected gates at zero over a declared horizon; one evolving radial statistic then replaces their factor directions in every projection, complete objective, and line-search decision, including rejected trials. An accepted-update family establishes why this radial information cannot all be discarded. The same construction separates when memory is reclaimed from which learning rule is executed: immediate, staged, and delayed release under one license preserve the same real training observations. A unified loss budget supplies computable certificates; a sharp cross-entropy bound admits 35 rather than 28 of 118 zero candidates in nonlinear confirmation. In a quality-first shared-backbone study, eight independently initialized adapters all admit release, removing 18 of 77 learned zero slots and 10.43% of their aggregate live state. All three release schedules match the full reference's trial signatures over 200 accepted blocks per adapter. Regression continuations retain 49.9–71.1% live-state savings, and directed rounding validates a complete nonlinear continuation. These results turn structural inactivity into a storage decision without adding a new real update rule.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.