acceptodds
Under review as a conference paper at ICLR 2027

When Does Quantization Form Reusable Concepts?

Abstract

Quantization encodes learned representations as discrete tokens. When do joint encoder and codebook updates organize them into reusable concepts? A preserved property such as shape may still require complex token combinations to read across colors and viewpoints. We study reuse through one rule shared across contexts. Our principle is that parameter updates move assignment boundaries, and inputs crossing them alter subsequent updates and shared-rule errors. Reference models show how boundary geometry constrains factor selection and relative update rates change local stability and relaxation. In a frozen residual model, we bound the time until one token test reads the target without error across contexts. Learned quantizers preserve the target perfectly at the same token budget, yet have certified minimum error-free rule lengths of 15 and 34 bits under one grammar. These results connect training conditions to factor organization and distinguish target recovery from concise reuse.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.