Nerve Faithfulness Certifies Data Geometry in Learned Dictionaries
Desk rejected at ICLR 2027, so left out of the paper lists and the map.
Abstract
Reconstruction error and sparsity do not determine whether a learned dictionary preserves the geometry of the activations it encodes. We introduce nerve faithfulness, a certified comparison between data geometry and code co-activation geometry. For a non-negative code , is exactly the Dowker -skeleton; after normalization it induces a metric , and certifies an interleaving through the identity on samples. The certificate is hypothesis-free and costs dense or for -sparse codes. It remains nearly invariant across a sweep of tuning width for a fixed antipodal polysemantic defect (–), while coherence varies by a factor of . As a training objective, a correlation form combined with coherence raises rank faithfulness on rotated MNIST from to , lowers the certificate from to , and improves reconstruction. On released GPT-2 dictionaries with FVU –, code–data rank correlations of only – show that good reconstruction can coexist with severe geometric mismatch. Nerve faithfulness adds a distinct, certifiable axis for evaluating and training learned dictionaries.
- Primary area:
- interpretability and explainable AI