From Update Geometry to Model Relatedness: Index-Only Salient Support Similarity
Abstract
Model similarity supports personalization, clustering, and retrieval, but standard methods typically require high-dimensional real-valued weights, updates, or representations. We study a layer-normalized Top-\(K\) index-support signal for client and checkpoint relatedness in a shared parameter coordinate system, which we call Salient Parameter Overlap Similarity (SPOS). Each client computes update-magnitude salience locally, normalizes salience within parameter blocks, and communicates only the identities of its selected coordinates. With \(K/M=0.1\) and 32-bit coordinate IDs, the similarity-plane payload is one tenth that of an \(M\)-entry FP32 vector. Across four vision, language, and speech model–dataset pairs, SPOS provides useful affinities for clustered personalization and neighbor retrieval. With label marginals held exactly fixed, the signal remains informative under feature shift, concept shift, and unequal client data and computation. For checkpoints adapted from a shared reference, SPOS obtains functional-agreement and cross-domain-transfer R@3 of \(0.716\) and \(0.702\), compared with \(0.733\) and \(0.720\) for signed dense-update cosine. These results characterize salient-support overlap as an index-only relatedness primitive when real-valued per-client affinity inputs are undesirable or unavailable.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.