acceptodds
Under review as a conference paper at ICLR 2027

From Update Geometry to Model Relatedness: Index-Only Salient Support Similarity

Abstract

Model similarity supports personalization, clustering, and retrieval, but standard methods typically require high-dimensional real-valued weights, updates, or representations. We study a layer-normalized Top-\(K\) index-support signal for client and checkpoint relatedness in a shared parameter coordinate system, which we call Salient Parameter Overlap Similarity (SPOS). Each client computes update-magnitude salience locally, normalizes salience within parameter blocks, and communicates only the identities of its selected coordinates. With \(K/M=0.1\) and 32-bit coordinate IDs, the similarity-plane payload is one tenth that of an \(M\)-entry FP32 vector. Across four vision, language, and speech model–dataset pairs, SPOS provides useful affinities for clustered personalization and neighbor retrieval. With label marginals held exactly fixed, the signal remains informative under feature shift, concept shift, and unequal client data and computation. For checkpoints adapted from a shared reference, SPOS obtains functional-agreement and cross-domain-transfer R@3 of \(0.716\) and \(0.702\), compared with \(0.733\) and \(0.720\) for signed dense-update cosine. These results characterize salient-support overlap as an index-only relatedness primitive when real-valued per-client affinity inputs are undesirable or unavailable.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Model relatedness, model similarity, federated learning, salient parameter support, parameter overlap, communication efficiency, value-free similarity, personalized federated learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_mn8xOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes SPOS, a client-relatedness measure for federated learning: each client sends only the indices of its top-K (with K=10% typically) layer-normalized update coordinates, and similarity is their overlap. It’s evaluated on relatedness retrieval, clustered personalized FL and checkpoint retrieval, with four model/dataset pairs and 30 clients.

Critical strengths

Estimating client relatedness when secure aggregation hides individual updates is a real problem in FL, and SPOS is a simple primitive that plugs into existing pipelines.

Critical weaknesses
  1. SPOS doesn’t beat what it replaces. In clustered FL it matches dense-update cosine within about 1 point, often inside the confidence interval, and on ResNet18 Dir(0.1) retrieval it’s clearly worse (R@6 0.56 vs 0.70). It’s not clear what SPOS brings over cosine besides hiding values, which secure aggregation already does for the aggregation itself.

  2. The main intuition behind the proposed method isn’t new. IndexGuard already compares clients through the overlap of their top-K salient update indices, and FedSelect (CVPR 2024) and TIES-Merging (NeurIPS 2023) select parameters by update magnitude. The contribution reduces to using this overlap as a clustering score.

  3. Key baselines are missing. PACFL (Vahidian et al., 2023) is cited but not compared, although it clusters clients from a few principal vectors per client and targets heterogeneity beyond label skew, which is exactly SPOS’s claimed goal. Without it, the case for sending tens of MB of indices per client per round (~44 MB for BERT) isn’t made.

  4. The evaluation mostly measures label-skew recovery. The oracle is label-histogram divergence on synthetic Dirichlet/pathological splits, where relatedness is by definition label similarity. Feature and concept shift, where SPOS should matter, get one small table. I suggest to highlight it instead.

  5. The scale is small: 30 clients, 10 rounds for the similarity experiments, synthetic splits, and CIFAR-10 accuracy around 33% on pathological splits. There’s no natural-partition FL benchmark and no PEFT/LoRA or foundation-model setting, which is where an index-only signal would matter (which is also the current standard).

Questions for authors

N/A

Reviewer_rM8iOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper explores how to calculate model correlations between clients in a federated learning, efficiently and securely. The author proposed SPOS, which is a similarity measurement method based on the degree of overlap of important parameter indexes. Specifically, this method determines the importance of parameters based on the magnitude of local model updates; Subsequently, an index of a set of important parameters was extracted through hierarchical normalization and Top-K filtering. By measuring the degree of overlap of these index sets between different clients to estimate correlation, there is no need to transmit complete real value update information for calculating similarity. However, this method mainly focuses on engineering implementation and lacks substantial innovation.

Critical strengths
  1. This paper focuses on how to efficiently and securely compute model correlation between clients in federated learning settings, which is an important research question.
  2. This paper selected multiple backbone networks for experiment, including CNN, ResNet18, BETR, and Wav2Vec2. The experiments cover multiple scenarios and have a large quantity.
Critical weaknesses
  1. The core issue of this paper is the lack of innovation, and the method design is very engineering oriented. In the method design of this paper, both the normalization strategy and Top-K screening mechanism are existing strategies. This paper only utilizes their combination, and the innovation of the method is not clear enough.

  2. Although a large number of experiments were conducted in this paper, there are still the following issues: Firstly, the comparison method is somewhat outdated and lacks relevant methods from recent years. Most of the comparison methods selected in this paper are very classic similarity measurement methods (such as cosine similarity), and the latest comparison methods have only been published in 2023. Secondly, this paper focuses on efficient calculation of model similarity and evaluates it through indicators such as recall rate. But is the ground truth of model similarity reliable? Is this method of calculating true values widely recognized? Is there any other way to truly verify the effectiveness of the proposed method in practical applications, rather than relying solely on the recall rate of manually defined ground truth. Finally, the experimental results indicate that this method cannot consistently maintain its advantages in all experiments and metrics. The underlying reasons should be analyzed and explained. And further summarize the usage scenarios and prominent advantages of this method.

  3. The writing quality of this paper needs to be improved. The logical structure of the paper is not clear enough, making it difficult to read. Especially in the experimental part, both the motivation and analysis of the experiment are difficult to understand.

  4. The presentation of experimental results in this paper needs improvement. Display the results of multiple methods and metrics in each table, but do not highlight the best in bold. This makes it difficult for readers to grasp the core information while reading the table.

Questions for authors

See Weaknesses.

Reviewer_PY5KOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies client similarity estimation in federated learning and proposes Salient Parameter Overlap Similarity (SPOS), which uses the overlap between the indices of salient parameters as an alternative to similarity measures computed from full model update vectors. The approach aims to reduce communication costs and limit the exposure of real-valued model updates.

Critical strengths
  1. The proposed method is simple and intuitive with a clear motivation, making the paper easy to understand.
  2. The experiments are reasonably comprehensive and generally support the paper's technical claims, making the paper technically sound.
Critical weaknesses

My main concern is the limited novelty of the proposed method. The core similarity measure, defined as the overlap ratio between the Top-K salient parameter index sets, has already been introduced in IndexGuard (ICML 2026). That work also empirically demonstrates the effectiveness of index-overlap similarity for distinguishing clients in federated learning, although in the context of backdoor detection.

While this paper extends the application to general client relatedness and introduces additional techniques such as layer-wise normalization, the fundamental similarity computation remains essentially the same. The manuscript cites IndexGuard but does not sufficiently clarify the methodological differences or establish what constitutes a substantially new contribution beyond the prior work.

Questions for authors
  1. What are the key novel contributions of this paper compared with IndexGuard? Given that the core index-overlap similarity measure has already been proposed and empirically validated, could the authors clarify which methodological or theoretical contributions are fundamentally new?

  2. Can the proposed similarity measure provide more direct benefits to downstream model collaboration? Beyond client clustering and neighbor retrieval, it would be interesting to investigate whether the estimated similarity can predict or improve the actual benefits of collaboration between clients. Although the paper reports clustered federated learning results, the current experiments mainly demonstrate that the proposed metric can preserve competitive performance in similarity-driven tasks. I am not fully convinced that similarity recovery alone constitutes a sufficiently important contribution. Demonstrating that this metric offers tangible advantages in practical downstream tasks, such as improving federated learning performance or enabling more effective model collaboration, would substantially strengthen the paper's contribution.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.