Recovering Clean Representation Similarity with Independent Replicas

Sampling debiasing does not necessarily remove the effect of observation noise on centered kernel alignment (CKA). Even when noise leaves the cross-model HSIC numerator unbiased, it can inflate the two self factors and distort both similarity scores and layer rankings. We introduce Independent-Replica CKA (IR-CKA), which pairs Gram matrices from two conditionally independent observations inside each minibatch ${\text{HSIC}}_1$ factor and averages the factors before normalization. Under the stated mean-preserving noise assumptions, all three factors are unbiased for their clean finite-pool counterparts. Comparisons with feature averaging and $m$-replica cross-Grams identify the corrections retained when independence is used elsewhere in the construction. We establish concentration under random partitions of a fixed input pool and $O(n^{-1})$ mean-square error for PSD-stabilized IR with ridge ${\lambda }_n=O(n^{-1/2})$, under moment and positive self-factor margin conditions. Experiments with frozen ViT and ResNet representations show that IR and its variants recover clean similarity matrices and bidirectional layer relationships more accurately than shared-replica, feature-averaging, and cross-Gram baselines under moderate feature dropout. Under stronger corruption, PSD stabilization restores complete estimates and improves several retrieval measures, although accurate recovery is not uniform across layers or metrics. These results demonstrate the practical value of placing replica independence within all three HSIC factors for recovering clean representation similarity from noisy observations.