RelFed: Relative-Representation Federation of Heterogeneous Small Language Models for Multi-Site AIoT Sensing

Sites of one organization observe the same events through different sensor subsets while running small language models of different sizes, and such fleets cannot be federated by parameter averaging: the tensors have no common shape, and resize-and-average workarounds degrade accuracy. RelFed federates them exactly. Each client re-expresses its pooled features as cosine similarities to a shared public anchor set, which makes the representation dimension identical for every backbone width, and a single shared classifier head over this space is aggregated by plain FedAvg at roughly sixteen kilobytes per client per round. Across text and sensor benchmarks under literal feature partitions, RelFed beats the paired local-only floor in 37 of 41 seed comparisons, reaching +3.69 points on 20 Newsgroups with a bootstrap CI of [+2.98, +4.11]. Gains peak near twenty federation rounds and stay positive at longer horizons, concentrate where local training is weakest, and vanish under label-disjoint partitions, delimiting the method to what it actually transfers: feature-view knowledge.