acceptodds
Under review as a conference paper at ICLR 2027

RelFed: Relative-Representation Federation of Heterogeneous Small Language Models for Multi-Site AIoT Sensing

Abstract

Sites of one organization observe the same events through different sensor subsets while running small language models of different sizes, and such fleets cannot be federated by parameter averaging: the tensors have no common shape, and resize-and-average workarounds degrade accuracy. RelFed federates them exactly. Each client re-expresses its pooled features as cosine similarities to a shared public anchor set, which makes the representation dimension identical for every backbone width, and a single shared classifier head over this space is aggregated by plain FedAvg at roughly sixteen kilobytes per client per round. Across text and sensor benchmarks under literal feature partitions, RelFed beats the paired local-only floor in 37 of 41 seed comparisons, reaching +3.69 points on 20 Newsgroups with a bootstrap CI of [+2.98, +4.11]. Gains peak near twenty federation rounds and stay positive at longer horizons, concentrate where local training is weakest, and vanish under label-disjoint partitions, delimiting the method to what it actually transfers: feature-view knowledge.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
federated learning, heterogeneous architectures, small language models, relative representations, feature-view heterogeneity, AIoT sensing, communication efficiency

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_zFm3OpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper targets multi-site AIoT federation: sites of one organisation observe the same machines through different sensor subsets, each running a small language model of width tied to its hardware budget. The authors formalise this as feature-view heterogeneity (every client covers all classes but none observes all evidence) and a pre-registered 756-cell campaign shows weight-space resize-and-average and logit-consensus distillation both fail. They propose RelFed: each client re-expresses its pooled hidden state as cosine similarities to a shared anchor set of M=200 task-domain texts, producing a representation whose dimension depends only on M. A single shared linear head (C × M) is aggregated by plain sample-weighted FedAvg, with no resize, no interpolation, no distillation; communication is ~16 KB/round. On 20 Newsgroups RelFed gains +3.69 pp, with 37/41 paired wins and an honest operating envelope over ~120 cells.

Critical strengths
  1. Conceptual contribution: feature-view heterogeneity as a distinct regime, with the right diagnostic pre-registered campaigns. Naming feature-view heterogeneity and showing that the two natural workarounds fail (756-cell resize-and-average campaign + logit-distillation campaign) is a clean piece of conceptual work. The 11 resize operators and the +0.16 pp best noise-level result are unusually thorough.

  2. Methodologically tight and exactly-parameter-federating. Building on Moschella et al. (2023), the paper lifts relative representations from a post-hoc stitching tool to a train-time substrate. The single linear head W_{\text{rel}} \in \mathbb R^{C \times M} is aggregated by plain sample-weighted FedAvg; the 16 KB per client per round budget (229–542× smaller than shape-incompatible LoRA exchange, 4–6 orders of magnitude below full-model exchange) is rare and very concrete for AIoT deployment.

Critical weaknesses
  1. The confirmed-effect subset is narrow. Table 1 reports +3.69 pp on 20 Newsgroups with CI [+2.98, +4.11] and 5/5 seeds, but Yahoo (+1.27, CI [-0.10, +2.70], 3/5) and HAR (+0.94, CI [-0.78, +2.42], 6/8) are explicitly labelled "consistent trends, not confirmed effects". The 41 paired runs and the 37/41 sign test are genuinely strong paired evidence, but the more conservative reading is that the confirmed improvement is concentrated on 20 Newsgroups. The operating envelope (Table 10) reinforces this: AG-News at -0.98 pp CI [-1.57, -0.39], Yahoo-seq256 at -0.33, HAR-Qwen-det at -0.89, and DBpedia +0.61 against an 82.3 % floor.

  2. The protocol scaling is narrow and the proprietary industrial pilot is unquantified. The main protocol uses N=5 clients with 400 examples each; client-count scaling is deferred. The report explicitly defers client-count scaling, and the claim "federating frequently doubles the gain of federating rarely" only varies aggregation round rather than client count. The proprietary multi-site industrial pilot that motivates the paper is included as motivation only — no reproducible number rests on it — so the generalisation from public benchmarks (20NG / Yahoo / HAR) to a real factory fleet is unverified.

Questions for authors
  1. Could you explicitly report a confirmed-effect subset — e.g. all rows in Table 10 whose bootstrap CI excludes zero and whose per-seed sign-consistency is \ge 4/n? The 37/41 figure is correct as a paired count, but the headline narrative is more interpretable next to a count of confirmed-cell wins.

  2. The main protocol uses N=5 clients with 400 examples each. Could you add a small client-scaling experiment (e.g. N \in 5, 10, 20 on 20NG) and a longer-input + same-data arm so the reader can separate “more data” from “more clients”?

  3. The proprietary industrial pilot is mentioned only as motivation. Could you report order-of-magnitude numbers (federated floor, paired delta, sign-consistency across the available deployments) without revealing the system? That would let the reader judge how far the public-benchmark results travel toward the real multi-site AIoT deployment that motivates the paper.

3 comments

We thank the reviewer for the careful reading and for the precise suggestions, which we have adopted.

Q1 / W1 (confirmed-effect subset). This suggestion strengthens the paper, and we now report the confirmed-effect subset explicitly. Applying the reviewer's criterion (bootstrap 95% interval excludes zero and per-seed sign consistency at least 4 of n) to all 17 configurations with at least five paired seeds, four configurations are confirmed positive: 20 Newsgroups with the mixed fleet (+3.69 pp, [+2.98, +4.11], 5/5), movie-review sentiment with the Qwen fleet (+2.83 pp, [+0.95, +4.71], 5/5), 20 Newsgroups with the Qwen fleet (+2.79 pp, [+2.04, +3.31], 5/5), and the sensor task with the Qwen fleet (+1.39 pp, [+0.64, +2.14], 5/5). Every one of these passes with all five seeds positive, and together they span two modalities (text and serialized sensor windows) and two independent backbone fleets, so the effect is neither specific to 20 Newsgroups nor to one architecture mix. This is exactly the behaviour the method predicts: RelFed transfers feature-view evidence, so the gain is largest where partial views leave local training short of the task ceiling, as the operating envelope in the appendix maps out. We now report this subset in the abstract and the main results, alongside the 37/41 paired count.

W2 / Q2 (client scaling, more data vs more clients). We ran exactly this experiment on 20 Newsgroups with five paired seeds per cell. With 400 examples per site, so that total data grows with the fleet, RelFed gains +3.69 pp at N = 5 (CI [+2.98, +4.11], 5/5), +1.90 pp at N = 10 (CI [+0.10, +3.39], 4/5) and +3.67 pp at N = 20 (CI [+3.02, +4.42], 5/5). With 2,000 examples in total, so that each site holds 200 and then 100 examples, it gains +2.83 pp at N = 10 (CI [+2.29, +3.52], 5/5) and +0.86 pp at N = 20 (CI [+0.56, +1.15], 5/5). All five cells have an interval above zero. The gain therefore persists as the fleet quadruples at fixed data per site, so it is not a data effect, and it survives even when each of twenty sites keeps only 100 examples. All fleets remain architecture-heterogeneous, with backbones assigned round-robin over the same four-model pool.

Q3 (industrial pilot). Beyond the accuracy range already stated in Section 4.1, we are not able to release numbers from the industrial deployment under the terms of that collaboration, and we do not have the right to publish them, since the deployment data partly include healthcare data collected in Tokyo and are subject to the corresponding privacy regulations. We have therefore revised the paper so that the pilot appears only as motivation for the problem setting, and we state explicitly that no claim in the paper rests on it. Every reported number comes from public benchmarks and regenerates from the released artifacts.

Reviewer_zFm3OpenReview ↗

The statement of AI documentation

I have filled out the AI documentation. The review was independently assessed by me. An LLM was only used for language polishing; all technical evaluation and the final score are my own judgment. There is no mismatch between my human assessment and the AI-assisted drafting. I will actively join the discussion with authors and the AC during the discussion period.

Thank you for your prompt reply and for carrying out the extra experiments.

W1/Q1 (Confirmed-effect subset). This is the analysis that has been performed. Applying the criterion to the 17 configurations with at least five paired seeds yielded four confirmed positive cells: 20NG mixed (+3.69, 5/5), movie-review sentiment with the Qwen fleet (+2.83, 5/5), 20NG with the Qwen fleet (+2.79, 5/5), and the sensor task with the Qwen fleet (+1.39, 5/5). Two of the four are still 20NG; however, the movie-review and sensor results from the two modalities and fleets are in line. The confirmed subset is no longer 20NG-only. The 37/41 paired count remains a useful summary; the full table plus the operating envelope is what I will read when judging the general claim.

W2/Q2 (Client Scaling). The N=5/10/20 experiment is the appropriate design, and the two arms are "more data" and "more clients". At 400 examples per site, the gains are +3.69 at N=5, +1.90 at N=10 (CI +0.10 to +3.39, 4/5), and +3.67 at N=20; at a fixed total data, they are +2.83 at N=10 and +0.86 at N=20, both with intervals above zero. This supports the claim that the effect is not merely a data effect, and that it survives 100 examples per site. Two additions would help here: please include the table with per-seed values in the response, and explain the N = 10 dip at 400 examples per site, since the non-monotonicity between N = 10 and N = 20 is not obvious. The fixed-total arm is the simpler test, and the expected shrinkage is observed.

W3/Q3 (Industrial Pilot). I accept that the industrial data will not be disclosed. The paper now presents the pilot as motivation only and states that there is no claim based on it; that is the correct solution. I do not need the proprietary numbers and will not penalise the paper for not including them. Do not state in the abstract or introduction that the pilot study provides quantitative verification of the method.

General. The additional experiments are real work, and I appreciate the care taken in running them. After re-reading them, however, I do not think they yet change my overall assessment. The new experiments answer the questions I asked, and I do not dispute their execution. What they do not change is that the confirmed evidence for the reference mixed fleet still comes from one dataset, while the two new confirmed cells use a single model family; the fixed-total arm also shrinks to +0.86 pp. I therefore keep the rating at 2 for this round, and the decisive item is a confirmed cell on a second dataset under the mixed fleet rather than further reporting of the existing ones.

Reviewer_Nr6UOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes RelFed, a federated learning method that trains a heterogeneous model architectures across clients. Clients have access to public data referred to as anchors (200 anchors versus 400 private training samples); clients compute cosine similarity between local data embeddings and anchor embedding and then trains a classifier head over these representations and the classifier weights are shared and aggregated via fedavg. The paper compares the method against a pool of baselines (FedAvg + weight resize, FedGH, FedProto, FedMD, FedDF).

Critical strengths

Strengths

  • The paper addresses a relevant problem in federated learning, namely training across clients with heterogeneous model architectures.
  • The proposed approach is conceptually interesting and enables communication-efficient federation through a shared representation space.
  • The experimental evaluation considers different datasets and heterogeneous model architectures, with ablation studies investigating some design choices.
Critical weaknesses
  1. Insufficient and questionable baseline comparisons. Comparisons with competing federated learning methods are relegated to the appendix, where nearly all adapted baselines underperform local-only training. The explanations for these negative results are not sufficiently convincing, raising concerns about the fairness and reliability of the comparisons.

  2. Limited experimental validation. The improvements over local-only training are modest and statistically inconclusive on two of the three main datasets (Table 1). Experiments involve only five clients with a largely fixed architecture composition, limiting the generalizability of the findings.

  3. Unclear contribution of public data and federation. The method relies on a substantial amount of shared public data (200 anchors versus 400 private examples per client). Although a non-federated relative-head ablation is provided, the paper does not establish whether similar improvements could be achieved by exploiting the same public data through conventional semi-supervised learning, without federation.

Questions for authors

Q1. Can the authors provide detailed descriptions of baselines tuning that allow complete reproduction of results.

Q2. Can the authors Include a local-only baseline that uses the same unlabeled public examples (without federation) to augment local data?

Q3. Can the authors provide a cost-matched comparison (communication and/or computation) with FedMD and FedDF or other work representative of the state-of-the-art of anchor-based federated training.

Q4. Related work does not seem complete and very updated: Table 9 (appendix) compares the proposed approach with methods from 2023 or previous. Can the authors discuss on more recent (or on the absence of) work that can be used as competitive baselines?

3 comments

We thank the reviewer for the critical reading. We address the three weaknesses and four questions in turn and have added the requested controls.

Q2 / W3 (public data without federation). This is the right control, and we have run it. Each client applies conventional self-training to the same 200 public anchor texts that RelFed uses: from round 2 on it pseudo-labels the anchors with its own current classifier, keeps those whose confidence is at least 0.5, and trains on them alongside its private data. Nothing is exchanged between clients. On 20 Newsgroups under the reference protocol, with five paired seeds, self-training on the anchors falls to 26.2% against a paired local-only floor of 34.3% (-8.04 pp, 95% CI [-9.28, -6.80], 0/5 seeds positive), whereas RelFed improves on its paired floor by +3.69 pp (CI [+2.98, +4.11], 5/5). Together with the non-federated relative-head ablation already in the paper (identical architecture, aggregation switched off), this separates three sources of gain: access to public text, the relative-representation architecture, and federation itself. The reading is unambiguous: the public text by itself does not help, and used through conventional semi-supervised learning it hurts, because under partial views a client's confident pseudo-labels are often wrong and self-training reinforces them. The gain therefore comes from using the anchors as a shared coordinate system for an aggregated head, not from extra training data. We also note that the anchors are unlabelled and that RelFed never trains on them; they serve only as reference points for the cosine coordinates.

W1 / Q1 (baselines and tuning). We agree, and we have updated the baseline experiments accordingly. The distillation families (FedMD-style and FedDF-style) and the strongest weight-space operator have been rerun at exactly the reference protocol of the main results (20 Newsgroups, featureskew, 20 rounds, one local epoch, the same five-client fleet, five paired seeds against the same local-only floors), and the appendix lists every baseline configuration we ran across all campaigns. At this protocol, logit consensus (FedMD-style) with a small distillation weight reaches +4.82 pp over the floor (alpha = 0.1, 5/5; +4.21 at alpha = 0.5, +2.96 at alpha = 1.0), the strongest weight-space operator +2.39 pp (5/5), and server distillation (FedDF-style) +1.88 pp (4/5), against +3.69 pp (5/5) for RelFed. RelFed obtains its gain at a markedly lower cost than the strongest baseline: 140 s per round and 12.8 GB peak memory, against 226 s and 17.9 GB for FedMD-style and FedDF-style training (1.6x the time, 1.4x the memory), because it needs no per-round inference over a public probe set, no consensus computation, and no distillation pass; and it is the only method in the comparison that federates an exact parameter, with no resizing and no distillation target. The two mechanisms are complementary rather than competing: RelFed combined with FedMD-style distillation reaches +6.16 pp over the floor (CI [+5.65, +6.67], 5/5), the best result of the whole comparison, and adds +1.34 pp (CI [+0.50, +2.25]) on top of FedMD alone in a same-seed comparison. RelFed is therefore an exact, cheap channel that strengthens the leading baseline rather than a replacement for it. We summarize these results in Section 4.2 of the revision and report the full table with their costs in the appendix.

Q3 (cost-matched comparison). The measured costs answer this directly. Per client and per round, RelFed uploads one C x M head (16 KB at C = 20, M = 200), and logit consensus uploads P x C logits (16 KB at P = 200 public probes), so their communication is matched. Compute is not: FedMD-style and FedDF-style training take 226 s per round and 17.9 GB against 140 s and 12.8 GB for RelFed on the same hardware, since every round they run every client over the full probe set and add a distillation pass, and FedDF additionally trains a server model. The weight-space operator takes 186 s per round and uploads the full shape-incompatible LoRA adapters, 229 to 542 times RelFed's payload.

Q4 (recent related work). We thank the reviewer and have extended the related work with 2024 to 2026 methods. The most recent heterogeneous federated LoRA methods, such as SeFoRA (Xia et al., 2026) and FedPA-LoRA (Jeon et al., 2026), relax the adapter rank but aggregate updates of one shared weight shape; they do not apply when clients run different backbones of different widths, which is our setting. Prototype methods for heterogeneous models, such as FedTGP (Zhang et al., 2024), exchange class prototypes in a common feature dimension, which reintroduces the shape problem RelFed avoids. Distillation-based transfer through public data belongs to the logit-consensus family, which we now evaluate at the full protocol and show to be complementary to RelFed. We position these methods in the revised related work.

W2 (modest effect, five clients). We now scale the federation from five to twenty sites on 20 Newsgroups with five paired seeds per cell: at fixed data per site RelFed gains +3.69, +1.90 and +3.67 pp at N = 5, 10 and 20, and at fixed total data +2.83 and +0.86 pp at N = 10 and 20, with every interval above zero. On the strength of the evidence: four configurations meet the strict criterion of a bootstrap interval excluding zero with all five seeds positive, spanning text and sensor tasks and two independent backbone fleets, and 37 of 41 paired comparisons improve overall (one-sided sign test p = 5.1e-8). The effect is therefore statistically robust, and the revision states both figures in the abstract.

Reviewer_Nr6UOpenReview ↗

Assessment of the rebuttal and remaining concerns

I thank the authors for their detailed rebuttal and the substantial additional experimental effort.

While some concerns have been satisfactorily addressed, others remain only partially resolved. I elaborate in the following.

W1 / Q1: Baseline comparisons and reproducibility (partially addressed).

The new experiments in Table 11 substantially improve the evaluation, particularly as several baselines now outperform local-only training, alleviating my initial concerns about their reliability. The additional tuning details are also appreciated.

However, some concerns remain:

First, FedMD-style now outperforms RelFed (+4.82 vs. +3.69 pp), although at a higher computational cost, shifting RelFed's main advantage toward efficiency and complementarity. The combined RelFed + FedMD results (+6.16 pp) are interesting, but I wonder why these important comparisons are still relegated to the appendix rather than presented in the main manuscript.

Second, the adaptations of FedGH and FedProto remain questionable. Table 9 describes both through resized classifier-head aggregation, whereas their original mechanisms rely on server-side head training and prototype aggregation, respectively. It is unclear whether these adaptations faithfully preserve the original methods, potentially affecting comparison fairness.

Finally, the new comparisons with competing FL methods (FedMD, FedDF, and weight resize-and-average) are restricted to 20 Newsgroups (Table 11).

W2: Experimental validation and statistical evidence (partially addressed).

The additional experiments with 10 and 20 clients, alternative configurations, and tasks strengthen the evaluation and mitigate my original concerns. However, the improvements on Yahoo Answers and UCI-HAR remain statistically inconclusive under the main protocol. I also have reservations about the 37/41 positive comparisons and the associated sign test, as most originate from the same dataset 20 Newsgroups.

W3 / Q2: Contribution of public data and federation (largely addressed).

I appreciate the new self-training baseline and the non-federated relative-head ablation, which largely address my concern by better isolating the contribution of federation. However, the negative self-training result (-8.04 pp) should not be generalized to semi-supervised learning more broadly, as only one pseudo-labeling configuration is evaluated.

Q3: Cost-matched comparisons (partially addressed).

The new communication and computational cost measurements are useful and support the efficiency advantages of RelFed. In particular, the comparison with FedMD under nearly identical communication budgets provides valuable evidence. I note, however, that the FedMD/FedDF runtime and memory measurements mentioned in the rebuttal do not appear in Table 14 of the revised manuscript (as far as I can see).

Q4: Recent related work (addressed extending related work coverage).

Reviewer_prU2OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

To address the challenge of directly aggregating model parameters in multi-site sensing scenarios—where clients employ language models of varying scales and can only observe partial features—the authors propose RelFed. This framework transforms clients' hidden representations into cosine similarities relative to shared public anchor points, thereby constructing representations with consistent dimensionality, and aggregates the associated small classification heads using FedAvg. Local classification heads are trained jointly with the relative representation-based heads, and predictions from both are fused during inference. The authors validated the method using text and serialized sensor tasks and analyzed the sources of performance gains through experiments varying communication rounds and anchor point counts, as well as comparisons against non-aggregation baselines. In my view, this work offers an elegant and insightful approach to heterogeneous model collaboration; its low communication overhead and proactive analysis of applicability make it practically valuable.

Critical strengths

1.The design of shared objects is not only elegant but also resolves interface issues between heterogeneous models.The authors effectively decouple model heterogeneity from the scale of shared parameters; this represents the paper's most significant practical contribution.

2.The dual-head joint training scheme enables the local classification head to utilize raw hidden features while the relative classification head utilizes a shared reference space; the losses from both heads jointly update the local LoRA.

Critical weaknesses

1.The construction of RelFed's relative representations does not appear to rely on LoRA parameterization. Have the authors considered applying it to full fine-tuning on the client side? Or have they examined RelFed's impact across various fine-tuning schemes?

2.Scalability regarding the number of clients remains to be further addressed. The authors could attempt testing using small models with a large number of clients.

Questions for authors

1.Could the authors provide additional data on the per-round execution time, anchor point refresh latency, and peak GPU memory usage on the existing hardware? This would help readers better evaluate RelFed's actual efficiency.

2.Could the authors provide additional ablation studies regarding the anchor refresh frequency, comparing the accuracy and computational overhead of refreshing every round, refreshing every few rounds, and using fixed initial anchor representations? This would help readers understand the necessity of the refresh strategy and demonstrate the balance achieved between performance and efficiency.

2 comments

We thank the reviewer for the positive assessment and for the concrete efficiency questions.

Q1 (runtime, refresh latency, memory). We instrumented the training loop and report, on a single NVIDIA RTX PRO 6000 Blackwell GPU: at the reference protocol (20 Newsgroups, five clients, 20 rounds; mean over five seeds), a federation round takes 140 s for RelFed against 121 s for the local-only floor (+16%). The anchor refresh, one forward pass over the M = 200 anchors under the current adapter, takes 2.76 s per client per round. Peak allocated GPU memory is 12.8 GB per training process for the whole five-client fleet, identical to the floor, because the anchor embeddings are a 200 x d matrix per client. Wall times were measured with five runs sharing the GPU, so they are upper bounds on dedicated-hardware cost.

Q2 (refresh frequency). We ran the requested ablation on 20 Newsgroups with five paired seeds: refresh every round (paper default), every five rounds, and never (anchor embeddings fixed at their initial values). Every arm improves on its paired floor on all five seeds, and accuracy rises monotonically with refresh frequency: fixed anchors give +2.16 pp (CI [+0.78, +3.54], 5/5), refreshing every five rounds +2.61 pp (CI [+1.45, +3.58], 5/5), and refreshing every round, the paper default, +3.69 pp (CI [+2.98, +4.11], 5/5). The cost side is small: one refresh is a 2.76 s forward pass per client, so per-round refresh adds about 16% wall time over the floor, against about 3% with refresh every five rounds and 1.6% with fixed anchors. The refresh is therefore useful but not critical: RelFed already beats the floor with fixed anchors, and per-round refresh buys the full gain for a modest cost; deployments with tight compute can refresh every few rounds and keep most of it.

W1 (LoRA vs other fine-tuning schemes). The reviewer is right that the relative representation does not depend on LoRA: it needs only a pooled hidden state and the same encoder applied to the anchors. LoRA matters only for the client-side cost and for keeping the backbone frozen. We state this independence explicitly in the method section.

W2 (number of clients). We scaled the federation from five to twenty sites on 20 Newsgroups with five paired seeds per cell. At fixed data per site RelFed gains +3.69, +1.90 and +3.67 pp at N = 5, 10 and 20; at fixed total data, where each site keeps 200 and then 100 examples, it gains +2.83 and +0.86 pp at N = 10 and 20. Every interval excludes zero.

Thanks for the authors' responses, I think 3 is too enough for this paper.

Comments on the paper

1 comment

We thank all three reviewers. The reviews agree on the contribution (a dimension-free shared object that lets clients with different backbones federate exact parameters at about 16 KB per round) and raise three common requests, which we have addressed with new experiments and revisions.

  1. Number of clients (all reviewers). New experiment: N in {5, 10, 20} on 20 Newsgroups, five paired seeds per cell, with per-client data held fixed (total data grows with N) and with total data held fixed (per-client data shrinks with N), so the effect of more clients is separated from the effect of more data. The gain holds at every fleet size: +3.69, +1.90 and +3.67 pp at N = 5, 10 and 20 with fixed data per site, and +2.83 and +0.86 pp at N = 10 and 20 with fixed total data, every interval above zero.

  2. What the gain comes from (Nr6U, prU2). New control: local self-training on the same 200 public anchor texts, with no federation. It falls 8.04 pp below the local-only floor (0/5 seeds), whereas RelFed gains +3.69 pp over its paired floor (5/5), so the gain does not come from the public text itself. New ablation: anchor refresh every round, every five rounds, or never; all three beat the floor on every seed, with the gain rising monotonically with refresh frequency (+2.16, +2.61, +3.69 pp). New measurements: RelFed adds 16% wall time per round and no peak memory over the local-only floor, with a 2.76 s anchor refresh per client per round.

  3. Strength of the evidence (zFm3, Nr6U). Four configurations meet the strict criterion of a 95% interval above zero with all seeds positive (20 Newsgroups with two backbone fleets, movie-review sentiment, and the sensor task), spanning both modalities and two independent fleets, and 37 of 41 paired comparisons improve (sign test p = 5.1e-8). The abstract and main results now state both.

We also reran the distillation baselines and the strongest weight-space operator at exactly the reference protocol, summarized in Section 4.2 of the revision with the full table and costs in the appendix. RelFed combined with the strongest of them, FedMD-style distillation, gives the best result overall (+6.16 pp, 5/5), +1.34 pp above that baseline alone, so the two are complementary. The appendix lists every baseline configuration we ran, and the related work now covers 2024 to 2026 methods. The industrial pilot is now presented as motivation only, and no claim rests on it.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.