acceptodds
Under review as a conference paper at ICLR 2027

Decentralized Split Federated Learning with Asynchronous Rotating Committees

Abstract

Split Federated Learning (SFL) enables collaborative training of machine learning models on resource-constrained IoT devices by combining model splitting with federated optimization. Despite its promise, existing SFL systems fundamentally lack scalability due to centralized server bottlenecks, synchronous training barriers, and reliance on fixed infrastructure, that are major limiting factors in IoT environments. We propose ARC-SFL, a fully asynchronous and decentralized framework for SFL that eliminates permanent servers while preserving accuracy and efficiency benefits. ARC-SFL distributes server-side computation across replicated committee members via our load balancing scheme, and supports asynchronous aggregation to remove straggler blocking. To further reduce communication overhead, ARC-SFL employs event-triggered gossip, enabling peer-to-peer information exchange only when local training has made progress and the learned representation has drifted. Across multiple vision, medical, and audio benchmarks, and on real homogeneous and heterogeneous clusters of IoT devices, we show that ARC-SFL: (i) scales linearly with the number of devices, reaching an order of magnitude higher throughput than synchronous SFL at 100 devices, while remaining robust to device and committee failures; (ii) significantly outperforms decentralized gossip learning on all six tasks by up to 31.2% and, without any centralized server, remains on par with the best centralized task-dependent baselines; and (iii) speeds up the convergence by  3.5x and reduces gossip communication by up to 55% without degrading accuracy on compact models.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Federated Learning, Split Federated Learning, SFL, Decentralized Learning, Decentralized Federated Learning, TinyML, Asynchronous Training, Rotating Committees, Tiny IoT Devices, Edge AI, Gossip Learning, Scalability, Robustness, Early Exit

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_XwoAOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The work presents a framework for federated learning that extends existing SOTA with the ability to do split federated learning in a decentralized and asynchronous mode. The use case is to enable low-capability edge devices to participate in model training when devices have access to only a subset of the training data and also the model is too large for a single device so its training is split across two devices. Experimental results indicate that it is still possible to achieve an acceptable accuracy even under these constraints and requirement.

Critical strengths

The proposed approach is interesting because it allows to split model under training across multiple devices and at the same time eliminate the centralized and synchronous training schema. This comes with a significant amount of complexity though and the provided descripting and algorithms do not make it particularly easy to follow. Nevertheless, the work seems to be relatively novel and interesting.

Experimental results seem to indicate that a satisfactory performance can be achieved with this approach. Experiments carried out with multiple datasets and multiple models are compared to other federated learning techniques and indicate a superiority of the proposed method.

The paper is structured around answering critical questions to demonstrate the performance and scalability of the proposed training mode.

Critical weaknesses

One weakness of this work is that only toy datasets and small enough models have been used in conducting experimental study. It is not clear how the datasets were split between each subset of federated devices. For a real-life scenario there are probably other considerations that need to be taken into account, such as a disbalance between the amount of data in each federation and frequency with which a new data arrives (since these are IoT devises and presumably they receive new data from sensors.) This study is still valuable even under the idealized conditions, but its real application needs to be tested.

Questions for authors

Why stopping at splitting the model only across 2 devices? How would the proposed method work if the model is too large for two devices? How scalable is the proposed solution in this dimension?

1 comment
AuthorsOpenReview ↗

Review Response

We thank the reviewer for the positive assessment and for finding the approach novel and interesting. We address the questions and concerns below.

We have uploaded a revised version of the paper. It clarifies the communication experiment: the gossip volumes in Table 2 are estimates of the traffic each trigger would send, while the accuracies are measured and test whether filtering communication degrades the model. It also fixes a broken equation reference and corrects several grammatical errors.

Datasets and model size

Our target devices are tiny resource constrained microcontrollers such as the ESP32-S3, which have only a few hundred kilobytes of on-chip memory. To evaluate ARC-SFL across different model complexities and maintain comparability with prior SFL work, our simulation experiments include architectures up to ResNet-18 with approximately 11.2M parameters, of which 15–25% reside in the client-side partition depending on the architecture (Table 6). These larger model experiments evaluate the framework's behavior in simulation and should not be interpreted as demonstrating the feasibility of training ResNet-18 on tiny devices. Separately, we evaluate ARC-SFL on real IoT clusters using UCI-HAR and FashionMNIST (Section 4). UCI-HAR consists of actual device-attached sensor recordings and is particularly relevant to the IoT scenario described by the reviewer. Our simulation benchmarks follow established SFL evaluation practice, facilitating comparison with prior work, while also covering additional modalities such as medical imaging (MedMNIST) and audio (Speech Commands).

How the data is split

Each simulated benchmark is partitioned across 50 clients using a Dirichlet distribution with concentration parameter α=0.5\alpha = 0.5 (Section 4 and Table 3), a common approach for simulating non-IID federated data. This introduces heterogeneous label distributions across clients and can also result in unequal local dataset sizes. The hardware experiments on UCI-HAR follow the same Dirichlet configuration, with four training clients in the five-device cluster and eight training clients in the ten-device cluster (Fig. 14). ARC-SFL accounts for differences in client contributions through its aggregation mechanisms, i.e., gossip aggregation weights each peer by the number of samples it has processed (Eq. 2), while committee consensus weights each member by the number of activations it has served (Eq. 3). These mechanisms account for differences in participation and processed data volumes without requiring equal contributions from all devices.

To clarify the answers to the questions explicitly:

Q1: Why stop at splitting the model across two devices? Two segments is the standard SFL setting and already allows training models that do not fit on a single microcontroller. Every additional split adds a wireless communication hop, which is the main bottleneck on such devices, so two segments keep this cost low.

Q2: How would the method work if the model is too large for two devices? First, ARC-SFL already reduces how often this case occurs. Because clients and committee members are devices of similar capability, we split the model more evenly than typical SFL, where the client holds only a small part and a powerful server holds the rest (Section 3.1). Both devices' memory is therefore used, so a model has to be considerably larger before two devices are no longer enough. If it still does not fit, the model would be cut into more than two parts, and each additional part would get its own committee. For example, with three parts, each client holds the first part, a first committee holds replicas of the middle part, and a second committee holds replicas of the last part. In the forward pass, the client sends its activations to a member of the first committee, which computes the middle part and forwards its activations to a member of the second committee. That member computes the loss and sends the gradient back along the same path. The middle committee would work exactly like a client and a committee member at the same time: it receives activations from below and gradients from above. The existing mechanisms carry over to each of these links. Gradients that arrive late are down-weighted with the same staleness discount, and the middle part could use its own auxiliary head to keep training while it waits for gradients from the last part, just as clients do today. Both committees would rotate and synchronize their replicas as described in Section 3.

Q3: How scalable is the method in this dimension? Each extra segment adds one activation and gradient exchange per batch, staleness accumulates along the chain, and every additional committee is one more point where a failure can interrupt training. We therefore expect a small number of segments to be practical, but we have not evaluated more than two in this paper.

Reviewer_sL7UOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes ARC-SFL, a serverless and asynchronous split federated learning (SFL) framework for resource-constrained IoT devices. Instead of relying on a central server, the top model is hosted by a committee of M devices. Each committee member keeps a replica of the top model and serves a subset of clients, and the replicas are periodically aligned through participation-weighted averaging. The committee rotates every E_rot rounds by letting each member swap roles with one of its assigned clients, so that the server-side role is not permanently assigned to the same devices. On the client side, they use an auxiliary model allowing the bottom model to keep training while waiting for gradients from the committee. Bottom models are aggregated through gossip with staleness-aware weights, and an event-triggered rule based on local loss improvement and representation drift is used to reduce gossip traffic. For experiment, they evaluates the method on six benchmarks in simulation, as well as on homogeneous and heterogeneous ESP32-S3 clusters.

Critical strengths
  1. The motivation is clear and practically relevant. The reliance of existing SFL methods on a powerful, always-available server is a real limitation in IoT settings, and removing it is a meaningful goal.
  2. The paper goes beyond simulation and implements the method on real microcontroller clusters, which requires non-trivial engineering effort and is valuable for the community.
  3. The system design is complete, and the individual components, including the auxiliary head, the event trigger, the committee size and the staleness exponents, are studied in separate experiments.
Critical weaknesses
  1. The main accuracy comparison does not evaluate the full method. According to Section 4.1 and Table 4, the auxiliary network and committee rotation are disabled in the main comparison (Fig. 3), and gossip is triggered every round instead of by event as claimed in methodology. That being said, none of the three claimed key components is used in the main results. Moreover, Fig. 8 shows that rotation reduces accuracy to some extent, and Table 2 shows that the Loss+Drift trigger costs 2.1 points on TinyImageNet. Therefore, the claim that ARC-SFL “removes the server without giving up accuracy” is not directly supported.
  2. The fairness of the comparison in main experiment should be clarified. Table 4 shows that gradient clipping is applied only to ARC-SFL, while the baselines run without it, and all methods share the same learning rate without per-method tuning. In addition, the accuracy curves plot accuracy against communication rounds, which may favor ARC-SFL, since ARC-SFL performs additional auxiliary steps in each round. It would be more convincing to have curves against wall-clock time or total computation.
  3. Some important baselines and related works are missing. SCFL [1] is identified by the authors as the only existing serverless SFL method, but it is not compared for the lack of public code. This does not seem to be a sufficient reason, since the method could be re-implemented. In addition, ComDML[5] is also an existing serverless SFL method that focuses on the straggler issue. The discussion of recent work is also limited. For example, GAS [2], MU-SplitFed [3] and HASFL[4] also address the straggler problem, but neither is mentioned in this paper. The related work section should include more discussion of recent SFL studies.
  4. The scalability experiment (Fig. 5) does not provide a fair comparison. The claimed 11x throughput of ARC-SFL includes update from the auxiliary model, while the baseline Sync-SFL does not. When this update from auxiliary model is removed, the improvement drops to 2.9×. Moreover, ARC-SFL has multiple committee members while Sync-SFL has only one single server. The two systems have different amounts of computation power, so even this 2.9× improvement is not fair. A fair comparison should implement Sync-SFL with multiple servers like in SFLv1, or report time-to-accuracy as a function of the number of devices.
  5. The benefit of committee rotation is not validated. The paper states that the purpose of rotation is participation rather than speed, namely letting the data of every device shape the top model. However, no experiment measures this benefit. The only evidence in Fig. 8 shows that rotation undermines the accuracy.
  6. The communication-efficiency evaluation is incomplete. Table 2 only reports gossip traffic. In SFL, however, the transmission of activation and gradient often dominates the total communication, as can also be seen in Fig. 14c. The total communication reduction in simulation, including split-layer traffic, is not reported. In addition, contacting K=16 peers out of 50 devices in each gossip phase is a relatively dense communication pattern, and its cost should be discussed in relation to the communication-efficiency claim.

Minor:

The caption of Fig. 12 contains a broken reference (“Eq. (??)”), and there are some grammatical issues, e.g., “and but also reach to close proximity” on page 2. τ_k in Eq. (2) is not explicitly defined. Also, the sample count s_k used in the gossip weights is cumulative, so clients that have participated longer may dominate the aggregation. This effect should be discussed.

References:

[1] B. C. Tedeschini, M. Brambilla, and M. Nicoli. Split Consensus Federated Learning: An Approach for Distributed Training and Inference. IEEE Access, 12:119535–119549, 2024.

[2] J. Yang and Y. Liu. GAS: Generative Activation-Aided Asynchronous Split Federated Learning. AAAI Conference on Artificial Intelligence (AAAI), 2025.

[3] D. Liang, J. Zhang, E. Chen, Z. Li, R. Li, and H. Yang. Towards Straggler-Resilient Split Federated Learning: An Unbalanced Update Approach. In Advances in Neural Information Processing Systems (NeurIPS), 2025.

[4] Z. Lin, Z. Chen, X. Chen, W. Ni, and Y. Gao. HASFL: Heterogeneity-aware split federated learning over edge computing systems. In IEEE Transactions on Mobile Computing, 2026.

[5] S.M.S. Mohammadabadi, L. Yang, F. Yan, and J. Zhang. Communication-efficient training workload balancing for decentralized multi-agent learning. In IEEE 44th international conference on distributed computing systems (ICDCS), 2024.

Questions for authors
  1. Do the conclusions still hold if gradient clipping is also applied to the baselines and each method is tuned separately? Can the authors provide accuracy curves against wall-clock time or total computation?
  2. Can the authors compare with [1] and [5] in experiment, and discuss the differences from [2-4]?
  3. Can the authors provide a compute-matched scalability comparison, e.g., against SFLv1 with M server replicas, or report time-to-accuracy as the number of devices grows?
  4. Can the authors provide experimental evidence that rotation improves participation or fairness?
  5. What is the total communication cost in simulation, including split-layer activations and gradients, rather than gossip traffic alone?
4 comments
AuthorsOpenReview ↗

Review Response - Part 1

(1/2)

We respectfully disagree with the assessment that the paper offers little or no new insight. Our central contribution is a working system for decentralized training of models that exceed the capacity of a single tiny device, using ordinary devices rather than a dedicated server. The committee design, asynchronous aggregation, and hardware evaluation substantiate this contribution. We recognize that the presentation could distinguish the core configuration from the optional mechanisms more clearly, and address this below.

Revised paper. We have uploaded a revised version of the paper. It clarifies the communication experiment: the gossip volumes in Table 2 are estimates of the traffic each trigger would send, while the accuracies are measured and test whether filtering communication degrades the model. It also defines τk\tau_k and discusses the cumulative sks_k next to Eq. (2), fixes the broken equation reference in the Fig. 12 caption, and corrects several grammatical errors, including the one on page 2.

W1: The main comparison does not use the full method.

Fig. 3 intentionally isolates the core serverless asynchronous committee with staleness-aware aggregation: the auxiliary head, rotation, and event trigger are disabled because the baselines lack equivalent mechanisms. Their effects are evaluated separately (Figs. 4, 8–10; Table 2).

The system with the auxiliary head and rotation is also evaluated on the ESP32 clusters. It matches or exceeds SplitFed v1 in accuracy: 73.6% versus 71.1% on five boards and 72.8% versus 71.6% on ten boards, while completing 100 rounds in 38 minutes rather than 128–132 minutes. The statement “removes the server without giving up accuracy” refers to the core configuration of Fig. 3.

W2: Fairness (clipping, tuning, rounds vs. time).

Rounds axis. In Fig. 3, the auxiliary network is disabled, and ARC-SFL performs no additional auxiliary training steps. Every method processes the same number of split-learning batches per round. The round axis therefore compares equal batch counts, although it does not imply equal computation or elapsed time.

Wall-clock. This is reported on real devices clusters (Fig. 14d, Fig. 16). On the 10-board heterogeneous cluster, ARC-SFL reaches 70% accuracy after 27 min, compared with 76 min for SplitFed v1 and 35 min for SplitFed v2.

Clipping. Clipping is a standard stabilizer for asynchronous updates, not part of the contribution. The Fig. 3 baselines follow the original SplitFed setup, which does not clip. The component ablation (Fig. 8) is a matched comparison: SFL-Sync, Sync-Committee, and both ARC-SFL variants run in the same framework with the same clipping, and ARC-SFL still converges faster and ends higher (59.5% vs. 55.0% and 55.4% at round 90).

Tuning. All methods share one optimizer configuration. ARC-SFL's own hyperparameters are also fixed across all tasks (Table 5), and therefore no method, including ours, receives a larger tuning budget.

W3: Missing baselines and related work.

GAS, MU-SplitFed, and HASFL address stragglers in SFL, but all three rely on a central server or edge server that hosts the server-side model and coordinates training. They solve a different problem, which is making the server-based setup more efficient, rather than removing the server. ComDML targets workload balancing in decentralized learning by offloading work from slow to fast agents, and does not use a replicated, rotating server-side model or asynchronous staleness-aware aggregation. SCFL relies on synchronous, sequential consensus without failure handling; we characterize it in Table 1 and Section 2, and its synchronous design is exactly the bottleneck that ARC-SFL is built to avoid.

(continued in the next comment)

AuthorsOpenReview ↗

Review Response Part 2

(2/2)

W4: Scalability comparison.

We want to stress that the paper already separates the auxiliary contribution: lines 412–418 report split learning batches only, and ARC-SFL still processes 249 vs. 87 batches/s at 100 clients (2.9×2.9\times). Regarding compute, the same paragraph contains the decisive data point: with a single committee member (10 clients), ARC-SFL and Sync-SFL are on par (32 vs. 35 batches/s). The gain appears only when the server role is spread across several devices. This is not an unfair advantage but the central idea of the method: in a serverless deployment there is no powerful machine, so the only way to scale the server-side work is to distribute it across ordinary devices, which a single-server SFL design cannot do. Sync-Committee in Fig. 8 shows that distribution alone is not enough. With several committee members but synchronous operation, it reaches only 9 batches/s per client against 18.5 for ARC-SFL without rotation.

W5: Benefit of rotation.

The rotation ablation is in Fig. 8 (ARC-SFL with and without rotation). It shows that rotation raises throughput (18.5→20.918.5 \rightarrow 20.9 batches/s per client) at a small accuracy cost, which we report openly. Its main purpose follows from the design rather than from an accuracy gain. Without rotation, the devices that host the top model never train on their own data, and the same few devices permanently carry the heavier server-side workload. Rotation removes both problems. The hardware timeline (Fig. 15a) shows the top-model role moving between boards, and the robustness experiments run with rotation enabled (Table 5) without any loss in accuracy after 30% of the clients drop out (Fig. 6).

W6: Communication.

The total communication, including split-layer activations and gradients, is reported in Fig. 14c: ARC-SFL sends 35 MB vs. 49 and 47 MB for SplitFed v1 and v2 on 5 boards, and 71 MB vs. 107 MB on 10 boards, a total reduction of 29–34%. Split-layer traffic per batch is identical for ARC-SFL and SFL, since both use the same cut layer, and the auxiliary steps send nothing. Table 2 complements this by estimating how much the event trigger alone reduces gossip traffic; the revised paper states explicitly that these gossip volumes are estimates. Regarding K=16K = 16: the exchange is pull-based, and with the event trigger a client serves its model in only about 45% of rounds (Fig. 10).

Minor. The broken reference and the grammar on page 2 are fixed in the revised version. τk\tau_k is the round lag of peer kk relative to the receiving client, τk=max⁡(0,ri−rk)\tau_k = \max(0, r_i - r_k); it is now defined next to Eq. (2). The cumulative sks_k does give more weight to clients that have processed more data. This is intended, since such clients carry more information, and the staleness discount ϕβ(τk)\phi_{\beta}(\tau_k) reduces the weight of clients that have fallen behind, which prevents stale but long-participating clients from dominating.

We thank the reviewer again for the careful review and hope these clarifications address the concerns about novelty, experimental comparisons, and the roles of the individual components.

To clarify the answers to the questions explicitly:

Q1: Do the conclusions hold with clipping for the baselines and per-method tuning? Wall-clock curves? In the matched-clipping comparison of Fig. 8, ARC-SFL still converges faster and ends higher. We did not tune each method separately; all methods, including ours, share one fixed configuration. Accuracy against wall-clock time is reported on the real device clusters (Fig. 14d, Fig. 16); see W2.

Q2: Comparison with SCFL and ComDML; differences from GAS, MU-SplitFed, HASFL? We do not provide an experimental comparison with SCFL or ComDML. SCFL is synchronous and sequential by design, which is the bottleneck ARC-SFL avoids, and ComDML does not host a replicated server-side model. GAS, MU-SplitFed, and HASFL all depend on a central or edge server; see W3.

Q3: Compute-matched scalability? With a single committee member, ARC-SFL and Sync-SFL are on par (32 vs. 35 batches/s, Fig. 5), and the synchronous multi-member Sync-Committee reaches 9 batches/s per client against 18.5 for ARC-SFL (Fig. 8). Time-to-accuracy is reported on the 10-board cluster (Fig. 16), not as a function of the number of devices; see W4.

Q4: Evidence that rotation improves participation? We do not measure participation directly. The benefit follows from the design: without rotation, the committee devices never train on their own data and permanently carry the heavier workload; see W5.

Q5: Total communication in simulation? The total including split-layer traffic is measured on the real device clusters (Fig. 14c, 29–34% less than SplitFed), not in the simulation. In simulation, Table 2 estimates the reduction in gossip traffic from the event trigger; see W6.

Thank you for the detailed response. While some minor issues has been explained, most of my concerns remain unresolved, as detailed below.

  1. Unfair experimental comparison. The response is self-contradictory. If clipping is “a standard stabilizer” and not part of the contribution, it should be applied to all methods. If it is withheld from the baselines, it acts as part of ARC-SFL’s advantage. Either way, the current Fig. 3 comparison does not isolate the effect of the proposed method, and its conclusions rest on an unfair setup. Moreover, the response claims that Fig. 8 is a matched comparison with the same clipping, but the manuscript does not state this clearly.
  2. Missing baselines. ComDML is a serverless method with public code that, like ARC-SFL, uses local-loss split training to address stragglers. It is the most directly comparable baseline, yet it is not compared, and the paper states that “no decentralized SFL method is publicly available” (L315), which is factually incorrect. Moreover, although GAS, MU-SplitFed, and HASFL rely on a server, they target the same straggler problem; the paper should discuss them, but the list of revisions does not mention in the related work.
  3. Unvalidated benefit of rotation. The response confirms that the claimed benefit of rotation is not measured (Q4). The evidence illustrated does not fill this gap. The throughput increase in Fig. 8 comes from auxiliary steps that clients perform while devices swap roles, not from additional split-learning progress, and accuracy decreases. In Fig. 6, rotation is enabled in both the failure and no-failure runs, so the result says nothing about rotation. Moreover, Table 5 enables rotation only in the client-dropout experiment, not in the robustness experiments in general. Thus, the only measured effect of rotation is a loss in accuracy, while its intended benefit rests on a design argument alone.
AuthorsOpenReview ↗

Reviewer Response - Part 3

We thank the reviewer for the follow-up and address the three points directly.

1. Clipping. During development, we also ran the SFL baselines with the same gradient clipping as ARC-SFL. Clipping did not consistently improve them, and in some cases it noticeably decreased their performance or made training less stable, so Fig. 3 reports the baselines in their original SplitFed setup without clipping. In ARC-SFL, clipping serves to stabilize the asynchronous updates. Because we cannot confirm that these earlier runs used exactly the same settings as Fig. 3, we do not report their numbers here, as they would not constitute a controlled comparison. In the camera-ready version, we will add a controlled comparison of all baselines with and without clipping under the exact settings of Fig. 3. The matched comparison already in the paper is Fig. 8, where all variants run in the same framework with the same clipping.

2. ComDML and related work. We thank the reviewer for pointing us to ComDML. We were not aware of it: our literature search targeted split federated learning, and ComDML presents itself as workload balancing for decentralized multi-agent learning rather than as an SFL method. The statement in L315 reflects this search and is not accurate with respect to ComDML. Beyond terminology, the objectives also differ: ComDML balances training workload between heterogeneous agents, whereas ARC-SFL removes the server role by distributing a replicated, rotating top model across ordinary devices with asynchronous, staleness-aware aggregation. GAS, MU-SplitFed, and HASFL address the straggler problem within a server-based architecture, which is the dependency ARC-SFL is designed to remove.

3. Rotation. The primary purpose of rotation is a system-level property that holds by construction: without rotation, the M committee devices never train on their own data (6% of all devices in our default setting with M = 3 of 50), and the same devices permanently carry the heavier top-model workload. We initially also expected a small accuracy gain from including this data, but did not observe one, and we report the small accuracy cost openly. We hypothesize two reasons. First, with M = 3 of 50 devices, the additional data is a small share and its effect is hard to measure within 100 rounds. Second, each swap hands a device a bottom model that was trained on another device's data, which briefly perturbs training after every rotation. Rotating less frequently or handing over models more gradually could reduce this cost, which we consider future work.

We thank the reviewer again for the careful reading, which has improved the fairness of the main comparison.

Reviewer_uUY6OpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes ARC-SFL, a decentralized split federated learning framework for resource-constrained devices. Committee members execute replicated top models, while clients train bottom models and exchange parameters through gossip. Role rotation distributes server-side responsibilities, auxiliary heads support local training during waiting periods, and staleness discounts regulate asynchronous updates.

The evaluation includes six simulated benchmarks and homogeneous and heterogeneous ESP32-S3 deployments. The authors report competitive accuracy, improved throughput, reduced communication, and resilience to failures. A conditional stationarity analysis characterizes optimization under assumptions on disagreement, stale gradients, auxiliary updates, and coordination drift.

Critical strengths
  1. The microcontroller implementation is valuable. Experiments on homogeneous and heterogeneous ESP32-S3 clusters provide practical evidence beyond simulation. Communication breakdowns and time-to-accuracy measurements are useful additions.
Critical weaknesses
  1. Section 3.3 closes a committee round once a fraction ρ of assigned clients finishes, and Tables 4–5 specify ρ = 0.7. However, Appendix A, lines 990–996, explicitly states that the simulated implementation waits until all clients assigned to a committee return. It distinguishes this barrier-based implementation from the hardware version. This affects more than the terminology “fully asynchronous.” The two protocols have different straggler sensitivity and failure behavior. In particular, the dropout discussion attributes continued progress to not waiting for every client, which conflicts with the stated simulation barrier unless an additional timeout or membership-removal mechanism is used. The discrepancy also affects interpretation of gradient-staleness handling. Under strict within-committee round synchronization, the defined delay τi = max(0, R − ri) would ordinarily remain zero, absent additional counter-management mechanisms. The gradient discount would then equal one regardless of α. The authors should reconcile this implication with the interpretation of the α ablation in Figure 12 and identify which implementation generated each result.
Questions for authors
  1. Does the simulation close rounds at ρ = 0.7 or wait for all assigned clients? How are permanent dropouts handled? Under the implemented protocol, when does τi become nonzero?

  2. Which implementation and round-counter semantics generated the α ablation? How do they relate to the within-committee barrier described in Appendix A?

  3. Why is clipping enabled only for ARC-SFL? Are there existing matched-clipping results or ablations that establish its contribution?

1 comment
AuthorsOpenReview ↗

Response to Reviewer

We thank the reviewer for the detailed reading. While we acknowledge the potential inconsistencies identified in the presentation, we respectfully disagree that these issues undermine the paper's central contribution. We proposed a working system for decentralized training of models that exceed the capacity of a single tiny device, without a dedicated server. We clarify each point below.

We have uploaded a revised version of the paper. It clarifies the communication experiment, explains in the robustness section that failed clients are removed from their committee, fixes the broken equation reference in the Fig. 12 caption, and corrects several grammatical errors.

Q1 (round close, dropouts, τi\tau_i)

Once ⌊ρ⋅N/M⌋\lfloor \rho \cdot N/M \rfloor clients of a committee have finished their split learning batches, the committee member signals the close. The remaining clients then stop auxiliary training and move on to gossip. This value is also used as the minimum number of clients a committee needs in order to continue. The DFL round itself, however, only ends once all connected clients have returned. This is the barrier described in Appendix A. The simulation does not wait for the slow clients' extra auxiliary steps, but it does wait for their split learning work. On the real devices clusters there is no such barrier. We call it asynchronous across committees and on the client–committee link, with a barrier inside each committee.

For dropouts, the failed clients' processes are terminated. Their connections close, and they are removed from the committee's client set, and the round finishes with the remaining clients. In the robustness runs ρ=0.4\rho = 0.4, which means a committee keeps going as long as 40% of its clients are alive.

Q2 (α\alpha ablation)

Fig. 12 was not produced by the Flower simulation. We therefore wrote a separate simulator with the same update rules (both staleness discounts, committee consensus every round, the same gossip weighting), in which clients can genuinely fall behind. In the Fig. 12 runs, 60% of the clients train only every fifth round, so their round counter lags further and further behind (τi\tau_i reaches about 40 by the end). These clients also never receive gossip. The setup is CIFAR-10 with MobileNetV2, 20 clients, a committee of three, 50 rounds, and two seeds.

Figure 12 examines the interaction between the two staleness discounts. At the default β=2\beta = 2, performance is insensitive to the tested values of α\alpha; sensitivity to α\alpha emerges when β\beta is reduced. Removing the gossip staleness discount reduces accuracy to 13%, highlighting its importance under the evaluated heterogeneous setting.

Q3 (clipping)

Clipping is there to keep the asynchronous updates stable; it is not part of the contribution. For the Fig. 3 baselines we followed the original SplitFed setup, which does not clip. There is, however, already a matched comparison in the paper. In the component ablation (Fig. 8), SFL-Sync, Sync-Committee, and both ARC-SFL variants run in the same framework with the same clipping (max norm 1.0). There, ARC-SFL processes 33–4×4\times more batches per second, reaches 45% at round 40 instead of round 70, and ends higher (59.5% vs. 55.0% and 55.4% at round 90).

We thank the reviewer again for the careful review. We maintain that the system design and hardware evaluation substantiate the paper's central contribution, and hope these clarifications support reconsideration of the assessment.

To clarify the answers to the questions explicitly:

Q1.1: Does the simulation close rounds at ρ=0.7\rho = 0.7 or wait for all assigned clients? Both, at two levels: ρ\rho ends the auxiliary (reactive) phase, while the round itself ends only once all connected clients have returned.

Q1.2: How are permanent dropouts handled? Disconnected clients are removed from the committee's client set, and the round finishes with the remaining clients.

Q1.3: When does τi\tau_i become nonzero? In the main simulation almost never; on the real device clusters whenever a client lags behind the fastest client of its committee; and in the straggler simulator of Fig. 12 by design.

Q2.1: Which implementation generated the α\alpha ablation? A separate straggler simulator, not the Flower simulation.

Q2.2: Which round-counter semantics? Each client keeps its own round counter rir_i, and τi=R−ri\tau_i = R - r_i with RR the current global round. Slow clients advance rir_i only every fifth round, so τi\tau_i grows over training.

Q2.3: How does this relate to the barrier in Appendix A? This simulator has no within-committee barrier; we wrote it precisely because the barrier keeps τi\tau_i at zero.

Q3.1: Why is clipping enabled only for ARC-SFL? To stabilize the asynchronous updates; the baselines follow the original SplitFed setup, which does not clip.

Q3.2: Are there existing matched-clipping results? Yes, the component ablation in Fig. 8.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.