acceptodds
Under review as a conference paper at ICLR 2027

PyramidSFL: Resource-aware Split Federated Learning via Adaptive Model Splitting

Abstract

Split Federated Learning (SFL) has emerged as a promising distributed learning paradigm that offloads part of the on-device training workload to a resource-rich server while keeping raw data at the edge. However, its effectiveness is often challenged by heterogeneity in both data silos and client resources, which jointly undermine model performance and training efficiency. This paper proposes PyramidSFL, a system-efficient Stacked-Activation SFL (SA-SFL) framework that dynamically partitions the model between clients and the server according to each client's communication and computation profile. To counteract the accuracy drop caused by dynamic model splitting, PyramidSFL combines a warm-up strategy that stabilizes early global optimization with a weight-shrinking mechanism that reduces cross-layer update inconsistency. This enables efficient training without sacrificing model quality, leading to a better time-to-accuracy performance. Extensive experiments on real-world benchmarks show that PyramidSFL consistently improves the accuracy-latency trade-off over existing FL and SFL baselines, improving global model accuracy by up to 15.53% under the same wall-clock budget and reducing convergence latency by up to 72.86%. Its gains become more significant under tighter client resource constraints and deeper task models, highlighting its effectiveness for heterogeneous and resource-constrained edge training.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Federated Learning, Split Federated Learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_hFaoOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes PyramidSFL, a resource-aware stacked-activation Split Federated Learning framework that dynamically selects client-specific cut layers according to computation and communication resources. To address the optimization inconsistency introduced by heterogeneous cut layers, the method combines a bandwidth-prioritized warm-up phase with layer-wise weight shrinking, while retaining a shared server-side model. The paper also provides a convergence analysis of fixed-cut SA-SFL and experimentally evaluates the method on CIFAR-10, CIFAR-100, and Tiny-ImageNet.

Critical strengths
  1. The paper addresses a practically relevant problem by jointly considering system heterogeneity and optimization degradation in split federated learning.
  2. The client-specific adaptive splitting mechanism provides a natural way to balance local computation and communication costs under heterogeneous device conditions.
Critical weaknesses
  1. The related-work section explicitly discusses several adaptive splitting approaches, but the main baselines are SFLV1/V2, SA-SFL, MergeSFL, ParallelSFL, and SMoFi. A direct comparison with existing resource-aware adaptive-splitting SFL methods would better establish the novelty and empirical advantage of PyramidSFL.
  2. Client computation and bandwidth are sampled from AI Benchmark and MobiPerf, while the experiments themselves are executed on NVIDIA A40/A100 GPUs. Real-device experiments would provide stronger evidence that the reported wall-clock improvements translate to practical SFL systems.
  3. Equation 4 optimizes the synchronization latency of client-side forward propagation, whereas the actual training latency also includes backward computation, server-side computation, and bidirectional communication. A clearer justification is needed for why minimizing this partial latency objective reliably minimizes end-to-end training time.
  4. The convergence analysis is derived for fixed-cut SA-SFL with uniform partial participation, whereas PyramidSFL relies on dynamic client-specific splitting, biased warm-up sampling, and layer-wise weight shrinking. Consequently, the theory provides useful background but does not formally explain the convergence behavior of the proposed method itself.
  5. The text states that client models of different depths must first be reconstructed into full models and then averaged, whereas Algorithm 1 directly describes averaging the client-side submodels and concatenating them with the server-side model. This discrepancy should be resolved.
Questions for authors

Please see Weaknesses.

Reviewer_GEjqOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a resource-aware split federated learning framework built on Stacked-Activation SFL (SA-SFL). Specifically, each client selects a latency-optimal cut layer according to its computation and bandwidth profile, and activations from different cut layers are inserted into a single shared server-side submodel. To mitigate the layer-wise update inconsistency caused by dynamic splitting, the paper introduces a bandwidth-prioritized warm-up phase with a fixed shallow cut, and a layer-wise weight-shrinking mechanism based on the gradient discrepancy across cut-layer groups. Experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet show improved time-to-accuracy over FL and several SFL baselines.

Critical strengths
  1. The paper identifies an optimization issue when combining client-specific dynamic splitting with SFL-V2: every layer on server side model updates in a different frequency, causing inconsistency in training and consequently affecting the convergence.
  2. The convergence analysis explains how different server-side update schemes affect within-round drift. It also shows that SA-SFL follows the same trajectory as SFLV1 with τˉ=1\bar{\tau}= 1 while maintaining only one server-side model.
Critical weaknesses
  1. The novelty is limited. Client-specific cut-layer selection based on resource profiles has been widely studied in SFL, and the paper itself cites several adaptive splitting methods [1, 2, 3, 4]. In addition, HSFL [5] assigns each client its own cut layer and maintains a per-client server-side model. DASFL [6] selects the cut layer for each client at every round based on its current computation and communication conditions. A discussion of the difference and comparison in experiment would be helpful. The weight-shrinking mechanism is adapted from FedLWS [7], and the warm-up phase is a heuristic. The main new element seems only to be applying dynamic splitting to SA-SFL with a shared server-side model, which is an incremental step.
  2. Comparisons with adaptive splitting baselines are missing. All compared SFL methods (MergeSFL, ParallelSFL, SMoFi) use a fixed cut layer L = 2, and none of the adaptive splitting methods discussed in the related work is included. In addition, warm-up and weight shrinking are general techniques that could also be applied to SA-SFL* or SFLV1*. Without such comparisons, it is unclear whether the gains come from the shared-server design or from these two add-on components.
  3. The shared server-side design does not seem to be helpful. In the ablation studies, SA-SFL with L∗L^* alone has lower accuracy than SA-SFL* on all three datasets. This suggests that the core design is weaker than the simpler alternative, and that the final improvement mainly relies on warm-up and weight shrinking. Moreover, warm-up alone adds 4.0 points and weight shrinking alone adds 2.4 points on CIFAR-100, but together they add 10.8 points. The paper does not explain this.
  4. Several modeling choices may bias the comparison. PyramidSFL increases latency in global aggregation as clients may need to receive additional layers. However, the reported training time excludes global aggregation latency, which makes the comparison unfair for the baselines. Moreover, cut-layer selection uses the forward-only latency in Eq. 3, while the reported time uses Eq. 9–11 that combines computation and communication. The selected cut may therefore not be optimal under the time model that is actually reported.

Minor

Section 2.3 and the ablation paragraph refer to “SA-SFL” where PyramidSFL seems to be intended. The aggregation in Algorithm 1 (lines 28–29) is inconsistent with Eq. 12–13 in Appendix B. Section 3 contains the typo "τ= 5 for CIFAR-10 and CIFAR-10". In Table 3, SMoFi on CIFAR-10 reaches 60.40% at 150h, yet its reported T@60% is 160.47h.

References

[1] M. Kim, A. DeRieux, and W. Saad. A bargaining game for personalized, energy efficient split learning over wireless networks. IEEE WCNC, 2023.

[2] W. Wu, M. Li, K. Qu, C. Zhou, X. Shen, W. Zhuang, X. Li, and W. Shi. Split learning over wireless networks: Parallel design and resource management. IEEE Journal on Selected Areas in Communications, 41(4):1051–1066, 2023.

[3] Y. Wen, G. Zhang, K. Wang, and K. Yang. Training latency minimization for model-splitting allowed federated edge learning. IEEE Transactions on Network Science and Engineering, 2025.

[4] Z. Lin, G. Qu, W. Wei, X. Chen, and K. K. Leung. AdaptSFL: Adaptive split federated learning in resource-constrained edge networks. IEEE Transactions on Networking, 2025.

[5] Y. Sun, G. Hu, Y. Teng, and D. Cai. Split federated learning over heterogeneous edge devices: Algorithm and optimization. arXiv:2411.13907, 2024.

[6] A. Li, Y. Li, G. Ye, D. Liu, T. Zhao, and Z. Guan. DASFL: Dynamic adaptive split federated learning for heterogeneous clients. IJCAI, 2026.

[7] C. Shi, J. Li, H. Zhao, D. Guo, and Y. Chang. FedLWS: Federated learning with adaptive layer-wise weight shrinking. 2025.

Questions for authors
  1. What is the difference between PyramidSFL and existing adaptive splitting methods such as [4, 5, 6]? Can the authors discuss the difference and provide experimental comparisons with them?
  2. What happens if warm-up and weight shrinking are applied to SA-SFL*? Without them, what is the advantage of the shared server-side submodel over SA-SFL*?
  3. Do the main conclusions still hold when aggregation latency and the extra model transmission caused by changing cut layers are included?
  4. Why do SFLV1* and FL report near identical accuracy in Table 4? Is SFLV1 equivalent to FedAvg in your implementation?
Reviewer_7R3SOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes PyramidSFL, a resource-aware split federated learning framework that dynamically partitions models according to client computation and communication capabilities. It introduces warm-up and layer-wise weight-shrinking strategies to mitigate optimization inconsistency, demonstrating improved accuracy and time-to-accuracy on 3 benchmarks under heterogeneous settings.

Critical strengths
  1. The paper identifies an important conflict between latency-optimal model splitting and optimization stability, providing a clear and meaningful motivation for the proposed framework.

  2. PyramidSFL achieves substantial improvements in accuracy and time-to-accuracy across multiple baselines and heterogeneous settings, with ablation studies supporting the effectiveness of its main components.

Critical weaknesses
  1. The reported training efficiency relies on modeled computation and communication latency rather than a complete deployment on heterogeneous edge devices. This limits confidence in the generalizability of the reported system-level improvements.

  2. The convergence analysis focuses on fixed-cut SA-SFL, while the proposed method relies on dynamic splitting and layer-wise weight shrinking. Consequently, the theoretical results do not directly characterize the convergence behavior of the complete algorithm.

Questions for authors
  1. Could the authors clarify how the reported time-to-accuracy results would change when actual communication, synchronization, and aggregation overheads are included, particularly in a real heterogeneous deployment?

  2. Could the authors clarify which theoretical findings directly support the proposed dynamic splitting and weight-shrinking mechanisms, beyond the convergence analysis of fixed-cut SA-SFL?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.