- Both main components of StreamSplit closely follow ideas from existing work, and the paper does not accurately describe these works, which makes its novelty unclear.
On the client side, the SSM is essentially an auxiliary network. The mechanism in Eqs. (5)–(8), where a small local network takes the cut-layer feature and produces a surrogate gradient from a local task loss, is the same mechanism used by CSE-FSL [3] and FSL-SAGE [2]. The paper argues that FSL-SAGE passively selects an auxiliary network rather than actively aligning it. However, FSL-SAGE already periodically aligns its auxiliary models to emulate the server-side model [2]. The remaining differences are the multi-branch architecture and the use of feature and logit alignment instead of aligning the cut-layer gradient itself. The paper does not explain why aligning intermediate features should be preferable to aligning gradients. This is particularly important because the authors’ own bound in Eq. (35) shows that the surrogate gradient error is governed by the Jacobian mismatch, which feature alignment controls only indirectly. In addition, the SSM increases client-side FLOPs by 53.0% for ResNet-18 and 66.7% for ViT-Tiny, which is not lightweight.
On the server side, the continuous server-side optimization in Section 3.2 is closely related to MU-SplitFed [1]. MU-SplitFed targets the same problem, namely server idleness caused by waiting for client activations under heterogeneous clients. It addresses this problem by letting the server performs multiple local updates on the available activations within each client round, instead of waiting for new ones. The LFT follows the same principle of letting the server keep updating on already received features. Its main addition is a utility-based rule for choosing which retained feature to reuse, which can be viewed as a variant of this multi-step server update. However, the paper does not discuss this connection and describes MU-SplitFed only as a zeroth-order method for client-side updates. This paper should clarify what the LFT contributes beyond multi-step server updates.
- The convergence analyses do not address the main problem of the method. The client-side analysis in Eqs. (79)–(90) only bounds the one-step deviation from a hypothetical standard SFL update by the gradient approximation error, and it assumes that this error is small without proving it. The server-side theorem is a standard analysis of SGD with bounded bias. The selection error and the maximum staleness are directly assumed to be bounded, and the utility-based selection itself is not analyzed. The derivation also does not use the unbiasedness of the stochastic gradient, so the variance term 9σ2 in Eq. (130) does not decay with the learning rate. Therefore, the claim in the discussion that the result reduces to the standard non-convex characterization when Δ=0 and ϵselection=0 is not correct. Most importantly, the paper analyzes the client side and the server side separately, but does not provide a convergence result for the coupled system.
- The experimental evaluation is not sufficient to support the claims. The paper only reports accuracy against time curves and accuracy at fixed time points, without the final accuracy at convergence or results over multiple random seeds. In Table 2, the accuracy difference between baselines and the proposed method is small. Without standard deviations, it is difficult to conclude that the utility-based selection is beneficial. Moreover, only one cut layer is evaluated for each model, and in both cases the client-side model is very shallow. This is the setting where noisy surrogate gradients are least likely to hurt the final accuracy. Finally, the training time is simulated on a single multi-GPU machine with assigned speed ratios, and the paper should explain how communication time is modeled.
References
[1] D. Liang, J. Zhang, E. Chen, Z. Li, R. Li, and H. Yang. Towards Straggler-Resilient Split Federated Learning: An Unbalanced Update Approach. In Advances in Neural Information Processing Systems (NeurIPS), 2025.
[2] S. Nair, M. Lin, P. Ju, A. Talebi, E. S. Bentley, and J. Liu. FSL-SAGE: Accelerating Federated Split Learning via Smashed Activation Gradient Estimation. In International Conference on Machine Learning (ICML), 2025.
[3] Y. Mu and C. Shen. Communication and storage efficient federated split learning. In IEEE International Conference on Communications(ICC), 2023.