acceptodds
Under review as a conference paper at ICLR 2027

LOPA: Learning Local Subspaces for Privacy Protection in Split Federated Learning

Abstract

Split Federated Learning (SFL) reduces the computational burden on clients through collaborative training between clients and a server. However, an honest-but-curious server can still reconstruct private images from uploaded intermediate representations. Existing defenses limit information exposure through activation dropping, quantization, or perturbation, but these operations do not necessarily suppress reconstruction-relevant information while preserving information needed for classification, and stronger restrictions may degrade task performance. We observe that low-rank channel projection combined with normalization and spatial pooling can reduce reconstruction quality, but random selection of subspace directions does not consistently preserve classification accuracy. To address this issue, we propose LOPA (Local Privacy-Aware Subspace Adaptation), which uses only local client data to initialize a task-relevant subspace based on between-class and within-class scatter statistics. It then calibrates the subspace using classification loss and distance correlation to preserve task-discriminative information while reducing the statistical dependence of representations on raw inputs. The clients' orthogonal projection matrices are aggregated into a shared subspace before training, which remains fixed during online training to filter uploaded representations without further subspace optimization. Experiments on CelebA, CIFAR-10, and FMNIST show that LOPA substantially degrades reconstruction quality under both the FORA and Inverse-Network attacks. Unlike the compared defenses, which trade accuracy for protection, LOPA matches undefended SFL on all three datasets. Furthermore, subspace initialization takes only seconds and requires less memory than client-side training itself.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Split federated learning, privacy preservation, differential privacy

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_gWRrOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

LOPA is a privacy-preserving framework for Split Federated Learning (SFL). It constructs task-relevant local subspaces from between-class and within-class scatter statistics, and calibrates them using classification loss together with distance correlation. The locally learned projection matrices are then aggregated into a shared subspace. During online training, this subspace is kept fixed and used to filter the intermediate representations uploaded from clients to the server. Experiments on CelebA, CIFAR-10, and FMNIST under FORA and Inverse-Network reconstruction attacks show that LOPA substantially reduces reconstruction quality while maintaining classification accuracy comparable to, or slightly better than, undefended SFL.

Critical strengths

A clear strength of LOPA is its balance between privacy protection and classification utility. The method constructs subspaces from local class statistics and calibrates them with distance correlation, which reduces the risk of reconstructing sensitive inputs from intermediate representations while maintaining competitive classification accuracy.

Critical weaknesses
  1. The evaluation uses only three clients, which is not sufficient to establish scalability in large-scale split federated learning systems. Because LOPA aggregates locally learned projection matrices into a shared subspace, increasing the number of clients may raise communication overhead and lead to more heterogeneous local subspaces. However, the paper does not examine whether classification accuracy and reconstruction resistance remain stable as the number of participating clients grows.
  2. Appendix A.2.2 reports the cost of offline subspace construction, but the online overhead introduced by feature filtering and backward propagation is not analyzed. LOPA also requires clients to transmit local projection matrices, while the uploaded intermediate representations retain their original dimensions. A comparison with standard SFL in terms of computation, end-to-end training time, and communication would help assess whether LOPA is practical for resource-constrained clients.
Questions for authors
  1. How does the aggregation mechanism behave as the number of clients and the degree of data heterogeneity increase? Are classification accuracy and reconstruction resistance maintained under these conditions?
  2. The authors should report the end-to-end online overhead, including feature filtering, backward propagation, and projection-matrix communication, relative to standard SFL.
  3. The rationale for selecting δ = 1.0001 in Eq. (5) is not fully explained. Was this value adopted from prior work or determined empirically? A sensitivity analysis over δ would help clarify its effect on accuracy, reconstruction resistance, and training stability.
  4. Although the paper compares LOPA with Dropout and B-SL, it does not provide direct comparisons with NoPeek and ResSFL, which are discussed in the related work section. NoPeek also employs distance-correlation-based regularization, while ResSFL addresses reconstruction attacks in split federated learning. The authors should clarify why these methods were excluded and provide comparisons under matched experimental settings.
Reviewer_nXRsOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This work develops a privacy protection method for split federated learning, called LOPA. It constructs a shared low-rank subspace using local class statistics, distance correlation regularization, and projection-matrix aggregation. Experiments on three datasets show improved resistance to image reconstruction attacks while maintaining classification accuracy.

Critical strengths

This work studies an important and relevant problem in split federated learning. The method is reasonably well motivated. The experimental results are encouraging.

Critical weaknesses
  1. The abalation study shows that random subspaces without initialization or calibration achieve nearly the same privacy protection as the proposed LOPA. This suggests that most of the privacy benefit comes from the structural bottleneck rather than the proposed subspace learning procedure. The additional contribution of the privacy-aware calibration is not sufficiently demonstrated.
  2. This work does not compare against NoPeek or ResSFL, both are relevent to the reconstruction privacy in SFL. As such, the advantage of LOPA over existing privacy-preserving approaches has not been fully established.
Questions for authors

Please see the weakness.

Reviewer_9AwAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a privacy-preserving method, LOPA, for split federated learning (SFL), aiming to reduce the server's ability to reconstruct original images from uploaded intermediate feature representations while maintaining classification accuracy. LOPA first uses class-aware divergence statistics to initialize local subspaces, then jointly calibrates them through classification loss and distance correlation, and constructs a shared subspace by aggregating the projection matrices of individual clients. During subsequent training, the shared subspace remains fixed and is used to filter uploaded intermediate feature representations. The authors conduct experiments on three image datasets and evaluate the effectiveness of the method using two attacks, FORA and Inverse-Network.

Critical strengths

S1. The research problem addressed in this paper is clear and important.

S2. The experimental results demonstrate that LOPA achieves good privacy protection while maintaining classification accuracy.

Critical weaknesses

W1. The relationship between task-discriminative information and image reconstruction information is insufficiently justified. Feature directions with higher inter-class separability do not necessarily contain less image reconstruction information. In addition, Table 4 shows that a random subspace achieves almost the same reconstruction resistance as the full LOPA, suggesting that the privacy protection benefit may mainly come from a structural information bottleneck rather than task-aware subspace selection.

W2. The justification for the effectiveness of federated subspace aggregation is limited. LOPA constructs a shared subspace by aggregating local projection matrices, but it cannot guarantee that the aggregated subspace still preserves the classification and privacy protection properties optimized by each client. The paper lacks a direct comparison before and after aggregation, and in particular does not verify its effectiveness when local subspaces differ substantially.

Questions for authors

Q1: Can the authors further explain why subspace directions with task-discriminative ability can effectively suppress image reconstruction-related information? Given that the random subspace in Table 4 already achieves almost the same image reconstruction resistance, is there further evidence to demonstrate the specific contribution of task-aware subspace selection and distance correlation calibration to privacy protection?

Q2: Can the authors further explain how federated projection matrix aggregation affects the privacy protection and classification performance of locally optimized subspaces? In particular, can they provide comparative evidence on the relevant properties before and after aggregation to verify that the shared subspace effectively preserves the results of local optimization?

Q3: How does LOPA perform under more severe label distribution heterogeneity, such as Dirichlet α = 0.1? When some clients contain data from only a few classes, is class-aware subspace initialization still effective?

I would be willing to raise my score if the authors’ responses satisfactorily address and resolve my concerns regarding these issues.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.