acceptodds
Withdrawn from ICLR 2027

Adaptive Sparse Support as a Calibration-Dependent Label Channel in Split Learning

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

18 views

Abstract

Sparse gradients are usually discussed in terms of the values they retain. In split learning, however, the coordinates chosen for transmission can themselves reveal the client's label. We study when this support channel is exploitable and how it interacts with the aligned values. Across our experiments, label decoding is strong when the server has labeled calibration messages from the same checkpoint or training lineage, but it largely collapses across independent initializations. Our experiments also overturn our initial theoretical explanation. On Diamonds, same-label cross-fit fidelity is negative in every seed even though ordinary support decoding under the natural class prior reaches (balanced ); the attack is instead driven by class-conditional coordinate frequencies. A decoder-free test based on one auxiliary-selected coordinate gives 95% lower bounds of , , and bits on Digits, Synthetic-10, and Diamonds. At epoch 48 on compact CIFAR-10, support decoding remains and joint decoding . Data-independent supports close the support channel at small measured utility cost, yet values remain predictive in every useful-utility trainer we test. Support is therefore an additional privacy risk, not the whole leakage story. Our audit covers support and values for the declared row-wise sparse backward message, not the complete training transcript.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Split Learning, Label Leakage, Gradient Sparsification, Privacy Auditing, Information-Theoretic Privacy

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_PK53OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies how the support of sparse gradients forms a distinct "label channel" in split learning, separate from the values of those coordinates.

Critical strengths

This paper is hard to read, and I can't find any strengths.

Critical weaknesses

This paper is extremely difficult to read and understand. The presentation is unclear throughout, making it challenging to follow the motivation, methodology, and experimental evaluation. I am curious whether any human was involved in writing this paper.

The overall structure is confusing, and essential background information is largely missing. Many equations are presented without adequately defining the notation. For example, in Equation (1), symbols such as "Comp", "supp", and "U" are introduced without any definition or explanation, making the formulation difficult to understand.

The experimental design is extremely unclear. The paper states that "Breast Cancer, Iris, Wine, Digits, and Synthetic-10 use disjoint 60/20/20 model-training, auxiliary, and test partitions and seeds {1, 7, 19, 31, 43}", yet these datasets are neither properly introduced nor cited. Without sufficient background knowledge or domain expertise, readers may not even recognize that these names refer to datasets. The description of the model architectures is also difficult to interpret. For example, the statement "Nonlinear tails, a three-cut convolutional Digits model, residual/TinyViT controls, and LFW test architectural boundaries" does not clearly specify the architectures or experimental settings. Moreover, essential information regarding the baselines, evaluation metrics, and implementation details is missing.

Overall, the paper requires substantial restructuring and clarification. The authors should provide sufficient background, clearly define all notation, systematically describe the methodology, and present complete experimental settings to make the paper understandable.

Questions for authors

Refer to weaknesses.

Reviewer_tKgtOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies the label leakage problem in split learning, where a user sends gradients to a server to collaboratively train a classification model. In this setting, the gradients are sparsified to save communication cost, so both partial gradient values and their coordinate indices are shared. Previous work has already shown that gradient values, and even sparse coordinate patterns, can leak private information. This paper develops information-theoretic and statistical analyses to characterize how much label information is contained in the selected indices, and studies how a server can exploit this information using labeled calibration gradients. Experiments show that the attack can be strong when the labeled gradients come from the same checkpoint or training lineage, but transfers poorly across independently initialized models. The paper also argues that even when labels are not leaked through the selected coordinates, they can still be leaked through the gradient values.

Critical strengths
  1. The paper studies an interesting and relevant privacy problem in which the transmitted gradient coordinates can also leak private information, an issue that has received less attention than leakage from gradient values.

  2. The experiments report both positive and negative cases of label leakage from gradient indices, helping clarify when the leakage occurs.

Critical weaknesses
  1. The threat model is strong and appears disconnected from the standard honest-but-curious setting. The attack requires access to labeled gradients from a colluding party whose model is closely matched to the victim’s setting (line 105), which may be difficult to realize in many deployments. The experiments in Section 5.4 show that the attack largely fails when this condition is not met, which limits its practical significance.

  2. The paper summarizes four contributions in lines 44–53, but the overall technical contribution is limited. Specifically, for the first point, the experiments overturn the paper’s main theoretical explanation, making it less convincing as a contribution. For the second point, it is unclear whether the root cause is that the selected indices are class-dependent, or that the gradient values are class-dependent and gradient-dependent compression therefore selects different indices. The third point mainly highlights the method’s limited generalizability, while the fourth mainly lists the breadth of the evaluation without clearly explaining what general conclusion this breadth supports.

  3. The presentation is difficult to follow and requires substantial improvement for readability. 1) Several notations are introduced with little explanation (line 85), which makes the threat model hard to parse. 2) Section 3 is largely a sequence of theorems and propositions without a clear roadmap or enough intuition connecting them to the actual privacy question. 3) Important experimental details are not introduced clearly; for example, several datasets and model variants are listed without citations or sufficient context (line 246). 4) The paper also overuses jargon and compressed phrasing. Sentences such as “we prove an independent same-label cross-fit implication and then falsify it as an explanation of our strongest attack: the larger Diamonds gap is negative” (line 44) are unnecessarily difficult to interpret and obscure a relatively simple point. It is not clear on first reading what “cross-fit” means, what “strongest attack” refers to, what “Diamonds gap” denotes, or why a negative value matters.

Questions for authors
  1. If the attack requires labeled calibration gradients from a closely matched model and fails without them, in what practical deployment settings would this threat be relevant?

  2. If the main theoretical explanation does not account for the empirical attack, what concrete role does the theory play in understanding the observed leakage?

  3. Since gradient values are transmitted together with the indices and can already reveal labels, what additional practical risk or attack capability is introduced by the indices?

  4. What additional insight does the class-dependent index result provide beyond the relationship between class-dependent gradient values and gradient value-dependent coordinate selection?

Reviewer_o87tOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies label leakage through sparse gradient coordinate selections in split learning, separately from the transmitted values. Theoretical analysis and experiments show that class-dependent selection patterns enable support-only label inference under compatible labeled calibration, highlighting a privacy risk that value-only protection may miss.

Critical strengths
  1. Separating support leakage from value leakage: The work shows that protecting gradient values alone can leave coordinate-selection patterns exposed, which can still reveal private labels. This highlights the need for privacy evaluation and protection for both support and values in sparse split learning.
  2. Theoretical and empirical support for the leakage findings: The paper provides both theoretical analysis and empirical experiments across multiple settings, supporting the leakage findings and clarifying when support-only label inference is effective.
Critical weaknesses
  1. Unclear practicality of compatible calibration: The paper demonstrates that calibration compatibility is important for support-only label inference. However, it remains unclear how an attacker could obtain gradient messages with known labels, and how these remain useful when clients update their private model segments. Clarifying these requirements would strengthen the practical relevance of this work.
  2. Weak organization and clarity: The theoretical analyses and empirical findings are difficult to connect. A clearer explanation of how each analysis contributes to the overall argument and which experimental findings it supports would help improve the overall readability and comprehension of the work.
Questions for authors
  1. Role of each theoretical component: Could the authors clarify the specific role of the cross-fitting analysis, frequency certificates, and Fano certificates in the overall argument? What does each of them establish, and how does it connect to the experimental findings?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.