acceptodds
Under review as a conference paper at ICLR 2027

Reducing Training-Value Leakage in Split Inference via Learned Weight Perturbations

Abstract

Split inference avoids transmitting raw inputs, but the intermediate representations exposed at the split point may still reveal valuable information. In proprietary industrial, scientific, and other high-value data settings, the primary concern is not only whether individual samples can be visually reconstructed, but whether the exposed information can be recovered and reused to train another model. We refer to this risk as training-value leakage. To mitigate it, we propose learned client-side weight perturbations that modify selected pre-split weights while leaving the server model and deployment interface unchanged. The perturbations are optimized with reconstruction-aware feedback and selected under an explicit validation-accuracy constraint, so that reduced leakage is not obtained by simply degrading the original prediction task. We evaluate the method on CIFAR-10, CIFAR-100, and NEU-CLS using post-selection SplitNN-decoder and UnSplit reconstruction benchmarks, and measure reconstruction MSE and reconstructed-dataset trainability across three evaluator architectures. Learned perturbations reduce split-inference accuracy by only 2.3 percentage points on average, while reconstructed-dataset trainability shows an average positive drop of 23.2 points and a maximum drop of 65.7 points. These results suggest that split representations can retain intended inference utility while becoming substantially less reusable as downstream training data.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
split learning, privacy, reconstruction attack, model perturbation, trainability, representation leakage

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_K9kmOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduce an interesting defense against attacking a split inference settings. The defense learns a weights perturbation that keeps the necessary information such that the server can still predict while making it harder for the attack to gather enough information about the training set to train its own model. The defense only perturbs the client's weights so no server retraining is needed. The authors showed empirically that the learned perturbations only have a small degradation on the server accuracy while having a large degradation on the attacker's model.

Critical strengths
  1. The paper introduces a defense that does not need to retrain the server, only the client. This makes the defense practical.
  2. Using weights perturbations is a nice way of defending, and to the best of my knowledge is the first defense to use it for against privacy attacks.
Critical weaknesses
  1. The experiments are quite limited. The network are only ResNet20, VGG11-BN, MobileNetV2. There are not Transformer based networks. In addition, the datasets are very simple and it is not obvious if the defense holds in the real world settings.

  2. The benchmark is against old attacks. It would be interesting to add PCAT (Gao & Zhang, USENIX Security 2023)

  3. To my understanding, Figure 2 shows reconstructed input from the attack. To me it seems very similar to the original input which may suggest that the defense does not protect from classic reconstruction attack. It would be nice to add image similarity metrics such as SSIM or LPIPS to validate the scope of the defense to reconstruction attacks.

Questions for authors
  1. Does the defense work on other scopes than images? for example text classification (GLUE, IMDb)
  2. Is the code available to reproduce the experiments?
2 comments
AuthorsOpenReview ↗

Response to Reviewer K9km: Evaluation Coverage, Reconstruction Metrics, and Reproducibility

Thank you for recognizing the practical value of preserving the server model and the interest of learning client-side weight perturbations. We appreciate your suggestions and clarify the scope of the current evidence below.

1. Architectures and datasets. The evaluation distinguishes the protected model from downstream evaluators. ResNet20 is the victim, while ResNet20, VGG11-BN, and MobileNetV2 test reconstructed-data utility across classifier architectures. This checks whether the observed effect is specific to one evaluator. The datasets cover natural images and industrial surface defects through NEU-CLS, providing evidence beyond a single visual domain. These controlled experiments support the reported settings. Transformer victims and larger-scale deployment distributions would provide complementary evidence; defense effectiveness in those settings is not established by the current evaluation.

2. Additional attacks, including PCAT. We appreciate PCAT as a relevant additional attack. Our evaluation examines a freshly retrained decoder and a clean-map input-optimization procedure. The decoder is trained after defense selection using features from the selected perturbed client, so evaluation is not limited to the defense-training surrogate. Input optimization provides a distinct but restricted reconstruction setting. PCAT is not included, and our conclusions concern the evaluated procedures rather than comprehensive resistance to every reconstruction or model-stealing attack.

3. Visual similarity, SSIM, and LPIPS. Visual recognizability and downstream training utility measure different properties. A sample may remain recognizable to a human without preserving the consistency required to serve as effective training data. We quantitatively test the latter by training evaluators on reconstructed images and testing them on real held-out data. Visible similarity therefore does not by itself contradict a measured trainability reduction.

Conversely, reduced trainability does not imply that recognizable or sensitive content has disappeared. MSE is reported as a pixel-level diagnostic; SSIM and LPIPS would characterize additional structural and perceptual aspects but are not reported here. Figure 2 should illustrate reconstruction changes rather than establish visual confidentiality. Our central evidence concerns the usefulness of reconstructed datasets for downstream learning.

4. Text tasks. We have also conducted preliminary experiments on text tasks and observed encouraging initial results, suggesting potential for extending the approach beyond image classification. The mechanism modifies model parameters rather than image pixels, although text requires different reconstruction procedures and task-specific utility criteria. These experiments are time-consuming, and the current results remain exploratory rather than a comprehensive evaluation. We therefore regard them as preliminary evidence of broader applicability, while the conclusions supported by the reported systematic evaluation remain focused on images.

5. Code availability. The implementation exists in our experimental suite and includes defense training, reconstruction, data partitioning, and downstream trainability evaluation. We plan to open-source the implementation and experimental configurations after refining the manuscript and completing the supplementary experiments. This release will support independent reproduction and further investigation of the proposed approach.

I thank the authors for their response. I feel like the contribution of the paper is limited. The paper is empirical, and propose an actual defense that should work in the real world. The paper does not cover modern architectures and split attacks, and it seems that even the current experiment present a very narrow defense against a specific attack while showing that the defense does not work for the more dangerous, reconstruction attack.

I believe the paper would benefit if it shows that the defense is more robust, either to other types of attack, or even to more modern split attacks.

I would maintain my current score.

Reviewer_J3t6OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies training-value leakage in split inference: whether intermediate representations enable reconstruction of datasets useful for training new classifiers. It proposes reconstruction-aware perturbations of client-side convolutional weights, keeping the server fixed and selecting perturbations under a validation-accuracy threshold. Leakage is assessed using reconstructed-dataset trainability rather than visual fidelity alone. Across three image datasets, the paper reports a mean 2.3-percentage-point inference-accuracy drop and substantial trainability reductions in several settings.

Critical strengths
  1. Relevant leakage metric. Evaluating classifiers trained on reconstructed images against real held-out data captures training utility beyond pixel MSE (Section 6.3).

  2. Useful experimental controls. Fixed server weights, separated data partitions, and post-selection SplitNN decoder retraining improve evaluation reliability (Sections 5–6).

  3. Multiple evaluation settings. Three datasets, three split points, two reconstruction benchmarks, and three downstream evaluator architectures provide evidence across multiple settings.

Critical weaknesses
  1. Unclear defense configurations. For the same dataset and split point, Table 2 reports different defended accuracies under SplitNN and UnSplit (e.g., 88.0% vs. 85.1% on CIFAR-10/L2). It is unclear whether separate perturbations were selected.

  2. Baseline comparison. The paper compares learned perturbations with random weight perturbations. However, the classification accuracies after random perturbations and the magnitudes of learned perturbations are not reported.

  3. Unclear averaging criterion. The abstract reports an average trainability drop of 23.2 points, while Figure 1 shows an overall average of 16.6 points. The former appears to average only the 38 cases with drops exceeding 4 points.

Questions for authors
  1. Can UnSplit adapt to the defended client map?

  2. Can the authors clarify the accuracy thresholds, perturbation norms, and averaging criteria?

1 comment
AuthorsOpenReview ↗

Response to Reviewer J3t6: Defense Configurations, Baselines, and Statistical Summaries

Thank you for recognizing the relevance of the trainability metric, the separated evaluation protocol, and the evaluation across multiple settings. We appreciate the opportunity to clarify the configuration and reporting details.

1. Different defended accuracies. The difference reflects two defense-training protocols within the same framework: an auxiliary-surrogate protocol and a target-surrogate protocol. For CIFAR-10/L2, the SplitNN entry reports the target-surrogate configuration (88.0% defended accuracy), whereas the UnSplit entry reports the auxiliary-surrogate configuration (85.1%). These configurations produce separately learned perturbations. For a fixed perturbation checkpoint, forward accuracy is independent of the reconstruction attack subsequently evaluated. The difference therefore comes from the defense configuration, not the attack changing victim accuracy. Explicit protocol labels would clarify this distinction. Within each configuration, the clean-versus-defended comparison characterizes its privacy-utility outcome; comparisons across attacks using different configurations should be interpreted accordingly.

2. Perturbation magnitude and random baselines. The random baselines apply prescribed relative weight norms of 0.02 and 0.05 to the same selected client-side tensors. Learned perturbations are optimized with a magnitude regularizer, task-preservation losses, and validation-based selection; their norms are optimization outcomes rather than fixed at those values. Random conditions provide reference points for unguided weight perturbations, while the forward-accuracy evaluation separately records the utility cost of learned configurations. These comparisons characterize the evaluated operating points. Reporting perturbation norms alongside forward accuracy would make the comparison more transparent. Equal-norm or equal-utility superiority is a separate claim that these reference conditions do not establish.

3. Averaging criteria. The two averages summarize different sets of settings. For the submitted table, 16.6 percentage points is the unconditional average over all 54 displayed drops, whereas 23.2 points is the conditional average over the 38 entries whose drops exceed 4 points. The latter describes the reduction magnitude in that subset, not the overall effect. The abstract should distinguish these quantities explicitly and use the unconditional average as the general summary. All results in the final evaluation are averaged over three completed seeds. The difference between 16.6 and 23.2 arises from aggregation across settings, not the number of seeds. The unconditional summary retains a positive average reduction while including small, zero, and negative drops.

4. Attacker adaptation and thresholds. The post-selection decoder is explicitly adapted by training a fresh decoder on auxiliary features generated by the selected defended client. The UnSplit-style input optimizer instead uses a fixed clean inversion model, assessing transfer to that reconstruction procedure rather than learning the defended map. These evaluations provide different evidence and should be described separately.

The utility threshold is the minimum of the dataset-specific absolute threshold (85%, 55%, and 85% for CIFAR-10, CIFAR-100, and NEU-CLS) and clean validation accuracy minus 5 percentage points. This defines an effective validation threshold; held-out test accuracies separately quantify the realized inference cost. It should not be interpreted as simultaneous enforcement of both thresholds or a guaranteed test-accuracy bound.

Reviewer_PsEuOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies whether representations exposed during split inference can be reconstructed into data useful for training another model. It learns additive perturbations to selected client-side convolutional weights while freezing the original model and preserving the server interface. A reconstruction-aware objective and validation-accuracy filter balance inference utility against the usefulness of recovered images. Experiments on CIFAR-10, CIFAR-100, and NEU-CLS evaluate three split points, two reconstruction benchmarks, and three downstream classifier architectures. The results suggest reduced reconstructed-data trainability in several settings, but attacker adaptation, the utility constraint, and aggregate-result accounting limit the broader leakage claim.

Critical strengths
  1. An operationally meaningful leakage metric. Training a classifier on reconstructed samples and testing on real images measures a concrete form of unauthorized data reuse.

  2. A practical restriction on the defense interface. The method confines changes to pre-split client weights and retains the server model, making compatibility with the existing inference path an explicit goal.

Critical weaknesses
  1. UnSplit is not fully adapted to the defended forward map. Appendix A.3 explicitly computes defended target features but optimizes candidate images using the fixed clean client map. The large UnSplit reductions can therefore reflect inversion-model mismatch, and rerunning this optimization after selection does not establish robustness against an attacker using or learning the defended map.

  2. The metric covers only one route to training-value reuse. The reported trainability measures force the adversary through image reconstruction and then raw-image classifier training. They do not assess direct reuse of stored representations or reconstruction optimized for downstream learning instead of pixel MSE, even though the server retains a highly accurate classifier over those representations.

  3. The stated utility constraint has a logical gap. Equation 12 sets the threshold to min(tau_fixed, clean_validation_accuracy minus epsilon), which enforces only the weaker of the absolute threshold and relative-drop requirement. For example, an 85% absolute threshold, 90% clean accuracy, and a 2-point allowed drop yield an 85% threshold that permits a 5-point drop; enforcing both requirements would use max, with infeasibility handled explicitly. The method therefore cannot claim both requirements are satisfied without clarifying its intended rule and actual thresholds, especially given the 8.4-point test drop on CIFAR-100/L3/UnSplit.

Questions for authors
  1. What client weights, perturbations, auxiliary labels, and defended feature queries are available to the server-side attacker? Why does Equation 17 use the clean map, and what supports keeping that map fixed given the auxiliary defended-feature access granted to the adaptive decoder?

  2. Is Equation 12 intended to enforce both thresholds or to relax one when the other is infeasible? Please report the actual absolute thresholds, allowed drops, selected validation accuracies, and corresponding test accuracies for the evaluated settings.

1 comment
AuthorsOpenReview ↗

Response to Reviewer PsEu: Attack Scope, Training-Value Leakage, and Utility Selection

Thank you for recognizing the operational relevance of reconstructed-dataset trainability and the practicality of preserving the server model. We clarify the attack settings, metric scope, and utility criterion below.

1. UnSplit adaptation and attacker access. The two reconstruction evaluations provide different types of evidence. In Equation 17, target features come from the defended client, while candidate inputs are optimized through a fixed clean client. This is an UnSplit-inspired input-optimization benchmark, not a full implementation of the original alternating input-and-model optimization. Rerunning it after defense selection does not by itself establish adaptation to the defended map.

In contrast, the post-selection SplitNN decoder explicitly adapts to the selected defense: a fresh decoder is trained on auxiliary images and their defended features and selected on auxiliary validation data. This provides evidence beyond evaluating only the defense-training surrogate. The decoder does not require exact perturbed weights; auxiliary labels are not used in its reconstruction loss. Source labels are supplied to reconstructed target samples for downstream trainability evaluation. The input optimizer uses clean client weights without the learned perturbation. Keeping that map fixed defines the benchmark, rather than excluding stronger attackers that learn a replacement map. We agree that these distinctions should be explicit and that the input-optimization results do not establish comprehensive adaptive robustness.

2. Scope of training-value leakage. Our metric addresses a concrete reuse outcome: whether reconstructed images can substitute for private images when training classifiers that generalize to real held-out inputs. It complements pixel fidelity by measuring downstream usefulness directly. Preserving the original server classifier necessarily retains task-relevant information, so the objective is not to eliminate all semantic information. The empirical contribution concerns reduced reconstructed-image training utility under the evaluated protocols. Direct feature reuse and reconstruction optimized specifically for downstream learning are distinct routes that are not evaluated. This limits the breadth of the claim, while retaining the operational relevance of the measured outcome.

3. Utility selection. Equation 12 uses the minimum of the absolute threshold and the clean-relative reference. The absolute thresholds are 85%, 55%, and 85% for CIFAR-10, CIFAR-100, and NEU-CLS, respectively; the relative reference subtracts 5 percentage points from clean validation accuracy. You are correct that enforcing both requirements simultaneously would require a maximum. The implementation should therefore be described as selection under an effective validation threshold, not simultaneous absolute and relative guarantees.

The 8.4-point test drop is not excluded by this rule: the effective threshold can be less strict than the relative reference, and validation selection is distinct from held-out test performance. Table 2 separately reports the realized test-accuracy cost, making larger-drop settings visible alongside the average. The submitted tables do not explicitly list every selected validation accuracy; those values should be distinguished from test accuracies when documenting feasibility. Our utility evidence is the reported forward-accuracy evaluation, rather than a stricter guarantee inferred from Equation 12.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.