acceptodds
Under review as a conference paper at ICLR 2027

NIFT: NOISE-INVARIANT FINE-TUNING FOR PRIVACY- PRESERVING LLMS

Abstract

Privacy-preserving fine-tuning of large language models is appealing for many important applications, but remains challenging in practice. Adding differentially private (DP) noise to input representations can protect private data, but this noise is amplified by deep Transformer layers and sharply degrades performance. Prior work has primarily addressed this problem through noise-correction mechanisms that attempt to recover useful representations after amplification has already oc- curred, but these approaches still leave a substantial performance gap compared with the corresponding no-noise setting. In this paper, we propose a two-stage training framework that instead prepares the backbone to be less sensitive to the agreed DP noise distribution before downstream task fine-tuning. First, we train a LoRA adapter with a label-free pairwise consistency objective over multiple noisy embedding copies, encouraging the backbone’s hidden states to remain stable under DP perturbations. Second, we fine-tune a lightweight client-side ladder side network that fuses local features with selected hidden states returned by the robust server-side backbone. This separation of noise-invariance preparation from task adaptation reduces the burden on the client model, improves robustness un- der privacy-preserving noise. Our experiments show that NIFT outperforms the state-of-the-art HiddenEcho across multiple datasets and privacy budgets.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
privacy, edge computing, side tuning, LLM

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_A3QbOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies the problem of DP-noise amplification in privacy-preserving split fine-tuning of LLMs. Building on the problem setting explored by prior work such as HiddenEcho, the paper proposes NIFT, a two-stage framework that aims to mitigate this issue at the backbone level rather than correcting noisy representations only after amplification has occurred. The method is evaluated on multiple text classification datasets and two LLM backbones, showing improved utility over HiddenEcho in most reported settings.

Critical strengths

strength1. The paper studies a practically relevant and meaningful problem. Continuing the line of work represented by HiddenEcho, this is an important problem for practical client–server LLM adaptation.

strength2. The motivation of making the backbone itself less sensitive to DP noise is intuitive. Conceptually, this is a sensible alternative to post-hoc denoising or hidden state correction, since a more noise-invariant backbone could reduce the burden on downstream client-side modules.

strength3. The method is relatively lightweight and compatible with the split-learning setting.

Critical weaknesses

weakness1. The main problem is that the author claims that Stage 1 achieves "robust representations", but what it actually optimizes is noise invariance, not "noise invariance while preserving clean semantic utility". Stage 2 can at most compensate for the information remaining after Stage 1, and it cannot put back the task-relevant information that Stage 1 has erased into the backbone. The current paper indirectly proves the utility is not damaged through downstream experiments, rather than ensuring it through the objective function itself. At the method level, this assumption is currently mainly supported by empirical results rather than being explicitly guaranteed by the objective.

weakness2. While Table 1 provides a relatively comprehensive comparison against GAN-DP, LDP, Split-and-Denoise, and HiddenEcho on Qwen2-1.5B, Table 2 reports only HiddenEcho and NIFT on Llama-3.2-1B. This substantially weakens the evidence for the claimed cross-backbone generality of NIFT. In particular, since the paper itself notes that the embedding scale, clipping radius, and resulting privacy behavior differ across backbone families, the relative ranking of LDP/SnD/HiddenEcho observed on Qwen cannot simply be assumed to hold on Llama. At minimum, I would expect the strongest relevant non-HiddenEcho baselines, especially LDP and Split-and-Denoise, to be evaluated on the second backbone as well.

weakness3. There is a lack of a representation-level experiment that directly proves that "hidden states truly become noise-insensitive". This is the aspect that this paper should present the most.

weakness4. As discussed in weakness 1, the Stage-1 objective only encourages consistency across noisy views and prevents degenerate collapse; it does not explicitly preserve task-relevant semantic information. This concern is reflected empirically in Table 3. On Financial Phrasebank, adding Stage 1 consistently reduces AUC across all reported privacy budgets compared with the Stage-2-only variant (0.890 -> 0.886 at \eta=100, 0.901 -> 0.885 at \eta=1000, 0.892 -> 0.881 at \eta=5000, and 0.896 -> 0.886 at \eta=6000).

Questions for authors

question 1: Is the additional server-side perturbation in Stage 1 applied to the \tilde e_0 that has already been client-side DP-perturbed, rather than generating multiple DP views independently from the clean embedding?

question 2: HiddenEcho is also based on the filtering of key transformer layers and the compression of hidden states, and does the proposed split bottleneck adapter have an essential difference from it?

Reviewer_Sg4MOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

NIFT first adapts a model to noisy inputs, then uses it to help a small device-side network learn the task. The goal is to preserve privacy while improving performance and reducing device–cloud communication.

Critical strengths
  1. The paper tackles a relevant problem in privacy-preserving device-cloud fine-tuning, addressing both utility loss from input perturbation and communication overhead.
  2. The two-stage framework is clearly presented and achieves utility gains over HiddenEcho in many of the evaluated settings.
  3. The ablation studies, including comparisons with Stage 2 alone and a local-only baseline, help assess the contribution of the proposed components.
Critical weaknesses

1. Limited model scale and task difficulty

Experiments are limited to Qwen2-1.5B and Llama-3.2-1B on five text classification datasets. This scale provides limited evidence for the intended device-cloud scenario, where cloud assistance is motivated by the difficulty of running a larger backbone locally. I would expect evaluation with at least a 7B-class model and more challenging datasets and tasks. Such experiments should establish whether the utility gains and efficiency benefits persist as model size and task difficulty increase.

2. Unclear role of the backbone after Stage 1

The paper does not directly compare the backbone after Stage 1 with the original model. Is the adapted backbone expected to remain usable on its own, or is it only a preparatory model whose utility depends on Stage 2 fine-tuning? In the former case, please evaluate its performance relative to the original model on clean and noisy inputs. In the latter case, this raises concerns about the generality of the adapted backbone. Please clarify whether Stage 1 preserves general capabilities or produces a model whose utility is limited to a specific downstream task. This distinction would help interpret the role of Stage 1 and the downstream results.

3. Dataset dependence and the additional cost of Stage 1

As described, Stage 1 uses perturbed versions of the target training inputs, so the backbone has already seen the fine-tuning data without labels. It is unclear how much of the reported gain depends on this prior exposure. Keeping the noise mechanism and privacy budget fixed, can a Stage 1 LoRA adapter trained on dataset A be reused for Stage 2 fine-tuning on dataset B? Comparing adaptation on A, adaptation on B, and no Stage 1, all evaluated on B after the same Stage 2 procedure, would help determine how transferable the learned noise adaptation is. The paper should clarify whether Stage 1 must be repeated for each new dataset. Since Stage 1 constructs 16 noisy views per input and trains LoRA through the backbone, please quantify its additional overhead relative to Stage 2 using appropriate metrics, such as training time, and include both stages when evaluating the overall efficiency. This is necessary to assess whether the additional adaptation cost can be amortized across downstream tasks.

Questions for authors
  1. How does Stage 1 affect performance on clean inputs? Can its adapter be reused across datasets, or does each new dataset require separate adaptation?

  2. How much additional training time and memory does Stage 1 require? How does NIFT compare with HiddenEcho when both stages are included?

  3. Do you have evidence that the utility and efficiency gains persist with larger backbones or more demanding tasks?

Reviewer_JaTbOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes NIFT, a two-stage framework for privacy-preserving "split fine-tuning." In the first stage, a LoRA adapter is trained on the server side by leveraging pairwise consistency across multiple perturbed views of privacy-protected client embeddings, optimized with an anti-collapse penalty. In the second stage, a ladder network is constructed on the client side to fuse the original local embeddings with features returned by a frozen, adapted backbone network. By employing a split bottleneck, activation quantization, and sparse gradient transmission, the framework effectively reduces communication overhead. Experimental results across five classification datasets and two types of backbone networks demonstrate that, compared to HiddenEcho, NIFT improves AUC metrics while significantly reducing the volume of hidden states and gradients transmitted; however, the contribution of the first stage varies depending on the specific task.

Critical strengths
  1. Embedding perturbations may diminish the utility of a split-model; enhancing robustness prior to the downstream adaptation phase serves as a more sound alternative to correcting distorted features post-hoc.

  2. A "label-free consistency objective" decouples representation construction from supervised adaptation. Generating additional views on the server side—effectively a post-processing step applied to privacy-preserved embeddings—does not, in itself, lead to any further information leakage from the client. The "anti-collapse term" introduced in the paper effectively addresses the significant degradation issues inherent in this consistency objective.

  3. The study evaluates the proposed approach across five datasets, two backbone architectures, and various noise settings. The results presented in Table 3, the "local-only" training baselines, and the ablation studies on compression techniques help clarify the specific contributions of each component. Furthermore, the explicit allowance for zero training steps during "Stage 1," along with the reporting of negative results on the Financial Phrasebank dataset, provide highly valuable insights.

Critical weaknesses
  1. Sections 3 and 4.3 treat both forward embeddings and backward gradients as attack surfaces. While the forward mechanism protects specific embedding outputs—a protection maintained throughout the server-side view generation process—the gradients in Phase 2 also depend on the original local embeddings and private labels. Consequently, these gradients are not necessarily mere post-processing of the already privacy-protected forward information. Top-k sparsification (including the transmitted support sets and error-feedback history) does not, in itself, constitute a differential privacy (DP) mechanism. The attack evaluations reported in the paper focus on "embedding inversion" rather than attacks targeting gradients or the full training process trace. Therefore, the claim of "system-wide protection" exceeds what the current evidence supports; either the backward channel must be provided with corresponding safeguards or evaluations, or the scope of the privacy protection claim must be narrowed.

  2. The paper defines "per-token" clipping and an L2L_2-Laplace density governed by a parameter η\eta, yet it fails to explicitly derive the privacy guarantees under specific "token-level," "sequence-level," or "record-level" adjacency definitions. For any clipped embedding, the maximum replacement distance is 2c2c; thus, the relationship between the parameter η\eta, metric privacy, and traditional ϵ\epsilon-privacy guarantees must be explicitly articulated rather than simply assumed. Furthermore, the handling of repeated samples across the two phases and multiple training epochs remains ambiguous: reusing cached, privacy-protected embeddings versus regenerating new outputs carries different implications for the "training process trace." Without these specific clarifications, the numerical setting of η\eta alone is insufficient to establish the strength of the claimed "end-to-end protection."

  3. Table 3 indicates that for the Financial Phrasebank dataset, the number of steps in Phase 1 is zero; meanwhile, for the BBC News dataset, an AUC of approximately 0.99 is achieved using only Phase 2. The "local-only" baseline achieved AUC scores of 0.990 on BBC News and 0.898 on Financial Phrasebank—results very close to those of the full method. This suggests that the significant performance gains observed with HiddenEcho on these tasks are largely attributable to the local data stream or the architectural design of Stage 2, rather than the "noise-invariant" preprocessing mechanism. While the improvements seen in Stage 1 (for MRPC, RTE, and certain WNLI configurations) are encouraging, any attribution of these gains should be interpreted with caution, given the mixed evidence. A more robust attribution analysis could be achieved through "client-only" comparisons (with matched computational resources) and "HiddenEcho" comparisons (with matched local capacity and tuning parameters).

  4. The "views" in Stage 1 incorporate both shared client-side perturbations and additional server-side perturbations, whereas the input for Stage 2 consists solely of client-side perturbations. Although the method successfully learns useful forms of stability, the training distribution does not align with the deployment distribution. Furthermore, relying solely on "pairwise contraction" and "preserved pairwise diversity" is insufficient to demonstrate that task-relevant semantics are effectively maintained. The explanations provided in the paper would be better supported by including metrics such as "layer-wise noise amplification," an analysis of representation quality before and after adaptation, and sensitivity analyses regarding the "number of views" and "noise specifications."

  5. Equations 13–14 and Section 5.4 account only for the transmission of hidden states and gradients; they overlook the uploading of input embeddings, the traffic generated during Stage 1, and the indices or other metadata required for sparse gradients. If these additional shared communication overheads were factored in, the overall reduction factor would generally be lower. The comparison between 119 MB and 5.08 GB mentioned in the text applies only to the specific components in question and should not be generalized to the entire training process. Furthermore, the example of "network time" based on bandwidth estimates should be distinguished from the "end-to-end training latency" (measured value), which includes the overhead for server preparation.

Questions for authors
  1. What privacy guarantees exist regarding the complete view accessible to the server—including sparse gradient values, indices, error feedback history, and model updates? For the proposed architecture, have the risks of gradient-inversion, label-inference, or joint forward/backward attacks been evaluated?

  2. How are the forward privacy guarantees defined—specifically, in terms of what adjacency relation or metric? Following gradient clipping, how is the parameter η\eta mapped to these privacy guarantees? Are the privacy-protected embeddings cached and reused across different stages and training epochs, or are they regenerated each time? How are scenarios involving repeated releases addressed?

  3. Referring to Table 3 and the "local-only" baseline, which improvements can be explicitly attributed to "Stage 1," as opposed to the "clean local stream" or the design of "Stage 2"? Have any comparative experiments been conducted using "matched local capacity" or "matched hyperparameter tuning" (i.e., ensuring equivalent tuning effort)?

  4. Could the authors provide a full breakdown of the total data transmitted (in bytes)—covering embedding uploads, Stage 1, sparse indices, and metadata—along with the total training overhead? What specific components are included in the 119 MB data figure mentioned in the paper?

  5. Are the results presented in the main tables based on evaluations on a "held-out test set," or do they represent the "best results on the validation set"? Please clarify the partitioning of the training, validation, and test sets; the random seeds used; the tuning budgets allocated for the baseline models; and the uncertainty associated with the reported improvements.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.