acceptodds
Under review as a conference paper at ICLR 2027

GAP: Gradient-Aligned Dual Perturbation for Offsite-tuning Large Language Models

Abstract

Adapting language models across organizations sometimes requires raw data and complete original models to remain within their respective owners. Offsite-tuning addresses this situation. The model owner supplies an offsite model with a weakened emulator and trainable adapters; the data owner freezes the emulator, tunes only the adapters locally, and returns them for insertion into the complete model. The challenge is to preserve plug-in utility while maintaining an obvious performance gap over the offsite model and the tuned offsite model, which encourages the upload of the adapters to get better performance. Alignment between the emulator and the complete model is central to this trade-off. Prior approaches align representations or preserve gradients through prescribed compression operations. We introduce Gradient-Aligned dual Perturbation (GAP), a dual-perturbation approach that expands optimization beyond limited compression choices. Training-based gradient-aligned perturbation learns adversarial weight changes while matching adapter gradients at virtual gradient descent steps; compression-based gradient-aligned perturbation uses Block Influence, feature-direction change between the input and output of a layer, to select low-disruption layer for deletions. Both stages use only support set at the model owner side. Across eight benchmarks, GAP improves macro Plug-in performance over OT, the strongest baseline by 3.59% and 4.53% on Llama-3.2-3B and Llama-3.1-8B respectively with larger performance gap. We theoretically justify GAP's effectiveness and empirically demonstrate its generalization across dense and MoE backbones.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Offsite-tuning, Large Language Model

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_hgVwOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes GAP, a two-stage approach to constructing emulators for offsite-tuning large language models. The first stage perturbs the middle-layer weights while aligning adapter gradients with those of the original model at virtual update steps. The second stage removes low-importance layers using Block Influence. The goal is to improve the performance of the final plug-in model while maintaining a performance gap between the offsite and complete models. Experiments on several LLM backbones report improvements over existing offsite-tuning methods.

Critical strengths
  1. The paper addresses a practical problem in offsite-tuning: how to weaken the shared emulator without making the learned adapters ineffective when transferred back to the original model. Focusing on adapter gradient alignment makes sense in this setting, since the adapters are what ultimately need to transfer. The two-stage method is also reasonably easy to follow.

  2. The experimental results on Llama-3.2-3B and Llama-3.1-8B are encouraging, with average Plug-in improvements of 3.59 and 4.53 percentage points over OT. The paper also includes component ablations, compression-ratio comparisons, support-set analysis, and experiments on other model architectures, which help assess the method beyond a single setting.

Critical weaknesses
  1. The benefit of the training-based perturbation is less clear than the paper suggests. In Table 3, removing gradient alignment reduces Plug-in performance by only 0.87 points, while replacing BI with uniform pruning causes a larger drop of 2.79 points. It is unclear how much of the improvement comes from the proposed gradient-aligned perturbation rather than layer selection.

  2. The gap-utility trade-off is not consistently improved. On Qwen2.5-7B, GAP increases Plug-in performance from 59.36 to 61.16, but Offsite FTG drops from 11.39 to 3.85 compared with OT. A similar pattern appears on Qwen2.5-3B. These results weaken the claim that GAP can improve plug-in utility while maintaining larger performance gaps.

  3. The theoretical analysis depends on assumptions that are difficult to verify in practice. In particular, the virtual adapter states are assumed to cover the downstream training region, and the gradient difference between support and downstream data is bounded. The paper does not show whether these assumptions hold during actual downstream fine-tuning.

  4. The experimental comparison could be more convincing. The OT baseline omits its distillation stage, and ScaleOT is not included despite being discussed in related work. Also, the reported error bars reflect evaluation-sample uncertainty rather than variation across independent training runs, leaving the stability of the improvements unclear.

Questions for authors
  1. How much does the gradient-aligned perturbation contribute when the layer selection is kept fixed? Would its advantage remain consistent across different random seeds and model architectures?

  2. Why does GAP substantially reduce Offsite ZSG and FTG on the Qwen models while improving Plug-in performance? Is this trade-off sensitive to the perturbation strength or gradient alignment weight?

  3. Does gradient alignment remain effective after pruning and throughout downstream fine-tuning? The current analysis mainly uses support-set gradients and local loss slices. How closely do these reflect the actual downstream training trajectories?

  4. How would GAP compare against OT with its original distillation procedure and newer methods such as ScaleOT? Are the reported improvements stable across independent emulator construction and fine-tuning runs?

Reviewer_VPLqOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes GAP, a method for constructing weakened emulator models for offsite tuning of large language models. The objective is to enable a data owner to train adapters using a released emulator while preserving high downstream performance when those adapters are plugged into the original model.

GAP combines trainable adversarial weight perturbations with gradient-aligned block selection. The approach attempts to improve downstream plug-in accuracy while maintaining substantial performance gaps between the plug-in model and the independently usable offsite emulator, both before and after offsite fine-tuning.

The authors evaluate GAP on Llama, Qwen2.5, and DeepSeek-V2-Lite model families. The primary Llama experiments show improvements in plug-in accuracy, while the additional cross-model results reveal substantial differences in emulator-isolation performance.

Critical strengths
  1. An important practical problem.

Offsite tuning requires balancing useful downstream adaptation against the capability of the independently released emulator.

Improving this utility–isolation trade-off is an important research direction for controlled model sharing and efficient downstream adaptation.

  1. A technically plausible method.

The combination of gradient alignment and adversarial perturbation has a coherent motivation.

The gradient-alignment component attempts to preserve compatibility with downstream adapter training, while perturbation-based weakening reduces the usefulness of the emulator as an independent model.

The use of block-influence information for structured selection is also reasonable.

  1. Positive plug-in results and useful component analysis.

The Llama experiments report meaningful improvements in plug-in accuracy. The paper also includes ablations concerning perturbations, gradient alignment, block selection, and computational overhead.

These experiments provide preliminary evidence that the proposed components can improve plug-in accuracy under selected conditions.

Critical weaknesses
  1. The claimed improvement in the central utility–isolation trade-off is not consistently supported.

This is the most important empirical weakness.

The primary objective of offsite tuning is not simply to maximize downstream plug-in accuracy. It is to maintain useful plug-in performance while preventing the released emulator from becoming independently competitive.

However, Appendix Tables 5–6 (PDF page 15) show substantial reductions in both offsite-gap metrics for the Qwen2.5 models.

For Qwen2.5-3B:

  • Plug-in accuracy increases from 54.58 to 57.34.
  • Offsite ZSG decreases from 27.61 to 16.68.
  • Offsite FTG decreases from 10.86 to 2.74.

For Qwen2.5-7B:

  • Plug-in accuracy increases from 59.36 to 61.16.
  • Offsite ZSG decreases from 33.49 to 16.40.
  • Offsite FTG decreases from 11.39 to 3.85.

Thus, GAP achieves modest improvements in plug-in accuracy while substantially weakening the isolation between the plug-in model and the offsite emulator.

Positive gap values alone do not demonstrate a superior utility–isolation trade-off.

The paper should compare methods at matched minimum isolation constraints or report the Pareto frontier between plug-in accuracy and both offsite-gap metrics.

Without these comparisons, the central multi-objective advantage remains unestablished.

  1. A key baseline is incomplete, undermining the fairness of the main comparison.

Section 4.1 (PDF page 6) explicitly states that the OT baseline omits the post-compression hidden-state distillation stage from the original method because of computational constraints.

This is a consequential methodological issue.

Hidden-state distillation can affect the compatibility of the compressed emulator with the original model and consequently influence downstream plug-in performance.

Removing this stage creates a modified baseline rather than a complete implementation of the published OT method.

The paper therefore does not convincingly establish that GAP outperforms the original OT approach under comparable conditions.

Furthermore, matching nominal emulator compression ratios does not necessarily imply equivalent released parameter counts, FLOPs, construction compute, or memory requirements across different compression and sharing mechanisms.

A fair comparison requires the complete baseline and explicit resource matching.

  1. The entire model suite is outdated for an ICLR 2027 submission.

All model families used in the experiments were released in 2024:

  • Llama-3.1-8B — July 2024.
  • Llama-3.2-3B — September 2024.
  • Qwen2.5 series — September 2024.
  • DeepSeek-V2-Lite — May 2024.

No model released in 2025 or 2026 is evaluated.

This is a significant limitation for an ICLR 2027 submission, particularly because newer open-weight base models are publicly available.

For instance, Qwen3-8B-Base and Qwen3-30B-A3B-Base would enable evaluation on more recent dense and mixture-of-experts architectures without requiring prohibitively large computational resources.

The omission is not merely a cosmetic issue concerning model release dates.

GAP relies on assumptions about gradient alignment, layer importance, and the transferability of perturbations across optimization steps. These properties may depend substantially on model architecture, training procedures, layer redundancy, and scaling characteristics.

The already inconsistent emulator-isolation results on Qwen2.5 further motivate testing these properties on contemporary models.

Consequently, the evaluation does not provide sufficient evidence for generalization to current foundation models.

At minimum, the paper should include recent dense and MoE base-model evaluations using matched emulator sizes and both isolation metrics.

  1. The robustness and statistical reliability of the reported improvements are insufficiently established.

The reported uncertainties primarily characterize evaluation examples rather than independently repeated emulator construction and downstream adapter training.

Several reported improvements are relatively small and may be sensitive to perturbation initialization, optimization randomness, training subsets, or hyperparameter selection.

Furthermore, Section 4.1 indicates that hyperparameter and model-depth selection use the first, unshuffled half of evaluation splits.

This raises concerns about sensitivity to split composition and the stability of the selected configurations.

The paper should report paired multi-seed experiments that independently repeat emulator construction and adapter training, rather than relying only on evaluation-level uncertainty.

  1. The theoretical analysis does not sufficiently establish the claimed downstream behavior.

The gradient-alignment argument depends on assumptions concerning the relationship between support-set gradients and downstream optimization behavior.

However, these assumptions are not empirically verified or bounded in the evaluated settings.

Moreover, alignment is performed over only one to three virtual optimization steps.

This does not directly demonstrate that the gradient-alignment property persists throughout downstream offsite fine-tuning.

The theoretical discussion therefore provides intuition but does not establish the claimed long-term compatibility improvements.

  1. The efficiency claims do not adequately account for peak memory requirements.

Appendix Table 10 reports that GAP requires 84.11 GiB of peak GPU memory, compared with 32.18 GiB for GradOT.

Although GAP reduces emulator construction time from 7.84 to 2.26 minutes, its substantially higher memory requirement changes the practical efficiency comparison.

Memory consumption is particularly relevant because the method is motivated partly by efficient model adaptation and deployment.

The paper should explicitly evaluate trade-offs involving wall-clock time, GPU-hours, peak memory, released parameter counts, and downstream adaptation cost.

Questions for authors
  1. Can the authors rerun the complete original OT baseline, including post-compression hidden-state distillation, under matched emulator sizes and computational budgets?

  2. On Qwen2.5-3B and Qwen2.5-7B, can GAP outperform OT at fixed Offsite ZSG and Offsite FTG thresholds? Please provide the corresponding utility–isolation Pareto frontiers.

  3. Why does GAP substantially reduce both isolation gaps on Qwen models despite improving plug-in accuracy? Does this indicate a fundamental limitation of the proposed perturbation mechanism?

  4. Why does the entire evaluation rely on 2024 model families? Can the authors evaluate recent publicly available dense and MoE base models, such as Qwen3-8B-Base and Qwen3-30B-A3B-Base, using the same methods and isolation metrics?

  5. How sensitive are the reported results to perturbation initialization, training seeds, downstream data splits, and emulator-construction hyperparameters?

  6. Can the authors report complete resource comparisons, including GPU-hours, released parameter counts, FLOPs, and peak GPU memory?

  7. Can the authors empirically measure whether the gradient alignment established during the virtual optimization steps persists throughout actual downstream offsite fine-tuning?

Reviewer_t1H6OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes GAP for offsite-tuning. It perturbs the emulator middle layers with an adversarial loss while keeping adapter gradients aligned with the full model, then prunes layers by Block Influence. It shows higher Plug-in than OT, CRaSh and GradOT on eight tasks with Llama models.

Critical strengths
  1. Learning the emulator weights directly is a good idea beyond fixed compression choices.
  2. Plug-in gains are consistent across several backbones.
Critical weaknesses
  1. Baselines look weakened. OT is run without distillation, and ScaleOT is not compared.
  2. The emulator is not really weaker. On Llama-3.2-3B, GAP has actually higher Offsite ZS and Offsite FT than OT, so the gaps look good mainly because Plug-in goes up.
Questions for authors

The paper mentions exploratory depth adjustments and prior test exposure. Which results were affected, and how?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.