acceptodds
Under review as a conference paper at ICLR 2027

CORE: Identifying and Protecting Critical Outliers in LLMs against Functionality Stealing

Abstract

Protecting proprietary large language models (LLMs) on user-controlled devices requires preventing unauthorized reuse while preserving normal inference. We investigate which components to prioritize based on their importance to model functionality and how to hinder their reconstruction from observable input–output pairs. We find that removing a small subset of outlier-associated weights from the MLP of one specific decoder layer severely degrades model performance, whereas removing the same number from other layers has much less effect. This finding motivates prioritizing the corresponding critical MLP for protection. We propose CORE, a learning-based defense algorithm that combines fixed encodings with learned selective recovery to hinder reconstruction of this component. With the original LLM parameters frozen, the recovery module is trained to support normal inference, while occasional output modifications shift the MSE-optimal surrogate prediction away from the output used in most executions. The fixed encodings and trained module can be reused without preparing fresh masks and corresponding recovery values for each inference forward pass. Across three LLMs, we show that the proposed CORE retains 96.0-98.8% of the original average zero-shot accuracy, while an MSE-trained surrogate replacement performs 17.8-31.2% below the protected model.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Model, Security and Privacy

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_kZGvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper identifies a "critical MLP" via layer-wise weight removal and proposes CORE, which protects it using two fixed sign–Hadamard encodings and a recovery module trained only on the primary encoding. Returning the secondary output with 1% probability shifts the MSE-optimal surrogate prediction. On three LLMs, CORE retains 96.0%–98.8% of the original zero-shot accuracy, while MSE-trained surrogates degrade substantially.

Critical strengths

S1. The paper raises an important point: hiding weights alone does not prevent surrogate training on observable input–output pairs. Addressing this at an internal interface while keeping the LLM frozen is a thoughtful design choice.

S2. The empirical effect is clear (e.g., LLaMA-2-7B: 65.98% → 34.78% after surrogate replacement), and Proposition 1 provides a characterization of the MSE-optimal prediction under mixed outputs.

Critical weaknesses

W1. The benefit of critical-layer selection could be validated more directly.

Figure 1 measures degradation after weight removal, which is not the same as resistance to reconstruction; a functionally important component may still be easy to reconstruct. It would be helpful to include an experiment applying CORE to non-critical layers. Since Sun et al. (2024a), which the paper cites, have already reported the importance of massive activations in specific early layers, demonstrating a protective benefit would make this contribution considerably stronger.

W2. Additional mechanism controls and simple baselines would strengthen the claims.

(1) Table 2 replaces the learned recovery with the exact inverse, which changes both the form and the complexity of the mapping, and Table 4 covers only 1%–5%. A run with the same recovery module at zero secondary probability would help clarify whether surrogate failure stems from the mixed outputs or from the learned mapping being harder to fit.

(2) The mean shift in Proposition 1 is not unique to CORE; a low-probability fixed offset would produce a similar effect. A comparison against such a baseline would help establish the necessity of the dual encoding and learned recovery.

W3. The robustness of the protection deserves further examination.

(1) With the recovery-supervision depth k = 0, Qwen3's surrogate accuracy (69.55%) slightly exceeds that of the protected model (68.34%); a clear gap appears only at k = 4. It would help to discuss how k (0, 5, and 4 for the three models) can be chosen without knowledge of the attack outcome.

(2) The evaluation does not consider an attacker who knows the CORE mechanism. Such an attacker might adopt a robust objective (under a coordinate-wise L1 loss, the optimum is the primary output whenever its probability exceeds one half) or filter out suspected secondary outputs. The paper lists this only as future work; even a simple adaptive evaluation, or at least a discussion of why such strategies would or would not succeed, would substantially strengthen the security claims.

W4. The cost–utility trade-off could be quantified more fully.

The added recovery module incurs training and inference costs, and reporting latency, trusted-memory, and communication costs would clarify the practical benefit; the 0.02% removal ratio does not reflect the actual protection cost. In addition, LLaMA-3-8B's WikiText-2 and PTB perplexity increase by about 56% and 54%, a cost that retained zero-shot accuracy alone does not fully capture.

Questions for authors
  1. Under matched budgets, how does CORE at the critical layer compare with CORE at non-critical layers?
  2. With the same recovery module, how does the surrogate perform at zero secondary probability? How does CORE compare with a low-probability fixed offset?
  3. How might k be selected in practice? Can an attacker who knows the CORE mechanism bypass the protection through simple adaptations, such as a robust training objective or filtering suspected secondary outputs?
  4. What are CORE's latency, trusted-memory, and communication costs relative to per-inference masking?
8 comments

We thank the reviewer for the constructive comments. Please find our point-by-point responses below.

W1. The benefit of critical-layer selection could be validated more directly.

Figure 1 measures degradation after weight removal, which is not the same as resistance to reconstruction; a functionally important component may still be easy to reconstruct. It would be helpful to include an experiment applying CORE to non-critical layers. Since Sun et al. (2024a), which the paper cites, have already reported the importance of massive activations in specific early layers, demonstrating a protective benefit would make this contribution considerably stronger.

Q1. Under matched budgets, how does CORE at the critical layer compare with CORE at non-critical layers?

We thank the reviewer for suggesting this direct test. To compare the critical layer with non-critical layers at matched cost, we apply CORE with the same configuration to the 5th, 9th, and 17th layers of LLaMA-2-7B. All MLPs of this model have the same size, so the protected size and the TEE cost are identical across layers.

As the reviewer notes, a layer that is important to the model is not necessarily difficult to steal. We therefore evaluate the two strategies available to an attacker and report the stronger one. Option 1: reconstruct the MLP with MSE. Its best possible result is S∗=(1−ϵ),y0+ϵ,y1S^*=(1-\epsilon)\\,y_0+\epsilon\\,y_1, the optimum of MSE reconstruction (Proposition 1). We compute it exactly by replacing the MLP's output with S∗S^*, so it does not depend on how well a particular attacker is trained. At the critical layer, the trained attacker matches it within 4%. Option 2: remove the MLP and use the remaining model unchanged.

Table R1: CORE at the critical and non-critical layers of LLaMA-2-7B (WikiText-2 PPL).

Protected MLPReferenceMSE optimum S∗S^*MLP removedStronger attack / Reference
2nd layer (critical)13.742910611909867×867\times
5th layer13.621392014.321.05×1.05\times
9th layer13.49575312.680.94×0.94\times
17th layer13.4159.413.781.03×1.03\times

At the 5th, 9th, and 17th layers, no reconstruction is required. Removing the protected MLP leaves the perplexity within 6% of the deployed model, so the attacker obtains a comparable copy at no cost. CORE still causes MSE reconstruction to fail at these layers (by up to three orders of magnitude at the 5th layer), but this does not prevent the attack, because the attacker can omit the layer entirely.

At the critical layer, neither strategy succeeds. Both removing the MLP and reconstructing it by MSE increase the perplexity by about three orders of magnitude, so the attacker's best copy remains 867×867\times worse than the deployed model. The results on PTB and on held-out text show the same pattern.

The layer that is most important to the model is therefore also the layer at which protection is effective. Sun et al. [1] identify where massive activations arise, and Yu et al. [2] link them to a small number of “super weights” in an early MLP. We add the security consequence of this location: the MLP whose outlier weights are indispensable to the model (Figure 1) is the only MLP that the attacker can neither remove nor reconstruct by MSE, and it is therefore where protection is most effective. The revision adds Table R1 and this discussion, with both citations.


References

[1] Massive Activations in Large Language Models. COLM 2024.

[2] The Super Weight in Large Language Models. arXiv:2411.07191, 2024.

W2. Additional mechanism controls and simple baselines would strengthen the claims.

(1) Table 2 replaces the learned recovery with the exact inverse, which changes both the form and the complexity of the mapping, and Table 4 covers only 1%–5%. A run with the same recovery module at zero secondary probability would help clarify whether surrogate failure stems from the mixed outputs or from the learned mapping being harder to fit.

Q2. With the same recovery module, how does the surrogate perform at zero secondary probability?

We thank the reviewer for suggesting this control. To determine whether the surrogate failure stems from the mixed outputs or from the learned mapping, we repeat the MSE attack of Table 1 against the same recovery module with the secondary probability set to zero. Specifically, the surrogate architecture and the query budget are unchanged, and the only difference is that ϵ\epsilon is reduced from 1% to 0. As reported in Table R2, the surrogate learns a working copy that is close to the copy stolen from the unprotected model (within 10% in perplexity and 1 point in accuracy), whereas the same attack at ϵ=1\epsilon=1\\% yields a perplexity about 2000×2000\times higher and roughly half the accuracy. We therefore conclude that the surrogate failure stems from the mixed outputs, not from the learned mapping being harder to fit. The role of the learned recovery is to make the mixing effective. By Proposition 1, the MSE optimum shifts by ϵ(y1−y0)\epsilon(y_1-y_0). The learned recovery is trained only on code 0, so it maps code 1 to a distant output, and the shift is large. With the exact inverse, the secondary output remains close to the primary one, the shift is small, and the same attack succeeds (13.98 PPL, Table 2).

Table R2: MSE stealing at zero secondary probability (LLaMA-2-7B).

Released outputsWikiText-2 PPLCMQA (%)
No defense13.0265.55
Same recovery module, ϵ=0\epsilon=014.2264.61
CORE, ϵ=1\epsilon=1\\%2805634.78

The revision adds this control to the appendix.

W2. Additional mechanism controls and simple baselines would strengthen the claims.

(2) The mean shift in Proposition 1 is not unique to CORE; a low-probability fixed offset would produce a similar effect. A comparison against such a baseline would help establish the necessity of the dual encoding and learned recovery.

Q2. How does CORE compare with a low-probability fixed offset?

We thank the reviewer for this valuable suggestion. To compare CORE with a low-probability fixed offset, we construct a baseline in which the TEE releases h+ch+c instead of the original output hh with the same probability ϵ\epsilon. Here cc is a fixed vector scaled so that ∣c∣2\\|c\\|^2 equals CORE's D=E∣y1−y0∣2D=\mathbb{E}\\|y_1-y_0\\|^2, so that both defenses shift the MSE optimum by the same magnitude. The results in Table R3 support two conclusions. First, they confirm Proposition 1: the mean shift induced by mixing is what degrades the stolen copy, so a fixed offset also defeats MSE stealing. Second, they establish the necessity of the dual encoding and the learned recovery, as the reviewer asks. A fixed offset displaces every output by the same vector, so the stolen copy deviates from the original layer by a constant. Subtracting ϵc\epsilon c restores the base model exactly, and no protection remains. In CORE, the dual encoding provides the recovery with a second input, and the learned recovery, trained only on code 0, maps it to an output that is far from the primary one and depends nonlinearly on the input. The MSE optimum is displaced as far as with the fixed offset, but in an input-dependent direction. Only 0.2% of this displacement is constant, and even the best constant correction leaves the copy unusable. This input dependence is what the dual encoding and the learned recovery contribute beyond a fixed offset.

Table R3: CORE vs. a fixed offset on LLaMA-2-7B (WikiText-2 PPL).

DefenseMSE copyMSE optimum S∗S^*S∗S^* after constant correctionConstant share of the shift
CORE28056291063347620.2%
Fixed offset10608810568512.19100%

The revision adds Table R3 and this comparison.

Reviewer_rJKXOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies selective protection of on-device LLMs against functionality stealing through observable intermediate representations. It identifies MLPs whose outlier-associated weights are particularly sensitive to removal, then proposes CORE, which combines fixed rotations with learned selective recovery and occasional secondary outputs. An analysis characterizes the resulting bias in MSE-optimal surrogate predictions. Experiments on three LLMs show that CORE largely preserves average zero-shot accuracy while substantially degrading the evaluated MSE-trained surrogate replacements.

Critical strengths
  1. The layer-wise ablation across four LLMs reveals pronounced differences in sensitivity to removing outlier-associated weights. This provides a concrete empirical basis for prioritizing particular MLPs for protection.

  2. CORE demonstrates a substantial functionality gap between protected models and the evaluated surrogate replacements across three LLMs, while keeping the original model parameters frozen. The exact-inversion comparison provides useful evidence that learned recovery contributes to the observed protection.

Critical weaknesses
  1. Security evidence is limited to the evaluated MSE reconstruction attack. The analysis establishes a shift in the population MSE optimum, but does not establish resistance to filtering, robust objectives, or downstream training. These adaptive attacks remain unevaluated, while clean-interface controls and direct evaluation of the mixture mean would help separate the proposed mechanism from surrogate capacity and optimization limitations.

  2. The practical utility–cost tradeoff is insufficiently characterized. Aggregate perplexity and commonsense accuracy do not distinguish primary-path recovery errors from secondary-output failures or establish reliability during extended generation. Without real TEE measurements or comparable protection baselines, the deployment benefits that justify these losses remain unclear.

Questions for authors
  1. How is branch selection handled during autoregressive generation? What is the frequency and severity of user-visible errors caused by secondary outputs, particularly in workflows involving repeated model calls?

  2. Could you report primary-only performance separately from mixed-output performance, including long-form generation or multi-turn tasks? This would help distinguish recovery errors from the effects of occasional secondary outputs.

  3. How robust is CORE to repeated-query filtering, robust regression losses, or surrogate training through the accessible downstream layers? These strategies could avoid fitting the mixture mean targeted by the current analysis.

  4. Could you provide clean-interface reconstruction controls for all three models under matched attack budgets? Evaluating the exact mixture mean through the downstream network would also help establish whether mean bias explains the observed degradation.

  5. What are the total protected parameter count and inference overhead, including the recovery MLP? How does CORE compare with existing protection methods and alternative protected layers under comparable resource budgets?

5 comments

We thank the reviewer for the constructive comments. Please find our point-by-point responses below.

W1. Security evidence is limited to the evaluated MSE reconstruction attack. The analysis establishes a shift in the population MSE optimum, but does not establish resistance to filtering, robust objectives, or downstream training. These adaptive attacks remain unevaluated, while clean-interface controls and direct evaluation of the mixture mean would help separate the proposed mechanism from surrogate capacity and optimization limitations.

Q3. How robust is CORE to repeated-query filtering, robust regression losses, or surrogate training through the accessible downstream layers? These strategies could avoid fitting the mixture mean targeted by the current analysis.

Q4. Could you provide clean-interface reconstruction controls for all three models under matched attack budgets? Evaluating the exact mixture mean through the downstream network would also help establish whether mean bias explains the observed degradation.

We thank the reviewer for these precise suggestions. Both controls and the adaptive analysis are provided below.

Clean-interface controls (Q4). To separate the effect of CORE from surrogate capacity and optimization, we repeat the MSE attack on the unprotected interface of all three models, with the surrogate architecture and the budget unchanged. As reported in Table R1, the attack yields a working copy in every case, whereas under CORE the copy has between 2.8×2.8\times (Qwen3-8B) and more than 2000×2000\times (LLaMA-2-7B) higher perplexity and 15–31 points lower CMQA accuracy. Because the two settings differ only in the interface, we attribute the gap to CORE's outputs rather than to surrogate capacity or optimization.

Exact mixture mean (Q4). We additionally evaluate the exact MSE optimum S∗=(1−ϵ),y0+ϵ,y1S^*=(1-\epsilon)\\,y_0+\epsilon\\,y_1 (Proposition 1) in Table R1. Specifically, we replace the MLP's output with S∗S^* and evaluate the resulting model, so the result is independent of how well a particular attacker is trained. On LLaMA-2-7B, the trained copy matches S∗S^* within 4%. On all three models, S∗S^* itself is worse than the deployed model, by a factor that ranges from about 1.5 (Qwen3-8B) to more than 2000 (LLaMA-2-7B). Even a perfectly trained MSE attacker therefore cannot match the deployed model, and the trained copies perform worse.

Table R1: MSE-stolen copy without and with CORE.

Model (kk)WikiText-2 PPL — ReferenceWikiText-2 PPL — Copy, no defenseWikiText-2 PPL — Copy, COREWikiText-2 PPL — S∗S^*CMQA (%) — Copy, no defenseCMQA (%) — Copy, CORE
LLaMA-2-7B (0)13.7413.02280562910665.5534.78
LLaMA-3-8B (5)22.0918.60533.2127.667.4438.61
Qwen3-8B (4)24.1629.6783.3335.7165.1250.56

Robust losses, filtering, downstream training (Q3, W1). We agree that an attacker who uses a robust loss or filters the outputs recovers the primary output y0y_0. We formalize this result and verify it empirically.

Proposition 2 (new). If the branch is drawn independently of rr with ϵ<1/2\epsilon < 1/2, the coordinate-wise median and the geometric median of the released output equal the primary output y0(r)y_0(r), and the Huber minimizer lies within ϵδ/(1−ϵ)\epsilon\delta/(1-\epsilon) of y0y_0.

Table R2 confirms the proposition. On both models, the ℓ1\ell_1 copy is within 4% of the primary path's WikiText-2 perplexity. Secondary outputs are also readily identified by their norm (AUROC ≥0.9996\geq 0.9996), so filtering them out has the same effect.

Table R2: MSE vs. robust reconstruction objectives.

ModelConfigurationPPL (WikiText-2)PPL (PTB)CMQA (%)
LLaMA-2-7B (k=0k=0)Primary path12.7149.4566.23
LLaMA-2-7B (k=0k=0)Reference13.7553.2065.98
LLaMA-2-7B (k=0k=0)MSE copy280566676334.78
LLaMA-2-7B (k=0k=0)ℓ1\ell_1 copy13.1786.3766.08
LLaMA-2-7B (k=0k=0)Unsquared ℓ2\ell_2 copy13.5075.9965.85
LLaMA-2-7B (k=0k=0)Huber copy14.61113.2264.64
LLaMA-3-8B (k=5k=5)Primary path20.4540.1367.51
LLaMA-3-8B (k=5k=5)Reference22.1043.0967.33
LLaMA-3-8B (k=5k=5)MSE copy533.44728.1738.61
LLaMA-3-8B (k=5k=5)ℓ1\ell_1 copy21.0941.2567.04

This behavior follows from the structure of the released outputs. CORE shifts their mean, whereas the majority of them remain equal to y0y_0. MSE regression converges to the mean and therefore fails. MSE is the standard objective for learning a layer replacement from hidden states [1] and is the threat model of the submission. Robust losses, filtering, and voting instead recover the majority output and therefore succeed.

This analysis also indicates how the design can be strengthened. These attacks rely on y0y_0 being the majority output, and two modifications address this dependence.

(a) Keyed selection. The TEE chooses the branch with a keyed pseudorandom function of the input, so a repeated query returns the same output and yields no additional information.

(b) Majority-free release. The TEE uses K≥3K\geq 3 codes, each drawn with probability 1/K1/K, and trains the shared recovery MLP so that every code gives a usable output after the next kk layers (Eq. 4), with an additional loss term that separates the outputs of different codes. No output is then a majority, so Proposition 2 no longer applies: the median, like the mean, is no longer one of the usable outputs, and there is no rare branch to filter out. We expect this to remove the advantage of robust losses and filtering. To succeed, the attacker would have to model a multi-modal distribution and select a single mode.

Training the copy through the downstream layers constitutes a different class of attack. In this case, the attacker re-learns the layer from text through the frozen remainder of the model, rather than from the outputs that CORE releases. The cost of this attack equals the cost of re-training the layer, which no defense at the layer's interface can alter.

The revision adds the clean-interface control to Table 1, reports the exact mixture mean in the appendix, and states the MSE threat model explicitly. The extension of the protection beyond this threat model is identified as a direction for future work. To inform this direction, the revision includes Proposition 2 and Table R2 as an initial analysis and outlines (a)–(b) as possible extensions.


References

[1] Streamlining Redundant Layers to Compress Large Language Models. ICLR 2025.

W2. The practical utility–cost tradeoff is insufficiently characterized. Aggregate perplexity and commonsense accuracy do not distinguish primary-path recovery errors from secondary-output failures or establish reliability during extended generation. Without real TEE measurements or comparable protection baselines, the deployment benefits that justify these losses remain unclear.

Q1. How is branch selection handled during autoregressive generation? What is the frequency and severity of user-visible errors caused by secondary outputs, particularly in workflows involving repeated model calls?

Q2. Could you report primary-only performance separately from mixed-output performance, including long-form generation or multi-turn tasks? This would help distinguish recovery errors from the effects of occasional secondary outputs.

Q5. What are the total protected parameter count and inference overhead, including the recovery MLP? How does CORE compare with existing protection methods and alternative protected layers under comparable resource budgets?

We thank the reviewer for asking us to separate the two error sources and to quantify the deployment cost.

Primary-only vs. mixed outputs (Q2). To separate the two error sources, we evaluate the model with every request served by the primary path, which isolates the recovery error (Table R3). The primary path retains 96–99% of the base zero-shot accuracy on the three models, and its perplexity is within about 5% of the base model on LLaMA-2-7B and Qwen3-8B. The 1% secondary requests account for the remaining gap to the deployed model (Reference), which is below 0.3 accuracy points and 8–9% in perplexity. To assess extended generation, we further evaluate GSM8K with chain-of-thought decoding on Qwen3-8B (Table R4). The primary path keeps 97% of the base accuracy, and the deployed system, which draws one branch per request, keeps 96%, whereas the MSE-stolen copy retains only 53%.

Table R3: Primary-only vs. deployed (Reference) performance.

Model (kk)WikiText-2 PPL — BaseWikiText-2 PPL — Primary onlyWikiText-2 PPL — ReferenceCMQA (%) — BaseCMQA (%) — Primary onlyCMQA (%) — Reference
LLaMA-2-7B (0)12.1912.7113.7466.7966.2365.98
LLaMA-3-8B (5)14.1420.4522.0970.1767.5167.33
Qwen3-8B (4)21.1322.2324.1669.6768.5368.32

Table R4: GSM8K accuracy of Qwen3-8B (5-shot chain of thought).

ConfigurationAccuracy (%)Relative to base
Base88.8100%
Primary only86.397%
Deployed (ϵ=0.01\epsilon=0.01)85.496%
Secondary only0.0–
MSE-stolen copy46.953%

Branch selection (Q1). The branch is drawn once per request and kept for every decoding step, so the KV cache stays consistent. A fraction ϵ=1\epsilon=1\\% of requests is affected. Over nn independent calls, the chance that at least one is affected is 1−(1−ϵ)n1-(1-\epsilon)^n (9.6% for n=10n=10). An affected request produces clearly invalid output (perplexity above 10410^4; on GSM8K, a string of zeros), so the client can detect it immediately and retry. A single retry lowers the failure rate to ϵ2=10−4\epsilon^2=10^{-4}.

Reviewer_mTDBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper protects LLMs on user devices by hiding one important MLP layer. The hidden layer sometimes (about 1% of the time) gives a wrong output on purpose. An attacker who copies the layer by training on its outputs with MSE then learns a bad copy.

Critical strengths
  1. The problem matters, and the idea is simple.
  2. The paper is honest that stronger attacks were not tested.
Critical weaknesses
  1. The defense is only tested against a simple MSE attacker. Since the wrong output is random, an attacker can query the same input twice and keep the matching answer. Other simple attacks are also not tested, such as L1 loss, removing outputs with very large values, or training the copy through the rest of the model.
  2. The setting is not clearly described. The paper does not say what is hidden in the secure hardware, what the attacker can see, or how many queries they can make.
  3. There are no baselines. There is no result for a stolen copy with no defense, no comparison with other ways of protecting models on devices (e.g., TEESlice, ShadowNet), and no comparison with protecting a different layer.
  4. One setting is picked separately for each model using the reported test numbers. For Qwen3 with k = 0, the stolen copy is even better than the protected model.
Questions for authors
  1. Could the wrong branch be chosen based on the input (e.g., a keyed hash) instead of at random, so that repeated queries do not help?

  2. Can the attacker simply start from the public model's weights for this layer?

  3. How slow is it to run this layer inside real secure hardware?

I would consider raising the score if the authors answer the questions and add more experiments.

5 comments

We thank the reviewer for the constructive comments. Please find our point-by-point responses below.

W1. The defense is only tested against a simple MSE attacker. Since the wrong output is random, an attacker can query the same input twice and keep the matching answer. Other simple attacks are also not tested, such as L1 loss, removing outputs with very large values, or training the copy through the rest of the model.

Q1. Could the wrong branch be chosen based on the input (e.g., a keyed hash) instead of at random, so that repeated queries do not help?

We thank the reviewer for identifying these attacks and for the keyed-hash suggestion. We agree that an attacker who uses a robust loss, filters the outputs, or votes over repeated queries recovers the primary output y0y_0 and obtains a working copy. We formalize this result and verify it empirically.

Proposition 2 (new). If the branch is drawn independently of rr with ϵ<1/2\epsilon < 1/2, the coordinate-wise median and the geometric median of the released output equal the primary output y0(r)y_0(r), and the Huber minimizer lies within ϵδ/(1−ϵ)\epsilon\delta/(1-\epsilon) of y0y_0.

To validate the proposition, we train the surrogate with the ℓ1\ell_1, unsquared ℓ2\ell_2, and Huber losses under the budget of Table 1 (Table R1). The copies trained with robust losses closely match the primary path: the ℓ1\ell_1 copy is within 4% of its WikiText-2 perplexity on both models, whereas the MSE copy is about 2000×2000\times worse on LLaMA-2-7B. This agrees with the proposition. Secondary outputs are also readily identified by their norm (AUROC ≥0.9996\geq 0.9996), so an attacker can filter them out, or vote over repeated queries, and train on primary outputs only.

Table R1: MSE vs. robust reconstruction objectives.

ModelConfigurationPPL (WikiText-2)PPL (PTB)CMQA (%)
LLaMA-2-7B (k=0k=0)Primary path12.7149.4566.23
LLaMA-2-7B (k=0k=0)Reference13.7553.2065.98
LLaMA-2-7B (k=0k=0)MSE copy280566676334.78
LLaMA-2-7B (k=0k=0)ℓ1\ell_1 copy13.1786.3766.08
LLaMA-2-7B (k=0k=0)Unsquared ℓ2\ell_2 copy13.5075.9965.85
LLaMA-2-7B (k=0k=0)Huber copy14.61113.2264.64
LLaMA-3-8B (k=5k=5)Primary path20.4540.1367.51
LLaMA-3-8B (k=5k=5)Reference22.1043.0967.33
LLaMA-3-8B (k=5k=5)MSE copy533.44728.1738.61
LLaMA-3-8B (k=5k=5)ℓ1\ell_1 copy21.0941.2567.04

This behavior follows from the structure of the released outputs. CORE shifts their mean, whereas the majority of them remain equal to y0y_0. MSE regression converges to the mean and therefore fails. MSE is the standard objective for learning a layer replacement from hidden states [1] and is the threat model of the submission. Robust losses, filtering, and voting instead recover the majority output and therefore succeed.

This analysis also indicates how the design can be strengthened. All three attacks rely on y0y_0 being the majority output, and two modifications address this dependence. The first corresponds to Q1.

(a) Keyed selection (Q1). Yes. The TEE can choose the branch with a keyed pseudorandom function of the input sequence. The same input then always receives the same output, so repeated queries yield no additional information. ϵ\epsilon and utility are unchanged.

(b) Majority-free release. The TEE uses K≥3K\geq 3 codes, each drawn with probability 1/K1/K, and trains the shared recovery MLP so that every code gives a usable output after the next kk layers (Eq. 4), with an additional loss term that separates the outputs of different codes. No output is then a majority, so Proposition 2 no longer applies: the median, like the mean, is no longer one of the usable outputs, and there is no rare branch to filter out. We expect this to remove the advantage of robust losses and filtering. To succeed, the attacker would have to model a multi-modal distribution and select a single mode.

Training the copy through the downstream layers constitutes a different class of attack. In this case, the attacker re-learns the layer from text through the frozen remainder of the model, rather than from the outputs that CORE releases. The cost of this attack equals the cost of re-training the layer, which no defense at the layer's interface can alter.

The revision states the MSE threat model explicitly and identifies the extension of the protection beyond this threat model as a direction for future work. To inform this direction, it includes Proposition 2 and Table R1 as an initial analysis and outlines (a) and (b) as possible extensions.


References

[1] Streamlining Redundant Layers to Compress Large Language Models. ICLR 2025.

W2. The setting is not clearly described. The paper does not say what is hidden in the secure hardware, what the attacker can see, or how many queries they can make.

Q2. Can the attacker simply start from the public model's weights for this layer?

We thank the reviewer for pointing this out. We now state the setting explicitly and add the following threat-model paragraph to Sec. 3.

We consider a threat model involving two entities: a defender and an attacker. The defender owns a model deployed on an edge device, while the attacker aims to extract or replicate the model through unauthorized access to the device.

Attacker's Goal and Capability:

The attacker aims to obtain a functionally equivalent model that can be deployed on an unauthorized device. We assume a strong adversary who controls the host software and has access to all model parameters outside the TEE. The attacker knows the model architecture, queries the model with arbitrary inputs, and observes the input and the released output of the protected component. From these input–output pairs, the attacker trains a surrogate for the protected component by adapting the layer-replacement method of LLM-Streamline [1]. We consider surrogates trained with the MSE loss, the standard objective for learning replacements of LLM layers.

Defender's Goal and Capability:

Following prior work [2, 3], we assume a hardware-enforced TEE that protects the confidentiality and integrity of the code and data inside it, even when the attacker controls the host. The defender aims to preserve inference utility for authorized users while preventing unauthorized reuse of the model without the protected unit.

Regarding query access, our experiments use 29,995 sequences of 2,048 tokens, with 20 queries per sequence and a fresh branch draw for each query, yielding approximately 1.2 billion token-level input–output pairs. Further details are provided in the experimental setup.

The protected components include the critical MLP's weights, the two sign vectors, the recovery MLP, and the per-request branch-selection state. Intermediate encoded representations remain inside the TEE.

Regarding Q2, CORE protects weights that are not public, so every surrogate in our experiments starts from random initialization. If the proprietary model is a fine-tune of a public base model, the public layer is already available, and the asset to protect is the change introduced by fine-tuning. Starting MSE training from the public weights does not change its optimum, which is still the mixture mean of Proposition 1. We add this case to the threat-model paragraph.


References

[1] Streamlining Redundant Layers to Compress Large Language Models. ICLR 2025.

[2] No Privacy Left Outside: On the (In-)Security of TEE-Shielded DNN Partition for On-Device ML. IEEE S&P 2024.

[3] ShadowNet: A Secure and Efficient On-device Model Inference System for Convolutional Neural Networks. IEEE S&P 2023.

W3. There are no baselines. There is no result for a stolen copy with no defense, no comparison with other ways of protecting models on devices (e.g., TEESlice, ShadowNet), and no comparison with protecting a different layer.

We thank the reviewer for requesting these baselines. All three are provided below.

Stolen copy with no defense. To quantify what the attacker obtains in the absence of any defense, we apply the MSE attack of Table 1, with the same surrogate and budget, to the unprotected model of all three LLMs. As shown in Table R2, the attacker then obtains a working copy on all three models, which retains 93–98% of the base accuracy. With CORE, the same attack with the same budget yields a copy whose perplexity is between 2.8×2.8\times (Qwen3-8B) and more than 2000×2000\times (LLaMA-2-7B) higher and whose accuracy is 15–31 points lower. Table R2 also reports S∗=(1−ϵ),y0+ϵ,y1S^*=(1-\epsilon)\\,y_0+\epsilon\\,y_1, the optimum that any MSE attacker can reach (Proposition 1). We compute it exactly by replacing the MLP's output with S∗S^*, so it does not depend on how well a particular attacker is trained. On LLaMA-2-7B, the trained copy matches it within 4%.

Table R2: MSE-stolen copy without and with CORE.

Model (kk)WikiText-2 PPL — ReferenceWikiText-2 PPL — Copy, no defenseWikiText-2 PPL — Copy, COREWikiText-2 PPL — S∗S^*CMQA (%) — Copy, no defenseCMQA (%) — Copy, CORE
LLaMA-2-7B (0)13.7413.02280562910665.5534.78
LLaMA-3-8B (5)22.0918.60533.2127.667.4438.61
Qwen3-8B (4)24.1629.6783.3335.7165.1250.56

Protecting a different layer. To examine whether the choice of the protected layer matters, we apply CORE with the same configuration to the 5th, 9th, and 17th layers of LLaMA-2-7B (Table R3). All MLPs of this model have the same size, so the TEE cost is identical. The attacker has two strategies: reconstructing the protected MLP with MSE (reaching S∗S^* at best), or removing it. The table reports the stronger of the two. At the 5th, 9th, and 17th layers, removing the MLP suffices: the perplexity stays within 6% of the deployed model, so the attacker obtains a comparable copy without any reconstruction. At the critical 2nd layer, neither strategy succeeds. Both removing the MLP and reconstructing it by MSE leave the copy about three orders of magnitude worse than the deployed model (867×867\times for the stronger attack). The critical layer is thus the only MLP that the attacker can neither remove nor reconstruct by MSE.

Table R3: CORE at the critical and non-critical layers of LLaMA-2-7B (WikiText-2 PPL).

Protected MLPReferenceMSE optimum S∗S^*MLP removedStronger attack / Reference
2nd layer (critical)13.742910611909867×867\times
5th layer13.621392014.321.05×1.05\times
9th layer13.49575312.680.94×0.94\times
17th layer13.4159.413.781.03×1.03\times

ShadowNet [1] and TEESlice [2] Both methods consume fresh masks and corresponding recovery terms for each inference. Although these materials can be precomputed offline, they cannot be reused and must be replenished as inference proceeds, incurring preprocessing, storage, or communication costs. CORE does not require these per-inference masking materials. Table R4 compares the three designs.

Table R4: Design comparison with ShadowNet and TEESlice.

ShadowNetTEESliceCORE
Model preparationweight quantization and transformationprivate-slice training and pruningrecovery-MLP training
Intermediate featuresOTP-maskedOTP-maskedreleased mixed outputs
Fresh masking material per inferencerequiredrequirednot required
Communication patternexchanges across outsourced operationsexchanges across outsourced operationsone selected interface
TEE-resident componentsmasking, restoration, and non-linear operationsprivate slices and backbone non-linear operationscritical MLP, encoding, and recovery MLP (about 4% of the parameters)

Choosing what to protect (the critical MLP) is independent of how it is protected, so the critical-layer choice can also be combined with masking-based schemes (Sec. 2.1). The revision adds Tables R2 and R3 to the experiments and this comparison to Sec. 2.1.


References

[1] ShadowNet: A Secure and Efficient On-device Model Inference System for Convolutional Neural Networks. IEEE S&P 2023.

[2] TEESlice: Protecting Sensitive Neural Network Models in Trusted Execution Environments when Attackers Have Pre-Trained Models. ACM TOSEM 2025.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.