- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper identifies a "critical MLP" via layer-wise weight removal and proposes CORE, which protects it using two fixed sign–Hadamard encodings and a recovery module trained only on the primary encoding. Returning the secondary output with 1% probability shifts the MSE-optimal surrogate prediction. On three LLMs, CORE retains 96.0%–98.8% of the original zero-shot accuracy, while MSE-trained surrogates degrade substantially.
Critical strengths
S1. The paper raises an important point: hiding weights alone does not prevent surrogate training on observable input–output pairs. Addressing this at an internal interface while keeping the LLM frozen is a thoughtful design choice.
S2. The empirical effect is clear (e.g., LLaMA-2-7B: 65.98% → 34.78% after surrogate replacement), and Proposition 1 provides a characterization of the MSE-optimal prediction under mixed outputs.
Critical weaknesses
W1. The benefit of critical-layer selection could be validated more directly.
Figure 1 measures degradation after weight removal, which is not the same as resistance to reconstruction; a functionally important component may still be easy to reconstruct. It would be helpful to include an experiment applying CORE to non-critical layers. Since Sun et al. (2024a), which the paper cites, have already reported the importance of massive activations in specific early layers, demonstrating a protective benefit would make this contribution considerably stronger.
W2. Additional mechanism controls and simple baselines would strengthen the claims.
(1) Table 2 replaces the learned recovery with the exact inverse, which changes both the form and the complexity of the mapping, and Table 4 covers only 1%–5%. A run with the same recovery module at zero secondary probability would help clarify whether surrogate failure stems from the mixed outputs or from the learned mapping being harder to fit.
(2) The mean shift in Proposition 1 is not unique to CORE; a low-probability fixed offset would produce a similar effect. A comparison against such a baseline would help establish the necessity of the dual encoding and learned recovery.
W3. The robustness of the protection deserves further examination.
(1) With the recovery-supervision depth k = 0, Qwen3's surrogate accuracy (69.55%) slightly exceeds that of the protected model (68.34%); a clear gap appears only at k = 4. It would help to discuss how k (0, 5, and 4 for the three models) can be chosen without knowledge of the attack outcome.
(2) The evaluation does not consider an attacker who knows the CORE mechanism. Such an attacker might adopt a robust objective (under a coordinate-wise L1 loss, the optimum is the primary output whenever its probability exceeds one half) or filter out suspected secondary outputs. The paper lists this only as future work; even a simple adaptive evaluation, or at least a discussion of why such strategies would or would not succeed, would substantially strengthen the security claims.
W4. The cost–utility trade-off could be quantified more fully.
The added recovery module incurs training and inference costs, and reporting latency, trusted-memory, and communication costs would clarify the practical benefit; the 0.02% removal ratio does not reflect the actual protection cost. In addition, LLaMA-3-8B's WikiText-2 and PTB perplexity increase by about 56% and 54%, a cost that retained zero-shot accuracy alone does not fully capture.
Questions for authors
- Under matched budgets, how does CORE at the critical layer compare with CORE at non-critical layers?
- With the same recovery module, how does the surrogate perform at zero secondary probability? How does CORE compare with a low-probability fixed offset?
- How might k be selected in practice? Can an attacker who knows the CORE mechanism bypass the protection through simple adaptations, such as a robust training objective or filtering suspected secondary outputs?
- What are CORE's latency, trusted-memory, and communication costs relative to per-inference masking?
8 comments
W2. Additional mechanism controls and simple baselines would strengthen the claims.
(1) Table 2 replaces the learned recovery with the exact inverse, which changes both the form and the complexity of the mapping, and Table 4 covers only 1%–5%. A run with the same recovery module at zero secondary probability would help clarify whether surrogate failure stems from the mixed outputs or from the learned mapping being harder to fit.
Q2. With the same recovery module, how does the surrogate perform at zero secondary probability?
We thank the reviewer for suggesting this control. To determine whether the surrogate failure stems from the mixed outputs or from the learned mapping, we repeat the MSE attack of Table 1 against the same recovery module with the secondary probability set to zero. Specifically, the surrogate architecture and the query budget are unchanged, and the only difference is that is reduced from 1% to 0. As reported in Table R2, the surrogate learns a working copy that is close to the copy stolen from the unprotected model (within 10% in perplexity and 1 point in accuracy), whereas the same attack at yields a perplexity about higher and roughly half the accuracy. We therefore conclude that the surrogate failure stems from the mixed outputs, not from the learned mapping being harder to fit. The role of the learned recovery is to make the mixing effective. By Proposition 1, the MSE optimum shifts by . The learned recovery is trained only on code 0, so it maps code 1 to a distant output, and the shift is large. With the exact inverse, the secondary output remains close to the primary one, the shift is small, and the same attack succeeds (13.98 PPL, Table 2).
Table R2: MSE stealing at zero secondary probability (LLaMA-2-7B).
| Released outputs | WikiText-2 PPL | CMQA (%) |
|---|---|---|
| No defense | 13.02 | 65.55 |
| Same recovery module, | 14.22 | 64.61 |
| CORE, | 28056 | 34.78 |
The revision adds this control to the appendix.
W2. Additional mechanism controls and simple baselines would strengthen the claims.
(2) The mean shift in Proposition 1 is not unique to CORE; a low-probability fixed offset would produce a similar effect. A comparison against such a baseline would help establish the necessity of the dual encoding and learned recovery.
Q2. How does CORE compare with a low-probability fixed offset?
We thank the reviewer for this valuable suggestion. To compare CORE with a low-probability fixed offset, we construct a baseline in which the TEE releases instead of the original output with the same probability . Here is a fixed vector scaled so that equals CORE's , so that both defenses shift the MSE optimum by the same magnitude. The results in Table R3 support two conclusions. First, they confirm Proposition 1: the mean shift induced by mixing is what degrades the stolen copy, so a fixed offset also defeats MSE stealing. Second, they establish the necessity of the dual encoding and the learned recovery, as the reviewer asks. A fixed offset displaces every output by the same vector, so the stolen copy deviates from the original layer by a constant. Subtracting restores the base model exactly, and no protection remains. In CORE, the dual encoding provides the recovery with a second input, and the learned recovery, trained only on code 0, maps it to an output that is far from the primary one and depends nonlinearly on the input. The MSE optimum is displaced as far as with the fixed offset, but in an input-dependent direction. Only 0.2% of this displacement is constant, and even the best constant correction leaves the copy unusable. This input dependence is what the dual encoding and the learned recovery contribute beyond a fixed offset.
Table R3: CORE vs. a fixed offset on LLaMA-2-7B (WikiText-2 PPL).
| Defense | MSE copy | MSE optimum | after constant correction | Constant share of the shift |
|---|---|---|---|---|
| CORE | 28056 | 29106 | 334762 | 0.2% |
| Fixed offset | 106088 | 105685 | 12.19 | 100% |
The revision adds Table R3 and this comparison.
We thank the reviewer for the constructive comments. Please find our point-by-point responses below.
We thank the reviewer for suggesting this direct test. To compare the critical layer with non-critical layers at matched cost, we apply CORE with the same configuration to the 5th, 9th, and 17th layers of LLaMA-2-7B. All MLPs of this model have the same size, so the protected size and the TEE cost are identical across layers.
As the reviewer notes, a layer that is important to the model is not necessarily difficult to steal. We therefore evaluate the two strategies available to an attacker and report the stronger one. Option 1: reconstruct the MLP with MSE. Its best possible result is S∗=(1−ϵ),y0+ϵ,y1, the optimum of MSE reconstruction (Proposition 1). We compute it exactly by replacing the MLP's output with S∗, so it does not depend on how well a particular attacker is trained. At the critical layer, the trained attacker matches it within 4%. Option 2: remove the MLP and use the remaining model unchanged.
Table R1: CORE at the critical and non-critical layers of LLaMA-2-7B (WikiText-2 PPL).
At the 5th, 9th, and 17th layers, no reconstruction is required. Removing the protected MLP leaves the perplexity within 6% of the deployed model, so the attacker obtains a comparable copy at no cost. CORE still causes MSE reconstruction to fail at these layers (by up to three orders of magnitude at the 5th layer), but this does not prevent the attack, because the attacker can omit the layer entirely.
At the critical layer, neither strategy succeeds. Both removing the MLP and reconstructing it by MSE increase the perplexity by about three orders of magnitude, so the attacker's best copy remains 867× worse than the deployed model. The results on PTB and on held-out text show the same pattern.
The layer that is most important to the model is therefore also the layer at which protection is effective. Sun et al. [1] identify where massive activations arise, and Yu et al. [2] link them to a small number of “super weights” in an early MLP. We add the security consequence of this location: the MLP whose outlier weights are indispensable to the model (Figure 1) is the only MLP that the attacker can neither remove nor reconstruct by MSE, and it is therefore where protection is most effective. The revision adds Table R1 and this discussion, with both citations.
References
[1] Massive Activations in Large Language Models. COLM 2024.
[2] The Super Weight in Large Language Models. arXiv:2411.07191, 2024.