- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
BG: Users may not want a cloud LLM provider to see their private inputs. To protect privacy, existing covariant transformation methods change the way of user inputs and model weights represented, to hide the privacy inputs and intermediate results from the service provider.
RP: However, to keep the model utility, the covariant transformations should follow certain rules. These rules can give attackers a way to recover the private inputs.
Method: Firstly, the paper introduces CROSSWEIGHT, an attack that recovers private inputs. It cancels out the transformations on user input by multiplying weights from different parts of the model. Then the attacker can use the publicly available original model to work backward the user inputs. Secondly, the paper proposes a defense using low-rank transformations, which discard some information so that the cancellation exploited by CROSSWEIGHT no longer works.
Critical strengths
- The paper exposes the structural weakness in covariant transformations. It identifies an exploitable invariant arising from the model’s structural constraints.
- The CROSSWEIGHT is solid. Table 2 reports 100% text-token recovery across all three protection schemes on Qwen3-1.7B, Llama3-8B, and the MoE model Qwen3-30B-A3B.
Critical weaknesses
The weakness is mainly on the proposed defense. It is designed to defend against only one specific attacker (the proposed CrossWeight). However, making CrossWeight ineffective doesn’t mean that the private inputs cannot be recovered. In particular, the low-rank transformations are initialized deterministically from the identity and then trained only lightly, so the deployed weights likely remain close to the public plaintext weights, and an attacker who simply compares the two directly (without the cancellation step CrossWeight relies on) may still recover the token permutation. The paper does not evaluate any such adaptive attacker, so the claim of privacy guarantee may be not supported.
Questions for authors
In Table 2, the Llama3-8B / EIA row and ConjFormer Column reports 0.00 TTRSR but 100.00 BLEU-4. Is this a typo? Why BLEU-4 got such different result only on this setting?
2 comments
Response to Reviewer uExm (Part 2/2)
W1 (Part II): Scope of Privacy Guarantees
We agree with the reviewer that our defense is designed to counter a specific attack (i.e., the proposed CrossWeight), and that robustness against a particular attack does not imply a universal cryptographic guarantee against all possible adaptive adversaries. Our primary contribution lies in identifying a fundamental structural vulnerability shared by existing lossless covariant transformation schemes through CROSSWEIGHT, and proposing a targeted low-rank remedy that disrupts the cancellation mechanism, achieving a favorable empirical privacy–utility trade-off against CROSSWEIGHT and commonly studied privacy attacks. In the revised manuscript, we have tempered the broad privacy claims in the Abstract and Section 4 to more accurately position our defense as an empirically validated mitigation rather than a universal privacy guarantee. We have also explicitly acknowledged this limitation in the added Limitation section and identified broader theoretical security analyses, including robustness against potentially stronger adaptive attacks, as an important direction for future work.
Q1: Typo in the Llama3-8B / EIA result in Table 2.
Response: Thank you for catching this typo. The BLEU-4 value was a transcription error and has been corrected in Table 2.
Response to Reviewer uExm (Part 1/2)
We sincerely thank the reviewer for the valuable and thoughtful comments, and for recognizing the structural vulnerabilities in covariant transformations revealed by our work, the soundness of our CROSSWEIGHT attack, and the 100% text-token recovery demonstrated across three protection schemes and multiple dense and MoE model architectures.
Please find the point-by-point response below.
Response: We thank the reviewer for this insightful observation. We agree that when choosing the identity basis (Pˇ(0)=B=[Ir;0]) for initialization, light fine-tuning may leave individual factors close to B, potentially allowing an adaptive attacker to directly compare the deployed weights (e.g., W~e) against the public plaintext weights (We) without relying on CROSSWEIGHT's cancellation step.
We clarify that this is specific to the identity-aligned initilazation, which is not necessary for our proposed low-rank framework.. Importantly, this issue can be addressed through random orthogonal initialization or an equivalent basis reparameterization of the trained checkpoint, without modifying the underlying defense mechanism.
1. Random Orthogonal Initialization:
In Appendix E, B=[Ir;0] was adopted as a simple and convenient initialization for the low-rank initialization, rather than as an essential component of our defense. We may instead adopt a secret random orthogonal initialization with R∈Rr×r, setting Pˇ(0)=BR and Qˇ(0)=R⊤B⊤, where RR⊤=Ir. Since Pˇ(0)Qˇ(0)=BRR⊤B⊤=BB⊤, this modification preserves exactly the same initial low-rank projection while randomly rotating the low-rank coordinates, thereby preventing the direct coordinate alignment exploited by the adaptive attacker. Importantly, this change does not alter the underlying defense mechanism. Retraining Qwen3-0.6/1.7B with this random orthogonal initialization yields the following results:
As shown above, random orthogonal initialization can reduce direct comparison recovery to 0.00% while preserving comparable utility and CROSSWEIGHT resistance. This confirms that the observed vulnerability is specific to the identity-aligned initialization rather than inherent to our low-rank framework.
2. Inherent Basis Equivalence of Existing Results:
Furthermore, the low-rank transformations (Pˇ,Qˇ) naturally admit an orthogonal change of basis: for any trained pair (Pˇ,Qˇ) and any secret orthogonal matrix R∈Rr×r, the rotated pair (PˇR,R⊤Qˇ) preserves exactly the same composition, (PˇR)(R⊤Qˇ)=PˇQˇ. As a complementary verification, this reparameterization can be applied directly to already-trained adapters without retraining, removing the direct coordinate alignment between the deployed embedding weights W~e=ΠWe(PˇeR) and the truncated plaintext weights. The results obtained by directly reparameterizing the already-trained Qwen3-1.7B adapters without retraining are shown below, with similar findings on Qwen3-0.6B and Llama3-8B.
Post-training rotation reduces direct comparison recovery from 98.56% to 0.00% while leaving all reported utility metrics and recovery results against CROSSWEIGHT, VMA, IA, and IMA unchanged. Together with random orthogonal initialization, this demonstrates that the observed coordinate-alignment leakage can be mitigated without modifying the proposed low-rank defense or its training objectives.
We have added the random orthogonal initialization results to revised Tables 3 and 8, and extended Appendix G.3 with evaluations of the reviewer's suggested direct weight-comparison attack and the extended basis reparameterization.