- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper studies the backdoor attack problem in large language models (LLMs). The authors propose ThoughtSeal, which is a supply-chain attack that targets LLMs’ reasoning prior. Compared to baselines that target LLMs’ surface actions, ThoughtSeal focuses on changing the deep mindset from which they reason. To defend against that threat, they propose ThoughtSeal Preference Distillation (TPD), which contains three main components: (1) Prior Canonicalization to construct priors for teacher models; (2) On-Policy Preference Distillation (OPPD) to internalize the priors; and (3) Context-Agnostic Persistence Optimization. The experiments demonstrate the effectiveness of TPD with Qwen2.5 and Llama3.1/3.2 series models.
Critical strengths
(1) Interesting and important problem. The paper investigates backdoor attacks targeting the judgment preferences of open-source LLMs. Understanding these threats and developing methods to defend against these attacks are both important for developing trustworthy LLMs.
Critical weaknesses
(1) The writing needs to be cleaned up. The method section contains long paragraphs with equations. However, many of the equations, for example, those in Section 4.2 introducing on-policy distillation, can be simplified. The saved space can be used to provide more intuition about the method or more discussion of the experiments.
(2) Experiments are limited to outdated models. The experiments rely on Qwen2.5-7B, Llama3.1-8B, and Llama3.2-3B, which were released two years ago, in September 2024. I suggest that the authors perform more experiments on recent models, for example, models from the Qwen3.5/3.6 series, to validate the generalizability of their method.
Questions for authors
(1) In the experiments, the baselines use different hyperparameters from those used for ThoughtSeal. For example, the baselines use a learning rate of 2e-4 and a LoRA rank of 8 (based on the description around line 1167), while ThoughtSeal uses a learning rate of 5e-6 and a LoRA rank of 64. There are also some differences in the update steps. I would like to request clarification on how to select these hyperparameters and how to ensure a fair comparison between the baselines and the proposed method.
Dear Reviewer,
Thank you again for your time and constructive feedback. We have carefully addressed your concerns in our rebuttal and would greatly appreciate it if you could take a moment to review our responses.
We would be happy to provide any further clarification or additional evidence if needed. We look forward to hearing your thoughts and continuing the discussion.
Best regards, The Authors