- Rating
- 1
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The manuscript addresses the problem of agent memory poisoning and proposes a supervised fine-tuning method termed Task-Preservation Training. The method uses the model to generate responses from clean memories and subsequently filters these responses to construct training targets. It then builds four types of contexts, including clean, blocker, redirect, and marker contexts. The authors conduct extensive experiments on Qwen and other models to validate the effectiveness of the proposed method, which substantially reduces the attack success rate. The manuscript also provides detailed analytical experiments.
Critical strengths
-
The manuscript addresses the important problem of memory poisoning and presents a clear research motivation.
-
The proposed Task-Preservation Training method is simple and straightforward, with relatively low training cost.
-
Experimental results show that the proposed method effectively reduces the attack success rate, and the manuscript also provides detailed analytical experiments.
Critical weaknesses
-
The proposed Task-Preservation Training is essentially based on supervised fine-tuning over contexts containing malicious instructions, using responses generated from clean contexts as training targets. Therefore, its methodological novelty appears limited.
-
The training data used in the manuscript may involve leakage of the correct answers, which requires further clarification from the authors.
-
The proposed method may primarily learn specific formatting rules. The authors should further clarify the definition of Compatible Instruction.
-
The experimental results indicate a substantial degradation in model performance, as shown in Table C12. The authors should explain the reasons for this decline.
-
The attacks considered in the manuscript are all fixed. It would be beneficial for the authors to further evaluate the proposed method against adaptive attacks.
Questions for authors
Please refer to Weaknesses.