- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper proposes TraceTilt, a framework that combines context distillation, activation steering, and model editing to retain context-induced behaviors and knowledge after the context is removed. The method constructs protected low-rank interventions by matching predictions from a context-conditioned teacher, and explores their application to behavioral steering and document-based knowledge editing. The paper also presents question-rewriting and activation-patching experiments to motivate its design. Experiments show strong performance on three behavioral steering tasks (92.2% mean accuracy), while the knowledge-editing experiments recover 28.6% and 26.9% of the Context Teacher's improvement on BFCL and FictionalQA, respectively.
Critical strengths
The combination of context distillation and model editing is reasonable and potentially insightful. Using contextual predictions as supervision for targeted parameter updates provides a promising direction for retaining information and behavior without requiring the original context at inference time.
Critical weaknesses
-
The paper lacks the mandatory AI Use Statement. The ICLR 2027 Author Guidelines explicitly require an AI Use Statement section, but this section is missing from the manuscript.
-
The paper suffers from serious clarity and organization issues, making it difficult to understand the main contributions and how different components relate to each other. In particular, the connections among behavioral steering, question rewriting, mechanistic analysis, and knowledge editing are not clearly established.
-
The knowledge-editing evaluation is insufficient to support the paper's broader claims. Tables 4 and 5 contain substantial pending results, including the entire LiteraryQA evaluation and complete capability-retention measurements. The completed knowledge-editing evaluation covers only 10 BFCL questions and 50 FictionalQA questions, which is too limited to convincingly demonstrate the effectiveness and generality of the proposed method.
Questions for authors
-
The discussion of related work appears insufficient. Given that this paper lies at the intersection of context distillation, activation steering, and model editing, could the authors better clarify its contributions relative to closely related works such as Learning by Distilling Context, SAKE, and Steer Like the LLM: Activation Steering that Mimics Prompting?
-
What motivates the specific design choices of the knowledge-editing method? In particular, why are separate shared task updates and context-specific updates necessary, and what advantages does this design offer over simpler context-distillation approaches or established model-editing methods?