- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper asks whether information about a character's earlier success or failure is kept active across intervening text or retrieved again when it becomes relevant. The authors test this in Qwen2.5-1.5B-Instruct using a staged pipeline: first training a probe, then using activation patching, then localizing candidate heads, and finally testing what happens after later updates. They conclude that both maintenance and retrieval are involved, and propose a compact writer-reader pathway that helps retrieve source-bound affective information.
Critical strengths
- Locations are chosen on development families and then tested on new ones, while the decision rules are fixed in advance. Family-level inference, multiple-testing correction, and reporting of failed predictions make the analysis less exploratory than many interpretability studies.
- The protocols, code, and family-level results made it possible to check many of the claims directly. Results that did not match the predictions, such as the outdating result, were still reported.
Critical weaknesses
- The source-patching effects are around 0.04–0.05, below the paper's own 0.20 smallest effect of interest from Experiment 1. The selected layer and heads also depend strongly on how the effect is scored, and the rescue test is identical to necessity by construction. I also could not see evidence connecting the layer 4 result in Experiment 2 to the heads identified in layers 12 and 13 in Experiment 3. This makes the proposed compact writer-reader pathway difficult for me to conclude from the current results.
- The probe is the only outcome, but it is not validated on the Experiment 2 and 3 story formats and performs poorly at several Experiment 4 positions. Figure 4 does not visibly show cue-time recovery: all of the lines drop or stay flat at the cue, and the positive retrieval score comes from the neutral condition dropping more. The Experiment 4 result also changes depending on starting orientation, with R coming mostly from negative-start stories while positive-start stories show outdating. Since no model behavior is measured, the claims about retrieval after suppression and what later information does or does not overwrite seem stronger than what the current readout supports.
Questions for authors
- What is the probe's accuracy on the Experiment 2 and 3 story formats, and at the different positions in Experiment 4? The low accuracy in Experiment 4 makes it hard for me to know how much weight to put on those results.
- Does source patching change what the model actually says after the cue, rather than only changing the probe score? A behavioral effect would make the causal result much more convincing to me.
- How stable is the head localization under other reasonable scoring rules, for example ranking by cue shift alone? How do the selected heads compare with random heads?
- Is there evidence connecting the layer 4 source effect in Experiment 2 to the heads in layers 12 and 13? For example, does ablating those heads reduce the earlier source-patching effect?
- Could you explain the different Experiment 4 results by starting orientation? In particular, R seems to come mostly from negative-start stories, while positive-start stories show outdating.
The maintenance-versus-reinstatement framework could use more explanation and connection to its psychology roots. "Affect" could also be better justified, since the stories manipulate success and failure rather than emotion. Terms such as "source-bound", "writer-reader", "causal restoration", and "diagnostic-TRAIN SD" were not clear to me on first reading. Since all experiments use Qwen2.5-1.5B-Instruct, I would also state the single-model scope more clearly as a limitation. A short limitations paragraph in the discussion would help.