- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper extends function vectors (FVs), which are activation vectors derived from in-context demonstrations, to feedback vectors (FBVs). These are vectors extracted from natural-language rules, explanations, and critiques that contain no input--output pairs. Four extraction methods are proposed: a residual readout (A), a contrastive mean (B), a causal-mediation construction mirroring the FV pipeline (C), and a supervised optimized readout (D). The authors evaluate them on six short-answer tasks across seven models (1B–9B). They reported that (i) feedback vectors can induce the tasks zero-shot and, under a Q/A template, often outperform critiques given as text; (ii) the vector pipeline is less sensitive to the prompt template than text feedback; (iii) Method-C heads overlap little with FV heads; (iv) FVs and FBVs generalize differently out of distribution; and (v) vector accuracy correlates with text-feedback accuracy (r = .86).
Critical strengths
- The "showing vs. telling" framing is interesting and intuitive.
Critical weaknesses
[W1] The four extraction methods are not justified
- The paper uses four constructions (Methods A–D) but never explains why this set is needed or what each contributes to the research question.
- Each method gets a lineage (Hendel-style, CAA/Stolfo-style, Todd-style, Yang-style) and a one-line description (Line 167–196), but no argument for why feedback requires all of them or what we learn when they disagree.
- Furthermore, there is no explanation of method-by-task differences and the injection mode differs between methods.
[W2] The problem the paper aims to solve is unclear
- The paper opens with an appealing frame, but the reader is left without a clear statement of the problem being tackled.
- The contributions read as four separate observations rather than evidence for a central claim.
[W3] The distinction from Davidson et al. (2025) is not clear
- The paper does not show any result specific to explanations or critiques that differs from instructions.
- There is no direct comparison with Davidson's instruction vectors on the same tasks and models.
- The authors should state precisely what is learned here that could not be inferred from Davidson et al.
- Because of this concern, I cannot find out implications of this work.
[W4 (Minor)] Sensitivity on experiments
- Template sensitivity is listed as one of four contributions (l.86), but it is not a substantive finding in that sensitivity of language models to prompt format is well known, as the paper itself acknowledges.
- The results may be highly sensitive to how the feedback is written
[Typos]
- Line 176–177: "Risk: captures a generic 'an instruction is present' direction…" is informal for the main text. Consider rephrasing as a sentence.
- Line 215: The citation "(Llama Team, AI @ Meta, 2024)" is formatted incorrectly. The Gemma citation on the same line has the same issue.
Questions for authors
See Weaknesses.