- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes the CTA triplet as a framework for describing in-context learning across model layers through context preparation, task-constraint writing, and output alignment. The three coordinates measure distance to final context representations, sensitivity to demonstration perturbations, and the margin between the final winning label and its runner-up. The authors use activation interventions to examine these interpretations and explore applications to prompt repair and attention replacement across classification and generation tasks.
Critical strengths
- The paper addresses an important interpretability question and provides explicit operational definitions that make its hypotheses testable.
- The experiments combine layer-wise readouts with activation restoration, random-position controls, shuffled-state comparisons, and held-out evaluation, providing several complementary perspectives on the proposed coordinates.
- The reporting is also transparent about ordering reversals, weak intervention effects, and unsuccessful baseline comparisons, which helps clarify where the framework is informative and where its interpretation remains uncertain.
Critical weaknesses
- The main concern is whether the coordinates identify distinct ICL functions or partly reflect their construction.
- C measures distance from a final-layer reference, while A uses the retrospectively selected final winner, so their early and late behavior may arise partly from generic representation dynamics. T also responds to formatting changes and irrelevant demonstrations, making its interpretation as task-constraint writing insufficiently specific.
- Evidence for a sequential decomposition is mixed: mean T peaks follow A on two Llama tasks, and window-center ordering holds in only 23.8–58.9% of examples. The practical results provide limited additional support: Table 8 shows no improvement over the final output margin, and combined CTA repair produces a −0.23-point gain compared with +0.55 points for random repair.
Questions for authors
- Please examine whether the proposed ordering persists under no-demonstration and random-label controls, alternative intermediate readouts, and different normalization choices.
- Stronger source-selectivity tests would help distinguish task-rule processing from formatting sensitivity; the association between A and restoration effects should also be evaluated after controlling for depth and excluding final-block restoration.
3 comments
Thank you for your response and the new experiments. The label and whitespace comparison is useful, but it does not fully show that T captures task rules. The two tests of A use the same output head, so I am still unsure whether the results would be similar if using a different way to read the middle layers.
Follow-up response to Reviewer jDMZ: T and A
Thank you for your follow-up comments. They prompted us to refine our interpretation of T and conduct an additional experiment addressing A’s dependence on the intermediate readout.
- T: demonstration sensitivity and task rules.
The matched-count label/whitespace experiment shows greater sensitivity to the tested label changes across all six model–task groups. This provides targeted evidence of label-content sensitivity. Establishing sensitivity specifically to task rules requires further controls that separate rule changes from other consequences of modifying labels. Accordingly, we interpret T as demonstration-perturbation sensitivity and evaluate demonstration usefulness separately through task accuracy.
- A: dependence on the intermediate readout.
We compared native readouts with per-layer label probes, final-logit distilled heads, and residual distilled heads on Llama3-8B and Qwen3-4B across SST-2, RTE, and CB. Each group used 20 previously fixed evaluation queries, with separate 96-query training and 24-query calibration sets. Hidden states and the native final winner/runner-up pair were held fixed.
The residual head preserves the native final distribution by construction. Its mean per-query A-curve Spearman correlation with native A was 0.244–0.609 across the six clean-prompt groups. Calibrated peaks shifted 12.00 layers earlier on Llama SST-2 and 9.75 earlier on Qwen SST-2, while Llama RTE showed a small, uncertain shift of 0.55 layers later. Untempered logit peaks also shifted earlier across all six groups. Broad high-margin plateaus warrant interpreting exact peaks alongside full curves.
Holding native C/T fixed and replacing only A reduced strict-order counts in all six groups. Recomputing T through the same residual readout retained strict CTA order in 13–18/20 queries per group. Thus, frequent ordering can persist while A’s precise profile depends on the readout.
Following your comments and these results, we interpret A specifically as the native output margin for the retrospectively fixed candidate pair. Its peak identifies the strongest separation expressed by that readout. This targeted exploratory check reuses historical evaluation queries and one training/calibration split; its scope is the tested lightweight, task-restricted readouts.
Author response to Reviewer jDMZ
Thank you for the proposed controls. They materially sharpen the conclusion: in the tested settings, these observables need not peak at the same depth; peak ordering also reflects general depth structure and endpoint definitions. T measures response to demonstration perturbations, not demonstration usefulness or correctness. The revised paper makes that distinction explicit.
1. Endpoint construction and ordering. Strict CTA order alone is insufficient evidence for three sequential functional stages. C is zero at its final-layer reference, and classification A uses the retrospectively fixed final winner/runner-up. In targeted post-review checks with 20 fixed examples per model–task group, random labels retain frequent order: clean→random strict counts are Llama SST-2/RTE/CB 12→11, 13→15, 14→14 and Qwen 16→14, 15→15, 12→13. Without demonstrations, C precedes A in 18–20/20 examples per group; T is identically zero and its localization is undefined (Table A11). These results support a general-depth contribution to the pattern. Source interventions supply the additional, coordinate-specific evidence: BH-positive query-to-A 18/18, label-to-T 12/18, input-to-C 4/18 (Section 4.1).
2. Native-margin scale and readout scope. The completed probability/logit comparison checks scale under the same native head, retaining the retrospective candidate pair. Only Llama CB strict counts change, 14→13 for clean and random labels; the other group counts remain fixed. Positive affine scaling preserves individual maxima but can alter aggregate peaks through reweighting (A.3/A.7). Intermediate-layer readout validity and alternative candidate definitions require separate tests; an independently trained readout comparison remains future work. We label the completed test as a margin-scale check.
3. Label versus format response. The targeted contrast supports greater response to the tested label changes than to the tested whitespace changes; it does not establish demonstration usefulness or correctness. We fixed this targeted post-review exploratory comparison before examining its new outputs; it was not preregistered. It uses four changed tokens per condition with sequence length preserved. Across two models/three tasks, all 120 model–example records qualify. Mean JS differences range from 0.00281 to 0.04931; individual bootstrap intervals are positive, and 10,000-draw sign-flip tests give Holm p≤0.0048 under null sign exchangeability (Table A12). Position and embedding-change differences bound the interpretation. Correctness is assessed separately: cyclic label changes improve Llama RTE from 17/20 to 18/20 and Qwen CB from 15/20 to 16/20. Section 4.1/A.8 ties the conclusion to the specific label/whitespace contrast.
4. Restoration after depth adjustment. The analysis shows substantial shared rank-depth structure, with limited evidence for correspondence beyond the fitted trend. Excluding the final block changes correlations only modestly, from 0.797–0.938 to 0.818–0.935; regressing both ranked curves on ranked depth then reduces them to −0.150 to +0.279 (Table A13). This descriptive adjustment separates the observed across-depth association from the causal output effect of replacing states. Section 4.3 and the discussion now interpret the result at that scope.