- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies the distinction between training exposure, behavioral source choice, and intervention-based control in retrieval-augmented language models. It introduces Latent Trajectory Shift (LTS), a layerwise projection of paired hidden-state differences onto a first principal component fitted exclusively on training data. Controlled conflicts between contextual and parametric answers provide the behavioral endpoint.
Verified-exposure experiments on Pythia and OLMo do not establish reliable exposure detection by the primary LTS feature. For source-choice prediction, state-difference magnitude is often stronger than signed PC1 projection. However, the reported activation interventions favor PC1 for changing source preference while preserving congruent and no-context behavior. The study includes matched-norm controls, candidate-gradient steering, dataset transfer, aligned cross-model transport, and a paragraph-level OLMo experiment.
Critical strengths
-
The paper makes a useful distinction between exposure detection, behavioral source choice, and controllability. It avoids interpreting answer agreement as evidence of causal source use and does not turn inconclusive membership results into an equivalence claim. The finding that magnitude can outperform PC1 for prediction while PC1 performs better under the tested intervention operator is an informative empirical question for representation analysis.
-
The intervention evaluation goes beyond demonstrating a change in a candidate score. It includes bidirectional updates, held-out direction fitting, norm-matched controls, discrete answer changes, non-target preservation, and transfer. The candidate-gradient comparison and paragraph-level experiment are particularly relevant to assessing whether the effect extends beyond a local short-answer margin change. These components provide a stronger basis for evaluating a controller than predictive probe accuracy alone.
Critical weaknesses
-
Several central results cannot currently be reconciled from the reported endpoint definitions. For example, Table 12 gives a Llama middle-block, toward-parametric PC1 effect of 0.314 at unit dose. In Table 15, the corresponding candidate-gradient effect is 0.1407 and the paired LTS-minus-candidate effect is 0.2536, implying an LTS effect of 0.3943. Both sections describe the same 120-item within-dataset setting, but any difference in dose, score normalization, policy aggregation, or cohort is not clearly specified. Table 20 also reports a contextual-direction “target success” of 0.243 over 779 Llama items, although only 43 baseline answers are parametric. This cannot be interpreted as a full-cohort parametric-to-contextual flip rate; a different endpoint would need an explicit definition. These ambiguities affect the central control evidence rather than peripheral reporting.
-
The controls do not yet establish why PC1 is the preferred learned direction. Random, orthogonal, shuffled-layer, and radial updates are useful controls, and the candidate-gradient baseline is valuable, but the paper does not clearly report the most direct alternative: the normalized mean paired displacement, which is already computed as g_l in Equation (6). A mean-difference steering baseline, following the principle of Contrastive Activation Addition (Rimsky et al., ACL 2024), could be evaluated under exactly the same additive operator, position, layer blocks, and norm budget. A source-choice probe direction would also connect the prediction-versus-control comparison more directly. Without such comparisons, the evidence supports superiority over the tested controls but leaves the specific value of the PC1 construction insufficiently established.
Questions for authors
-
Please explain the relationship between Tables 12 and 15. For Llama's middle-block toward-parametric condition, why does the candidate effect plus the paired LTS advantage equal 0.3943 rather than the 0.314 reported for PC1? Please identify the exact dose, candidate-score definition, cohort, and aggregation rule for each table.
-
What precisely is “target success” in Table 20? If it is a flip rate or an increase in target-answer frequency, how is the contextual-direction value of 0.243 compatible with only 43 initially parametric answers among 779 items? If it instead uses margin-based eligibility, a soft score, or a conditional denominator, please provide its formula and distinguish it from generated-answer flips. A clean-versus-steered transition table would resolve this directly.
-
How does PC1 compare with the normalized mean congruent-minus-conflict displacement under the identical operator and preservation criteria? Please also give the exact mathematical definitions of the signed and unsigned radial updates, including their reference vector and the information available at intervention time. These details are necessary to interpret the claimed separation between a magnitude diagnostic and a directional controller.
-
How were the 500 members and 500 nonmembers established for OLMo-2-1124-7B-Instruct? Please specify the training-data provenance, deduplication and contamination checks, coverage of relevant training stages, and how these examples were converted into the shared conflict-choice cohort. The Pythia checkpoint construction is substantially clearer than the corresponding OLMo verification.
2 comments
Dear Reviewer aneo,
Thank you again for your time and constructive feedback.
With 1 day remaining in the author–reviewer discussion period, we would greatly appreciate it if you could take a moment to review our rebuttal and revised manuscript. We have addressed your concerns through additional analyses, experiments, and clarifications.
Thank you for your time and consideration.
Best regards,
The Authors
Dear Reviewer aneo,
Thank you again for your time and constructive feedback.
With 1 day remaining in the author–reviewer discussion period, we would greatly appreciate it if you could take a moment to review our rebuttal and revised manuscript. We have addressed your concerns through additional analyses, experiments, and clarifications.
Thank you for your time and consideration.
Best regards,
The Authors