- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
To raise the expected value of some token-level score by a target amount, the authors ask what the smallest possible change to the model's next-token distribution is. The authors show that the least-distorting target distribution is a simple re-weighting of the original one toward high-scoring tokens, which gives a benchmark that doesn't depend on any particular steering method. Their main theorem shows when an intervention just before the model's final normalization and output layer can reach that target exactly, based on three necessary and sufficient conditions. When any condition fails, they prove a lower bound on the extra distortion. Empirically, they find that four simple lexical scores in three small language models all fall largely outside the expressible space, so exact attainment is impossible. Even artificially constructed scores that pass this test are exactly reachable only for very small score increases. A toy model and checks at earlier layers support the theory.
Critical strengths
- A clean, method-independent definition of "selective" steering. The minimum-KL target is the exponential tilt (Eq. 2). Identity (3), , splits any intervention's excess KL into distortion and overshoot. This gives the steering literature a precise, intervention-agnostic yardstick for side effects, which I found very informative.
- A rigorous and fairly complete attainability theory for the final normalized readout. Theorem 1 gives necessary and sufficient conditions for with a quantitative excess-KL lower bound. Theorem 2 gives the minimum intervention norm for every readout rank, and Example 10 shows that attainability can be non-monotone. Each assumption is shown to be needed by a counterexample (Appendix: D2, D4, D6), and the results I checked hold. The theory appears correct and carefully hedged.
Critical weaknesses
- The practical size of the benchmark gap is never measured. The paper's motivation is evaluating steering, yet no steering method (e.g. ActAdd/CAA) is compared against , no value of is computed for a pretrained model, and the constant is never evaluated. The authors themselves note that this constant "may be small for a large vocabulary" (p.5), and since with -, the bound is likely negligible. Without such a measurement, the claim to distinguish reaching a target score from reaching it with minimal distributional change has no practical size. Solving at the final layer () and scoring one standard steering vector would address this.
- The main empirical finding follows from dimension counting. The centered readout has rank , so almost any score direction is out of range. A random direction would give a relative residual -, which is higher than the reported 0.888-0.980 (Table 1). App. G.1 acknowledges that the dimension argument "does not bound the residual", yet the abstract presents "large range mismatches in all twelve model–score pairs" as a finding. There is no null baseline, and the softmax-bottleneck literature (Yang et al., 2018; Demeter et al., 2020; Grivas et al., 2022) is not cited. Reporting random-direction residuals next to Table 1 and positioning the result relative to that work would fix this.
Questions for authors
- What residuals do random isotropic directions in , or random token sets of the same sizes, give for the readouts in Table 1? Is anything about the lexical scores specific beyond ?
- For the Table 1 scores at a representative and , what is the numerical value of the bound in Eq. (16), and what is the actual obtained by optimizing over the final-layer ball?
- In Sec. 5.3 the in-range direction satisfies and . Is the "narrow range" in the abstract set by this construction and by RMSNorm's near-unit-sphere regime rather than by the scores? How do the endpoints change for directions not orthogonal to ?
- Can you share results for scoring standard mid-layer steering methods (ActAdd/CAA, dual steering) by their excess KL over at matched score gains? Perhaps also extending the benchmark to multi-token properties (Appendix G4)