- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
WAQS derives activation-dependent steering from the gradient of a quadratic concept probe and absorbs the resulting affine intervention into compatible model projections. The paper characterizes quadratic scores as the smooth functions whose fixed Euclidean-gradient step is affine, provides a Gaussian discriminant interpretation, and introduces low-rank probe estimation. Experiments evaluate truthfulness, refusal, sycophancy, and corrigibility, with ablations of state dependence and measurements of inference throughput.
Critical strengths
-
A useful connection between probe expressiveness and deployment. A quadratic score induces an affine gradient field, allowing input-dependent interventions to be compiled at compatible affine layers. Proposition 1 correctly characterizes this scalar-gradient construction, and the Gaussian interpretation clarifies when covariance differences introduce state dependence. The low-rank parameterization makes probe estimation and weight folding practical.
-
Targeted evidence for adaptivity and efficiency. Frozen-QP retains the fitted quadratic probe and replaces its gradient field with its training-set average, providing a meaningful control for state dependence. Table 4 reports higher True*Info and MC1 for dynamic QP. Table 6 reports essentially unchanged prefill and decoding throughput after absorption. Appendix C.1 also specifies held-out validation selection and once-only test evaluation.
Critical weaknesses
-
The demonstrated compilation scope does not establish the general deployment claim. Equations (7)–(8) require both weight and bias updates; a weight-only edit of a bias-free projection cannot generally realize the nonzero offset. The RMSNorm derivation in Section 3.1(c) likewise retains an additive after updating the scale. More substantially, Section 5 acknowledges that residual-stream steering generally requires an explicit runtime transformation because it changes the weightless skip path, whereas Appendix C.1 and Table 9 claim absorption across residual-stream locations. Task performance at those locations does not establish equivalence between an intervention hook and a merged checkpoint. The paper needs to identify the compiled operators and reconcile these statements before its central zero-additional-operations claim can be assessed.
-
The evidence supports useful trade-offs, but not the claimed general superiority without utility loss. Table 2 reports GCG attack success rates of 0 for Spherical Steering and 9 for WAQS, and GSM8K accuracy of 22.59 for DiffMeans versus 22.36 for WAQS. These results contradict the accompanying claims of lowest attack success across all attacks and matching or exceeding both constant baselines on every utility benchmark. Table 1 also reports Llama-2 cross-entropy of 2.41 for WAQS versus 2.13 for Linear Probe. Without unsteered utility results in these tables and uncertainty for the refusal comparisons, the extent of capability preservation and the significance of small gains remain unclear. This limits the evidence for replacing linear steering as the default baseline.
Questions for authors
-
Which exact weights, biases, and runtime operators are used for the main task results and Table 6? Please provide hook-versus-compiled activation or logit discrepancies for prompt processing and cached decoding, explain the treatment of additive offsets, and identify residual-site results that require hooks. Establishing equivalent merged implementations for the central results, with a consistent architectural scope, would resolve the principal deployment concern.
-
Can you provide unsteered utility measurements and uncertainty or paired comparisons for the refusal results, and revise the claims to reflect the reported trade-offs? Evidence that the compiled intervention delivers a meaningful steering benefit at comparable capability preservation would support raising the rating; correcting the prose alone would not establish that benefit.