- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper extends spherical activation steering with truthful and hallucinatory prototypes obtained from an entropy-regularized optimal-transport coupling. Task-specific gating controls the intervention, which is evaluated on two language models using TruthfulQA and five additional multiple-choice benchmarks.
Critical strengths
The improvements over the task-tuned single-prototype control are worth taking seriously. Compared with Variant 3, the full method raises LLaMA's generation from 60.28 to 73.63 percent and Qwen's multiple-choice average from 56.50 to 60.62 percent. The gain on LLaMA multiple-choice is smaller, from 61.42 to 62.74 percent (Tables 2 and 6). Tables 3 and 7 also show improvements over spherical steering across the six benchmarks.
The method is inexpensive to use: two stored directions, less than 0.14 MB of prototype storage, and no backbone updates. The rotation preserves activation norms. Table 4 puts LLaMA generation latency at about 1.29 seconds per prompt for both methods, with little added cost over spherical steering.
Critical weaknesses
-
Theorem 2 is too strong as stated. Take unit vectors on a circle at positive-class angles and negative-class angles . With the intended pairwise geodesic cost, uniform marginals, and , applying Equations (5) to (8) reduces prototype separation from about to . This small construction should be straightforward to check directly. The proof in Appendix C.4 assumes centroid-norm growth that the weighting does not guarantee. A conditional theorem would need explicit, justified assumptions; otherwise the separation claim should be treated as an empirical hypothesis.
-
I cannot reconcile one ablation with the stated gate. Table 5 gives Qwen Variant 6 generation and . Algorithm 2 uses when , and zero otherwise, so this setting leaves activations unchanged. Yet Table 6 gives of 77.36 percent, compared with 74.40 percent for the unmodified model. Appendix A also leaves the common tuning grid and selection objective unspecified.
Questions for authors
Is Theorem 2 meant to assume additional conditions on the activation distributions? If so, what rules out the example above, and what evidence supports those conditions in the studied models?