- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper aims to build a mathematical explanation for the J-lens, and proposes a filtering step to increase its effectiveness.
Critical strengths
- Exploring the J-lens is a timely topic, and getting a foundational understanding of how the method works would be useful for the field.
Critical weaknesses
-
The notion that the J-lens is an expectation over future readouts is not particularly novel, since it is what the original J-lens paper intends to isolate [1]. Further, the central theorem assumes Gaussian inputs, but transformer activations are not Gaussian [2]! This in my view very much weakens the strength of the analysis in this work.
-
The paper does not explain how much additional time/FLOPs are required to implement their proposed method atop the J-lens, or justify how it would be actually desired in practice. How much compute is needed for the energy calculation and ranking them?
-
The paper does not check whether improvements from the proposed method consistent with different noise seeds, nor does it evaluate whether the improvements are statistically significant and meaningful.
Minor Weaknesses:
-
Claims will generalize poorly because all experiments have been conducted on small checkpoints on just 2 model families.
-
There are some potential mathematical issues with Eq. 18 and 20, please verify.
[1] https://arxiv.org/abs/2607.15495 [2] https://arxiv.org/abs/2402.17762
Questions for authors
-
There are no controls to show that ranking high energy (source, target) and pruning is what improved the ICR score. What would have happened if they were ranked randomly?
-
Further, how is the proposed method actually supposed to help down the line? Does pruning while preserving high energy source, target pairs result in J-lens readouts that reveal more ideal steering targets?