- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper aims to expand the J-lens into multimodal models. They propose the MJ-lens, which aims to perform J-lens like early layer readout on activations on image and text latents for Qwen-VL, and image-text, text-image, etc. channels for Janus Pro.
Critical strengths
-
The idea of applying a J-lens like readout to models which output both text and images is interesting.
-
The authors explore methods to steer model behavior with their multimodal lens.
Critical weaknesses
-
The title asks "Does the Workspace See?", but the paper contains very little (weak and noisy) information about whether the "workspace" induced by J-lens is actually visual. Of course we can apply a visual unembedding module to the activation if one exists for the model. But the authors do not creatively extend these multimodal unembedding modules to explore whether the latent space is "thinking" visually.
-
The main finding in this paper goes against the original J-lens finding that steering to alter the J-lens readout changes the model behavior. The authors actually find that in Qwen-VL and Janus Pro, the two model families they study, this is not the case. But why would this be? Is there a control on LLM-only models which show that steering through the J-lens does change outputs in LLMs, so the finding is unique to multimodal models? Why is difference-of-means not necessary for Claude?
-
The writing of this manuscript is very obscure. For example, this sentence that closes the abstract: "The bound is on the instrument, not the representation: the lens can read the subspace, but as a readout operator it does not provide directions that control what the model draws.” --> it could have just been "We find our MJ-lens can perform readouts but does not extend to steering".
Minor Weaknesses:
- The idea of applying a logit-lens like readout on multimodal activations is not entirely new. It has been performed previously in [1], for example. The Jacobian Lens is just a small modification on the logit lens setup. I say this is minor because this manuscript also investigates Janus Pro, but I still find the contributions to be quite narrow.
References: [1] https://arxiv.org/pdf/2410.07149
Questions for authors
- If the MJ-lens does not enable steering - what might we use it for? Is it a reliable monitoring tool for misalignment? How does adding the extra modality expand what abilities we would have had with just the text monitoring? Why should we be interested in the MJ-lens?