acceptodds
Under review as a conference paper at ICLR 2027

Verbalizing Multi-Token Concepts in LLMs

10 views

Abstract

Lens methods inspect model computation by reading hidden states as ranked vocabulary tokens. Yet the concepts humans need to read out often span multiple tokens—entities, phrases, intermediate objects—making token-level readouts incomplete. Reliable multi-token readout with little model-specific preparation remains challenging. We introduce Concept Lens: token-level lens clues guide candidate search, while the model recovers a representation for each candidate and scores it against the source activation in the same readout space. Across 2,400 multi-hop clozes on five LLMs (8B–70B), Concept Lens instantiated with J-lens and R-lens achieves average Rank@10 scores of 36.6% and 54.5%, respectively, compared with 21.7% for Template Lens. Controlled tests show that recovered representations preserve concept identity and that concept rankings track changes in the source activation. Concept-swap interventions further shift model answers toward those associated with the replacement concepts. Our results move lens-based interpretability beyond token-level inspection toward the concept-level structure of model computation. Our code is available at https://anonymous.4open.science/r/ConceptLens.

Primary area:
interpretability and explainable AI
Keywords:
Large Language Models, Mechanistic Interpretability, Representation Analysis, Jacobian Lens, Multi-Token Representation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_pUpROpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper extends logit lens readouts from single tokens to multi-token concepts. Concept Lens is training-free: PROPOSE generates candidates from the source prompt, and SCORE ranks them via ridge-projected concept vectors, independently of generation probability. The method is evaluated on a newly constructed multi-hop cloze benchmark across several models, with controls that fix candidates while swapping source states to verify activation-dependent ranking. Calibration costs a fraction of what library-based alternatives require.

Critical strengths
  1. The method extends readouts from individual tokens to complete concepts. Complete entities and phrases make intermediate representations easier to interpret than isolated token fragments.

  2. It supports concept readout and intervention with little preparation. The method produces both concept labels and representations that can modify model activations, without training an auxiliary model.

Critical weaknesses
  1. The main comparisons do not clearly establish where the method’s advantage comes from. The main baseline in Table 2 is Template Lens, whose fixed concept library is missing many target concepts. Better results may therefore come from finding more candidates rather than scoring them more accurately. Even after adding the correct target, the remaining candidates differ between methods. The main text also lacks direct comparisons with methods that generate multi-token descriptions, leaving the claimed faithfulness advantage over generative methods unsupported by direct evidence.

  2. The paper’s organization makes the method and evaluation difficult to follow. Figure 1 repeats the Read and J-lens lists but leaves out how candidate vectors are constructed and compared with the source activation. Readers must piece together these steps from the equations. The experimental setup is also scattered: models are introduced in Section 4, while the dataset and evaluation protocol appear in Section 5.1, making the overall setup harder to grasp. Finally, several valuable control experiments in the appendix directly support the paper’s reliability claims and should be included in the main text.

Questions for authors

1.With the same source activations and output budget, is Concept Lens more accurate than existing lightweight generative methods, and does it better reflect changes in the activations?

2.In Table 2’s oracle setting, does Concept Lens retain its advantage over Template Lens when both use identical candidate sets and the same vocabulary competitors?

7 comments
AuthorsOpenReview ↗

(1/2) [Q2, W1, W2] Identical candidates and competitors, and the reorganization

TL;DR:

Thank you for this precise question; it isolates where our advantage comes from. With identical candidates (one shared 39,930-concept bank containing every target) and the same vocabulary competitors, Concept Lens beats Template Lens by 4.6–14.3 points on four models and trails it by 6.4 points on Llama-70B. The advantage therefore does not come only from finding more candidates: on identical candidates, our representations rank the target higher on most models. We also agree with your points on organization and will restructure the paper as described in Section 3.


1. Identical candidates and competitors (Q2)

Both methods rank the same 39,930-concept bank together with the full vocabulary, so every target is available on all 2,400 clozes. Concept Lens does not use its own proposals here, and the scoring rule is shared. So only the concept representations differ: Template Lens uses its template representations, and Concept Lens uses its carrier representations. This setting differs from the oracle-candidate setting in Table 2 of the paper, where each method ranked its own candidates plus the target (Template Lens: its 38,400-concept bank; ours: at most 25 proposals per layer), so the numbers are not comparable to that table.

Table 1. Rank@10 (%) with identical candidates and vocabulary competitors. Best rank across the workspace band.

MethodLlama-8BGemma-12BQwen3-14BQwen3.6-27BLlama-70B
Template Lens62.163.972.177.480.3
Concept Lens (J-lens)76.472.877.182.073.9

This test answers your question by isolating the representations, and it helps trace where the advantage comes from. As the main comparison, however, it would be unfair to Concept Lens: it imposes a fixed concept bank, the very limitation of Template Lens, on a method designed to need none. In normal use, Concept Lens proposes its own prompt-specific candidates, so the end-to-end comparison in Table 2 of the paper reflects how the two methods are actually used. Even on Template Lens's terms, Concept Lens leads on four of five models.


2. Where the advantage comes from (W1)

Both stages contribute. On identical candidates, SCORE ranks the target higher on four of five models (Table 1). In ordinary use, PROPOSE also supplies targets that a fixed library lacks: the Template Lens library contains 25.7% of our targets, while PROPOSE covers 30.1–75.5% with at most 25 prompt-specific candidates per layer and no library (Table 2 of the paper). The direct comparison with generative methods is in comment 2/2.


3. Organization (W2)

We agree and will revise as follows:

  • Figure 1 will show the full pipeline: the proposed candidates, their carrier representations, and the comparison with the source activation that produces the joint ranking.
  • One setup section before all experiments will introduce the models, benchmark, metrics and workspace bands together.
  • Reliability controls in the main text: the fixed-candidate source swap (App. D.3), the same-first-token test (App. D.4), fixed-bank scoring against generation probability (App. D.2), and the comparisons with Patchscopes and SelfIE in comment 2/2.
AuthorsOpenReview ↗

(2/2) [Q1, W1] Generative methods under the same activation and budget: more accurate and activation-faithful

TL;DR:

Thank you for asking for a direct comparison with generative methods; we agree it belongs in the main text. At the same activations and with a matched output budget, Concept Lens is more accurate than Patchscopes and SelfIE on our cleaned benchmark (49.1% Rank@10 with R-lens vs at most 15.8%) and on two of three public benchmarks; SelfIE leads on Hopping Too Late. When the activation changes, our ranking follows it, while Patchscopes often returns the same text for different activations and both generative methods rarely name the injected concept.


1. Accuracy

We compare Concept Lens with Patchscopes (Ghandeharioun et al., 2024) and SelfIE (Chen et al., 2024) at the same activations, following their official implementations and released prompts. Each baseline gets greedy decoding plus 10 samples per layer, one output more than our top 10, and any whole-word mention of the target counts as a hit, which is more lenient than the exact match we require of our own method.

Table 2. Rank@10 (%) on our cleaned benchmark and three public benchmarks. Our target must rank in the top 10 at any layer of the workspace band; for Patchscopes and SelfIE, a whole-word mention in any output at any layer counts as a hit. Averaged over five models on our cleaned benchmark (the 1,031 clozes left after removing those whose answer is a prefix of the concept, following Reviewer yzCg) and over three models on the public benchmarks (1,546 queries, each passing separate bridge-answer and final-answer checks on all three models).

MethodOurs (cleaned)TwoHopFactHopping Too LateMQuAKE
Template Lens26.89.07.319.6
Patchscopes14.25.815.08.2
SelfIE15.86.930.127.4
Concept Lens (J-lens)31.224.124.639.7
Concept Lens (R-lens)49.129.329.246.5

2. Reflecting activation changes

A reader that reads the activation must change its output when the activation changes, and change it to the right concept. We test this in two ways.

(a) Swap in the partner's activation. We use the counterfactual pairs of App. D.3: two prompts share a relation but require different concepts. With the partner's activation, the readout should move to the partner's concept.

  • Concept Lens (candidates and representations fixed): the injected concept ranks in the joint top-10, vocabulary included, in 71.1% of cases (Llama-8B, L27, 204 pairs), 87.0% (Gemma-12B, L43, 131 pairs) and 64.9% (Qwen3-14B, L32, 124 pairs), while the source concept drops to 0.7%, 3.8% and 10.1%.
  • Patchscopes and SelfIE, given the same activations at the same layer, name the injected concept in at most 3.8% of greedy outputs (9.3% with samples). Patchscopes returns identical greedy text for the two different activations of a pair in 36.3–41.1% of pairs.

Table 3. Generative readouts after swapping in the partner activation. Percentages for greedy decoding; parentheses: greedy or any of the 10 samples, omitted when equal. Pairs follow: both sides name the concept of their injected activation. Identical greedy text: both sides produce the same greedy string.

ModelMethodNames injected conceptNames the other conceptPairs followIdentical greedy text
Llama-8BPatchscopes2.7 (7.1)0.00.5 (2.0)36.3
Llama-8BSelfIE2.0 (3.7)0.00.0 (0.5)0.5
Gemma-12BPatchscopes3.8 (8.4)0.0 (0.4)1.5 (3.8)40.5
Gemma-12BSelfIE0.00.0 (0.4)0.08.4
Qwen3-14BPatchscopes2.0 (4.8)0.00.0 (0.8)41.1
Qwen3-14BSelfIE2.8 (9.3)0.00.0 (1.6)4.0

(b) Feed an activation that holds no specific concept. We replace each activation with the mean over all prompts of these pairs at the same layer; the readout should at least change. Patchscopes returns exactly the same greedy text on 30.4–47.2% of prompts across five models: its output often comes from its prompt, not from the activation. SelfIE's text usually changes, but as Table 3 shows, it rarely names the right concept. The top-10 of Concept Lens changes on every prompt; a ranked list changes more easily than a single string, so this test mainly exposes Patchscopes.

AuthorsOpenReview ↗

Follow-up: Revised Manuscript and Method Clarifications

Dear Reviewer pUpR,

We would like to let you know that we have uploaded a substantially revised version of the manuscript, incorporating the changes discussed in our rebuttal.

We have also posted an Official Comment summarizing the major revisions, including the clearer method presentation and additional reliability experiments addressing your concerns.

We would greatly appreciate your feedback on the revised manuscript and would be happy to address any remaining questions.

Best regards, The Authors

Reviewer_Gf9DOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes a lens method for multi-token concepts. The method first generates candidate "complete concepts" that the model may be internally representing. It uses single token lens methods' output as "first token clue" and complete the concept by generating the rest. Then it scores the concepts by comparing the model’s internal representation of each candidate with the model's original hidden state. The method is evaluated on synthetic benchmark, which is created by the authors, and compared with an existing baseline method.

Critical strengths
  • Motivation and flexibility. The idea of extending lens methods from individual tokens to multi-token concepts is well-motivated. Moreover, unlike Template Lens, the method does not require a predefined set of candidate concepts.
  • Interesting applications. The examples in Section 6 show that the method can reveal differences in the model’s internal interpretation even when the final outputs are the same.
Critical weaknesses
  • Clarity of presentation. The paper, especially the method part, is hard to follow. Key terms such as “complete concept” and “first-token clue” are not clearly defined, and some definitions appear only in the appendix. It is also unclear what exactly the input and output of the method are, which makes the main contribution hard to grasp. A simple step-by-step illustration of the method with a concrete example would greatly improve readability.

  • Evaluation. The purpose of the analyses in Section 4 was not entirely clear to me. It would be helpful if the authors could clarify what each analysis is intended to validate and how it supports the effectiveness of the proposed method. In addition, the main experiments in Section 5 are conducted on a synthetic benchmark constructed by the authors. Could the authors comment on whether the method has been, or could be, evaluated on existing benchmarks? Such results would help assess how well the findings generalize beyond the authors’ own dataset.

Questions for authors

The application results in Section 6 are interesting. Could the authors clarify whether this type of analysis is uniquely enabled by Concept Lens, or whether similar insights could also be obtained with existing methods (e.g., token-level lenses, Template Lens, or Patchscopes)? A comparison on the same examples would help highlight the advantage of the proposed method. Moreover, in the Figure 4, what are the orange and grey colors refering to?

6 comments
AuthorsOpenReview ↗

(1/3) [W1] What Concept Lens takes, returns and does, with a worked example

TL;DR:

Thank you for pointing out that the method section was hard to follow and for suggesting a step-by-step example; we agree on both. Concept Lens takes one hidden state and the prompt that produced it, and returns a single ranked list in which complete concepts and vocabulary tokens compete. Below we define the two terms you flagged, state the input and output, and walk through Figure 1 step by step. In the revision, the definitions and this walk-through, as a step-by-step figure, will open Section 3.


1. Definitions

  • Complete concept: a concept written as two or more tokens and read as one unit, e.g., black hole. Its individual tokens (black, hole) are not complete concepts.
  • First-token clue: a token in the lens ranking of the hidden state that may begin a complete concept, e.g., black at rank 8 in Figure 1. It tells us where to start searching, not which concept it begins.

2. Input and output

  • Input: h, one hidden state at a chosen layer and token position, and the source prompt that produced it.
  • Output: one ranked list of complete concepts and vocabulary tokens, scored on the same scale, plus a representation for each concept that can be used in interventions.

3. Worked example

Figure 1 uses Llama-3.1-8B and the prompt:

Fact: The boundary around an object that has gravity strong enough to trap light is the ___

The answer is event horizon; the intermediate concept is black hole, which does not appear in the prompt. We inspect h at layer 24, at the last prompt token, before the model answers.

  1. Read tokens. J-lens ranks vocabulary tokens for h. black is 8th: it may begin black hole, but on its own it does not identify the concept.

  2. Propose complete concepts (PROPOSE). For each of several top-ranked tokens, we give the model the prompt and that token and let it complete a concept. From black, the model proposes black hole; other tokens lead to event horizon, Schwarzschild radius and others. These are only candidates; none has been checked against h yet.

  3. Build a representation for each candidate. We run the model on a fixed prompt: “Remember the following concept in your mind. Concept: black hole. Fact: The capital of France is Paris.” We take hidden states at fixed positions in the factual sentence, subtract their background component and average them into one representation. Every candidate uses the same prompt with only the concept changed. If h comes from another layer, we map the representation to that layer.

  4. Score and rank (SCORE). SCORE compares each candidate's representation with h through the lens, and scores every vocabulary token on the same scale, so concepts and tokens share one ranked list. The score depends only on the representation and h, not on how likely the candidate was to be generated. black hole ranks first, ahead of event horizon and every vocabulary token, whereas J-lens alone shows only the fragment black, at rank 8.

AuthorsOpenReview ↗

(2/3) [W2] Results on three public benchmarks, and what Section 4 validates

TL;DR:

Thank you for suggesting evaluation on existing benchmarks; we agree it was needed. We evaluated Concept Lens on three public multi-hop benchmarks (1,546 queries, three models). It leads on every model on TwoHopFact and MQuAKE; on Hopping Too Late, SelfIE leads on average (30.1% vs 29.2%). Section 4 checks the two stages of the method separately, where ground truth is clean: whether a concept's first token is readable and leads to the full concept (PROPOSE), and whether our representation captures the whole concept (SCORE).


1. Three public multi-hop benchmarks

We evaluate TwoHopFact (Yang et al., 2024), Hopping Too Late (Biran et al., 2024) and MQuAKE (Zhong et al., 2023): 534, 465 and 547 queries. A query is kept only when all three models answer separate prompts for the bridge entity and the final answer correctly; the answer to the first-hop prompt is the bridge target. We also include Patchscopes and SelfIE, as requested by Reviewer yzCg, following their official implementations and released prompts. All methods read the same activations at the same layers. Each baseline gets greedy decoding plus 10 samples per layer, one output more than our top 10, and any whole-word mention of the bridge entity counts as a hit, which is more lenient than the exact match we require of our own method.

Table 1. Rank@10 (%) on three public benchmarks. For our method, the bridge entity ranks in the top 10 at any layer of the workspace band; for Patchscopes and SelfIE, a whole-word mention in any output at any layer counts as a hit.

MethodLlama-8BGemma-12BQwen3-14BAvg
TwoHopFact (n = 534)
Template Lens10.59.76.99.0
Patchscopes8.27.71.55.8
SelfIE8.28.44.16.9
Concept Lens (J-lens)27.030.714.624.1
Concept Lens (R-lens)34.630.023.429.3
Hopping Too Late (n = 465)
Template Lens6.09.56.57.3
Patchscopes11.424.19.515.0
SelfIE15.349.026.030.1
Concept Lens (J-lens)16.141.915.724.6
Concept Lens (R-lens)20.240.926.729.2
MQuAKE (n = 547)
Template Lens14.324.519.919.6
Patchscopes10.17.37.18.2
SelfIE17.649.715.027.4
Concept Lens (J-lens)22.764.432.039.7
Concept Lens (R-lens)27.267.345.046.5

Concept Lens leads on every model on TwoHopFact and MQuAKE. On Hopping Too Late, it leads on Llama-8B and Qwen3-14B, while SelfIE leads on Gemma-12B and on average. Template Lens stays at 7.3–19.6% on average.


2. Why we also built our own benchmark

  • Our controls and causal swaps need counterfactual pairs: two prompts that share a relation but require different intermediate concepts. Public benchmarks are not organized into such pairs.

  • Their bridge entities are also mostly people and places, while our benchmark also covers abstract concepts, processes, scientific terms and everyday objects (App. A.2).

Following Reviewer yzCg, we also removed the 1,369 clozes whose answer is a prefix of the concept and re-ran the main comparisons on the remaining 1,031 (our response to yzCg, comment 1/3). The revision will report the public benchmarks alongside our own.


3. What Section 4 validates

Concept Lens can work only if two premises hold. Section 4 tests each where the ground truth is unambiguous; Section 5 then tests the full pipeline.

  • Section 4.1 tests PROPOSE. PROPOSE starts from a first token read from the activation and asks the model to complete it. This requires that (i) the first token of a multi-token concept is readable from the activation, and (ii) given that token and the prompt, the model completes the right concept. Figure 3 shows that first tokens remain readable in the lens top-10, though less often as concepts get longer, and that the model completes the full concept in 89.9% of cases.
  • Section 4.2 tests SCORE. SCORE needs a representation of the whole concept, not of its fragments. We split held-out single-token words into two tokens (airport → air + port), so the word's own lens direction is a known ground truth. Our representation retrieves the correct word in 98.0–99.8% of cases, versus at most 66.8% for the first fragment or the fragment mean (Table 1 of the paper).

In the revision, each analysis will begin by stating which stage it validates.

AuthorsOpenReview ↗

(3/3) [Q1, Q2] Section 6 with existing methods on the same examples, and the Figure 4 colors

TL;DR:

Thank you for suggesting a comparison on the same examples; it shows the advantage more clearly than our original figure. We ran the token-level J-lens, Template Lens, Patchscopes and SelfIE on the six Figure 4 prompts at the same activations. On these examples, only Concept Lens captures all three distinctions. The token lens captures two of them as single tokens, but on the pragmatic pair it shows only punctuation; Template Lens, Patchscopes and SelfIE capture none. Orange and grey mark the two prompts of each pair; we will fix the counting panel and add a legend.


1. Existing methods on the Section 6 examples (Q1)

Each pair gives the same or a similar final response, while the prompts differ in interpretation or problem structure. All methods read the same activation at the same layer and position: Llama-8B at layer 17 for the contradiction and pragmatic pairs, and Qwen3-14B at layer 32 for the counting pair. Patchscopes and SelfIE use their few-shot and interpretation prompts.

Table 2. Readouts on the Section 6 examples. Token lens: top J-lens token. Template Lens: top concept from the 39,930-concept bank (its 38,400 concepts plus our benchmark targets). Patchscopes and SelfIE: greedy description; "..." marks truncation. Concept Lens: the indicated concept and its rank in the joint concept–vocabulary list; for sincere praise, the rank is that of humour or irony, not of the top concept.

PromptToken lensTemplate LensPatchscopesSelfIEConcept Lens
Contradictory premisescontradictoryregular prism"Leonardo DiCaprio is not a real person""the importance of being ..."contradiction between the statements (#1)
Consistent premisesakestregular prism"Leonardo DiCaprio is an American actor, and Samsung""the new academic year"youngest person (#1)
Unordered counting_choosecleave"a real-time strategy game""a summary of the ..."combinations formula (#1)
Ordered counting_permsHooray Henry"I need to figure ...""a summary of the ..."permutations without repetition (#1)
Sarcastic praise.\nmagic bullet"star is used to ...""expressing approval or agreement"humour or irony (#3)
Sincere praise?\nstool test"the game is not ...""a phrase of encouragement"humour or irony (#4,930)
  • Concept Lens separates the contradiction and counting pairs with a complete concept ranked first; on the pragmatic pair, humour or irony ranks 3rd for sarcastic praise versus 4,930th for sincere praise.
  • Token lens separates the contradiction and counting pairs with single tokens, but both prompts of the pragmatic pair yield only punctuation, so the pragmatic distinction is invisible.
  • Template Lens returns the same concept for both prompts of the contradiction pair and unrelated concepts elsewhere.
  • Patchscopes often continues the examples in its few-shot prompt (e.g., the Leonardo DiCaprio outputs) instead of describing the activation. We follow the official implementation and prompt; this is a known failure mode, in which the prompt's language prior overrides the injected activation (Gong et al., 2026).
  • SelfIE does not name any of the distinctions: it reads sarcastic praise as approval, although some sampled completions call it sarcasm.

2. Colors in Figure 4 (Q2)

Thank you for pointing this out. Orange marks the first prompt of each pair (contradictory premises, sarcastic praise) and grey its matched control (consistent premises, sincere praise). In the counting panel, both prompts were orange because neither is a control, which broke this convention. In the revision, ordered counting will be grey, and the figure will include a legend.

Reviewer_yzCgOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a method named Concept Lens to extend existing token-level lens method (J-lens and R-lens) into concept level. Concept lens is able to identify the hidden concept in the activations of the early layers of LLM and decode that into a multi-token concept. However, the designed benchmark is overwhelmed by low-quality simple questions. The reported results are hard to support the claim and contribution of the paper.

Critical strengths
  • The proposed Concept Lens method extends J-Lens to multi-vocabulary, which is timely and significant.
  • Multiple LLM backbones are evaluated.
Critical weaknesses
  • The most significant flaw is the design of the evaluation set. In the source data provided by the authors (data/clozes.json), most of the answers are simply prefixes of the concept name (57% of all samples). For example:
  {
    "id": 77,
    "prompt1": "Fact: The first word in the conventional English name of the formation growth and maturation of the nervous system is",
    "concept1": "neural development",
    "answer1": "neural",
    "prompt2": "Fact: The first word in the conventional English name of the farming designed to remain productive while limiting environmental and social harm is",
    "concept2": "sustainable agriculture",
    "answer2": "sustainable"
  },

Such an evaluation protocol has a huge discrepancy to the original design in J-lens, making the claims in the paper less convincing.

  • In the proposed solution, J-lens/R-lens has done the heavy-lifting job, especially under the flawed evaluation, making the scientific contribution limited. Meanwhile, although Patchscopes and SelfIE are mentioned in the introduction as not reliable, none of them are evaluated in terms of the multi-token readout with lightweight adaption.
Questions for authors
  • Could the authors report revised results on a cleaned evaluation set that excludes the trivial items, i.e., the directed clozes whose answer is a prefix of the intermediate concept name? Please revise Table 2 and 3 on the remaining 1,031 items, ideally also restricted to items the model answers correctly.
  • Q2. On the same cleaned set, could the authors also evaluate Patchscopes and SelfIE?
8 comments
AuthorsOpenReview ↗

(1/3) [W1, Q1, Q2] Prefix items removed: readout on the cleaned set, with Patchscopes and SelfIE

TL;DR:

Thank you for inspecting the data and catching this flaw; you are right. In 1,369 of 2,400 clozes (57%), the answer is a prefix of the intermediate concept's name. We removed them all and will replace Tables 2 and 3 with the cleaned results. On the remaining 1,031 clozes, Concept Lens (R-lens) reaches 49.1% Rank@10, versus 26.8% for Template Lens and at most 15.8% for Patchscopes and SelfIE. On the 988 swap trials with both sides clean, Concept Lens representations swap the answer in about 28% of trials, versus 7.2% for Template Lens. On 1,546 queries from three public benchmarks built by others, Concept Lens (R-lens) also leads on average (both in comment 2/3).


0. What we removed

  • 452 ask for the concept's first word (neural for neural development).
  • 811 ask for its first morpheme or first five letters (life for lifelessness).
  • 106 are one-hop items whose annotated intermediate is the answer itself.

The first two groups came from carrying J-lens's first-letter relation ("The first letter of the month containing Halloween": October → O) over to multi-token concepts, where the answer becomes the concept's own prefix; we missed this. The third group consists of annotation errors.


1. Readout on the cleaned set (Q1, Q2)

Patchscopes and SelfIE read the same activation as our method (last prompt token, every layer of the workspace band), following their official implementations and released prompts. Each baseline gets greedy decoding plus 10 samples per layer, one output more than our top 10, and any whole-word mention of the concept counts as a hit, which is more lenient than the exact match we require of our own method. Without the prefix items, the lead of Concept Lens (J-lens) over Template Lens narrows to 4.4 points on average; it trails Template Lens on Qwen3-14B and ties on Gemma-12B, while Concept Lens (R-lens) stays ahead of every baseline on every model. With the same PROPOSE and SCORE procedure, switching the underlying lens from J-lens to R-lens adds 17.9 points: the lens is the main limit of Concept Lens. This also shows its flexibility: Concept Lens is a natural extension of any token-level lens, so it improves as lenses improve.

Table 1. On the cleaned set, Concept Lens (R-lens) reads out the concept more often than every baseline on every model. Rank@10 (%): for our method, the concept ranks in the top 10 at any layer of the workspace band; for Patchscopes and SelfIE, a whole-word mention in any output at any layer counts as a hit. The 532-item subset contains clozes answered correctly by all three of Llama-8B, Gemma-12B and Qwen3-14B.

MethodLlama-8BGemma-12BQwen3-14BQwen3.6-27BLlama-70BAvg (5 models)
Cleaned (n = 1,031)
Template Lens28.830.427.421.925.426.8
Patchscopes17.614.78.613.516.614.2
SelfIE12.523.911.917.213.415.8
Concept Lens (J-lens)40.830.425.422.137.331.2
Concept Lens (R-lens)58.047.152.935.052.449.1
Cleaned ∩ answered correctly (n = 532)
Template Lens35.737.833.632.634.934.9
Patchscopes22.520.412.017.823.619.3
SelfIE16.935.317.124.820.622.9
Concept Lens (J-lens)47.134.929.330.438.736.1
Concept Lens (R-lens)71.658.064.847.565.461.5
AuthorsOpenReview ↗

(2/3) [W1, Q1] Causal swaps on the cleaned set, and three public benchmarks

TL;DR:

Thank you for asking us to re-run the causal swaps (Table 3 of the paper) on the cleaned set. The prefix items inflated our swap rates as well: average CS@10 drops from about 40% to 28%. Still, on the 988 trials with both sides clean, our representations swap the answer about four times as often as Template Lens (7.2%) and ten times as often as random directions (2.6%). Beyond our own data, on 1,546 queries from three public benchmarks built by others, Concept Lens (R-lens) leads on average (35.4% Rank@10, versus at most 21.2% for the baselines), while SelfIE leads on Hopping Too Late.


1. Causal swaps on the cleaned set (Q1)

For causal swaps, we retain only pairs whose two sides both pass the prefix filter: 494 pairs, or 988 directed trials. The injected partner is therefore also clean. Swap rates are lower than on the full set (average CS@10 of 27.9% and 28.1%, versus 39.8% and 41.5%), so the prefix items inflated our swap results as well. Still, our representations swap the answer about four times as often as Template Lens and about ten times as often as random directions.

Table 2. Derived representations still swap the answer on the cleaned set. CS@10 (%): replacing the source concept's representation with the partner's moves the partner answer into the top-10 and the source answer out of it. Avg is over five models. The 504-trial subset contains trials whose clozes are answered correctly by all three of Llama-8B, Gemma-12B and Qwen3-14B.

MethodLlama-8BGemma-12BQwen3-14BQwen3.6-27BLlama-70BAvg
Both sides clean (n = 988)
Random3.03.02.72.51.72.6
Template Lens8.67.23.43.213.67.2
Concept Lens (J-lens)29.933.221.520.334.627.9
Concept Lens (R-lens)32.030.320.922.734.428.1
Both sides clean ∩ answered correctly (n = 504)
Random3.82.81.81.20.62.0
Template Lens8.17.92.21.615.37.0
Concept Lens (J-lens)37.744.027.225.242.935.4
Concept Lens (R-lens)39.940.326.428.242.935.5

2. Three public multi-hop benchmarks

We also evaluate the same readout on TwoHopFact (Yang et al., 2024), Hopping Too Late (Biran et al., 2024) and MQuAKE (Zhong et al., 2023). These benchmarks were built by others, independently of our data construction. Queries are retained only when separate prompts for the bridge entity and final answer are both answered correctly by all three models.

Table 3. Rank@10 on the three public benchmarks. Rank@10 (%) uses the same criteria as Table 1 in comment 1/3, averaged over Llama-8B, Gemma-12B and Qwen3-14B.

MethodTwoHopFact (n = 534)Hopping Too Late (n = 465)MQuAKE (n = 547)All (n = 1,546)
Template Lens9.07.319.612.2
Patchscopes5.815.08.29.4
SelfIE6.930.127.421.2
Concept Lens (J-lens)24.124.639.729.8
Concept Lens (R-lens)29.329.246.535.4
AuthorsOpenReview ↗

(3/3) [W2] Building on the lens keeps the method simple; SCORE picks the concept and makes it verifiable by intervention

TL;DR:

Thank you for pressing on what the lens contributes. Building on an existing lens is our design choice, so the lens is also our main limit: on the cleaned set, Concept Lens (J-lens) reaches 31.2% Rank@10 and Concept Lens (R-lens) 49.1%. We agree the prefix shortcut helped (Concept Lens (R-lens): 58.5% on the removed items). But a lens scores only single tokens. Concept Lens adds what it lacks:

  • Simple: no training and no concept library, yet more accurate than Template Lens, Patchscopes and SelfIE (§1).
  • Concept-level: it separates concepts that share a first token; the lens cannot (§2).
  • Verifiable: every readout comes with a representation we can intervene on (§3).

1. Simple by design, and still ahead

The lens shows black; the model completes it to black hole (PROPOSE), and SCORE checks it against the activation. Nothing is trained and no concept list is built.

  • Cost (Qwen3.6-27B, App. H): 0.1 A100-hours for all 52 layers, versus 20–30 A100-hours plus USD 1,500–2,000 for Template Lens and an estimated 2,000–4,000 A100-hours for one layer of Oracle Lens.
  • Cleaned set (Table 1 in comment 1/3): with at most 25 candidates per layer, Concept Lens (R-lens) reaches 49.1% Rank@10, versus 26.8% for Template Lens with 38,400 concepts and at most 15.8% for Patchscopes and SelfIE.
  • Public benchmarks (Table 3 in comment 2/3): 35.4% versus 12.2% for Template Lens.

2. The lens cannot separate concepts that share a first token; SCORE can

carbon begins both carbon dioxide and carbon monoxide, so the lens gives both the same score. We test all 37 counterfactual pairs whose concepts share a first token (App. D.4); a pair counts only if each prompt ranks its own concept higher. The prefix shortcut cannot help here.

Table 4. Pairs separated. Same source states, layer and score map; only the representation differs. Constituent mean: average of the concept's token representations.

RepresentationLlama-8BGemma-12BQwen3-14BQwen3.6-27BLlama-70B
First token (lens alone)0/370/370/370/370/37
Constituent mean18/377/377/3717/3721/37
Ours22/3726/3719/3726/3725/37

Ours beats the constituent mean on every model. Beyond these pairs, first-token lens scores give 39.5–63.5% Rank@10 on fixed candidate banks, versus 85.1–98.3% for SCORE (R-lens, full set, App. D.2).


3. Only a concept representation makes the readout verifiable

To test a readout, replace the concept in the activation and check the answer. This needs a representation of the concept. The lens has one only for single tokens (carbon, not carbon monoxide). Patchscopes and SelfIE produce text without one; a representation extracted afterwards played no part in the readout. In Concept Lens, the representation that SCORE ranks is the one we intervene with. On the 988 both-sides-clean trials, it swaps the answer in 27.9% and 28.1% of trials for Concept Lens (J-lens) and Concept Lens (R-lens), versus 7.2% for Template Lens and 2.6% for random directions (Table 2 in comment 2/3).

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revisions to the Manuscript

Dear Area Chair and Reviewers,

We sincerely thank Reviewers yzCg, Gf9D, and pUpR for your thorough reviews. We have revised the manuscript and expanded our experiments across all five LLMs (8B–70B). Below, we summarize our major revisions.

1. Step-by-step overview figure (Reviewers Gf9D, pUpR). Replaced the old Figure 1 with a positioning diagram (Figure 1) and a step-by-step overview (Figure 2, Steps 1–9) tracing a concrete example (black hole → event horizon) across two stages.


2. Renamed PROPOSE & SCORE to GUESS & CHECK (Reviewers Gf9D, pUpR). Renamed the two stages to GUESS and CHECK to better reflect their respective roles: GUESS generates candidate concepts from token-level seeds, while CHECK derives explicit concept representations and ranks them against the source activation.


3. Streamlined Section 3 aligned with Figure 2 (Reviewers Gf9D, pUpR). Moved detailed theoretical derivations from Section 3 to the appendix and reorganized Section 3 with descriptive headers and numbered steps (Steps 1–9), directly aligned with Figure 2.


4. Unified Experimental Setup and roadmap table (Reviewers Gf9D, pUpR). Removed the old Section 4, consolidated reliability tests in Section 4.3, and opened the revised Section 4 with Experimental Setup (Section 4.1) and Table 1 clarifying our evaluation logic, datasets, baselines, and metrics.


5. Revised Applications, Figure 3, and baseline comparison (Reviewer Gf9D). Redesigned Figure 3 in Section 5 (defining orange/grey matched prompts, yellow context differences, and boxed readout tokens) and added a comparison with existing readers (token-level lens, Template Lens, Patchscopes, SelfIE) on the same six activations.


6. Expanded datasets, baselines, and reliability evaluations (Reviewers yzCg, Gf9D, pUpR).

  • Cleaned ConceptCloze (Reviewer yzCg): Removed answer-prefix and invalid one-hop items, retaining 988 multi-hop prompts (494 pairs; Tables 2–4), and reported results on the correctly answered subset (62.8% Rank@10 and 35.5% clean swaps with R-lens).

  • Public benchmark PublicTwoHop (Reviewer Gf9D): Added 1,546 external multi-hop prompts from TwoHopFact, Hopping Too Late, and MQuAKE, where Concept Lens (R-lens) achieves 36.9% Rank@10, outperforming all baselines (≤24.6%; Table 2).

  • Generative & library baselines (Reviewers yzCg, pUpR): Evaluated Patchscopes and SelfIE on all five LLMs for end-to-end readout (14.5% / 15.8% vs. 49.5% R-lens; Table 2) and counterfactual pairs (53.6% / 50.4% vs. 80.7%; Table 3b), scaled Template Lens 3× to 38,400 concepts (24.5% Rank@10 on ConceptCloze), and compared oracle rankings (89.4% vs. 40.0%; Table 2).

  • Main-text reliability tests (Reviewers yzCg, pUpR): Section 4.3 (Table 3) highlights SplitWords (99.1% J-lens / 98.4% R-lens Top-1 vs. 52.3% token mean), counterfactual pairs (80.7% R-lens), and SameFirstToken (94.4% R-lens vs. 0.0% first token, 76.0% token mean, 77.2% token NNLS), demonstrating that CHECK captures whole-concept information beyond first-token shortcuts.


7. Explicit task formulation in Section 2.2 (Reviewer Gf9D). Explicitly defined Task, Input, and Output in Section 2.2.

We sincerely thank Reviewers yzCg, Gf9D, and pUpR again for their constructive feedback. Your comments helped us identify important clarity issues in the initial submission and guided substantial improvements to both the presentation and experimental evaluation. We hope the revised manuscript addresses your concerns, and we welcome any further suggestions. Thank you for your time and consideration.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.