acceptodds
Under review as a conference paper at ICLR 2027

Making Sense of Soft Tokens: Brain Captioning with Causally Testable Semantic Channels

Abstract

Brain captioning models map fMRI responses to continuous soft tokens that a frozen language model consumes to generate natural language. While effective for producing captions, these brain-derived tokens are primarily shaped to fit the language model's output space, without an explicit, text-like semantic interface connecting brain activity to language. Consequently, it remains unclear how semantic information is organized within these tokens, what individual tokens represent, and which representations the language model actually uses. We introduce **MindLens**, a lightweight semantic interface that transforms a subset of LLM-facing soft tokens into named semantic channels. Each channel is assigned a human-interpretable concept group and directly scores its concepts in the LLM's own token-embedding space, requiring only two additional scalar parameters in total and no architectural modification to the language model. By assigning each concept to a single channel, MindLens encourages channels to specialize in coherent yet distinct semantic content, transforming an opaque neural prefix into an explicit and structured semantic interface. Rather than adding information, MindLens realigns the semantic information already encoded in the fMRI tokens into interpretable directions that can be directly read and causally tested. On the Natural Scenes Dataset, MindLens achieves a concept mAP of without a learned probe, closely approaching the 0.420 obtained by a trained probe from the unstructured backbone, while preserving caption quality (CIDEr: 64.80 vs. 64.21). Targeted interventions further show that individual channels selectively control their associated concepts during generation. These results establish MindLens as an interpretable intermediate layer between neural activity and language, enabling brain captioning to move beyond fitting final language outputs toward explicitly representing, validating, and causally testing the semantic information carried by brain-derived tokens.

Primary area:
applications to neuroscience & cognitive science
Keywords:
brain captioning, interpretable neural decoding, semantic soft tokens

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_DbaeOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose MindLens, an interface that assigns explicit semantic roles to soft tokens to verify which soft tokens generated from fMRI represent and which expressions language models actually use, and to check whether the tokens are used in actual sentence generation.

Critical strengths

[1] The authors show that semantic role tokens have a substantial influence on generating captions from brain signals.

[2] The authors evaluate in detail the impact of representations on information and output through various ablation experiments.

[3] The authors use readout analysis to evaluate properties of intermediate representations that captioning metrics such as BLEU, CIDEr, and ROUGE alone cannot reveal.

Critical weaknesses

[1] The authors have not sufficiently demonstrated the added value of MindLens over post-hoc alignment or its generalisability across encoders. Although 8 semantic role tokens improve direct readout and intervention selectivity, the paper does not compare MindLens with post-hoc alignment applied to the backbone, and the interpretability analyses are limited to the authors’ own backbone.

[2] Test captions were used to construct the concept vocabulary, and the descriptions in the main text and appendix are inconsistent. Section 2.4 states that concepts are extracted from training captions, whereas Appendix A.5 explicitly states that the procedure, including vocabulary selection and normalisation, was applied to captions from all 73,000 stimulus images without a train/test split.

Questions for authors

[1] Regarding Weakness 1, could the authors compare MindLens with post-hoc alignment applied to the backbone, with the alignment fitted on separate calibration data and evaluated on an independent test set?

[2] Regarding Weakness 2, could the authors clarify the actual data scope and procedure used to construct the concept bank and groups?

[3] COCO dataset provides 80 object categories with 12 super-categories. Some of the author's categories have identical or similar name to COCO super-categories, but their memberships differ. Could the authors (i) clarify the criteria for assigning concepts to groups beyond the requirement that each group have a coherent name, and (ii) explain why a custom caption-derived taxonomy was chosen instead of given coco categories?

[4] [minor] In Table 1, METEOR for Mind-SA, Yo’Mind, the backbone, and MindLens (38–40) is about twice that of the other methods (16–22), while CIDEr and SPICE show no comparable gap. Were all scores computed with the same METEOR implementation? Were the baseline numbers reproduced under the same pipeline, or copied from the original papers?

[5] [minor]Section 2.4 describes clustering concept embeddings with K-means followed by manual adjustment, whereas Appendix A.5 describes a manually constructed grouping and presents K-means partitions as ablations. Could the authors clarify the relationship between these procedures and explain how the final groups were constructed?

5 comments

We appreciate Reviewer Dbae for your insights and careful reading, which led to the new results detailed below. The five parts below follow your questions and name the weakness each one addresses. (W and Q stand for Weaknesses and Questions, respectively.)


[Q1 + W1] Comparison with post-hoc alignment on the backbone

Thank you for suggesting this comparison. A post-hoc alignment fitted on the backbone finds group directions, but removing them changes the captions only slightly: the mean own-group drop is at most 0.81 points, compared with 6.52 for MindLens. This suggests that aligning the channels during training is what makes them causally important for the captions.

Post-hoc alignment. We kept the paper's backbone unchanged. On the 36,000 training rows, the same rows and labels MindLens is trained with, we computed for each group and each of the 48 tokens the group's direction: the mean token over images containing the group minus the mean over images without it. Each group's post-hoc token is the one whose projection onto this direction best separates the group. The alignment is evaluated on the 4,000 test rows, none of which is used for fitting. Each group chooses its token freely, which gives the backbone more freedom than MindLens's one channel per group.

Two post-hoc edits. Both use the paper's removal test and the same fill value as for MindLens: the token's mean over test images lacking the group.

  • (a) Remove the whole post-hoc token. This is the paper's channel test, and the only difference is that the token is chosen after training instead of being declared before it.
  • (b) Remove only the group's direction within that token, leaving the orthogonal part intact. This is the most targeted post-hoc edit.
MindLens channel(a) Whole post-hoc token(b) Group directionBackbone, tokens 0–7
Token chosenbefore trainingafter trainingafter trainingfixed
Mean own-group drop (points)6.520.370.810.05
Mean spillover (points)1.351.010.200.94
Control change (points)0.550.610.200.61

What this shows:

  • Post-hoc handles are weak. Removing the post-hoc token (a) or only its group direction (b) lowers the group's recall by 0.37 and 0.81 points on average, and by at most 1.58 points for any group under (b). Removing a MindLens channel lowers it by 6.52 points.
  • Removing a whole post-hoc token disturbs other groups more than its own. In (a), the same operation as MindLens's channel test, other groups change by 1.01 points, more than the group's own drop (0.37) and the control (0.61), as with the backbone's fixed tokens (0.94 against 0.05). For MindLens, the own-group drop is nearly five times the spillover (6.52 against 1.35).
  • The direction edit is targeted but small. In (b), other groups change no more than under the control (0.20 each), and the group's own drop is about four times the control, but only 0.81 points.
  • The backbone's captions barely depend on any single token. The post-hoc edits leave CIDEr between 63.85 and 64.45, against 64.21 unedited. This is consistent with our finding that a probe on any single backbone token recovers almost all the concept information (0.405 to 0.416 mAP, Section 4.3): in the backbone, the information is spread over the prefix.
  • Readout. On readout alone, post-hoc alignment matches MindLens: a linear probe fitted on the backbone reaches 0.420 mAP, against 0.410 for MindLens's direct readout (Section 4.3). The added value of MindLens is therefore not readability but channels that can be read without a fitted probe and are causally necessary.

Generalization across encoders. This is a fair point. Our analyses have so far been carried out on our own fMRI encoder, and whether the findings extend to other encoder architectures remains an open question that we have not yet been able to test. In the revision, we will make this scope clear and note evaluation on other encoders as future work.

[Q2 + W2] Data scope of the concept bank and groups

We agree that the vocabulary should not be built from test captions, and that Section 3.4 was inaccurate. Rebuilt without the test images, the vocabulary keeps the same 132 concepts and eight groups, and the results change only marginally.

Original scope. As Appendix A.7 states, vocabulary selection and normalization were run over the captions of all 73,000 NSD stimulus images, including the 1,000 shared test images. Section 3.4 wrongly said "training captions". The vocabulary is a frequency ranking over human captions and involves no model and no fMRI data. We apologize for this mistake.

Procedure. The full construction is given in our answer to Q5 below. The groups take only the concept list as input.

Rebuilt without the test images. We reran the same pipeline on the captions of the 72,000 non-test images and retrained MindLens on the resulting labels, with nothing else changed. The rebuilt vocabulary has the same 132 concepts, with the same names, order, and groups. Two member words change, both through the rank-200 cutoff: path falls below it and leaves concept sidewalk, and glasses rises above it and joins concept eyewear, as already assigned in the concept file. Labels change on 540 of the 73,000 images, 532 of which lose sidewalk.

Caption quality is essentially unchanged:

VocabularyB@1B@2B@3B@4METEORROUGECIDErSPICE
Paper (with test images)59.5641.4127.4918.0540.3345.0864.8013.77
Rebuilt (no test images)59.6241.5227.7418.5040.3745.1065.6913.75

Readability and selectivity. Selectivity is identical (55/56), and readout and removal effects change only slightly (drops in recall points):

VocabularyProbe-free mAPMean own-group dropMean spillover
Paper (with test images)0.4106.521.35
Rebuilt (no test images)0.4126.891.23

Revision. We corrected Section 3.4 and added this comparison to the appendix. The final version reports all results with the rebuilt vocabulary.


[Q3] Assignment criteria and the choice of taxonomy

Our groups follow the roles concepts play in a scene description, not object types, because the channels feed caption generation and the causal test counts what generated captions mention. COCO's supercategories group object types instead, so shared names differ in meaning.

Assignment criteria

Beyond nameability, three rules decided the assignment:

  1. Starting point. Groups start from k-means clusters of the concept-name embeddings (Q5), so concepts the language model represents similarly start together.
  2. Scene role. Following the structure of scene descriptions (Appendix A.7), the groups cover who is present (people), animals, food, the indoor or outdoor setting, vehicles, activities (sport), and other objects.
  3. Context words join the objects they host. For example, zoo joins animals group; kitchen, counter, stove, refrigerator and restaurant join food group, and train track, train station, airport and luggage join vehicle group.

Each concept belongs to exactly one group, and groups are not balanced by size.

Why not COCO's categories

Captions, not object annotations. The causal test (Section 4.4) counts which concepts a generated caption mentions, so the concepts must be the words captions use. COCO's 80 categories are annotated object instances and do not cover scene context such as street, field, water, or grass. By our count, 51 of our 132 concepts are COCO category names, and none of the 26 outdoor concepts is. Yet the outdoor channel shows the largest own-group drop in the paper (14.61 points).

Matched to a 3-second view. NSD participants viewed each image for 3 seconds, enough for the gist of the scene but not every small object. COCO's annotators had ample time to mark every instance of the 80 categories, including small objects a participant may never register, and a caption can describe a scene without naming any annotated object. Captions name a scene's salient content, which is closer to what a participant takes in.

Same names, different meanings. COCO's "outdoor" is street furniture (traffic light, fire hydrant, stop sign, parking meter, bench), four of which are in our object group, and its "indoor" is small household items such as books, clocks, and vases. Our indoor and outdoor groups describe the setting.

The conclusions do not rest on our taxonomy. They hold across five partitions, including k-means partitions with no manual step (Table 10): probe-free mAP stays between 0.402 and 0.410, and selectivity between 0.789 and 0.967, against 0.464 for the backbone (chance 0.49).

Revision. We state these criteria in Appendix A.7 and add the comparison with COCO's categories.

[Q4] METEOR in Table 1

Thank you for pointing this out. We share this concern and noticed the same discrepancy when preparing Table 1.

To our knowledge, Yo'Mind (ICLR 2026) remains the state-of-the-art brain captioning model built on a lightweight language model (GPT-2). As a peer-reviewed work, we considered its reported results a reliable reference and followed it closely: nearly all baseline numbers in Table 1 are copied directly from the Yo'Mind paper.

We also attempted a quick re-evaluation of Yo'Mind, but only its code is released, without trained checkpoints. The same holds for Mind-SA, MindGPT and MindArt, which share the same first author and research group. Interestingly, MindGPT and MindArt report METEOR around 20, whereas Mind-SA and Yo'Mind report around 40, although their other metrics show no comparable gap. We are therefore also unsure whether the METEOR values in these papers were computed with the same implementation. What we can confirm is that our own METEOR scores are computed with NLTK 3.9.4. We will contact the authors of Yo'Mind to confirm their implementation before the final revision.


[Q5] How the final groups were constructed

The eight groups were obtained by running k-means and then adjusting the clusters by hand. Section 3.4 and Appendix A.7 each described only part of this procedure; we clarify it step by step below:

  • Step 1: concepts. We extract candidate nouns from the stimulus captions, merge spelling variants and synonyms through a hand-authored lookup table (no embedding or clustering), and keep the 200 most frequent entries by image document frequency. These are collected into 132 concepts in a fixed file that assigns each surface form to exactly one concept (Appendix A.7). The variation that Section 3.4 illustrates with dog, puppy and canine is handled here, not by clustering.
  • Step 2: groups. We ran k-means (Nc=8N_c = 8) over the frozen GPT-2 input embeddings of the 132 concept names to obtain initial clusters, and then adjusted them by hand so that every group carries a single natural name; a channel whose group cannot be named is not interpretable.
  • Step 3: fixed before training. The concept file and the partition were fixed before any model was trained and are released with the code. Rebuilding the vocabulary without the test images (Q2) only changed which surface forms pass the rank cutoff: path fell below rank 200 and glasses rose into it, both already assigned in the file, so the 132 concepts and the eight groups are unchanged.

Where k-means enters. K-means enters twice: as the starting point of the hand-adjusted partition (Step 2 above), and on its own in Table 10, where partitions produced by k-means alone, without the manual adjustment, serve as an ablation at $N_c = $ 6, 8 and 10. These give the same readout (probe-free mAP 0.402 to 0.406, against 0.410) and selectivity far above the backbone (0.789 to 0.967, against 0.464; chance 0.49), so the conclusions do not depend on the manual adjustment. We keep the hand-adjusted groups because they can be named, and at Nc=8N_c = 8 they are also more selective (0.929 against 0.875).

Revision. We rewrote Section 3.4 to describe the two steps separately, added the k-means starting point to Appendix A.7, and updated Figure 1 accordingly, so that all three describe the same procedure.

Reviewer_tac5OpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper investigates the interpretability of continuous soft-token prefixes in fMRI-to-text brain captioning. The authors observe that while standard brain-derived prefixes contain rich semantic information that can be extracted by trained linear probes, this information is distributed across tokens and cannot be directly read in the frozen language model's native embedding space. To address this, they propose MindLens, a lightweight semantic interface that designates a subset of prefix tokens as named semantic channels tied exclusively to human-interpretable concept groups in the language model's token embedding space, while leaving the remaining prefix tokens unconstrained. Experiments on the Natural Scenes Dataset show that MindLens makes the designated channels directly readable without a separate probe while preserving overall captioning quality, and counterfactual token-replacement and cortical perturbation analyses show that individual channels selectively control their associated concepts in the generated captions.

Critical strengths
  1. The proposed semantic interface is remarkably simple and cleanly separates representational organization from model capacity. By anchoring a subset of soft tokens directly to frozen language model vocabulary embeddings and enforcing exclusive concept ownership across channels, the method introduces virtually no additional trainable capacity and requires no modification to the frozen language model. Because captioning performance remains on par with the unconstrained backbone and linear probes recover the same total semantic information from both models, the experiments convincingly show that the interface reorganizes existing fMRI-derived semantics into directly readable directions rather than adding external capacity.
  2. The paper also makes a compelling methodological shift from correlational probing to causal and neuroanatomical interventions. The authors demonstrate that linear probing is insufficient for evaluating soft-token prefixes because a probe can recover semantic content from almost any individual token position in the unconstrained model. By replacing individual channels with counterfactual group-absent means while preserving sequence geometry, and by performing in silico ablations of category-selective visual regions, they provide solid evidence that the language model causally relies on the designated channels and that these channels reflect underlying cortical selectivity.
Critical weaknesses
  1. Because the remaining residual tokens in the prefix are left completely unconstrained, they still redundantly encode salient scene concepts, which limits single-channel causal control at the per-image level. Suppressing a single semantic channel only alters the mention of its target group in a modest fraction of eligible test images, and the recall reduction on frequent categories such as animals is very small. This appears to stem from two factors: first, the concept loss weights every individual concept equally rather than balancing by group size, giving large concept groups several times the gradient share of smaller groups; second, placing the semantic channels at the very beginning of the prefix allows subsequent residual tokens to attend to and buffer their content under causal self-attention, which aligns with the appendix ablation showing that moving the channels to the end of the prefix more than doubles the causal recall drop upon channel removal.

  2. There is also a methodological hygiene issue in how the concept vocabulary was constructed, alongside a narrow language-model evaluation scope. According to the appendix, candidate noun extraction, manual synonym normalization, and document-frequency filtering were conducted across all stimulus captions in the dataset without separating the training and test splits. Even though no fMRI signals were involved in that step, deriving the target concept vocabulary on the union of training and test captions violates strict train-test separation. In addition, all experiments are conducted on a single legacy language model (GPT-2) with a single random seed, leaving open how well this alignment transfers to modern instruction-tuned language models.

Questions for authors
  1. For clarification, since candidate noun selection and the document-frequency cutoff were run over all stimulus captions without a train-test split, do the selected concepts change if the vocabulary is built strictly from the training split (excluding the shared test images), and do the probe-free readout and causal selectivity results remain unchanged?

  2. The position ablation shows that placing the semantic channels at the end of the prefix achieves slightly better caption quality while more than doubling the own-group recall drop under intervention compared to placing them at the beginning. Does placing the channels at the start allow the subsequent residual tokens to attend to and redundantly copy their information under the language model's causal attention mask, and would penalizing alignment between the residual tokens and the concept embeddings during training improve single-channel necessity?

  3. Removing the animals channel produces almost no drop in animal recall compared to large drops for outdoor concepts. Since the concept loss weights all individual concepts equally, giving large groups several times the gradient weight of smaller groups, does reweighting the loss so that each channel receives equal total gradient weight improve causal control on the smaller groups?

  4. For future work, beyond suppressing present concepts via mean replacement, it would be important to test positive counterfactual steering (injecting a concept direction into an image where that concept is absent to see if the language model incorporates it into the caption) and to validate MindLens on modern instruction-tuned language models across multiple random seeds.

4 comments

We thank Reviewer tac5 for your precise questions and valuable suggestions, which led to the new experiments described below. The four parts below follow your questions and name the weakness each one addresses. (W and Q stand for Weaknesses and Questions, respectively.)


[Q1 + W2] Does a vocabulary built without the test images change caption quality, readability, or selectivity?

No. The same 132 concepts remain; caption quality and probe-free readout move only marginally.

What we did. We reran the vocabulary pipeline of Appendix A.7 on the captions of the 72,000 NSD stimulus images outside the shared test set, so that none of the 1,000 test images contributes. We then retrained MindLens on the resulting labels with nothing else changed, and repeated the channel-removal test of Section 4.4 with that run's own labels and member-word lists.

Vocabulary. The rebuilt vocabulary has the same 132 concepts, with the same names and order. Of its 184 member words, 183 are identical. Two member words change at the rank-200 cutoff: path leaves sidewalk, and glasses joins eyewear. Labels change on 540 of the 73,000 images, 532 of which lose sidewalk. This only affects 252 training samples.

Caption quality. The caption metrics are essentially unchanged. "Paper" is the vocabulary built on all 73,000 images, while "Rebuilt" is the one built on the 72,000 non-test images.

VocabularyB@1B@2B@3B@4METEORROUGECIDErSPICE
Paper (with test images)59.5641.4127.4918.0540.3345.0864.8013.77
Rebuilt (no test images)59.6241.5227.7418.5040.3745.1065.6913.75

Readability and Selectivity. Selectivity is almost identical, and the probe-free readout and the removal effects change only slightly. Drops and spillover are in recall points, as in Figure 4.

VocabularyProbe-free mAPMean own-group dropMean spillover
Paper (with test images)0.4106.521.35
Rebuilt (no test images)0.4126.891.23

Revision. We agree that the vocabulary should not be derived from any of the test captions. Section 3.4 said the concepts came "from the training captions", which was inaccurate, and Appendix A.7 described the actual scope. We corrected Section 3.4 and added the comparison above to the appendix. It shows that excluding the test images changes neither the selected concepts nor the caption, readout, and selectivity results. The final version will report all results with the rebuilt vocabulary.


[Q2 + W1] Front placement and a residual-alignment penalty design

Front placement (first half of Q2)

We agree and attribute the effect mainly to the causal attention mask. As you note, placing the channels last raises the mean own-group drop from 6.52 to 16.17 points, while selectivity falls from 0.982 to 0.911 (Table 11). Under the causal mask, the 40 residual positions can attend to channels placed first but not to those placed last. This is a limitation introduced by the decoder-only language model, and we will explicitly disclose this in the ablation in Appendix A.6.

Penalty on residuals (second half of Q2)

Penalizing the alignment between residual tokens and concept embeddings nearly doubles single-channel necessity, with the channels still at the front. We retrained MindLens with an extra hinge loss, max⁡(a+−a−,0)\max(a^+ - a^-, 0), weighted by 10. Here a+a^+ is the mean cosine between the 40 residual tokens and the embeddings of concepts present in the image, and a−a^- the same for absent ones. It adds no parameters. Everything else is unchanged.

Own-group recall drop when one channel is removed w/o penalty (points):

peopleanimalsfoodindooroutdoorvehiclesportobjectmean
MindLens (paper)6.370.863.254.7414.619.434.008.866.52
With penalty10.8910.314.884.5542.866.696.379.7012.03
MindLens (paper)With penalty
Mean spillover (points)1.352.02
Drop when a residual token is removed (points)0.550.16
Mean CIDEr cost of removing one channel1.682.39
CIDEr64.8064.06
a+−a−a^+ - a^- (residual tokens)+0.007−0.016

What this shows. The penalty works as intended: the residual tokens lose their concept alignment and matter less, while each channel matters more, for its own group and for the whole caption.

Revision. We agree with W1 that single-channel control in the paper's model is selective but partial. We keep the model without the penalty as our main model and report the penalty variant in the appendix: it trades a little selectivity for stronger necessity, concentrated in two channels. The placement effect is stated as a limitation in Section 5.

[Q3 + W1] The weak animals channel and the weighting of the concept loss

We appreciate this insight and propose a data-driven alternative to fixed reweighting that strengthens the weak animals channel: its own-group drop rises from 0.86 to 12.79 points, and the mean over all channels rises from 6.52 to 10.14 points.

Per-concept bias. Instead of fixing the loss weights by group size, we let each concept adjust its own contribution in a data-driven way. We replaced the bias β\beta in Eq. 7, shared by all concepts, with one trainable bias βk\beta_k per concept, so that y^k=σ(γαk+βk)\hat{y}_k = \sigma(\gamma\alpha_k + \beta_k). Each concept then calibrates its readout to its own frequency, which in turn adjusts how much gradient it sends to its channel. The interface has 133 scalars (γ\gamma and 132 biases) instead of 2, and everything else is unchanged.

Own-group recall drop when one channel is removed (points) is provided below:

peopleanimalsfoodindooroutdoorvehiclesportobjectmean
MindLens (paper)6.370.863.254.7414.619.434.008.866.52
Per-concept bias5.1812.794.296.4126.488.337.5010.1210.14

Caption quality is preserved (CIDEr 64.57 against 64.80). Among the four smallest groups, animals and sport improve (by 11.93 and 3.50 points), while people and vehicle decrease slightly (by 1.19 and 1.10).

Revision. We see per-concept calibration as a useful design choice for stronger single-channel control. We report the per-concept bias variant in the appendix, as an option when stronger causal control matters more than keeping the interface to two parameters.


[Q4 + W2] An instruction-tuned language model, random seeds, and positive steering

Transfer to an instruction-tuned language model. We replaced GPT-2 with a frozen Vicuna-7B v1.5, the language model of LLaVA-1.5, and trained the backbone and MindLens with the same recipe and seed as the GPT-2 models.

Caption quality. With either language model, MindLens matches or slightly exceeds its backbone.

ModelB@1B@2B@3B@4METEORROUGECIDErSPICE
Backbone, GPT-2 (paper)59.6241.1426.8917.4840.0144.8364.2113.46
MindLens, GPT-2 (paper)59.5641.4127.4918.0540.3345.0864.8013.77
Backbone, Vicuna-7B60.0740.9826.8817.6240.6544.6364.2314.04
MindLens, Vicuna-7B60.5141.5927.6818.3641.0644.8266.5814.16

Readability and causal control. Readability transfers: probe-free concept mAP is 0.412, against 0.410 with GPT-2. The causal results transfer too, and become much stronger. For the backbones, the ownership map is imposed on positions 0 to 7, as in the paper.

Metric / PropertyBackbone GPT-2MindLens GPT-2Backbone Vicuna-7BMindLens Vicuna-7B
Selectivity0.2680.9820.3751.000
Mean own-group recall drop (points)0.056.520.2024.31
Mean CIDEr cost of removing one channel0.131.680.084.12

Necessity. Every Vicuna channel is strongly necessary: the own-group drops are people 36.43, animals 15.55 (0.86 with GPT-2), food 17.01, indoor 18.87, outdoor 48.31, vehicle 25.55, sport 11.75 and object 20.99 points, so even the weakest exceeds the GPT-2 mean of 6.52. With selectivity at 56/56, the larger drops stay specific to each channel's own group.

Random seeds. We trained the backbone and MindLens with two more seeds (2025 and 2027), using the paper's recipe, and evaluated all three seeds with the same pipeline. Caption quality is stable, and MindLens matches the backbone:

B@1B@2B@3B@4METEORROUGECIDErSPICE
Backbone59.61 ± 0.1941.03 ± 0.2526.92 ± 0.3817.57 ± 0.3739.91 ± 0.1244.84 ± 0.0563.82 ± 0.6613.44 ± 0.04
MindLens59.30 ± 0.3541.06 ± 0.3527.19 ± 0.2817.80 ± 0.2340.09 ± 0.2245.01 ± 0.1164.22 ± 0.5813.55 ± 0.18

Single-channel necessity varies more across seeds. Own-group recall drops when one channel is removed (points):

Seedpeopleanimalsfoodindooroutdoorvehiclesportobjectmean
20252.867.920.448.8310.353.1412.3810.036.99
2026 (paper)6.370.863.254.7414.619.434.008.866.52
20279.2916.326.219.2038.014.789.389.5312.84

The mean own-group drop is 8.78 ± 3.52 points, and single channels vary more. We add the three-seed results to the appendix and report the causal results per seed.

Positive steering. This is indeed an interesting and meaningful direction and the natural complement to suppression. As you suggest, we leave it to future work, and the paper's causal claim will keep its scope to selective suppression rather than active insertion.

Thanks for the clarification. I find the intervention experiments interesting, but I still have some questions about how the concept bank was built and how much the results depend on the particular backbone. I will keep my rating at 3.

Thank you for the follow-up, and we are glad you find the intervention experiments interesting. To make sure we address exactly what remains unclear, we summarize both points below, with where each now appears in the revised PDF.

How the concept bank was built

The concepts and their groups are built in two steps, both fixed before any model is trained (Section 3.4, Figure 1, and Appendix A.7 in full).

  1. Concepts. From the captions of the stimulus images, we extract candidate nouns, merge spelling variants and synonyms with a hand-authored lookup table (no embeddings or clustering), and keep the 200 most frequent entries by image frequency. A fixed concept file collects them into 132 concepts, each surface form belonging to exactly one concept.
  2. Groups. k-means (Nc=8N_c = 8) over the frozen GPT-2 embeddings of the concept names gives the starting clusters, which we then adjust by hand so that every group carries a single natural name. The adjustment follows the roles concepts play in a scene description, and context words join the objects they host (e.g., kitchen joins food).

Two checks show that the results do not depend on these choices:

  • Data scope. Rebuilding the vocabulary from the 72,000 non-test images alone yields the same 132 concepts, names and groups, and essentially the same results (probe-free mAP 0.412 vs. 0.410; selectivity 55/56 in both; Appendix A.4).
  • Taxonomy. Across five partitions, including k-means partitions with no manual step, the probe-free mAP stays between 0.402 and 0.410 and the selectivity between 0.789 and 0.967, against 0.464 for the backbone (chance 0.49; Appendix A.6).

How much the results depend on the backbone

  • Language model. Replacing GPT-2 with a frozen Vicuna-7B, trained with the same recipe, preserves readability (mAP 0.412 vs. 0.410) and strengthens the causal result: selectivity rises to 1.000 (56/56) and the mean own-group drop from 6.52 to 24.31 points, at a CIDEr of 66.58 (Appendix A.4).
  • Training runs. Across three seeds, selectivity stays between 0.911 and 0.982 (Appendix A.5).
  • The effect comes from the interface, not the backbone. With the same ownership map imposed on the backbone, selectivity is 0.268 for GPT-2 and 0.375 for Vicuna-7B, below chance. Group directions fitted on the backbone after training are also not causally necessary: removing them lowers recall by at most 0.81 points on average, against 6.52 for a MindLens channel.

Could you let us know which aspect remains unclear? For example, for the concept bank, is it the frequency cutoff, the manual adjustment, or the data the vocabulary is built from? For the backbone, is it the language model, the fMRI encoder, or the variation across training runs? We would be glad to clarify further, and your answer would help us improve the quality of our work.

Reviewer_xhgmOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

So the goal of this work is to attempt interpretation of intermediate representations used in brain activity (fMRI) to natural language decoding.

The authors achieve this by specifying a subset of trainable prefix tokens as "interpretable" tokens, and applying a cross-entropy based loss against a linearly transformed version of these tokens, trained jointly with caption reconstruction.

This framework yields little performance loss against other evaluated methods. The authors demonstrate that their concepts do not require linear probes to be interpretable. They further find that replacing individual concept tokens with the dataset average reduces in corresponding semantic group recall.

Critical strengths
  1. Interpretability of brain decoding models is always an aspect worth investing. For many brain decoders, it is unclear what information the brain supplies to the final decoded output. When the brain is combined with a sufficiently powerful inductive bias (diffusion/GAN models in image decoding, or LLM in language decoding), the brain information may not be critical to quality metrics that are commonly evaluated.
Critical weaknesses
  1. The text clarity in this paper is very poor in my opinion, on this basis alone I would recommend strong reject. I do not consider this paper anywhere near the quality of a typical work accepted at ICLR. The mathematical notation is extremely confusing, and the usage of technical terms does not align with their typical meaning. Overall I think the authors rely too much on vague intuition in many places. To give a few examples:

    • In Line 19, the authors use the word channels. By typical accepted definitions, this would represent a feature channel. Only after closely reading the paper, does it become clear that the authors actually mean a "token"
    • In Line 44, the "dominant paradigm" for what? Just one sentence prior the authors are talking about fMRI image viewing studies, where I would assume the authors would be discussing block-based designs or fast-event/slow-event based designs. Yet the authors are suddenly talking about mapping fMRI activations to tokens?
    • In Line 122, what is vs∈RLsv_s \in \mathcal{R}^{L_s}? Does it represent the response to a single stimuli? What is LsL_s? Similar notation issues exist for τ\tau.
    • In equation (1), it is not immediately clear where τprompt\tau_\text{prompt} is from. So far, we have only seen τ\tau defined as a "corresponding caption"
    • In equation (2), what is Rdshare\mathcal{R}^{d_\text{share}} referring to? Is it the bias? Is it referring to ff?
    • In Line 168, the definition for a "neural encoder" is completely unrelated for how an "encoder" is usually defined in fMRI studies. [1]. This suggests that the authors are deeply unfamiliar with the field.
    • In Line 208 onwards, the entire section about Concept bank and grouping is extremely difficult to understand
  2. The figure presentation is very poor in my opinion. For Figure 1, the authors uploaded a figure in raster format, causing the included text to be very blurry. All figures lack detailed descriptions that would explain the contents included in each figure.

  3. The authors never explain why they believe caption decoding from the brain is a scientifically interesting task, instead it is presented ipso facto. Selectivity across different higher visual cortical regions is well characterized at this point by the field, and decoding captions does not add scientifically to this.

[1] Encoding and decoding in fMRI (2011); Thomas Naselaris, Kendrick N Kay, Shinji Nishimoto, Jack L Gallant

Questions for authors
  1. Could you provide an explanation for the reasoning why text decoding from fMRI recordings during image viewing is scientifically useful?
  2. From my perspective, the brain decoder is a human construct, and ultimately this work has not furthered our knowledge about the brain. Why is it scientifically interesting to study a human construct that is restricted to such a narrow task?
3 comments

We note that the review does not identify an error in the method, the experiments or the results: its weaknesses concern wording, notation, figure format and motivation. Most of the wording and notation points are already defined in the paper, at the lines given below. We fixed the rest in the revision.


[W1] Clarity of the text and notation

  • Line 19, "channels". The sentence that introduces the term defines it: MindLens "transforms a subset of LLM-facing soft tokens into named semantic channels" (lines 18–19).
  • Line 44, "dominant paradigm". The paragraph is about brain captioning (lines 39–41), and the sentence itself states the paradigm: fMRI responses are mapped to soft tokens that form a prefix for a pretrained LLM, following prefix-tuning, LLaVA (lines 44–46). This is the prevailing formulation of brain captioning (Section 2.1), not an experimental design. We now write "the dominant brain-captioning paradigm".
  • Line 122, vsv_s, LsL_s and τ\tau. Lines 121–123 define vs∈RLsv_s \in \mathbb{R}^{L_s} as the voxel response to one stimulus; LsL_s, which varies across subjects, is its number of voxels. τ\tau is the caption; we now write the generated caption as τ^\hat{\tau}.
  • Equation (1), the prompt. As in standard LLM prompting, it is the fixed text that follows the prefix: "Scene description:" (line 189). We now name it where eq.(1) first uses it.
  • Equation (2), Rdshare\mathbb{R}^{d_{\mathrm{share}}}. In f=Ws⊤vs+bs∈Rdsharef = W_s^\top v_s + b_s \in \mathbb{R}^{d_{\mathrm{share}}}, the membership applies to ff, and hence to bsb_s, as is standard. Line 162 gives Ws∈RLs×dshareW_s \in \mathbb{R}^{L_s \times d_{\mathrm{share}}}, and Appendix A.2 gives dshare=2048d_{\mathrm{share}} = 2048. We use the default notation of the ICLR template, in which R\mathbb{R} is the set of real numbers; dshared_{\mathrm{share}} is the dimension of the representation shared across subjects.
  • Line 168, "neural encoder". This is EvoxE_{\text{vox}} in eq.(3), which maps the shared fMRI representation ff to neural tokens FF. "Encoder" has its machine-learning meaning, as in the "frozen image encoders" of BLIP-2, whose querying transformer we adopt. It is not an encoding model in the sense of Naselaris et al., which predicts brain responses from stimuli.
  • Line 208, concept bank and grouping. The review does not say which step is unclear. Appendix A.5 gives the full procedure step by step, as does our response to Reviewer Dbae (Q5), and we would be glad to clarify any specific step.

[W2] Figures

Figure 1 is now a vector graphic, and each figure caption now states what is plotted and what it shows.

[Q1 + W3] Why decode captions from the brain

Brain captioning is a widely studied task. A caption describes a visual experience as people do: one open-vocabulary sentence names the objects, scene, and how they relate, which category labels cannot. Decoding such multi-component mental content is a central challenge in neuroscience [1]. Models trained with natural-language supervision better predict high-level visual cortex [2], making language a natural target for decoding it. Non-invasive language decoders are pursued for communication, and that of [3] also described events in silent videos.

The field is accordingly active (lines 39–41): Table 1 compares with 8 brain captioners from 2023 to 2026, including work at NeurIPS, ECCV, ICML, and ICLR. Our contribution is to make such decoders testable, so one can verify which content of a caption comes from the brain rather than the language model (Section 3.4).

This paper does not study cortical selectivity and makes no claim to add to its characterization. Section 3.5 uses that characterization as a reference to validate the decoder: perturbing the face, body, place and food regions selectively disrupts the paired channels and the paired groups' mentions in the captions.

[1] Horikawa T. (2025). Mind captioning: Evolving descriptive text of mental content from human brain activity. Science advances.

[2] Wang, A. Y., Kay, K., Naselaris, T., Tarr, M. J., & Wehbe, L. (2023). Better models of human high-level visual cortex emerge from natural language supervision with a large and diverse dataset. Nature Machine Intelligence.

[3] Tang, J., LeBel, A., Jain, S., & Huth, A. G. (2023). Semantic reconstruction of continuous language from non-invasive brain recordings. Nature Neuroscience.


[Q2] Why study a decoder

Learned models are human constructs, and making their representations interpretable is a core topic at ICLR; our contribution is of this kind. MindLens makes the soft tokens of a brain decoder readable without a probe and causally testable, with two scalar parameters and no change to the language model. When decoders are used to study the brain, this separates what the brain supplies from what the language model supplies.

If you have a specific concern about the soundness of the method or the results, we would be glad to address it.

I maintain a strong recommendation for rejection.

I have serious concerns about the novelty, validity, or scientific value of this work.

The authors state Brain captioning is a widely studied task, but ultimately this paragraph does not convince me. For one, the NMI Wang paper does not actually study captioning, and in fact does not study decoding at all. They study encoding models under a linear projection, and the interpretability that can come from a linear parameterization. They derive concrete scientific conclusions, namely the partition of higher visual cortex & early visual cortex, and derive the localization of food selectivity via PCA factorization of those weights (Fig S12 in the biorxiv version of the work). I see no such similar scientific conclusions from this work. Nothing that advances our knowledge of either the brain or more broadly to AI.

The Tang work, focuses significantly on imagined speech. This has obvious applications in certain patient populations (for example ALS, which may lose their ability to verbalize or move). The current ICLR submission focuses on captions from a fast-event based fMRI task, it is not clear to me that decoding these captions has any scientific or medical value.

So given the lack of scientific insight related to the brain, and the lack of potential real-world patient applications, we must ask what do we learn related to brain decoders? Again, I would argue nothing. Clearly given prior works on functionally selective regions, we know that broad categories can be decoded via linear probes. Given a high quality event based task, you can decode broad categories using a linear model. In fact, NSD actually includes these functional localizer masks (faces, bodies, places, words). So ultimately, this work is creating a construct of unknown scientific value, and does not explicate why studying this leads to any scientific insight, medical insight, or engineering insight.

In my view this work is a clear rejection.

Minor issue:

  • Fig 1 is still raster in the latest revision of this work

1. What we mean by "widely studied"

By "widely studied", we do not mean that brain captioning is a mature field with established applications, concrete systems or well-defined branches. We mean that it is an active research frontier of AI for neuroscience: Table 1 compares eight brain captioners from 2023 to 2026, including work at ICLR, NeurIPS, ICML, ECCV, ICCV and CVPR, all evaluated on the same NSD benchmark as ours. As at any frontier, basic questions remain open, including the one the review lists as its first strength: what the brain supplies to the decoded output. Our paper addresses this question.

We did not present Wang et al. [2] as a captioning or decoding study. We cited it for one finding: models trained with natural-language supervision better predict high-level visual cortex, which makes language a natural target for decoding it. Its conclusions concern cortex; ours concern decoders (point 3). We cited Tang et al. [3] because, besides perceived and imagined speech, their decoder described silent videos, decoding language from visual input as in our setting.

2. The potential of brain captioning

The communication goal that the follow-up credits to [3] is also pursued with visual stimuli. Horikawa [1] generates descriptions of viewed and recalled videos and presents this as a potential communication pathway for people with aphasia. Tang and Huth [4] transfer semantic decoders to new participants using only their responses to silent movies, because people with impaired language production often also have impaired comprehension.

The full value of a technology is rarely clear at the frontier stage. The internet began as a network linking a few research sites, mobile phones as heavy and costly devices, and chatbots as research demonstrations; few foresaw that they would be everywhere. A mature brain captioner could likewise go beyond communication:

  • Experiences that cannot be reported. Decoders trained on responses to viewed images already predict the content of dreams at sleep onset as object categories [5]; a captioner trained the same way could describe it in sentences. One in four behaviorally unresponsive patients with disorders of consciousness performs cognitive tasks detectable by fMRI or EEG [6]; a captioner could go beyond detecting such covert awareness to describing what these patients perceive.
  • A tool for neuroscience. An open-vocabulary readout lets researchers test what a brain region contributes to an experience, as Section 4.5 does by perturbing category-selective regions.

We claim none of these uses for our model. All of them, however, require what our paper provides: a decoder built on a language model can produce a fluent sentence whatever the brain supplies, so one must be able to verify which content comes from the brain.

3. What we learn about brain decoders

We never deny that broad categories can be decoded with linear probes, and the paper starts from this (Section 4.3). Category decoding, however, is like answering true/false or multiple-choice questions from a fixed list. A caption is an open-ended answer: it draws on an open vocabulary and can describe any object, action, or scene, and how they relate, in flexible and varied ways. Moreover, a probe shows what is present, not what the decoder uses: in a standard brain captioner, a probe on almost any single token recovers the concepts, yet removing the most informative token barely changes the caption. MindLens makes designated tokens readable without a probe, and removing one selectively lowers mentions of its concepts, at no cost in caption quality (Sections 4.2–4.4). We use the localizer regions and the follow-up mentions as ground truth to verify these channels (Section 4.5), not as a new finding.

4. Novelty and validity

The follow-up states concerns about novelty and validity but names none. If the reviewer knows of prior work that makes the soft tokens of a brain decoder readable without a probe and causally testable, or sees an error in our experiments or statistics, we will address it directly.

5. Figure 1

Figure 1 is a vector graphic in the revision uploaded on 18 November; only the photograph and the small icons in it are bitmaps. Our previous comment preceded this upload.

[1]–[3] are as in our previous comment.

[4] Tang J., & Huth A. G. (2025). Semantic language decoding across participants and stimulus modalities. Current Biology.

[5] Horikawa T., et al. (2013). Neural decoding of visual imagery during sleep. Science.

[6] Bodien Y. G., et al. (2024). Cognitive motor dissociation in disorders of consciousness. New England Journal of Medicine.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.