acceptodds
Under review as a conference paper at ICLR 2027

Super Neurons Answer Before VLMs Speak

Abstract

Vector-level representations in pretrained vision-language models can be repurposed for prediction, but how small can these representations be? We show that direct scalar readouts from a handful of neural units (e.g. 10) in the LLMs of VLMs can outperform the full, billion-parameter model from which they are selected on categorical visual question-answering tasks. Notably, these super neurons can be identified in a gradient-free setting, using a single GPU. We thoroughly analyze super neurons and find that they satisfy key properties of effective predictors: they do not seem to exploit spurious biases and generalize well. Therefore, they enable diverse applications. Since they are discovered during prefill encoding, their predictions are able to bypass the autoregressive generation process of the LLM, dramatically reducing runtime by up to 27.8× on Qwen3-VL-8b-Instruct. Since they are sparse, they enable lightweight finetuning, preserving some of the VLM’s general capabilities while improving its performance on a target task. Our code will be released upon acceptance.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Vision-language model, Pretrained Representations.

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_JzErOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper picks 10 hidden-state units ("super neurons", SNs) inside a VLM using 3,000 labeled examples, and reads answers from their signed average before decoding. On seven yes/no VQA datasets, SNs beat the zero-shot model by 2.8 average F1 points on LLaVA-v1.5-7b and 16.0 on Qwen3-VL-8b, and guide sparse finetuning.

Critical strengths
  1. A very simple, gradient-free readout of 10 units gives a real gain on balanced POPE. LLaVA's SNs raise both precision and recall there (Table 12), and gains hold at every Qwen3-VL size from 2b to 32b (Table 14).

  2. The method is cheap and easy to inspect. Selection usually takes under 30 minutes on one GPU, and the POPE gains hold across five probing sets (Fig. 7).

Critical weaknesses
  1. The headline comparison seems uneven, and this is the main reason for my rating. SNs are fit on 3,000 balanced labeled examples, but the base model is zero-shot and the other baselines see far fewer examples (App. A.3). With no dense linear probe or re-tuned threshold on the base model's yes/no score, the gain may mostly reflect the 3,000 labels, not "underutilized" signal.

  2. Much of the gain seems to come from avoiding the base models' answer bias. For Qwen, about 90% of the average F1 gain comes from both InstaOrder splits and CLEVR, where Qwen mostly gives the same answer (Table 13). For LLaVA, average accuracy rises 14.0 points but F1 only 2.8, which matches a fix of its habit of answering yes too often (Table 15a).

  3. The step beyond earlier work seems small. SNs act like a sparse probe with plus-or-minus-one weights, a modest step from sparse attention vectors (SAVs, Mitra et al. [1]). Skill neurons (Wang et al. [2]) and sparse probing (Gurnee et al. [3]) are also close, but the paper cites them without comparing.

  4. The claim that SNs challenge the view that VLMs lack 3D understanding seems overstated for real images. On InstaOrder (Occ.), Qwen's SNs get 33.7 F1, not far above the 25.7 of a 3-shot run that answers yes nearly every time (Table 11). Their 68.4% accuracy is also below the base model's 84.6%, which comes from mostly answering no.

  5. The 27.8× speedup seems to come mostly from the decoding setup. Qwen generates up to 128 tokens with no stop strings, averaging 80.72 tokens per POPE question (Table 4). Reading its first-token yes/no logits, as App. E already does for confidence, would need only the same prefill as SNs.

  6. The generalization and finetuning claims rest on narrow tests, such as one POPE-to-VOC transfer (Table 2) and one finetuning task on LLaVA. The LoRA and SFT baselines keep default settings (App. A.4), so their collapse on MM-Vet may not be a fair contrast. In Table 5, 31.1% accuracy and 42.0 F1 on InstaOrder (Occ.) do not seem possible together at its yes rate of about 15%.


References:

[1] Chancharik Mitra et al. Enhancing few-shot vision-language classification with large multimodal model features. ICCV, 2025.

[2] Xiaozhi Wang et al. Finding skill neurons in pre-trained transformer-based language models. EMNLP, 2022.

[3] Wes Gurnee et al. Finding neurons in a haystack: Case studies with sparse probing. TMLR, 2023.

Questions for authors

Questions:

  1. On the same 3,000 samples, how do SNs compare with a dense logistic-regression probe, SAVs [1], and the base model's yes/no margin under a re-tuned threshold? If SNs beat the threshold baseline by a good margin and come close to the dense probe, I would raise my rating.
  2. Could you report each validation set's yes rate, a balanced metric such as AUROC, and standard multiple-choice accuracy on A-OKVQA and ScienceQA? Gains that hold under these would support the spatial and multiple-choice claims.
  3. Does SN selection work as well after each layer's hidden states are rotated at random? A large drop would show that single units matter and would raise my view of the novelty.
  4. Can SN inference stop at the deepest selected layer, and how much time does that save over reading the base model's first-token logits? A real saving would make the speedup an SN contribution.

Suggestions:

  1. The shuffle test (Fig. 3) shows SNs use both image and text, but a test on subtler biases like object co-occurrence would say more.
  2. Reporting spread over several probing sets for all seven datasets, not only POPE, would make the smaller gains easier to judge.
4 comments
AuthorsOpenReview ↗

Rebuttal reply 1/2

Dear reviewer JzEr,

thank you for your informative and thoughtful comments on our super neurons manuscript. We are pleased that you found our approach simple and efficient.

We updated the manuscript with the detailed answers to your questions. We colored the amended text in orange for you. We nonetheless concisely reply to your questions below.

1. Comparison with more baselines

Following your suggestion, we compare our super neurons to more baselines, notably:

1.1. Logit classification from the calibrated LLM last layer's hidden state

The macro-average confirms that super neurons, even when k=1 outperform the baseline by 3.0 AUROC score on average for LLaVA-v1.5-7b and 1.4 AUROC score for Qwen3-VL-8b.

We provide all the details in sec. C.6 (Fig. 10 for the AUROC macro-average) of the appendices and detail how we construct that baseline in appendix A.3, L. 1031.

1.2. Sparse probing

On the mean-diff. variant, which is most competitive, super neurons are on average 0.71 AUROC score ahead for LLaVA-v1.5-7b and 0.84 AUROC score ahead for Qwen3-VL-8b at k=100k=100.

We invite you to refer to section C.6 of the appendices, where we analyze these baselines.

1.3. Traditional linear probing

This baseline mostly serves as an upper bound since it leverages many learnable parameters, while ours is completely gradient-free. None of the assessed baselines are able to outperform it, yet, ours comes the closest. Please refer to appendix C.6 for more information.

1.4. Budget-aligned SAV baseline

We re-evaluate SAVs under the same sample budget as our super neurons (using NN=3000).

In supplementary C.5 (most specifically, on Fig. 10), we show that super neurons outperform AUROC scores of SAVs most of the time, despite only leveraging a handful of scalars. Moreover, SAV relies on orders of magnitude larger representations since it uses tensor clustering.

1.5. Main manuscript

To help reader understand how we position against all baselines, we report the macro-average of super neurons against SAVs, sparse probing, linear probing and the classification from the logits of the model in Fig. 8 of the main manuscript and comment it on L. 423.

2. More statistics

We report the ROC curve and the AUROC score of the base model's logits and our super neurons on all datasets.

We visually observe that AUROC of super neurons is at or above the first-token logit margin of the VLMs. We comment these figures in details in supplementary materials C.11 (Fig. 14). Moreover, we measured the standard multiple-choice accuracy on A-OKVQA and ScienceQA. We find that super neurons match or slightly exceed the model’s own answer in accuracy (+0.0 to +0.5).

3. Super neurons extraction under hidden states rotations

Following your request, we select the super neurons after hidden states are rotated at random. At evaluation, we observe a sharp performance degradation in the performance of the probed units on all the datasets, for both models. We agree with the reviewer that this large drop shows that single units matter. You can find the results of this experiment in appendix C.13 (table 18).

AuthorsOpenReview ↗

Rebuttal reply 2/2

4. Efficient inference using super neurons

Reviewer JzEr is right in thinking that super neurons can be stopped right at the deepest selected layer.

To measure this, we constrain the search of super neurons up until a given layer and probe the units up until that layer. We find that we can significantly reduce the runtime of the model by exiting inference at that given layer, while still staying within 0.5 AUROC of full depth super neurons. This provides substantial gains of 1.38× on Qwen and 1.79× on LLaVA, averaged over 7 datasets. We show a case for Pope in C.14 (fig. 17).

5. Spread of the gains

Following your suggestion, we compute the full sample ablation over a number of probing samples from 10 to 3000 on 5 randomly sampled probing sets and report the 95% confidence interval on these runs. This helps assessing the variation in results.

We observe that super neurons outperform the model most of the time when probed at 100 samples. At 3000 samples, the confidence interval tightens around our reported measures. You can find the details results for both models on each datasets in appendix C.12 (figs 15 and 16).

6. MCQ results

Following your suggestion, we report the results of the standard MCQ evaluation, where we feed the complete prompt to the VLM.

We also design a simple approach to make super neurons work in this setting. Assuming CC (ex. If the MCQ contains A, B, C, and D, then C=4C=4) possible classes in the dataset, we compute CC sets of super neurons, each responsible for answering that particular letter.

Results in table 13 and analyses in C.3 highlight that super neurons are still on par or better than the base model (L. 1236). This substantiates our claims that super neurons are effective on MCQ data.

We apologize for not covering the co-occurrence experiment, but due to the high volume of experiments requested for this rebuttal, we preferred prioritizing more critical experiments (such as the comparison to other baselines), which were also reported by other reviewers. We will try our best to update the rebuttal before the deadline to include this experiment, otherwise, we will include it in the final version of the manuscript.

We would like to thank reviewer JzEr once again for their thoughtful and useful feedback. We believe that their comments were instrumental to improve the quality of the manuscript.

Thanks for running the extra experiments. On the same 3K samples, SNs now outperform the re-thresholded yes/no logits on both models and come close to the dense probe. That addresses my main concern, so I’m raising my score from 2 to 3.

I’d still soften the claims about 3D understanding and MCQ performance. Re-thresholding alone seems to explain about 60–70% of the gain over the zero-shot model. The clearest improvements are on InstaOrder and a few LLaVA datasets, while the standard MCQ gains are only 0.0–0.5 points.

The 27.8× speedup in the abstract is measured against full autoregressive decoding (Table 4), while Fig. 17 shows 1.5–2.7× over first-token logits, so I’d mention the latter as well. I’d also double-check Table 18. In 8 of the 14 rows, the rotated SNs predict "no" for almost every sample, and the A-OKVQA numbers match exactly across models.

Finally, lines 129–131 should be corrected: skill neurons also use each neuron’s mean activation as a threshold and keep the better-performing direction.

AuthorsOpenReview ↗

Final details fixes. Thank you

Dear reviewer JzEr,

Thank you for your prompt reply, we appreciate that we could mitigate the concerns you had.

Softening the claims about 3D understanding and standard MCQ. We understand your concern, we have updated the PDF to reflect these softened claims.

27.8x speedup gain. We clarify here that these gains come from the comparison between the normal autoregressive inference of Qwen3-VL-8b and its super neurons on Pope. You are right in pointing out that Fig. 17 computes the runtime of super neurons against the calibrated yes/no logit classification baseline. This is to provide a more challenging baseline.

We have updated the abstract to reflect both these settings.

Rotation experiments. Thank you for pointing this out to us, we re-ran the rotation experiment under 10 random seeds and double checked the A-OKVQA results. We have updated table 18 with the correct results and the experimental description to reflect the seeding setup.

Skill neuron addition in the related works section. Thank you for this additional detail. We have added this suggestion in the manuscript.

We would like to thank you once again for the quality of your review and the promptness of your replies. We believe your suggestions made the manuscript more comprehensive and substantially stronger.

Authors of "Super Neurons Answer Before VLMs Speak".

Reviewer_pYawOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies whether a very small number of scalar activations inside the LLM component of a vision-language model (VLM) can serve as accurate task predictors. The authors propose a simple gradient-free probing method to identify “super neurons,” i.e., individual hidden units whose normalized activations correlate strongly with binary labels on a small probing set. At inference time, they aggregate the signed activations of the top k selected neurons (typically k=10k=10) to make predictions directly from prefill hidden states, without relying on the model’s autoregressive decoding process.

The paper evaluates this idea on several categorical visual question answering benchmarks, including POPE, InstaOrder, CLEVR, VizWiz, A-OKVQA, and ScienceQA, using LLaVA-v1.5-7b and Qwen3-VL-8b-Instruct. The main empirical claim is that as few as 10 selected neurons can outperform the full base model on these categorical tasks. The paper further reports ablations on the number of selected neurons and probing examples, analyses of causal influence and out-of-distribution transfer, and applications to extreme early exiting and sparse fine-tuning.

Critical strengths

1. A simple yet compelling approach to extracting predictive signals from individual neurons.

The paper presents an interesting empirical finding that a small number of individual neurons in pretrained VLMs can provide highly predictive signals for categorical visual question answering, sometimes outperforming the predictions of the full model. Unlike conventional probing methods that require training additional classifiers, the proposed approach directly uses normalized scalar activations without introducing learnable prediction heads. The simplicity of the method, together with its strong empirical performance, makes this a potentially valuable contribution to understanding and utilizing representations within VLMs. In particular, the results suggest that task-relevant information can be directly accessible from a surprisingly small number of individual units, even when the model's final predictions do not fully exploit this information.

2. Extensive empirical evaluation and practical applications beyond neuron identification.

The paper provides a relatively comprehensive empirical investigation across multiple categorical VQA benchmarks, model architectures, and model scales. The authors examine the effects of the number of selected neurons and probing samples, evaluate robustness through shuffled-image and shuffled-text controls, investigate out-of-distribution generalization, and conduct causal intervention experiments. These analyses strengthen the empirical evidence that the selected neurons capture meaningful predictive signals rather than merely fitting the probing examples. Furthermore, the paper demonstrates practical applications through prefill-only inference and sparse fine-tuning. In particular, the reported inference speedups and the preservation of general capabilities after sparse fine-tuning highlight the potential utility of the approach beyond the initial classification benchmarks.

Critical weaknesses

1. Insufficient comparisons with supervised probing baselines.

My main concern is that the paper's central performance claim is not sufficiently supported by comparisons under equivalent supervision. The proposed method identifies super neurons using up to 3,000 labeled probing examples per task, whereas the primary comparison is against the base VLM performing zero-shot autoregressive inference. Although the method is gradient-free and does not train an additional classifier, the selection of neurons and their prediction orientations is still explicitly supervised. Therefore, it is difficult to determine whether the reported improvements arise from a distinctive property of the selected neurons or simply from task-specific supervised feature selection.

The authors compare against few-shot prompting and sparse attention vectors (SAVs), which is useful, but these comparisons do not fully address this concern. In particular, comparisons with regularized linear probes and sparse linear probes trained on the same hidden states and labeled examples are missing. Such baselines are important for determining whether the identified super neurons provide advantages beyond standard supervised probing, especially given the paper's emphasis on their strong predictive performance. Without these comparisons, the empirical finding remains interesting, but the extent to which the proposed method offers a distinctive advantage over alternative readout strategies is unclear.

2. Limited experimental controls for the claimed advantages of super-neuron fine-tuning.

The fine-tuning experiments are promising, but the current comparisons make it difficult to attribute the observed improvements specifically to the selection of super neurons. According to Appendix A.4, the LoRA and full fine-tuning baselines use their default training configurations, whereas super-neuron fine-tuning uses a different batch size, learning rate, and optimization setup. Consequently, the substantial performance degradation observed for LoRA and full fine-tuning on POPE and MM-Vet may be influenced by differences in training configurations rather than reflecting an inherent limitation of these approaches.

Furthermore, the paper does not provide a direct control experiment in which an equivalent number of randomly selected neurons or parameters are fine-tuned under the same conditions. Such a comparison is particularly important for determining whether the reported improvements result from identifying task-relevant super neurons or simply from restricting the number of updated parameters. Without hyperparameter-matched and parameter-matched baselines, the evidence is insufficient to establish that super-neuron selection itself is responsible for the reported advantages in target-task performance and preservation of general capabilities.

Questions for authors

Q1. Fairness of the comparison and the role of supervised probing.

One of my primary concerns is whether the experimental comparisons sufficiently isolate the contribution of the proposed super neurons from the benefits of supervised task adaptation. The proposed method uses up to 3,000 labeled probing examples per task to identify neurons with low binary cross-entropy, determine their prediction orientations, and estimate the normalization statistics used during inference. In contrast, the principal performance comparisons are made against the original VLMs operating in a zero-shot autoregressive generation setting. Although the proposed procedure is gradient-free and does not require training an additional prediction head, it nevertheless uses task-specific supervision to construct a decision rule. Therefore, I am not convinced that the reported improvements over the original VLMs, by themselves, establish the distinctive predictive capabilities of the selected neurons.

The comparisons with few-shot prompting and sparse attention vectors (SAVs) are appreciated, but they do not fully resolve this concern. In particular, the paper does not establish how the proposed method compares with standard supervised readout approaches under an equivalent data budget. I would like the authors to provide comparisons with regularized linear probing, sparse linear probing, or other relevant feature-selection approaches using exactly the same probing examples and hidden states. It would also be useful to compare against a task-calibrated readout of the base model's output logits.

Importantly, I do not expect the proposed method to necessarily outperform every trained probing baseline, since its ability to operate without an additional learnable classifier is itself a meaningful distinction. However, a controlled comparison is necessary to understand the trade-off between predictive accuracy, supervision, readout complexity, and computational efficiency. Without such evidence, it remains unclear whether the reported performance improvements primarily reflect a special property of super neurons or a more general benefit of supervised feature selection from pretrained representations. I would appreciate a more thorough justification of the current evaluation protocol and additional evidence supporting the paper's central performance claims.

Q2. Distinctiveness, stability, and interpretation of super neurons.

The paper argues that a very small number of individual neurons contain highly predictive task-specific information. However, since the proposed method searches over a large number of candidate scalar activations and selects those with the lowest supervised BCE, I believe further evidence is needed to determine whether the identified neurons constitute genuinely distinctive computational units or simply highly predictive coordinates in a large representation space.

I acknowledge that the authors evaluate the method on held-out validation sets, examine different probing-set sizes, and report confidence intervals across multiple probing sets. These experiments provide useful evidence that the resulting predictions are not merely artifacts of fitting the probing examples. Nevertheless, prediction stability does not necessarily imply that the identities of the selected neurons are stable or that the corresponding information is uniquely localized to those units.

Could the authors provide an analysis of the overlap between selected neuron sets across different probing samples, as well as the stability of their layer-wise distributions and individual predictive performance? It would also be useful to compare against randomly selected neurons, alternative supervised feature-selection criteria, and potentially label-permutation controls. More fundamentally, the authors could clarify what distinguishes a super neuron from an ordinary highly predictive feature discovered through a large-scale search.

This distinction is important because the paper's broader interpretation concerns the locality and sparsity of task-relevant information in pretrained VLMs. I would like the authors to substantiate this interpretation more carefully and distinguish the empirical observation of highly predictive scalar coordinates from the stronger claim that the model contains identifiable task-specific expert neurons.

Q3. Attribution and fairness of the sparse fine-tuning results.

The sparse fine-tuning results are among the most practically interesting findings of the paper, particularly the reported improvements on the target task while maintaining relatively strong performance on POPE and MM-Vet. However, I have significant concerns about whether the current experiments adequately isolate the contribution of super-neuron selection from the effects of restricting the number of trainable parameters and using different optimization configurations.

According to Appendix A.4, the LoRA and full fine-tuning baselines use their respective default training configurations, whereas super-neuron fine-tuning employs a different batch size, learning rate, and optimization setup. Given the substantial performance degradation reported for LoRA and full fine-tuning on the general-purpose evaluation benchmarks, it is difficult to determine whether these results demonstrate an inherent advantage of the proposed method or are partly attributable to suboptimal training configurations for the baselines. The fact that all methods use the same training data and number of epochs does not necessarily make the comparison equivalent.

I would therefore ask the authors to provide more carefully controlled comparisons, ideally including hyperparameter-tuned LoRA and full fine-tuning baselines under comparable training budgets. More importantly, an experiment that fine-tunes an equivalent number of randomly selected neurons or parameters under exactly the same optimization conditions would be particularly informative. Such a control would directly test whether identifying super neurons is essential for the reported improvements, rather than the improvements arising primarily from sparse parameter updates.

In addition, the paper should clarify the exact trainable parameter count and parameter-selection procedure, especially given the statement that the selected units include contributions from the residual stream. Without these controls and implementation details, I find it difficult to attribute the observed improvements in target-task F1 and retention of general capabilities specifically to the proposed super-neuron selection strategy. Since sparse fine-tuning is presented as an important application of the method, I believe this issue requires a more substantial explanation and stronger experimental support.

Q4. Interpretation of inference efficiency and the reported speedups.

The reported inference speedups of 6.35× and 27.8× are impressive and represent a potentially important practical benefit. However, I believe the current evaluation does not sufficiently distinguish the efficiency gained specifically from super-neuron readout from the more general benefit of avoiding autoregressive generation on categorical prediction tasks.

As shown in Table 4, the prefill computation remains unchanged, while the decoding stage is entirely removed. In particular, the Qwen3-VL-8B baseline generates an average of 80.72 tokens, despite the underlying evaluation being formulated as a binary classification problem. Consequently, a substantial portion of the reported speedup may arise from comparing prefill-only classification against relatively lengthy autoregressive responses, rather than from an efficiency advantage unique to the proposed neuron-selection mechanism.

Could the authors compare the method against stronger inference-time baselines, such as classification using the first-token logits, direct comparison of the logits corresponding to candidate answers, or constrained yes/no decoding? These approaches would provide a more appropriate comparison for determining the additional latency reduction achieved by reading out super neurons. Ideally, the comparison should report both prediction accuracy and end-to-end latency under the same inference conditions.

I agree that eliminating autoregressive decoding is a legitimate and useful advantage of the proposed approach. My concern is not that the reported speedup is incorrect, but rather that its practical significance should be established relative to efficient alternatives designed for the same categorical prediction setting. A more carefully controlled evaluation would help clarify the extent to which the proposed method offers a distinctive efficiency advantage.

Q5. Causal influence versus localization and necessity of task-relevant information.

The causal intervention experiments provide useful evidence that neurons with lower probing BCE tend to exert a stronger influence on the model's output logits. In particular, the reported relationship between predictive performance and changes in the yes/no logit margin is an interesting observation that strengthens the paper beyond a purely correlational probing study. Nevertheless, I believe the interpretation of these results requires further clarification.

The current experiments demonstrate that perturbing selected neuron activations can influence the model's output. However, a strong causal effect under a particular intervention does not necessarily establish that the corresponding neuron is necessary for the model's normal predictions, nor does it demonstrate that the relevant information is exclusively or uniquely localized to that neuron. The same task-relevant information could potentially be represented redundantly across multiple neurons, with the selected units serving as particularly accessible or influential readout coordinates.

Could the authors provide additional evidence through neuron ablation or mean-replacement experiments, measuring the effect of removing the selected neurons on the base model's prediction accuracy? Comparisons against randomly selected neurons or similarly activated but less predictive neurons would help determine whether the identified units play a distinctive causal role. It would also be useful to examine whether the remaining representation can still support accurate predictions after the selected super neurons are ablated.

I would appreciate a clearer distinction between predictive sufficiency, causal influence, and causal necessity in the discussion. These are related but different properties, and the current evidence appears strongest for predictive sufficiency and intervention-based influence. Since the paper draws broader conclusions about the locality of task-relevant representations, I believe a more careful interpretation or additional experimental validation is necessary to substantiate those claims.

Q6. Consistency between the mathematical formulation and the proposed algorithm.

I noticed a potential inconsistency between the mathematical formulation of super-neuron selection in Equation (5) and its implementation in Algorithm 1. Equation (5) appears to formulate the selection of kk neurons as a constrained optimization problem that minimizes the probing loss while enforcing pairwise correlation constraints. However, Algorithm 1 implements a greedy procedure that sequentially selects neurons in ascending order of BCE, discarding candidates that violate the correlation threshold with previously selected neurons. In general, this greedy procedure does not guarantee a globally optimal solution to the corresponding constrained subset-selection problem and may not always return exactly kk neurons.

Could the authors clarify whether Equation (5) is intended to define a global optimization objective or merely describe the greedy selection heuristic? If the latter, I believe the formulation should be revised to reflect the actual algorithm more precisely. It would also be useful to explain whether the authors have encountered cases where the correlation constraints prevent the selection of kk neurons, and how such cases are handled in practice.

Additionally, Algorithm 1, Line 4 appears to omit the necessary broadcasting or expansion of the ground-truth labels when computing BCE over tensors of shape [2,N,LD][2,N,LD]. Although this is likely a pseudocode simplification, clarifying the intended tensor operations would improve the reproducibility of the method.

2 comments
AuthorsOpenReview ↗

Rebuttal reply 1/2

Dear reviewer pYaw,

We appreciate your thoughtful and comprehensive comment related to our submission. We are glad that your interest stimulated numerous of relevant questions.

We amended the manuscript with your suggested changes in light blue colored text.

Q1. Additional baselines for Super Neurons

We thank reviewer pYaw for suggesting additional baselines, this allows to further frame our contribution's performances against them.

Notably, we now compare super neurons to linear probing, classification on the model's output logits and sparse probing on the same 3,000 probing samples and hidden states for a fair benchmark (macro-average on all the datasets in Fig. 8 and an analysis in L.423 of the main paper. Per-dataset analyses are in appendix C.6, L. 1373, and Figs. 10, 11 and 12).

As reviewer pYaw expected, all the neuron-extraction approaches do not perform as well as standard linear probing since this is a gradient approach that leverages a larger amount of parameters and a larger amount of input features for its learning. On the other hand, we recall that our approach does not rely gradients at all. This comes with many advantages that we have also now detailed in our response to reviewer t2xQ. Nonetheless, our approach outperforms the model's output logits most of the time positively compares to sparse probing.

We also obtain the ROC curves of Super Neurons against the base model (Sec. C.11 in the appendices and Sec. C.6 with Fig. 14 and 10 for comparison to other baselines). Our Super Neurons show a larger area under the curve, in almost all cases, indicating a better predictability than the base model. We note an interesting result on InstaOrder (Occ.) in which LLaVA has near-chance discrimination (0.52 AUROC score) whereas our Super Neurons significantly improve the prediction capability.

We thank reviewer pYaw for giving us the opportunity to better frame our baselines. We believe that these positive results help strengthen our approach.

Q2. Distinctiveness, stability, and interpretation of super neurons

We agree with reviewer pYaw that the stability of super neurons is an interesting and important question.

We followed the reviewer's suggestion and computed the stability of super neurons probed on 5 different probing data, for all datasets except VizWiz. These probing sets consist of 3,000 randomly sampled data points, except for VizWiz which does not contain enough data.

Results are in figs. 24 and 25, with a discussion in Sec. E, L. 2006, in the appendices. One can see that super neurons are relatively stable across probing sets, with at least one neuron selected on 5/5 probing sets on all models and datasets except Clevr. As requested, we also report their individual respective performance in supplementary materials section C.10, Figs 22 and 23.

We originally performed image- and text-label permutation experiment in Figure 3 of the main paper (L.278), where we show that the performance of super neurons collapse under image permutation and text permutation. This seems to indicate that super neurons are not overfitting artifacts, or do not seem to rely on spurious relations from text or images.

Following the suggestion from another reviewer, reviewer JzEr, we also conducted an experiment in which we randomly rotate the hidden states before obtaining the super neurons (refer to C.13). In that setting too, we observe a large drop in performance on the validation set for all datasets, on all models. Such results continue to suggest that super neurons hold genuine predictive information related to their task.

Q3. Attribution and fairness of the sparse fine-tuning results

According to your suggestions, we re-ran the finetuning results and aligned the experimental settings for all our baselines (LR=2e-4, global batch size of 8, same 3,000 training samples). In this 3k samples setting, super neurons outperform the LoRA tuned and SFT model (Table 5 in the main paper).

Despite a larger learning rate for super neurons than in our original manuscript, we observe that super neurons are still the only method that allows to maintain a reasonable performance on MM-Vet (Table 6 of the main paper) and improve the spatial performances of the model (Table 19, in C.15).

Following your advice, we also ran finetuning on random units, still with the same experimental setting, and observe that they hurt the performances of the model on the target dataset (Table 5 of the main paper). We report the number of trained parameters as well as the selection procedure of the finetuned weights for SN-finetuning in more details in appendix A.4. (L. 1049).

AuthorsOpenReview ↗

Rebuttal reply 2/2

Q4. Interpretation of inference efficiency and the reported speedups

We agree with you that the inference speed claims can be assessed in a tighter way.

Following your suggestion and reviewer JzEr's we constrain the super neuron search to a given layer and run inference up until that layer. We compare it to classification from the first generated logits of the model in Fig. 17.

Not only super neurons still provide appealing runtime improvements under this setting, they are also able to outperform the baseline in starting from the middle layers of the LLM. We provide more explanations and discussions in appendix C.14, and visualizations in Fig. 17.

Q5. Causal influence versus localization and necessity of task-relevant information

As clarified in our revision, we frame Super Neurons through predictive sufficiency and causal influence rather than exclusive causal necessity.

We also follow reviewer JzEr’s suggestions and provide additional experiments by comparing super neurons against random neuron sets in appendix D (table 20 for the sign flipping experiment and table 21 for the signal amplification experiment). When activations of super neurons at τ=1.0\tau = 1.0 are flipped, the resulting logit difference is above 9.3 standard deviations compared to the random sets. Moreover, if we amplify the signals of the τ=1.0\tau=1.0 set by z=50z=50 towards the correct direction, the model’s accuracy for Qwen Pope increases from 92.8% to 99.8%. If we amplify them in the incorrect direction, the model’s accuracy decreases from 92.8% to 32.2%.

This confirms that these specific units serve as highly concentrated, causally influential readout coordinates.

Q6. Consistency between the mathematical formulation and the proposed algorithm

We thank reviewer pYaw for catching this inconsistency. We would like to clarify that Eq. (5) was meant to describe the greedy procedure, not a global optimum.

We replace it with the sequential definition: s1=arg min⁡ℓ;sj=arg min⁡ℓs_1 = \argmin \ell; s_j = \argmin \ell over units whose ∣corr∣|\text{corr}| with every previously selected unit is below τ\tau, for j≤kj \le k. Selection stops at kk or when no candidate remains, in which case fewer than kk units are used. An early stopping case where fewer than kk neurons survive the decorrelation algorithm doesn't happen in our current setting at τ=0.7\tau=0.7. Line 4 of algorithm 1 now broadcasts the labels to [2,N,LD][2, N, LD].

We hope that our answers will help clarifying our Super Neurons' effectiveness, novelty and positioning against more baselines. We believe that your feedback was critical in improving the quality and the positioning of our manuscript.

Thank you.

Reviewer_t2xQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes “super neurons”: by using a small number of labeled samples, approximately 10 of the most predictive neurons are selected from the hidden states within the vision-language model, and their values are averaged to answer “yes/no” type questions. The authors verified on seven visual question-answering datasets using LLaVA and Qwen3-VL, and the results showed that it outperformed the zero-shot original model, few-shot prompts, and SAVs in most tasks. In addition, the authors also demonstrated that merely fine-tuning the relevant parameters can enhance the target task while minimizing the damage to the model’s general capabilities.

Critical strengths

1.The mathematical expression of the method is clear and complete, accompanied by pseudocode, which is convenient for readers to understand and reproduce.

2.The paper as a whole is easy to understand, with a simple and intuitive method and clear illustrations. Readers can quickly grasp the core idea.

Critical weaknesses

1.Limited novelty. The idea of “picking out a few neurons from the hidden states for prediction” has emerged in existing works such as sparse probing and skill neurons.

2.The scale of the experiment and the robustness of the conclusion are insufficient. The paper only examined the impact of sample size on a single dataset, and the performance tended to be saturated at approximately 3,000 samples. It remains unclear whether this method still has an advantage over supervised methods such as fine-tuning when more labeled data is available. Furthermore, in some tasks, the performance of super neurons is even inferior to that of the original zero-shot model.

3.Limited scope of application. Only “yes/no” type questions can be answered. Multiple-choice questions need to be split, and the method cannot be directly applied to open-ended tasks. Moreover, for each new task, new labeled data are required to select neurons.

Questions for authors

What is the core innovation of this paper compared with existing methods such as sparse probing and skill neurons? When more labeled data is available, does this method still have an advantage, and can it be extended to open-ended tasks?

2 comments
AuthorsOpenReview ↗

Rebuttal reply 1/2

Dear reviewer t2xQ,

We first would like to thank you for recognizing the clarity and completeness of our paper. Your feedback enables us to specify details of our approach, especially related to its novelty.

We amended the manuscript to take into consideration the feedback you kindly provided. The green colored text describes our modifications based on your requests and advice.

1. Novelty of our approach

We specifically compare our methods to skill neurons and sparse probing. We clarified some of these differences in the related work section (L. 125) of the manuscript.

1.1. Multimodality

In both Skill neurons (Wang et al., 2022) and sparse probing, units are found on unimodal LLMs trained purely for NLP. We emphasize that while it would make sense to observe strong individual units on NLP tasks from an NLP model, it is not straight forward that super neurons appear in VLMs since the vision encoder and LLMs were pre-trained separately and only lightly adapted via a linear projection alignment stage. Yet, we show that super neurons exist in this situation too.

1.2. Scale

We show that super neurons appear from 2b to 32b parameters models, a larger scale than Wang et al., 2022 and Gurnee et al., 2023. We therefore provide evidence that these units emerge at large scales (Fig. 5, main paper).

1.3. Gradient-free approach

Skill neurons (Wang et al., 2022) are identified after prompt-tuning RoBERTa-base on each task.

Sparse probing, from Gurnee et al., require training probes iteratively until converging to the top-kk units. On the other hand, super neurons are readouts from a frozen VLM's prefill, using only per-unit statistics from a small labeled probing set, meaning no gradients. The gradient-free nature of our work also removes the need to tune optimizer-related hyperparameters. Both Wang et al. and Gurnee et al. require hyperparameters for training (optimizer, learning rate, initialization of the modules, etc.).

On the other hand, super neurons only require τ\tau and kk, for both of which we provide robust defaults (τ=0.7\tau=0.7, and kk provides better and better results as it scales to 100100).

1.4. Decorrelation Algorithm

Sparse probing (Gurnee et al., 2023) evaluates the selected neurons through a trained logistic probe. On the other hand, Super neurons are aggregated by an unweighted signed mean, which works the best if the selected units are not redundant.

We therefore propose a decorrelation algorithm which differs from both skills neurons and sparse probing. Contrary to Wang et al. and Gurnee et al., our approach directly reads the prediction from the activations, without any reliance on a learned module which could obfuscate the true underlying signal of the neuron (the soft prompt from Wang et al., and the logistic probe from Gurnee et al.).

1.5. Evidence

On the same hidden states and probing samples, under the equal-weight readout, SN selection outperforms mean-difference and adaptive-thresholding selection at k=100k=100 (7-dataset macro F1: Qwen3-VL 80.3 vs 77.9/78.3; LLaVA 64.6 vs 63.4/63.3). We provide a macro-average of AUROC score in fig. 10, in the appendices, where our method scores equals sparse probing at k=10k=10 and outperforms all baselines at k=100k=100.

Our contribution is therefore not a better sparse probe, but showing that a handful of units can be read out without any learned readout.

2. More labeled data

On Pope, SN performance saturates around N=3,000N=3,000 (Fig. 9, L. 457). We also report the results of the sample ablation for both models on all datasets (Fig. 15 and 16, with a discussion in supplementary C.12), where we observe similar trends as for Pope. With the same data, a dense logistic probe is stronger; we now present this explicitly as a trade-off between accuracy and readout complexity (100 units, no learned weights).

We would like to emphasize that super neurons perform worse than the model only for LLaVA's F1 on Clevr and InstaOrder (Occ.) on Qwen (Tab. 11 and 12 in the appendices). While we do acknowledge that we do not consistently outperform the model baseline, we still believe that super neurons obtain strong results in the majority of the cases, we remind the reviewers that these results were only obtained using k=10k=10 hidden state scalars.

We report that gains for super neurons compare the best at the 3k sample range, where we even outperform LoRA and SFT under the exact same experimental setup (table 5 of the main manuscript). We do not claim that super neurons outperform LoRA and SFT at larger scales, in which cases they are better alternatives. However, we believe that the low-data regime setting along with the capabilities of super neurons to maintain general capabilities of the network to a reasonable level make it an appealing alternative at that scale.

AuthorsOpenReview ↗

Rebuttal reply 2/2

3. Applications

We clearly define the scope of our work as categorical prediction. We have added a sentence about this in the limitation paragraph, in the conclusion of our main manuscript (Sec. 5, L. 524).

While open-ended generation remains outside the immediate scope of scalar readouts, structured output generation is recently gaining traction (Almeida, 2026). In these workflows, models are prompted to make fast, deterministic categorical decisions before generating structured fields.

To substantiate our claims, we additionally perform an experiment where we compare super neurons' performance against the logits classification of the last layer of the LLM of the VLM, during prefill-encoding (Fig. 8 of the main paper and Figs. 11 and 12 in the appendices, along with an analysis in appendices C.6 and L.423 in the main manuscript). In this setting, super neurons outperform the base model. We therefore strongly believe that super neurons could have a strong impact on future research related to these large classification models.

Following your suggestion, we also compare to a standard MCQ evaluation, that is, when the model is being fed the full questions all at once (Appendix C.3, table 13). We construct a super neuron baseline that aims at matching this baseline settings. We obtain on par or better performance than the VLM, even in this setting (L. 1236, supplementary materials C.3)

Because our multi-class formulation (Appendix C.3, Table 13) computes independent Super Neuron sets per discrete option directly during the prefill stage, it naturally aligns with and can serve as a lightweight, zero-decoding-latency readout mechanism for structured output pipelines.

4. New labeled data required for new tasks

We agree with reviewer t2xQ that our approach requires new data for each task, however, that is also the case of all the baselines we evaluate (sparse probing, SAVs, finetuning approaches). Therefore, this is a recurrent limitation of this particular branch of research rather than a specific issue of our approach.

We hope that our answers were clear and helped you assessing more carefully distinctions with prior works.

We remain open for conversation should any of our reply require more explanations.

We thank you again for your time and thoughtful feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.