acceptodds
Under review as a conference paper at ICLR 2027

Neural Selectivity as a Compatibility Problem: From Latent Coordinates to Explicit Features

Abstract

Whether a neuron is called purely selective, linearly mixed, or nonlinearly mixed is usually decided under a feature set fixed in advance by the experimenter. This label belongs to the pair of neuron and description, not to the neuron alone. Information-preserving re-descriptions of the same inputs can turn nonlinear dependence into linear dependence. We therefore treat the feature description itself as the object of discovery, organized by neural compatibility, the degree to which a feature description supports prediction of a target neural population through a specified simple-readout class. We introduce compatibility-guided feature discovery. A shared population encoder learns a response-predictive representation, and a sparse autoencoder (SAE) provides candidate activation patterns. Language-model-assisted feature proposals and shallow decision trees turn these patterns into binary features defined by executable sensory and behavioral rules. These rule-defined features are selected for their contribution to linear neural prediction. Across 575 neurons from 22 mouse primary somatosensory cortex sessions, full-data discovery produced a descriptive catalog of 164 rule-defined features. Under four-fold nested cross-validation, with encoder training and feature discovery restricted to outer-training trials, adding fold-specific prediction-selected features to inputs already augmented with input-informed LLM features increased mean linear prediction from 0.1079 to 0.1254. On these recordings, previously analyzed for nonlinear sensory integration, the discovered features made part of the nonlinear predictive structure linearly accessible, reducing the nonlinear-over-linear gap by 54.2%. Even with trial time in the baseline, these features retained 92% of their original gain, which was 2.4 times the gain from random features matched in vocabulary, tree structure, and prevalence. Features discovered and fixed in 15 sessions also improved held-out linear prediction in each of seven excluded sessions after within-session readout calibration. Treating descriptions as discoverable turns a known caveat of selectivity analysis into a procedure for discovering and testing interpretable features.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Neural selectivity, Mixed selectivity, Neural compatibility, Feature discovery, Neural encoding, Sparse autoencoders, Large language models, Rule extraction, Interpretable machine learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_2cLAOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper suggests to train an encoding model with linear readout and then to use latent features shared across animals and neurons to train a sparse autoencoder. Than the features from sparse autoencoders are mapped into interpretable rules using some behavioral and stimuli information and an LLM.

Critical strengths
  1. Relevant problem - it is very true that in the classic neuroscience setup the feature descriptions and neuron selectivities are paired and neuronal selecties might not be linearly predictable because the feature description was not formulated precisely.
  2. Testing on heldout animals- testing on heldout animals is a nice way to test if the rules generalize.
  3. Empirical improvement - They show that in the selected datasets the new rules make the neuronal activity more linearly predictable.
Critical weaknesses
  1. Linearity assumption: Thought the text the paper emphasised that the responses should be linearly predictable from the encoder features, which is not exactly correct, since most of the models would use a non-linearity after a dot-product with the readout to avoid negative responses (as a living organism cannot have this)
  2. Error bars While the paper reports some confidence intervals across data bootstrapping in the text - fig 3 and 4 contain no error bars. SAE are known to not recover exactly the same concepts across several model refits (https://arxiv.org/abs/2501.16615) and the paper does not estimate the stability of SAE reconstructions apart from one sentence (lines 360-361).
  3. Clarity: Its only stated in the appendix what are the 318 input features are and its not clear for me how they were selected. Same for the amount of SAE bits - its not clear why its eight of them (I have not found an ablation in the appendix). Its also not clear for me how exactly the LLM proposed input-informed features - what was the specific propmt?
  4. AI-assisted diagrams require polishing - for example for Fig 1 - small details are missing, like in scalar features only one zz probably should be colored with green, no? Also it feels a bit overloaded - like why the contrasts are different in B but not in C and D? For Fig2A - the first panel implies that the 318 inputs are somehow transformed to neurons while in fact they are just paired (i dont think we can make a causal claim if neuronal signal or behavioural features come first).
  5. Related work missing - there are a few works analyzing neuroscience data with AI, which is missed in the related work a) https://arxiv.org/pdf/2605.12485 - summarises most exciting stimuli for V1 neurons using VMLs b) https://www.biorxiv.org/content/10.1101/2025.11.12.688086v3 claims to discover tuning laws for V1 neurons using LLMs
  6. Proof of concept example and bigger dataset - Since the paper misses the error bar and the improvement is noticeable but small - a paper would benefit a lot from a synthetic toy example as a validation / proof of concept for their pipeline (numerically simulated toy example). The current example would benefit from the estimated trial-to-trial noise ceiling - to contextualize the R^2 scores. I would also inspire the authors to repeat their pipeline on a bigger dataset since right now its ~550 neurons and ~160 rules, which is like 3 neurons per rule, so the rules might not be general enough.

Minor:

  • the choice of digits to be reported seems to be arbitrary - for example in line 376 4 digits are reported despite the 3rd one is already quite different
Questions for authors
  1. Linearity assumption: - In equation (1) the ϕn\phi_n function is a non-linearity, isnt it? Normally, this would be a non-linearity which would make the responses strictly non-negative. But then in equation (3) the ϕn\phi_n is dropped and responses y^n\hat{y}_n could be predicted negative which does not really happen in a living organism. Should there still be a non-linearity in eq. (3)?
  2. SAE stability evaluation: In lines 360-361 its said that 63 out of 164 rules got nice MCC scores across 20 SAE initializations. Just to make sure - were this the same rules or is it just at least 63 rules for each SAE initialization got MCC≥0.4MCC \geq 0.4. How similar are the rest 101 features across SAE initializations? (e.g. how many of the learnt features vary across initializations)? How stable is SAE across re-fits?
  3. Size-fold evaluation: Section 3.3 - Why do we need the 4-size evaluation fold? Where does size come from - isn't it cross-validation of ordering and amount of K needed? A schematics would help for this paragraph as its writing is confusing at least for me.
  4. SAE technicalities: In line 212 it says that "A sparse autoencoder (SAE) is then trained to reconstruct normalized hidden vectors." - why do we need to normalize them? In line 216 it says "We use 256 atoms and at most eight positive coefficients" - why only positive and not negative and select by the amplitude? neurons can be selectively suppressed by sth?
  5. Clarity 1: What is II in line 165 - mutual information?
  6. Clarity 2: Is formula 2 basically normalized MSE? like 1 - R^2?
  7. Clarity 3: what exactly is outer-training partition and inner-training partition? 15 and 7 other mice or what?
  8. Were LLM suggested featured needed?: For the LLM suggested features - how many of the LLM proposed features survived? what is the statistical significance of these? what are their semantic?
  9. Scalability: How scalable is this pipeline if I have much more base features to select from? What is the complexity in OO notation based on the amount of potential features for rule construction and rule selection?
  10. SAE dimensionality ablation: How has eight dimensionality for SAE been chosen? Have you run an ablation for more or less active features?
  11. Multicollinearity: Do I get it right that many of the 318 used for rule construction are strongly correlated? What is you would just remove strongly correlated features from the proposals?
9 comments
AuthorsOpenReview ↗

Response to Reviewer 2cLA (1/5)

We thank the reviewer for the careful reading. We have carried out additional analyses in response to the comments and report them below. The synthetic proof of concept and the application to additional data (W6) are in progress and will follow in a second response. The revised manuscript, with all changes, will be uploaded in a single update.

[W1] Linearity assumption

Response [W1/Q1]: Firing rates are nonnegative. Equation 3 predicts each neuron's standardized firing rate, so zero on this scale corresponds to that neuron's mean rate. A negative output predicts activity below the mean. Applying a nonnegative output function on this scale would prevent the model from predicting those decreases. We will state the target scale beside Eq. 3.

The gain from the discovered features also holds when the readout is constrained to be nonnegative or monotone. We replayed the held-out evaluation with a rectified readout on the firing-rate scale and with a monotone output function fitted without test data. Only 3.5% of the affine predictions fell below a firing rate of zero, and the gain remained under both readouts:

ReadoutBaseline R²With discovered featuresGain (95% CI)
Affine (paper)0.1080.1250.0175 (0.0157–0.0195)
Rectified, max(ŷ, 0)0.1100.1270.0167 (0.0153–0.0183)
Fitted monotone (isotonic)0.1130.1280.0150 (0.0137–0.0164)

The revision will report this analysis in Appendix C.6.

Equation 1 allows two sources of nonlinearity. The upstream transformation gng_n maps the measured inputs xx to latent features zn=gn(x)z_n=g_n(x). The monotone response function ϕn\phi_n then acts on a weighted sum of those features plus a bias. We will label the path from xx to zz in Fig. 1A with gng_n and explain in the caption that the figure uses an identity response function to show how the same response can be linear in one description and nonlinear in another.

Equation 3 puts linear compatibility (Eq. 2) to use. The encoder can learn nonlinear combinations of the inputs, and its affine readout asks how well a weighted sum plus a bias predicts the response from that representation. We use the same criterion to evaluate the sensory rules: we fit an affine readout to the original description and another to the description augmented with the discovered rules. The gain on held-out trials therefore tells us how much more of the S1 response a linear readout can explain once those sensory conditions are made explicit.

[W2] Error bars and SAE stability

Response [W2], error bars in Figures 3–4: We computed 95% confidence intervals for both figures and will add them in the revision, using the same session bootstrap as the CIs in the text (10,000 draws over 22 sessions, Appendix C.1).

  • Figure 3B–D: profile CIs are at most ±0.14 z for feature-positive windows. Feature-negative CIs are smaller than the markers. Positive and negative CIs are separated at all 14 plotted points.
  • Figure 4A: every claim in this panel compares input representations within the same neurons, so we show paired CIs. Each gap reduction will carry its 95% CI, for example −54.2% (46.1–61.9%) when prediction-selected features are added to the 440 inputs. Paired intervals are the informative ones here: intervals on the bar heights mainly reflect between-session differences in overall predictability and do not show whether two conditions differ. The corresponding paired differences are:
Paired comparison (same neurons)Mean95% CI
Linear gain from prediction-selected features0.01750.0157–0.0195
Gap reduction0.01370.0116–0.0157
SAE coefficients − prediction-selected features (linear)0.0031−0.0007 to 0.0064

The revised captions will state the CI method.

Response [W2/Q2], SAE stability: The same 63 rules: the 164 rules are fixed, and each was scored against each of 20 new SAE dictionaries; 63 reached best-match MCC ≥ 0.40 in all 20 runs. Of the remaining 101, 35 reached it in 15–19 runs, 16 in 10–14, 12 in 5–9, and 38 in 0–4 (Appendix C.2, Figure S1). Pooled median best-match MCC was 0.437, versus 0.276 for matched random rules and 0.120 under trial-block permutation. The atom directions themselves do vary across seeds (median matched decoder cosine 0.63 at 256 atoms; Appendix A.5), consistent with the paper the reviewer cites. Our method does not depend on recovering identical atoms. Rules are frozen and kept for their contribution to neural prediction, and correspondence to atoms only characterizes them. Seed-to-seed variation in SAE atoms therefore does not undermine the discovered features: 98 of the 164 rules find a matching atom in at least 15 of 20 independently trained dictionaries, their agreement is well above that of matched random rules, and the prediction results rest on the frozen rules, not on any particular set of atoms.

AuthorsOpenReview ↗

Response to Reviewer 2cLA (2/5)

[W3] Clarity: 318 inputs, k = 8, LLM prompt

Response, 318 inputs: We did not select these inputs. They are exactly the encoder design matrix of Nogueira et al. (2023), rebuilt with their public code; all 70 sensory columns matched their builder at r = 1.000. The matrix has 14 channels over five 0.1-s lags plus eight current/previous-trial variables expanded over 31 time bins: 14×5+8×31=31814\times5+8\times31=318 (Appendix A.2). The 14 channels are signed and absolute lick rate, and contact-start rate, contact angle, and angular position for whiskers C0–C3. Keeping their basis is deliberate: the question is whether the nonlinearity reported for this basis shrinks when features are added. We will state this in Section 3.

Response, prompts: In the submitted version, the 122 input-informed features (Table S1, Appendix B.1) were not generated with a single standardized prompt. For this response, we wrote a standardized prompt that generates this vocabulary and measured how much its output varies. The prompt gives the LLM only the input layout and the design principle: "an exhaustive enumeration of low-order aggregates over the whisker and lag structure", including counts of positive entries. In five independent runs it regenerated 106 or 107 of the 122 features every time; the features it missed were the thresholded whisker counts. The variation was equally small downstream. We replaced the 122 features with the features generated in each run and refitted the linear baseline, which contains only the 318 inputs and the input-informed features (no rules and no atom-informed features). Held-out R² was 0.1107–0.1124 across the five runs, against 0.1079 with the original 122 features and 0.1005 with the inputs alone; each run also generated further aggregates beyond the 122.

The atom-informed proposals are generated by a prompt, which is in the supplementary code with its output schema and the per-atom evidence records (neural_repro/proposal_features.py, function prompt). We will add the standardized prompt in full, with these results, to Appendix B.5 and the supplementary code.

Response, k = 8 (full answer in [Q10/W3] below): It was set during development, not tuned. We ran the requested ablation: the rules exceed matched random rules at every kk from 4 to 64 (to be reported in Appendix A.5).

[W4] Figures

Response: We will revise both figures.

  • Figure 1. Both z1z_1 and z2z_2 contribute to the single scalar feature f∗f^{\ast} in Fig. 1B. We will show their arrows converging on one node inside a green box. Following the convention in C, we will also draw the two upstream coordinates in D as two nodes inside a purple box. The difference in contour contrast in B has a specific meaning: only f∗f^{\ast} determines the response. We will explain in the caption that the faint contours mark the second coordinate, which can vary at fixed f∗f^{\ast} without changing the response.
  • Figure 2A. The inputs and the neural recordings are paired measurements. We will remove the arrow between them and state this in the caption.

[W5] Related work

Response: Thank you; both are relevant. In the revision, Section 5 will cite Tilbury et al. alongside LLM-guided program evolution and Lad et al. alongside language descriptions of cortical selectivity, and Appendix F will state the difference: they characterize responses to a known stimulus variable (orientation) or image set, validated by single-cell fit or in silico synthesis, whereas our rules are executable descriptions of multivariate inputs tested for linear compatibility on held-out recorded sessions.

AuthorsOpenReview ↗

Response to Reviewer 2cLA (3/5)

[W6] Proof of concept, noise ceiling, bigger dataset

Response, synthetic toy example: We agree that a simulated example with known ground truth is a valuable validation of the pipeline. We are preparing a numerically simulated example in which we generate the responses from features that we define ourselves, so that we can test directly whether the pipeline recovers them. We will report it in a second response and include it in the revision.

Response, noise ceiling: We estimated the noise ceiling at 0.36 (session-bootstrap 95% CI 0.31–0.40). Relative to it, feature discovery raises linear prediction from 30% to 35% of explainable variance, a gain of 4.9 percentage points (CI 4.1–5.8) that is far from negligible on that scale. The animals whisk actively, so no two trials repeat the same input and a repeat-trial ceiling is not available. We therefore used a split-half estimate within each 0.1-s bin: the bin is cut into interleaved 10-ms slices, alternate slices give two half-counts N₁ and N₂, and the ceiling is 4 Cov(N₁, N₂) / Var(N₁ + N₂), computed on the same held-out observations as the R² values. Slice widths of 5, 25 and 50 ms and a Poisson reference give gains of 4.6 to 5.3 points. The revision will report the analysis in Section 4.2 and Appendix G (sensitivity in Table G1, neuron-level diagnostics in Table G2).

Response, bigger dataset: We are applying the same pipeline to additional data beyond these recordings. We will report the results in the second response and include them in the revision. Within the present data, the rules already generalize beyond the sessions used to find them: rules discovered in 15 sessions and then held fixed improved prediction in all seven excluded sessions (190 neurons; R² from 0.102 to 0.119; Section 4.3). Every rule is selected for its contribution to predicting all 575 neurons through session-specific linear readouts, so each is supported by the whole population.

Minor

Response: We will follow, and state in Appendix C.1, a rule tied to the uncertainty: R² levels are given to three decimals in the main text, because their session-bootstrap CIs span about ±0.01 (line 376 will read 0.108 to 0.125), and paired differences with their CIs to four decimals, because paired CIs are several times narrower. Where two levels under comparison differ only in the fourth decimal, we give four.

[Q1] Linearity assumption

Response: See [W1/Q1] above.

[Q2] SAE stability evaluation

Response: See [W2/Q2] above.

[Q3] Size-fold evaluation

Response: Yes. "Size" is the number of selected rules, KK, and the size-evaluation folds are the cross-validation that chooses it. Forward selection orders the rules, and choosing KK on the trials that produced the ordering would be optimistic. We therefore split the selection trials into four trial-grouped folds, matching the four-fold splits used elsewhere. For each fold, the ordering is computed on the other three, and every prefix is scored on the held-out fold. KK is the maximum of the averaged curve. The ordering is then refitted on all selection trials, and its first KK rules are kept (Appendix B.3). The revision will add a schematic of this procedure (Figure B1, Appendix B.3), and Section 3.3 will state that these folds choose KK.

[Q4] SAE technicalities

Response: The SAE decomposes the direction of the hidden state rather than its size. Row norms of hh vary about fivefold across samples (1st–99th percentile 0.83–4.24), so without normalization high-norm samples would dominate the squared reconstruction loss. Downstream, only which atoms are active is used (binary rule targets), so magnitude plays no role there.

Nonnegative codes are the standard TopK SAE form (ReLU, then the kk largest entries), and they do not prevent suppression: atoms describe the shared representation, and a neuron is suppressed through negative readout weights on the selected features (Eq. 3). The reviewer is right that sign matters at the atom level: a bipolar direction would be split into two atoms that become separate rule targets. In our dictionary this is rare. Of 15,400 pairs of live atoms, the most opposite has decoder cosine −0.65 and only 5 fall below −0.5. Atoms in these pairs almost never co-fire (≤0.3% overlap), yet their rules overlap rather than mirror each other (rule MCC +0.29 to +0.44). Signed codes with selection by magnitude are a natural extension, and the Discussion will note it.

[Q5] Clarity 1

Response: Yes, I(⋅;⋅)I(\cdot;\cdot) at line 165 is mutual information; the revision will define it there.

[Q6] Clarity 2

Response: Yes. Eq. 2 is the population average over neurons of 1−MSE/Var1-\mathrm{MSE}/\mathrm{Var} for the best affine predictor, i.e. one minus the normalized MSE, which is the population R². Held-out R² is its finite-sample estimate.

Reviewer_avXtOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The authors propose a framework to identify sensory and behavioral features, through linear or nonlinear combinations of existing input variables, that make neural responses more linearly predictable. The approach involves a pipeline combining a neural population encoder, sparse autoencoder (SAE), LLM-assisted feature generation, and decision trees to extract explicit, rule-defined features. The discovered features improve linear neural response prediction (R² from 0.108 to 0.125). While the relative improvement is substantial, partly because of the poor baseline performance, the absolute improvement is quite modest. Conceptually, the idea that nonlinear relationships can become linear through appropriate feature transformations is not new. The feature-discovery algorithm is interesting, but the modest predictive gain raises questions about its practical utility. Furthermore, the computational or behavioral relevance of the discovered features remains to be systematically established.

Critical strengths
  1. Interesting conceptual perspective on nonlinear mixed selectivity, although the underlying idea is not entirely new.
  2. Potentially useful feature-discovery framework. The pipeline extracts explicit sensory and behavioral rules that could make neural encoding models more interpretable. However, the demonstrated benefit remains modest in absolute terms.
  3. Rigorous evaluation. The authors use nested cross-validation, several control analyses, and cross-session transfer to evaluate the predictive value of the discovered features.
Critical weaknesses
  1. Limited conceptual novelty. The observation that nonlinear selectivity can become linear through an appropriate transformation of input features is well known. The theoretical framework largely formalizes this existing insight.
  2. Uncertain neuroscientific significance of the discovered features. The rule-defined features are essentially nonlinear basis expansions of existing input variables and do not necessarily correspond to biologically meaningful computations. The three examples in Figure 3 involve temporal changes in whisker position, relative whisker positions, and combinations of whisker position and contact, all of which seem like natural candidate predictors of S1 activity. Can the authors provide stronger evidence that the discovered features reveal previously unrecognized neural selectivity or offer new insights into S1 computations, rather than simply improving prediction?
  3. Poor baseline prediction and modest absolute improvement. The baseline linear prediction performance is rather low (R² = 0.108), and the improvement to 0.125 is small in absolute terms. Why is the neural response prediction performance so low? How does it compare with the state of the art on this dataset? Without a noise ceiling or stronger reference model, it is difficult to assess the significance of the improvement.
  4. Unclear necessity of the SAE component. The ablation results show that fitting rules directly to neural principal components achieves nearly identical prediction performance when using the same atom-informed LLM feature proposals. This raises questions about the importance of the SAE and its latent components. Is the main benefit coming from LLM-assisted feature generation rather than the SAE? Could a simpler feature-engineering pipeline achieve comparable results?
  5. Limited generalizability. The method is evaluated on only one dataset from mouse S1. Although the cross-session transfer results are encouraging, it remains unclear how well the approach would generalize to different neural populations, sensory modalities, or behavioral tasks.
Questions for authors

One motivation of the paper is that the classification of neurons as purely, linearly mixed, or nonlinearly mixed selective depends on the choice of input features. However, the authors mainly evaluate improvements in population-level prediction. How does the classification of individual neurons change after feature discovery? How many neurons previously classified as nonlinearly mixed become linearly mixed or purely selective? Do substantial nonlinear interactions remain after the new features are introduced? Such analyses would more directly connect the empirical results to the paper's central conceptual motivation.

10 comments
AuthorsOpenReview ↗

Response to Reviewer avXt (1/5)

We thank the reviewer for the careful assessment. We have carried out additional analyses in response to the comments and report them below. The analysis for W5 is in progress and will follow in a second response. The revised manuscript, with all changes, will be uploaded in a single update.

[W1] Limited conceptual novelty

Response: Our contribution is a method for discovering explicit sensory descriptions that make neural population responses more linearly predictable. We measure how much these descriptions reduce the nonlinear prediction advantage. Building on relational accounts of representation (Pohl et al., 2026) and usable information (Xu et al., 2020), we use compatibility as a criterion for discovering these descriptions. Specifically, response-predictive latent patterns guide the search for shared, executable sensory rules. Each rule defines a sensory condition that can be read, computed from the measured inputs, and tested for its contribution to neural prediction.

The dependence of selectivity on the chosen variables is explicitly recognized in accounts of mixed selectivity (Kaufman et al., 2022; Tye et al., 2024) and in the analysis of the S1 recordings revisited here (Nogueira et al., 2023). Yet these accounts stop short of providing a systematic procedure for discovering alternative sensory descriptions and measuring how much they change the nonlinear prediction advantage. We therefore make the sensory description itself the object of discovery, while keeping the neural population and readout class fixed.

The table separates the established foundations from the discovery procedure and empirical result we contribute:

ContributionStarting point in prior workWhat this paper adds
Comparison criterionThe dependence of selectivity on the chosen description is established (Kaufman et al., 2022; Tye et al., 2024).Use compatibility as the objective for discovering a shared sensory description of a fixed neural population under a specified readout class (Eq. 2).
Description discoveryEncoding studies test alternative sensory or task descriptions (Naselaris et al., 2011; Campagner et al., 2016; Hirokawa et al., 2019).Use response-predictive latent patterns to discover shared, executable sensory rules, then select them for the linear predictive value they add.
Empirical resultNogueira et al. (2023) raised the possibility that other variables could reduce the observed nonlinearity in these recordings.Descriptions found on training trials reduce the nonlinear-over-linear prediction gap by 54.2% on unseen trials.

In the S1 data, this search leads to a substantially simpler explanation of the recorded responses. On the recordings studied by Nogueira et al. (2023), the nonlinear-over-linear R² gap fell from 0.0252 to 0.0116, a 54.2% reduction, and decreased in 455 of 575 neurons. Crucially, nonlinear R² also increased, from 0.133 to 0.137. The narrower gap therefore reflects the stronger linear explanation enabled by the discovered features (Section 4.2).

The controls further show that the discovered descriptions provide substantially more explanatory value than a matched random expansion. With trial time already included in the baseline, prediction-selected features reached R² 0.126 versus 0.117 for matched random features, from a baseline of 0.110 (Section 4.2; Appendix C.5). Beyond this comparison, their value also extends to new recording sessions. Features discovered in 15 sessions and then held fixed improved prediction in all seven excluded sessions, from 0.102 to 0.119 after within-session readout calibration (Section 4.3).

At the theoretical level, Proposition 1(ii) identifies the ideal limit of linear compatibility under information-preserving re-description as the fraction of response variance predictable from the inputs (Appendix E). In practice, our experiments use held-out nonlinear prediction as an empirical benchmark, fitting both predictors for each description and comparing their gaps (Section 4.2).

Compatibility-guided discovery therefore identifies sensory descriptions that explain neural responses more simply and establishes how much that choice matters in the data. We will sharpen this framing in the Introduction and Discussion.

AuthorsOpenReview ↗

Response to Reviewer avXt (2/5)

[W2] Neuroscientific significance of the discovered features

Response: We agree that the original manuscript did not bring out the biological significance of these features clearly enough. Our additional analyses show that, in these S1 recordings, comparisons of whisker position across time are particularly useful for explaining neural responses. A simple summary—whether any recent position average exceeds a reference—also improves prediction consistently across sessions. With the full set of discovered features, the gap between linear and nonlinear prediction falls by 54% (Appendix C.4, Table S5). This substantial reduction shows how strongly the apparent need for a nonlinear predictor depends on the sensory description.

The features match established aspects of S1 physiology. Whisking amplitude and set point, movement-dependent touch responses, and integration across whiskers are familiar components of S1 coding (Rodgers et al., 2021; Curtis & Kleinfeld, 2009; Lyall et al., 2021). Our procedure finds related structure from binned position and contact signals without receiving cycle-resolved amplitude or set-point predictors. This agreement supports the biological relevance of the features.

Comparisons of position across time make a particularly strong contribution. We removed these rules and refitted the linear model. Prediction on held-out trials declined 3.4 times as much as when we removed the same number of randomly selected other rules. The result also held when the comparison rules were matched for how often they activated (Appendix C.7, Table C2).

A useful summary is whether any recent position average exceeds the whisker's reference position. Motivated by the rule catalog, we compared simple summaries of recent position in a follow-up analysis. For each whisker, we considered five consecutive 100-ms bins, including the current bin, and used its training-set mean as the reference. Just four yes/no features improved prediction in all 22 sessions, recovering about 9% of the gain of the roughly 150 discovered rules (mean ΔR² = 0.0016). This summary outperformed testing whether the recent average exceeded the reference or whether most bins did. Its advantage remained after accounting for individual-bin thresholds and time within the trial. The discovered rule set already captured most of this effect (Appendix C.8, Table C3). Thus, a compact summary of recent position captures reproducible explanatory value with just four binary features.

The gains are specific to the discovered features and generalize across sessions. The selected features produced 2.4 times the improvement obtained from matched random features built from the same vocabulary (Appendix C.5, Table S6). Rules discovered on 15 sessions and then held fixed improved prediction in all seven excluded sessions (Section 4.3; Appendix D.2, Table S9). Together, these comparisons establish that the selected sensory relationships have substantially greater explanatory value than matched alternatives, and that this value extends to new sessions.

Temporal comparisons and a compact summary of recent position reveal structure in S1 responses that a linear readout of the original inputs misses. Compatibility makes sensory descriptions themselves objects of discovery and validation, selecting interpretable features by their contribution to prediction with a linear readout. Each rule specifies the whiskers, time intervals, thresholds, and contact conditions needed to test the resulting selectivity hypotheses experimentally. In the revision, we will relate the example features to known S1 physiology in Section 4.1, report the contribution of temporal comparisons in Section 4.2, state in the Discussion what these results show about S1 activity, and give the full analysis protocols, controls, and tables in Appendices C.7 and C.8.

AuthorsOpenReview ↗

Response to Reviewer avXt (3/5)

[W3] Poor baseline prediction and modest absolute improvement

Response: We agree that a noise ceiling is important for interpreting these R² values. We predict responses in 100-ms bins within individual trials, rather than responses averaged over repeated trials. The median neuronal firing rate is 5.2 Hz, equivalent to only about 0.5 spikes per bin on average. These single-trial counts retain spiking variability that averaging across repeated trials would reduce, so raw R² can be low even when a model captures meaningful changes in firing rate. We have therefore estimated a noise ceiling and compared our models with published results from these recordings.

Noise ceiling puts the size of the improvement in context. We estimated reliability by splitting each bin into interleaved 10-ms slices and using the covariance between the two half-counts. On the same held-out observations used to score the models, the estimated noise ceiling is 0.36 (session-bootstrap 95% CI, 0.31–0.40). Relative to this estimate, feature discovery raises linear prediction from 30% to 35% of explainable variance, a gain of 4.9 percentage points (95% CI, 4.1–5.8). Thus, relative to the estimated explainable variance in these recordings, the R² gain is far from negligible.

The baseline matches or exceeds the performance of published models of these recordings. Nogueira et al. (2023) report held-out R² of 0.089 for their linear encoder and 0.111 for their best nonlinear encoder. Using their 318-input design, rebuilt from public code, our reanalysis achieves 0.1005 and 0.1375, respectively (Table S5). With the discovered features, a linear readout reaches 0.1254, higher than even the best nonlinear performance reported by Nogueira et al.

The gain is reproducible and specific to the discovered features. Feature discovery and model fitting were confined to training trials in 4-fold trial-grouped nested cross-validation. On held-out trials, linear R² increased from 0.1079 to 0.1254: ΔR² = 0.0175, with a 95% CI of 0.0157–0.0195 (session bootstrap; Section 4.2, Table S5). With trial time included, the discovered features produced 2.4 times the gain of matched random features (Table S6). Rules discovered in 15 sessions and then held fixed also improved held-out prediction in all 7 excluded sessions, after fitting session-specific readouts (Section 4.3, Table S9). These results show that the improvement persists across test trials, control comparisons, and recording sessions.

The scientific gain is discovering interpretable features that support a simpler account of neural responses. Compatibility guided feature discovery by selecting sensory and behavioral descriptions for their contribution to linear prediction. The resulting features reduce the nonlinear-over-linear gap by 54%, from 0.0252 to 0.0116. The gap decreases in 455 of 575 neurons, and the linear readout reaches 92% of the nonlinear model’s R² (Section 4.2, Table S5). Compatibility-guided discovery thus makes much of the nonlinear model’s predictive advantage accessible through explicit sensory and behavioral features and a linear readout.

The revision will report the noise-ceiling analysis and the comparison with published models in the Results (Section 4.2), with the estimation procedure, sensitivity analyses, and neuron-level diagnostics in Appendix G (Tables G1 and G2).

Reviewer_V9edOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The authors assert that a neuron’s selectivity label depends on which input features you describe it with which motivates trying to find better features automatically. They train a network to predict mouse S1 firing from whisker and licking measurements, separate its hidden layer into sparse components, and fit small decision trees (with LLM-suggested derived features) that turn each component into an if-then rule over the labels. Rules that improve linear prediction on held-out trials are kept. On 575 neurons across 22 sessions this raises linear R2 from 0.108 to 0.125 and transfers to sessions not used for discovery.

Critical strengths

A pipeline for turning jointly recorded behavioral labels into descriptive rules for how neural activity is predicted is an inspiring idea.

Critical weaknesses

The ablation study seems to indicate almost all of the performance comes from the shallow decision-tree rules and forward selection. Swapping the SAE targets for neural PCs changes R2 by only 0.0004 (CI includes zero, Appendix C.6), so the SAE does not add predictive value.

The formalizations do little to help me understand the paper. Proposition 1 in particular seems to be a restatement of literature ideas.

Questions for authors

How sensitive is this method to the representation of numerical inputs? Since many quantities may have fixed or canonical ranges, min-max or fixed-range normalization may be suitable. Appendix A.3 uses pooled training-session standardization for transfer; how sensitive are the learned thresholds to the choice of standardization reference, and would a fixed-range scheme change which rules transfer?

Do rules appear to be hierarchical, in that there are near repeats across neurons or within neurons that could be collapsed lossily into fewer rules? Appendix B.7 groups the 164 rules into four clusters and 20 rules capture roughly 70% of the transfer gain; how much prediction is lost under a compressed rule set?

Is there any indication that rules correspond to biological functions?

Can you report a noise ceiling (split-half or repeat-trial) so the R2 values can be read as a fraction of explainable variance?

6 comments
AuthorsOpenReview ↗

Response to Reviewer V9ed (1/4)

We thank the reviewer for the careful reading. We have carried out additional analyses in response to the comments and report them below. The revised manuscript, with all changes, will be uploaded in a single update.

[W1] The SAE does not add predictive value

Response: The 0.0004 difference the reviewer cites is the paired difference in held-out R² between rules fitted to SAE atoms and rules fitted to neural PCs (atom minus PC, 95% CI −0.0001 to 0.0010). We agree that changing the rule-fitting target made little difference to prediction here. However, both routes used the same atom-informed proposals, sensory features that the LLM proposes from SAE activation patterns. To assess what those proposals added, we therefore kept the PC targets fixed and compared prediction with and without the proposed features (Table C1, Appendix C.6).

Rule targetInput-informed featuresAtom-informed featuresHeld-out R²Share of SAE-guided gainSAE-guided minus this route (95% CI)
SAE atoms — SAE-guided route✓✓0.1261100%—
SAE atoms–✓0.1262101%−0.0002 (−0.0005 to 0.0002)
Neural PCs✓✓0.125697%0.0004 (−0.0001 to 0.0010)
Neural PCs✓random features0.123987%0.0022 (0.0016 to 0.0028)
Neural PCs✓–0.123484%0.0027 (0.0021 to 0.0032)
Neural PCs––0.122678%0.0035 (0.0027 to 0.0043)
None — matched random features✓✓0.116641%0.0094 (0.0085 to 0.0103)
Baseline, no rules——0.11000%—

Gain is measured above the common baseline; the first row defines 100%. The feature columns show whether each rule's tree vocabulary contains the 122 input-informed features and the atom-informed proposals; "random features" replaces the proposals with an equal number of random features from the same 14-operation grammar. The readout baseline retains the input-informed features and time indicators in every condition. Confidence intervals use paired session bootstrap.

The strong PC result also shows the value of these proposed sensory features. With the same atom-informed proposals, PC-target rules reached R² = 0.1256 versus 0.1261 for atom-target rules, recovering 97% of the SAE-guided gain. The LLM obtains its clues from input differences between atom-active and atom-inactive samples, together with examples of where simple rules succeed or fail. It proposes executable features from this evidence. Rules built from them are then selected for their conditional contribution to linear neural prediction and tested on separate held-out trials. The useful sensory information in these proposals thus carries over to another rule-fitting target.

Choosing these sensory relationships makes a clear, consistent difference. Holding PC targets fixed, adding the atom-informed proposals increased held-out R² by 0.0022 (95% CI 0.0019–0.0025), with gains in all 22 sessions. They also outperformed an equal number of random features from the same 14-operation grammar by 0.00173 (95% CI 0.00141–0.00205). This is a gain from the content of the proposed features: the same number of additional features works substantially better when it captures informative sensory relationships.

A simpler PC route is a useful option, and the full method takes the improvement further. Rules fitted directly to neural PCs, without atom-informed proposals, atom activations, or encoder/SAE checkpoints, recovered 84% of the SAE-guided gain. This is a useful simplification: it retains compatibility-based selection and validation while making candidate rules more directly. The SAE-guided route nevertheless delivers about 20% more improvement over the common baseline (paired ΔR² = 0.0027, 95% CI 0.0021–0.0032). Thus, the framework supports a simpler implementation, while atom-informed proposals help it obtain more explanatory power.

Compatibility makes the sensory description itself the object of discovery and validation. We hold the neural prediction target and the linear readout class fixed while searching for interpretable input features that make more response structure linearly accessible. To implement this principle, we assess how much a candidate rule improves linear prediction beyond the inputs and rules already included. Even with the same vocabulary, atom-informed proposals included, and the same prediction-based selection, matched random features recover only 41% of the SAE-guided gain. Learning how to turn that vocabulary into rules that predict neural responses therefore adds substantial value. The SAE-guided route supplies evidence for finding useful features and keeps each rule linked to the latent component that motivated it. Together, these steps turn feature ideas into explicit sensory descriptions with tested explanatory value. We will make these roles clearer in Section 4.2 and Appendix C.6.

AuthorsOpenReview ↗

Response to Reviewer V9ed (2/4)

[W2] The formalization does not help; Proposition 1 restates the literature

Response: The formal section defines the objective that the method optimizes and the quantity that every result reports. We make the feature description itself the object of discovery. This requires a criterion for saying that one description of a fixed neural population is better than another, and linear compatibility (Eq. 2) is that criterion. We agree that Section 2 did not make this role clear, and in the revision it will open by stating it.

Proposition 1 builds on established ideas, which we cite: relational accounts of representation (Pohl et al., 2026), the distinction between information and usable information (Xu et al., 2020), and the recognized dependence of selectivity labels on the chosen variables (Kaufman et al., 2022; Tye et al., 2024). What it adds is a statement of what a search over descriptions can and cannot change, and the rest of the paper uses it in three places:

  • The objective. Linear compatibility is the quantity that forward selection increases and that every result reports. It gives "a better description" a definition.
  • The reading of the main result. Re-description leaves the information about the response unchanged (i) but can move linear compatibility from zero to one (iii), up to the fraction of response variance predictable from the inputs (ii). Accordingly, nonlinear R² also increased (0.133 to 0.137) and linear R² increased more (0.108 to 0.125), so the nonlinear-over-linear gap fell by 54.2%: the discovered features made existing predictive structure accessible to a linear readout.
  • The selection rule. A feature helps only through the part not already explained linearly by the current features (Appendix E, Property 3). Forward selection scores candidates by this conditional contribution.

The framework thus turns a recognized dependence into a quantity that can be measured (Eq. 2), bounded (Proposition 1), and searched over (Section 3).

[Q1] Sensitivity to the representation of numerical inputs

Response: The rules and their transfer are robust to the normalization scheme. We tested both suggestions in the transfer analysis that the reviewer cites (156 rules fixed in 15 sessions, evaluated in the seven excluded sessions). We replaced the pooled standardization with each scheme and refitted the trees. The encoder, the SAE atoms, and the feature definitions were held fixed (to be reported in Table D2, Appendix D):

NormalizationHeld-out R²Share of the gainDifference from the paper (95% CI)Agreement with the original rules
Mean and SD, pooled over the 15 sessions (paper)0.1190100%n/an/a
Min-max, range pooled over the 15 sessions0.118899%−0.0002 (−0.0006 to 0.0003)median MCC 0.86; 94 of 156 rules at or above 0.8
Fixed range where a canonical range exists0.118898%−0.0003 (−0.0010 to 0.0005)median MCC 0.85; 88 of 156 rules at or above 0.8

The baseline is 0.1024 (the held-out R² of the 440 base and input-informed features without any rule; the share is the gain over this value relative to the paper's gain). The fixed range is ±90° for contact angle and angular position, and the trial variables keep their coded range of −1 to 1. Contact-start and licking rates have no canonical range and keep the pooled standardization.

  • How sensitive are the thresholds? Most rules stay close to their original version under both schemes (median MCC 0.86 and 0.85).
  • Would a fixed-range scheme change which rules transfer? The set transfers equally well. Both intervals for the difference include zero, and all seven sessions improve under every scheme.

We agree that a fixed range is a natural choice where a quantity has a canonical range, and these results show that the discovered rules and their transfer are largely unaffected by the choice.

AuthorsOpenReview ↗

Response to Reviewer V9ed (3/4)

[Q2] Redundancy among rules and compression

Response: The rules are largely distinct: few are near repeats, and each contributes explanatory value. They are not hierarchical, and the more they are merged into general rules, the more prediction is lost (a fifth of the gain at 80 merged rules, two thirds at 20). We tested this in the transfer analysis the reviewer cites (156 rules fixed in 15 sessions, evaluated in the seven excluded sessions; to be reported in Table D1, Appendix D):

RulesReductionHeld-out R²Share of the gain
156None0.1190100%
140Near repeats removed, MCC > 0.80.118697%
95Near repeats removed, MCC > 0.60.116887%
47Near repeats removed, MCC > 0.40.113165%
80Merged into 80 groups0.115780%
20Merged into 20 groups0.107933%

The baseline is 0.1024 (the held-out R² of the 440 base and input-informed features without any rule; the share is the gain over this value relative to the gain of all 156 rules). Near repeats were removed by keeping, of two similar rules, the one that forward selection in the discovery sessions chose first. To merge, we formed groups with the clustering of Appendix B.7 and replaced each group by one rule that holds when any of its rules holds. Appendix D will give the full procedure and thresholds from 0.9 to 0.3.

  • Are there near repeats? Few. Among the 164 catalog rules, 18 of 13,366 pairs have an MCC above 0.8 and the median is −0.03 (to be added to Appendix B.7). Removing the near repeats among the 156 rules (MCC above 0.8) costs 3% of the gain, and the loss grows gradually as more loosely similar rules are removed (13% at 0.6, 35% at 0.4).
  • Can the rules be collapsed into fewer rules? Not without loss. Merging loses a fifth of the gain at 80 groups and two thirds at 20. The four groups of Appendix B.7 are loose (silhouette 0.22) and summarize the catalog; they are not general rules that the others are variants of.
  • Where do the 70% at 20 rules come from? From selection, not from merging. Those 20 are the first rules of a forward-selection order fitted within each excluded session (Appendix D), so they are not one fixed set and differ between sessions.

[Q3] Correspondence to biological functions

Response: Yes, several lines of evidence indicate this. The discovered rules match established aspects of S1 physiology, and comparisons of whisker position across time are particularly useful for explaining the responses.

  • The rules match known S1 physiology. Whisking amplitude and set point, movement-dependent touch responses, and integration across whiskers are familiar components of S1 coding (Rodgers et al., 2021; Curtis & Kleinfeld, 2009; Lyall et al., 2021). Our procedure finds related structure from binned position and contact signals without receiving cycle-resolved amplitude or set-point predictors.
  • Comparisons of position across time make a particularly strong contribution. Removing these rules and refitting the linear model reduced held-out prediction 3.4 times as much as removing the same number of randomly selected other rules (Appendix C.7, Table C2).
  • The content of the rules is specific. Relative to matched random features from the same vocabulary, the selected features more often use cross-whisker contrasts (26.0% vs 3.3%), temporal changes in position (10.9% vs 1.9%) and contact angle (15.8% vs 7.8%), with the same sign in all four folds (Appendix B.8).
  • Their value is most pronounced in the neurons with the largest nonlinear contribution. The advantage of discovered over matched random features grew with the contact–angle nonlinear contribution measured using the ablation approach of Nogueira et al. (2023) (Spearman ρ = 0.46; positive within 21 of 22 sessions; Appendix C.9).
  • They hold across sessions and animals. Rules fixed in 15 sessions improved prediction in all seven excluded sessions, two of them from animals not used for discovery (Section 4.3).

Each rule specifies the whiskers, time intervals, thresholds, and contact conditions needed to test the resulting selectivity hypotheses experimentally. The revision will relate the example features to known S1 physiology (Section 4.1) and report these analyses (Section 4.2; Appendices C.7 to C.9).

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response to all reviewers

We thank the reviewers for their careful reading. The comments converged on four points: what is new and what it reveals about the responses, what the SAE contributes, how to read R² values of this size, and whether the approach holds beyond one dataset. We ran new analyses for each, added a simulated example and two datasets, and revised the paper.

1. What is new (avXt W1, W2; V9ed W2, Q3). That selectivity depends on the chosen description is established (Kaufman et al., 2022; Tye et al., 2024), and we build on relational accounts of representation (Pohl et al., 2026) and usable information (Xu et al., 2020). Existing accounts offer no systematic procedure for discovering alternative descriptions and measuring their effect; we provide one. In mouse S1, where Nogueira et al. (2023) reported nonlinear sensory integration, we found interpretable features that let a linear readout capture response patterns previously better explained by a nonlinear model. A linear readout of the full description recovered 67% of the nonlinear model's advantage on the original inputs (95% CI 60–74%). Added to a baseline that already holds the input-informed features, the discovered features narrowed the remaining gap by 54.2%, and it shrank in 455 of 575 neurons. Features comparing whisker position across time were especially informative for linear prediction. Removing them reduced held-out R² 3.4 times as much as removing the same number of other rules at random.

2. The role of the SAE (avXt W4, V9ed W1, 2cLA Q8). Our original paper could have been clearer about why the PC route nearly matched the SAE route. PC-target rules recovered 84% of the SAE-guided gain without atom-informed proposals. They reached 97% when they reused features the LLM proposed from SAE activations, compared with 87% when the same number were drawn at random from the same grammar. SAE activations thus help identify which sensory features to test and link each rule to its latent pattern.

3. New analyses of the S1 data.

  • Noise ceiling (all reviewers): 0.36 (0.31–0.40). Feature discovery raises linear prediction from 30% to 35% of total explainable variance (4.9 pp, 4.1–5.8).
  • Per-neuron change (avXt Q1): Of 396 neurons with a nonlinear lead above the preset cutoff, 53% fell within it with discovered features vs. 23% with matched random features (cutoff 0.01 R²).
  • Robustness (2cLA W1, W3, Q11; V9ed Q1): Rule gains persist with monotone readouts, alternative normalizations, and the 122 input-informed features removed from the tree vocabulary. Rule–atom correspondence exceeded matched random controls at every tested k (4–64).

4. Simulation and new datasets (2cLA W6, avXt W5).

  • Simulated responses with planted features: the rules selected by the unchanged procedure predicted all four kinds of planted features better than matched random rules, prediction improved only in the neurons that carried each kind, and no rule was selected when none was planted. The rules added 0.0085 in R², 7 times matched random features, and the linear readout passed the nonlinear model on the base inputs in all five datasets.
  • Macaque motor cortex (MC_Maze, 182 units): with the full description, linear R² rose from 0.0187 to 0.0584, exceeding the nonlinear model on the base inputs (0.0525), and from 18% to 58% of explainable variance. On maze layouts withheld from every step it rose from 0.0107 to 0.0446 (nonlinear 0.0428).
  • Macaque somatosensory area 2 (Area2_Bump, 65 neurons): from 0.0600 to 0.0932, 85% of the nonlinear model's advantage; rules joining a movement and a load condition improved prediction beyond the two given separately.
  • In all three evaluations rules added 2.8 to 4.4 times the gain of matched random features selected the same way.

5. Changes in the revision (marked in blue).

WhereChange
Abstract, Sec. 1S1 gap reduction (54.2%), macaque results, overview and contributions
Sec. 2How Eq. 2 and Proposition 1 are used; mutual information
Sec. 3Source and scope of the inputs; target scale of Eq. 3; roles of the SAE
Figures 1–4Figures 1 and 2 revised; 95% CIs in Figures 3 and 4 (2cLA W2)
Sec. 4.1–4.3Link to S1 physiology; noise ceiling; rule families; PC variants; per-neuron result; transfer to new animals
Sec. 4.4 (new)Simulation and macaque datasets
DiscussionRewritten: S1 finding, new datasets, proposed experiments; signed codes
App. A.5, B.3, B.8Sparsity sweep (Table A1); size-fold schematic (Figure B1); use of atom-informed proposals
App. C.6–C.9Discovery routes (Table C1); monotone readouts; per-neuron analysis (Figure C1); rule-family deletion, recent-position summaries, link to the contact–angle nonlinearity
App. DCompressed rule sets and input normalization (D1, D2)
App. G–INoise ceiling (G), rule checks against planted signal and PC control (H, Table H3), macaque data (I)

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.