acceptodds
Under review as a conference paper at ICLR 2027

From Information to Conditional Contribution: Residual-Informed Sparse Decoding for Traceable Neural Representation Analysis

Abstract

Neural representations are organized patterns of population activity that carry information about external stimuli. Large-scale recordings make such information decodable, but it remains unclear which neural responses, at which times and temporal scales, support a given readout, or whether their conditional discriminative value changes with population context, stimulus, and decoding objective. Latent-component methods often form mixed coordinates that are difficult to trace to the original responses. Information-theoretic methods preserve traceability, but static rankings do not capture a response's value for the current readout. We introduce Residual-Informed Sparse Decoding (RISD), a residual-conditioned framework that constructs task-specific neural representations from traceable multiscale response coordinates. RISD first identifies an information core using conditional mutual information, then uses grouped out-of-fold residuals to add responses not captured by the current readout, yielding sparse, testable representations selected using training data only. We evaluate RISD on image-identity and semantic decoding using recordings from marmoset retina, the mouse visual pathway, and macaque visual cortex. Under matched support budgets, RISD outperforms baselines with compact response-coordinate sets and traces selected coordinates to specific neural regions, neurons, time windows, and scales. It reveals stimulus-specific response combinations and partially recoverable readout support in retinal activity, while showing across datasets that a response's discriminative value depends on selected support, stimulus, region, and decoding objective rather than its overall statistical information.

Primary area:
applications to neuroscience & cognitive science
Keywords:
neural decoding, neural population coding, sparse representation, multiscale neural responses, conditional mutual information, computational neuroscience, visual neuroscience

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_4zS5OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Residual-Informed Sparse Decoding (RISD), a supervised feature-selection framework that combines wavelet conditional mutual information (WCMI) with out-of-fold residual-guided feature completion. RISD first selects an information core using WCMI and then adds features correlated with the current decoder's prediction residuals.

Experiments on marmoset retinal recordings, the mouse visual pathway, and macaque visual cortex show modest decoding improvements over baselines. The authors further use RISD to investigate combinatorial neural coding, functional substitution, and task- and population-dependent neural representations.

Critical strengths

1. Interpretable neural feature selection.

RISD preserves the correspondence between selected features and their original neurons, temporal windows, and scales. This enables interpretable analyses of neural population activity without relying on mixed latent representations.

2. Evaluation across multiple neural systems.

The experiments cover three species, multiple visual processing stages, and different decoding objectives.

Critical weaknesses

The main weaknesses are stated below, and the authors can check the Questions For Authors part for the details.

1. Limited methodological novelty and insufficient comparisons.

The paper mainly combines established WCMI and residual-based feature-selection techniques, as the 1st contribution stated in the last part of introduction, but the improvement on RISD compared with WCMI is incremental.

Residual-based feature-selection methods are not new. The paper neither sufficiently establishes the necessity of its two-stage design, nor compares with enough modern baselines. Without these comparisons, it remains unclear whether RISD provides a meaningful advantage over existing supervised feature-selection techniques.

2. Insufficient evidence supporting the claimed neuroscience contributions.

The 2nd and 3rd contributions stated are the neuroscience contributions, however, most analyses in Figures 2–4, including combinatorial coding, neuron deletion, functional substitution, task-dependent representations, and population-context effects, can also be performed using existing methods such as WCMI. However, the paper does not systematically compare the neuroscientific findings obtained by different approaches, making it unclear what is special on RISD compared with other methods, if different methods are all available on those tasks with minor differences ?

Overall, the experiments demonstrate that RISD can support interpretable neural representation analysis, but do not convincingly establish that it identifies more reliable or distinctive neuroscientific findings than existing approaches. This is particularly important given that the central contributions are framed as moving from information to conditional contribution.

Questions for authors

Major:

  1. Baselines and ablations. How does RISD compare with (i) iterative residual-only selection, (ii) in-sample vs. OOF residual selection, and (iii) other interpretable supervised feature-selection methods such as L1/Elastic Net, LassoNet, GradEnFS, or standard forward selection? Please compare under matched feature budgets and a common decoder, including accuracy–sparsity–computation trade-offs. How do RISD-selected neurons and temporal coordinates differ from those identified by conventional stimulus-tuning or single-neuron encoding analyses? Does RISD reveal reliable conditional decoding contributions that cannot be explained by individual response selectivity?
  2. Distinctive neuroscience contributions. Can the authors repeat the key analyses in Figures 2–4 using WCMI or other interpretable baselines? Do these methods identify similar combinatorial responses, functional substitutions, and task-dependent representations? What neuroscientific findings can RISD identify more reliably than existing methods? A same-size WCMI continuation would help distinguish the benefit of residual completion from simply adding more features. Specifically, for Figure 3C, the comparison between S1S_1 and S1∪S2S_1 \cup S_2 uses different numbers of features. Moreover, the 23 repaired images represent a selected subset. Please report the total number of evaluated images and the full distribution of per-image accuracy changes, including unchanged and degraded cases.
  3. Statistical validation and stability. Can the authors provide confidence intervals and paired significance tests for Tables 1–2 and Figures 2–4 for neuroscience analysis? How stable are the selected neuron-time-scale features across independent splits or repeats? In particular, does the task-specific separation in Figure 4E–F exceed same-task selection variability and appropriate null baselines?
  4. Interpretation of neuroscience results. How are the information gains in Figure 2D and the evidence scores in Figure 4F defined? Do the reported joint gains establish genuine information synergy or simply complementary decoding information? Can the authors provide appropriate controls for neural response correlations and population size, particularly for Figures 2 and 4G?
  5. Feature-selection fairness and computational cost. The WCMI baseline uses the feature budget selected by RISD. How does WCMI perform with its own optimized feature budget? Can the authors provide accuracy-versus-feature-count curves and runtime comparisons, including WCMI ranking, cross-fitting, and residual scoring?
  6. Cross-fitting and potential leakage. Is WCMI feature ranking and K_1 selection repeated independently within each inner fold? Otherwise, OOF residuals may be affected by supervised feature selection using held-out labels. Please clarify the complete nested validation procedure.

Minor:

  1. Experimental protocol and reproducibility. What are the numbers of animals, sessions, trials, classes, and outer folds? How is StimID voting accuracy defined? Please clarify the differences between the results in Table 1, Table 2, and Figure 3B. What are the exact decoder architecture, training settings, and search ranges and selected values for K_1, K_2, and P? Are wavelet coefficients standardized before covariance-based scoring? Given the absence of a methodological appendix, could the authors provide a detailed appendix or supplement specifying the missing experimental and implementation details, together with anonymized code or complete reproduction configurations?

The reviewer would update the score if reasonable answers are provided.

Reviewer_xvjNOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Targeting at improving supervised stimulus decoding, this paper introduces Residual-Informed Sparse Decoding (RISD), a method that isolates a compact, interpretable set of (neuron, time, scale) coordinates. The approach begins by identifying a core subset of features based on conditional mutual information. It then evaluates the remaining coordinates just once by measuring their covariance with the grouped out-of-fold residuals from the initial core decoder. The top-ranked candidates are added to the core model before refitting, with all feature-set sizes optimized via nested grouped cross-validation. Evaluated across several datasets, RISD consistently achieves higher accuracy on image-identity and five-way semantic classification tasks compared to baseline methods like PCA, SCA, WI, and size-matched WCMI.

Critical strengths
  1. The paper is well motivated by a real gap: latent-component methods produce components that cannot be traced back to individual neurons and spike times, while information rankings ignore what the current decoder already captures. The residual score has a clear first-order interpretation as the negative cross-entropy gradient for adding a coordinate. Computing it from grouped out-of-fold predictions keeps it from chasing training-set errors.

  2. The numerical evaluation is comprehensive and covers several popular datasets on which RIS all performed well.

  3. The interpretability analyses are well designed and straightforward.

Critical weaknesses
  1. All baselines are either unsupervised (PCA, SCA) or filter rankings in Table 2. It remains unclear whether a WCMI core plus one non-iterative screening step beats standard sparse decoders on the same Haar coefficients at matched support. Examples could be L1L_1, elastic-net, or group-lasso multinomial logistic regression, or forward selection that refits after each addition.

  2. According to the Triple-N paper, the stimulus set consists of 1,000 Natural Scenes Dataset (NSD) images and the COCO scenes that typically contain multiple objects. However, the manuscript does not specify how the five semantic categories (like faces, bodies, etc.) in Figure 4 were assigned, whether the classes are balanced, or how many images belong to each category.

  3. Several key modeling choices are missing from the main text, most notably the specific architecture of the decoder model and its regularization scheme.

Questions for authors

See weakenesses

Reviewer_CRPXOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this work, the authors propose a framework named: Residual-Informed Sparse Decoding (RISD), which combines information-theory-based feature selection with guided support completion to doscover traceable neuron–time–scale features in neural datasets. Under three extensive neural recording datasets, RISD improves decoding under matched feature selections and analysis how selected responses’ contributions depend on population context and decoding objectives. The results show robust decoding gains over the evaluated baseline methods.

Critical strengths
  1. The research focus of the proposed framework, on sparse, traceable neural readout, plays an important role in neuroscience research.
  2. The scientific findings of the proposed RISD, as listed in Contirbution point 3 on Line 077, is scientifically-meaningful to me .
  3. The experimental results over three real-world neural datasets are abundant.
Critical weaknesses
  1. I think the overall contribution of this proposed RISD is okay to me. I am curious that is there any specific contributions on the methodology or algorithm novelty of the proposed framework.
  2. Adding more related-work discussion could better enhance semantic interpretability of latent representations. A relevant complementary approach not listed in the paper yet is MIG-Vis [1], which uses generative visualization to investigate the semantic selectivity of neural latents.

[1] Uncovering Semantic Selectivity of Latent Groups in Higher Visual Cortex with Mutual Information-Guided Diffusion, Wang et al., ICLR 2026

Questions for authors

I have no more questions here. Please refer to my Weaknesses section to see my other questions or concerns.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.