acceptodds
Under review as a conference paper at ICLR 2027

Orientation selective dendritic feature representation supports robust pattern identification

Abstract

Orientation-selective synaptic inputs in visual cortex motivate studying how feature selectivity is formed and represented within dendritic computations, and how dendritic representations influence network-level visual processing. We investigate this question using a Two-Stage Dendritic Network (TSDN), in which multiple dendritic branches process local inputs and their nonlinear responses are integrated at the soma. Hebbian and anti-Hebbian plasticity first learn feature detectors from unlabeled inputs; these detectors are then frozen while somatic integration and a classification readout are trained with supervision. Gabor analysis reveals orientation-selective feature structure, while ablations show that anti-Hebbian plasticity reduces detector redundancy and improves downstream classification relative to Hebbian-only learning. Across five image datasets, TSDN achieves higher mean classification accuracy than the tested backpropagation-trained baselines. It also exhibits improved robustness to Gaussian input noise on all tested datasets except CIFAR-10. Varying the anti-Hebbian learning rate reveals a discriminability–robustness trade-off, accompanied by non-monotonic changes in feature frequency content and branch-response sensitivity. These findings connect unsupervised dendritic feature formation with recognition and noise tolerance, highlighting plasticity strength as a factor shaping the computational properties of dendritic representations.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Dendritic neural networks, Unsupervised representation learning, Orientation selectivity, Hebbian and anti-Hebbian plasticity, Noise robustness

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_TJ91OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces a two-stage dendritic network to study how unsupervised local plasticity shapes visual representations. Dendritic branches learn filters from local image patches using Hebbian and anti-Hebbian updates with competitive sparsification. These filters are then frozen, while somatic integration weights and a classification readout are trained with supervision.

Critical strengths

The random-feature, Hebbian-only, and combined-learning comparisons show that the anti-Hebbian component improves downstream classification while reducing redundancy relative to Hebbian-only learning. TSDN outperforms the tested backpropagation baselines under Gaussian noise on four datasets, with the CIFAR-10 exception explicitly acknowledged.

Critical weaknesses

Unsupervised feature learning and supervised readout are an existing paradigm. Comparisons primarily against backpropagation baselines do not establish what the proposed dendritic organization or learning procedure adds beyond closely related unsupervised methods.

Gabor-like filters, reduced redundancy, and improved performance occur together, but the experiments do not establish whether orientation selectivity or dendritic grouping specifically contributes to the gains. Generic competitive learning, local connectivity, or freezing-induced regularization remain plausible explanations.

In addition, the paper should more clearly position TSDN relative to existing biologically inspired unsupervised learning approaches, including Unsupervised learning by competing hidden units; A two-stage strategy for brain-inspired unsupervised learning in spiking neural networks; A novel un-supervised burst time dependent plasticity learning approach for biologically pattern recognition networks; An unsupervised STDP-based spiking neural network inspired by biologically plausible learning rules and connections; Biologically plausible unsupervised learning for self-organizing spiking neural networks with dendritic computation.

Questions for authors

See the weakness

Reviewer_UK2JOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how unsupervised feature learning in a dendrite-inspired network affects representation structure, classification, and noise robustness. In the Two-Stage Dendritic Network, Hebbian learning with a competitive anti-Hebbian-inspired term learns local feature detectors, which are then frozen while somatic integration and a classification readout are trained with supervision. Gabor fitting identifies oriented feature structure, and ablations show reduced filter similarity, increased response dimensionality, and improved FashionMNIST accuracy with competition. Across five datasets, the method reports marginally higher accuracy than the tested backpropagation baselines but significantly improved robustness to clipped Gaussian test noise on four datasets, with CIFAR-10 the exception. A competition-strength sweep relates nonmonotonic changes in clean/noisy accuracy to feature frequency content and branch response sensitivity.

Critical strengths

The reported Gaussian-noise robustness gains are substantial on MNIST, FashionMNIST, KMNIST, and EMNIST, despite training only on clean images. Including a backpropagation-trained dendritic baseline makes the comparison more informative than comparison with an MLP alone. Results across five seeds and explicit acknowledgment of the CIFAR-10 limitation help delimit the empirical finding.

Critical weaknesses
  1. The novelty over prior work is not sufficiently established. The manuscript appropriately acknowledges existing dendritic architectures, but learning oriented features through nonlinear Hebbian learning is well established (Brito and Gerstner, 2016). Competitive Hebbian features with a supervised classifier have also previously shown Gaussian-noise robustness advantages over backpropagation on MNIST and FashionMNIST (SoftHebb; Moraitis et al., 2022). Redundancy reduction versus noise suppression and discrimination versus tolerance of noisy inputs likewise have explicit precedents (Atick and Redlich, 1992; Barak et al., 2013). These works do not duplicate the exact architecture or parameter sweep. However, the paper needs to identify what new computational insight its dendritic organization and learning rule establish beyond these known phenomena; the current comparisons and descriptive statistics leave that distinction unclear.

    Refs:

  • Brito & Gerstner (2016) — Nonlinear Hebbian Learning as a Unifying Principle in Receptive Field Formation.
  • Moraitis et al. (2022) — SoftHebb: Bayesian inference in unsupervised Hebbian soft winner-take-all networks.
  • Atick & Redlich (1992) — What Does the Retina Know about Natural Scenes?
  • Barak, Rigotti & Fusi (2013) — The Sparseness of Mixed Selectivity Neurons Controls the Generalization–Discrimination Trade-Off.
  1. The evidence for the proposed representational mechanism is incomplete. Lower whole-filter cosine similarity and higher response participation ratio support broader feature diversity, but do not specifically establish orientation diversity among branches belonging to the same neuron or its role in performance. The pool of competition is unspecified, and questionable angle annotations in Figure 3A further limit this interpretation. Similarly, Figure 6 shows that frequency content and response sensitivity vary with competition strength, but does not isolate any specific factor as the explanation for the accuracy changes. Even establishing this is not enough for a satisfying level of mechanistic interpretation, some version of analytical understanding is also central to the scientific contribution beyond the benchmark results.
Questions for authors
  1. Model specification: What is M in Section 2.2, and how does detector index m map to the neuron/branch indices (i,j)? Which units enter k-WTA and C: branches belonging to one neuron, units across neurons, or a separately learned feature bank? Are filters shared or reused across somas? Please also specify the populations used for cosine similarity and participation ratio. This determines how the learning analysis supports claims about organization among branches of a single neuron.

  2. Orientation analysis: In Figure 3A, d24 under combined learning receives angles of 105 degrees, 90 degrees, and 3 degrees at iterations 500, 2000, and 10000, although the filters appear to develop a similar upper-positive/lower-negative pattern. Could you provide the fitted reconstructions and fit correlations, and verify the angle conversion/annotations? Figure 2's 95-degree and 93-degree examples also do not by themselves establish functionally separated orientation channels. What population-level evidence distinguishes orientation diversity from differences in phase, frequency, or other filter properties? A fitting problem would affect the orientation interpretation without necessarily invalidating the separate diversity and accuracy measurements.

  3. Noise mechanism: How is the proposed high-frequency explanation reconciled with per-filter L2 normalization? Before clipping and ReLU, white noise are independent of frequency content at fixed norm. Equation 7 also normalizes by the clean-response magnitude. Can the observed sensitivity changes be attributed to clean-signal responses, nonlinear processing, or population/readout effects, rather than raw single-filter noise amplification? Please clarify what the evidence establishes, or temper the mechanistic interpretation.

  4. Empirical reporting: Please specify the selected model dimensions, model-selection procedure, and baseline optimization schedules. Figure 4A appears to reach approximately 91% FashionMNIST accuracy for TSDN, whereas Table 1 reports 89.98% for the best-performing configuration; are these different conditions? Please also reconcile the 2.0M/20.0M baseline parameter examples in Section 3.2 with the displayed sweep, whose axis ends near 1M. I recognize that the figures count gradient-trained parameters; the distinction from total stored parameters should be maintained when discussing efficiency.

Reviewer_9mj3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates how unsupervised plasticity shapes feature representations in a Two-Stage Dendritic Network (TSDN). Each soma integrates nonlinear responses from multiple branches receiving a fixed local image patch. Branch filters are first learned using Hebbian and anti-Hebbian updates with k-winner-take-all competition, then frozen while somatic integration and a classification readout are trained with supervision. The paper analyzes learned filters using Gabor fitting, feature similarity, and response participation ratio. It reports higher mean classification accuracy than the tested BP-MLP and BP-DN baselines on five datasets, and improved performance under Gaussian input noise on four datasets, with CIFAR-10 as the exception. Experiments on FashionMNIST further show a non-monotonic relationship between anti-Hebbian learning strength, feature diversity, clean accuracy, and noise robustness.

Critical strengths
  1. The random-feature and Hebbian-only baselines are useful. Comparing them with the combined Hebbian–anti-Hebbian rule helps show what the feature-learning procedure contributes within this architecture.
  2. The paper looks at more than classification accuracy. Gabor fits, cosine similarity, participation ratio, frequency content, and response sensitivity give several perspectives on the learned features. I found the non-monotonic changes with anti-Hebbian strength particularly interesting and worth exploring further.
  3. Reporting results over five seeds helps assess their variability. BP-DN is also a relevant comparison because it keeps the same overall architecture and changes how the model is trained. The authors clearly acknowledge that the robustness gains do not consistently hold on CIFAR-10.
Critical weaknesses
  1. My main concern is what we learn from this work beyond the results in this particular setting. The paper brings together an existing dendritic architecture and Hebbian/anti-Hebbian feature learning, but the additional insight is less clear. I would like a clearer explanation of how this advances our understanding of dendritic computation, or what practical benefit it offers for representation learning.
  2. It is difficult to tell how much the dendritic organization itself contributes. The ablations show that anti-Hebbian learning helps, but BP-DN keeps the same architecture, so that comparison mainly addresses the training procedure. Using the same learned filters with a different integration structure would help establish whether the branch-to-soma organization matters for the reported results.
  3. I am not fully convinced by the explanation of robustness. The Gabor analysis describes the learned filters, and Figure 6 shows how frequency content, response sensitivity, and accuracy change with anti-Hebbian strength. However, several properties change together. These results do not yet show whether orientation selectivity contributes to robustness, or whether frequency content explains the changes in performance.
  4. Some experimental details are needed to assess the comparisons. Section 3.2 says that model sizes were selected separately for each method, but the final configurations, search ranges, and selection procedure are not sufficiently clear. I would also like to know how the baselines were trained and whether they had converged. The parameter comparison is useful for measuring the supervised parameter budget, but total training and inference costs remain unassessed.
Questions for authors
  1. What do the authors see as the main new insight beyond existing local competitive learning methods and dendritic networks? The two-stage training procedure is clear, but I would appreciate a more specific explanation of what this combination adds.
  2. Have you tried keeping the learned filters fixed and changing only how branch responses are combined? This could help separate the effect of feature learning from that of dendritic organization.
  3. In Figure 6, several feature properties change with anti-Hebbian strength. Is there evidence that separates the effects of orientation selectivity, redundancy, and response statistics on robustness? Which parts of the proposed explanation should currently be understood as correlations?
  4. Could you provide the configurations used in Table 1, the model sizes considered, and the data split used to select them? Please also clarify the baseline training schedules and whether training curves indicate that the baselines had converged.
  5. How do the MM filters in Φ\Phi correspond to the branch filters ϕi,j\phi_{i,j}? Are any filters shared between somas? Also, are k-WTA competition and the co-activation matrix computed within each soma or over a larger group of filters?
1 comment

Thank you for your careful response which has addressed my concerns on experiments. However, I maintain my concern of the insight or the novelty of the proposed method. The paper may need further research to give better solution. I will maintain my initial score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.