A Dendritic Neuron with Intra-Branch Nonlinear Modulation for Sequence Modeling

Dendritic neurons enrich sequence models with multiple compartment states that encode temporal memory. However, common branch readouts based on a weighted sum of compartment states do not explicitly distinguish the different functional and computational roles of compartments, restricting how internal state relationships shape the branch output. Inspired by location-dependent, asymmetric interactions within cortical pyramidal dendrites, we propose the distal-modulated proximal readout, which assigns proximal and distal roles to the compartments within each branch and uses their state differences to modulate the proximal response. Building on this readout, we construct the Nonlinear-Readout Dendritic Neuron (NR-DN), which introduces a role-specific nonlinear branch readout while retaining parallel computation of compartment states. The soma of NR-DN complements this temporal processing with a short, learnable causal window and bounded integer firing, allowing diverse frequency responses to the dendritic output. NR-DN preserves channel dimensions and can be integrated into different sequence backbones. Experiments on image, speech, and text sequence tasks demonstrate competitive performance with only one active branch per neuron. S4 equipped with NR-DN achieves 87.53% average accuracy on the Long Range Arena benchmark, demonstrating that NR-DN is an effective neuron model for sequence modeling.