GliaLLM: Beyond Neuronal Spotlight —— Glia-Inspired Structural Learning in Large Language Models

Large language models (LLMs) typically rely on parameter scaling while maintaining fixed architectures, which leads to considerable redundancy in both model parameters and computation. Inspired by astrocyte-mediated bidirectional synaptic plasticity and microglia-mediated selective synaptic pruning, we propose the artificial Neuron–Astrocyte–Microglia (NAM), a new computational model for neuron-level structure learning in neural networks. Building on NAM, we develop the GliaLLM framework, in which Astrocyte Net (AS-Net) and Microglia Net (MG-Net) jointly learn layer- and projection-specific continuous scaling factors and binary gates from the LLM’s linear-layer weights during training. These modulation signals are then computed once offline and folded into the LLM to produce a deployable pruned subnetwork. We introduce a three-stage alternating optimization strategy to jointly learn model parameters and network structure. Relative to the pretrained Llama-style-1.1B baseline, GliaLLM achieves a 23.1% parameter reduction ratio (PRR), while incurring only a 0.006 decrease in aggregate $acc\_norm$ across six benchmarks. These results demonstrate that glia-inspired neuron-level structure learning can yield deployable LLM subnetworks that lower peak GPU memory usage across workloads and reduce TTFT for long-input inference.