SenSeed-LLM: Sensitivity-Aware Secure Seed-Based Weight Quantization for LLMs

Large language models (LLMs) incur substantial storage, memory-bandwidth and energy costs, motivating compact weight representations. Existing seed-based compression methods reconstruct weights from compact pseudo-random representations but do not explicitly account for the non-uniform sensitivity of model weights. We introduce SenSeed, a sensitivity-aware seed-based weight compression framework that uses lightweight Linear Feedback Shift Register (LFSR)-based weight generation with non-uniform bit allocation. SenSeed assigns larger representation budgets to sensitive weights while aggressively compressing less sensitive regions. Importantly, this non-uniform allocation requires no side-information: the decoder deterministically reconstructs the bit-allocation schedule, with no rung depending on the decoded weights, eliminating the need to store per-block metadata or use calibration data while preserving the baseline coding rate. Experiments on Llama-2-7B/13B, Llama-3-8B and Mistral-7B models show that SenSeed matches 4-bit perplexity of SeedLM with 3-6% fewer bits, while at the same 4 bits/weight it reduces perplexity degradation by 23-42% and zero-shot accuracy loss by 27-55% relative to SeedLM. We also show that SenSeed simultaneously achieves high security against bit-flip attacks on model parameters, as bit corruption affects multiple reconstructed weights, greatly amplifying its impact and making it easier to detect. We further implement SenSeed in an ASIC-based accelerator and demonstrate modest hardware overhead compared to prior seed-based approaches.