Beyond Reconstruction: Auditing Sparse Features for Physical Channel Pruning

Sparse autoencoders (SAEs) describe activations, but structured pruning must choose physical channels. We audit whether SAE features improve that decision in Qwen3-4B. Matched SAE and TopK-PCA representations generate fixed-width masks with the same joint search; statistical selectors receive common recovery rules. At 35.53% MLP-channel removal, SAE improves seven-task mean accuracy by 0.285 points and lowers WikiText perplexity by 0.834 versus RMS across three calibration seeds. PCA has comparable accuracy (49.52% versus 49.50% for SAE). Larger SAEs trained on over one million activation positions improve reconstruction and some local risk rankings, but neither greater sparsity nor active decoder subspaces consistently beat direct-gradient deletion scores. Common mean recovery changes rankings; FLAP-score and RCPU-score adaptations exceed SAE accuracy in the measured setting. A separate larger-calibration audit favors SAE on perplexity but favors a random dictionary on task accuracy. These results identify a limited reconstruction-proxy benefit without establishing SAE-specific functional preservation or superior physical deletion decisions.