Recursive Self-Improvement Agent for Adaptive Symbolic Pruning Strategy Discovery

Post-training pruning is indispensable for deploying large language models (LLMs), and the importance metric that ranks weights largely determines the quality of the sparsified model. Hand-crafted metrics such as Wanda and SparseGPT encode fixed human intuition and do not adapt to new architectures or high-sparsity regimes, while automated symbolic discovery methods such as Pruner-Zero run genetic programming (GP) with frozen hyperparameters and a frozen evaluation pipeline, which leads to premature convergence and a prohibitive GPU cost per candidate. We propose PrunerAgent, an agent-based recursive self-improvement (RSI) framework for symbolic pruning-metric discovery. PrunerAgent nests two loops: an inner GP loop evolves symbolic trees over weights, activations, and gradients, and an outer meta-agent monitors population statistics and adapts the GP hyperparameters, compute budget, seeding strategy, grammar constraints, and validation thresholds of the inner loop. The search process itself, not only its output, is therefore improved during the run. Two further components reduce cost: an agent-managed symbolic search space that canonicalizes expressions and prunes theoretically defective subexpressions before evolution, and a six-stage validation pipeline that combines symbolic reasoning with progressively more expensive proxy evaluations so that only promising candidates reach full GPU evaluation. On LLaMA-1/2/3 and Mistral models at 50–80% unstructured and 4:8/2:4 semi-structured sparsity, consistently outperforms Pruner-Zero and hand-crafted baselines while reducing search cost by 62%.