Beyond Memorization: Auditing and Generating Functional Neural Weights

Neural weight generation promises to replace per-task optimization with reusable distribution learning over trained checkpoints, yet rigorous evaluation remains underdeveloped. Task accuracy alone cannot distinguish genuine synthesis from memorization and interpolation. Conversely, parameter-space novelty can arise from nonfunctional dispersion. We introduce WeightAUDIT, a five-metric diagnostic framework that jointly assesses parameter dependence, behavioral dependence, affine interpolation, task utility, and predictive diversity against independently trained references. Auditing nine generators reveals three dominant failure signatures, namely parameter memorization, interpolation, and ineffective generation. We also introduce Weight Diffusion with Instantiated Network Supervision (WINS). It instantiates complete networks from predicted clean weights and backpropagates task loss through these networks to train the denoiser. On KMNIST, WINS achieves 76.9% accuracy, exceeding the training mean of 72.8%, while maintaining weight novelty and predictive diversity near the reference level. Across MNIST, EMNIST, SVHN and CIFAR10, WINS retains utility without severe checkpoint contraction, interpolation, or predictive collapse, although performance varies by dataset. Varying the checkpoint pool size reveals an accuracy dip, followed by higher utility and lower training dependence in larger pools.