A Functional Shrinkage Approach for Input Optimization

Input optimization is a major methodology in deep learning that has been widely applied in adversarial attacks, neural network verification, and deep reinforcement learning. An important topic is how to inject a perturbation with a limited budget to the crucial part of a network, in order to avoid unstable optimization paths or meaningless outcomes. In this work, we formulate a network as a functional of activation functions, and propose a functional shrinkage approach to extract the principal component of the network. Then the residual component can be vulnerable to perturbations, which is exploited for perturbation injection. Experimental results show that the proposed method achieves state-of-the-art performance in several input optimization tasks. It provides a new insight into the structural decomposition and weakness exploitation of neural networks.