WHERE THE NOISE HIDES: BLIND DENOISING FROM THE SCORE JACOBIAN’S SPECTRUM

Blind denoising, i.e., recovering a clean signal when the noise level is unknown, is a long-standing challenge in signal processing and machine learning. Differentiating Tweedie's formula ties the Jacobian of the score function to the posterior covariance of the clean signal, and hence to the noise variance. However, conventional methods built on it assume the true score function, ignoring the approximation error that neural network training introduces. In this work, we target the noise estimation and denoising problem by analysing the effect of that error on the spectrum. We first show that when the clean signal is supported on a low-dimensional manifold, the true score Jacobian has a cluster of eigenvalues at $-1/{\sigma }^2$ displaced only slightly by signal manifold curvature, making the smallest eigenvalue an optimal noise estimator with the exact score. Using neural tangent kernel theory, we illustrate that score matching perturbs the learned score Jacobian with zero mean initialization randomness, even for a network that interpolates its training samples exactly. In an idealized case the resulting distribution of the Jacobian noise eigenvalues is symmetric about $-1/{\sigma }^2$ for any covariance of the perturbation, implying that the centre of the eigenvalue cluster is robust to learning perturbation while the smallest eigenvalue drifts to underestimate noise. A practical algorithm, JAcobian-spectrum Noise Estimation (JANE), is hence proposed to estimate ${\sigma }^2$ from the leftmost mode of the eigenvalue density, with no knowledge of the noise level, the signal dimension, or a clean reference. Across six dataset configurations and different noise levels, JANE attains $3.3$ % mean relative error in ${\sigma }^2$ estimation against $9.4$ % for Noise2Score and $28.1$ % for UNSURE, and matches an oracle step-size denoising to within $0.05$ dB PSNR in all dataset and noise level settings.