CoRe-INR: Coordinate Rescaling for Spectral Expansion and Convergence Improvement for Implicit Neural Representations

Implicit Neural Representations (INRs) create continuous representations for a broad range of signals by mapping the signal coordinates and corresponding values. However, INRs have limitations in learning high-frequency components due to the spectral bias. In this research, we explore the advantage of coordinate rescaling to mitigate spectral bias. We theoretically explain that scaling would improve the spectral representation characteristics using frequency atoms with a trade-off of introducing a spectral jitter. This spectral-aligned scaling improves convergence, explained through the Neural Tangent Kernel and the novel perspective of the spectral representation of tangent features. Specifically, we observe that some signals demonstrate a delayed learning behavior while scaling reduces this barrier to reach high-frequency content, explained through the tangent features. Further, through experimental results, we argue that there is a potential to outperform a deeper network with a shallow network, with the support of scaling. Moving further, we introduce CoRe-INR that demonstrates axis-based scaling could improve representation and inverse task modeling of high-dimensional signals.CoRe-INR outperforms the state-of-the-art algorithms, notably obtaining a +1.7 dB improvement in image representation. Our groundwork provides a new opening for future research in this domain.