- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates how dense-stage optimization affects the compressibility of implicit neural representations for video compression. The authors identify a phenomenon termed "spectral drift," where prolonged dense fitting shifts weight energy toward weakly excited input directions, making the model more sensitive to pruning and quantization. Based on a spectral analysis of optimization dynamics, they propose SpecFit, which combines decoupled weight decay with a larger learning rate during dense fitting.
Critical strengths
-
The paper provides a useful perspective on why better dense reconstruction quality does not necessarily translate into better compressed performance. The proposed spectral analysis offers an intuitive explanation connecting weight energy distribution with sensitivity to pruning and quantization.
-
SpecFit requires only modifications to dense-stage optimization without introducing additional architectural complexity. The reported BD-rate improvements across multiple INR backbones are encouraging and suggest that dense-stage optimization is an important but relatively underexplored factor in INR-based video compression.
Critical weaknesses
The theoretical analysis relies on simplifying assumptions that are not sufficiently validated. The analysis in Section 3.2 approximates AdamW using a scalar preconditioner and assumes stationary input statistics. However, actual AdamW uses coordinate-wise adaptive preconditioning, and intermediate feature distributions evolve throughout training. These assumptions limit how directly the proposed spectral dynamics explain the observed behavior. Moreover, the conclusion that weakly excited modes accumulate disproportionately large weight energy depends on the relationship between the target cross-covariance and the input eigenspectrum, which is not explicitly characterized.
Questions for authors
-
How well does the proposed spectral analysis explain the behavior of actual AdamW with coordinate-wise preconditioning and non-stationary intermediate features? In particular, can the authors clarify under what conditions the target cross-covariance causes weakly excited input modes to accumulate disproportionately large weight energy?
-
Could the authors provide an unregularized NVRC-net baseline under the same compression pipeline and fine-tuning schedule as NVRC-net+SpecFit?