- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper proposes Spike Spectral Acceleration Regularization (SSAR), a regularizer for deep RL critics based on random matrix theory. The core idea is to boost singular values of the critic’s penultimate layer that sit just above the edge of the random spectral bulk to help task-relevant directions break out sooner. It switches off once enough modes have separated. Using the BBP phase transition, the authors prove that this extra update has no guaranteed effect on the loss below the threshold, but reliably lowers the Bellman loss at and above it. Synthetic experiments match the predicted scaling, with 98% of updates lowering the loss in those regimes. With SAC on six DeepMind Control tasks, SSAR beats vanilla SAC and L2 regularization in sample efficiency.
Critical strengths
The idea of using Random Matrix theory as a framework to analyze and regularize the critic might be potentially useful but the results do not support the motivation and conclusions.
Critical weaknesses
- The paper is difficult to read. Details are presented before the big picture which makes it difficult to understand the methodology.
- Results are reported on only 6 simple control tasks with no clear benefit, even over limited baselines. See for example: https://arxiv.org/abs/2602.19373
- Comparisons should be provided to many other methods addressing regularization in DRL, and highlighting the core advantages.
- Experiments only used 3 seeds resulting in very large standard deviations and no clear statistical significance.
Questions for authors
I don't have questions to the authors. I think the paper needs to be rewritten with less LLM use, and more human language and style. Results should be significant and demonstrate the benefits of the approach. At the moment, there is no clear advantage in the method presented.