- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This work proposes SpikeARC, a framework for continual reinforcement learning that aims to balance rapid adaptation to changing environment dynamics with the retention of previously acquired behaviors. The method combines a Stable pathway for the mature policy with recurrent spiking Fast and Fusion modules that generate and gate state-dependent action corrections. Experiments are conducted on continuous-control environments using PPO, additional CARL benchmarks using SAC.
Critical strengths
-
Balancing adaptation to changing dynamics with the retention of previously learned control behaviors is interesting and useful for long-term robot deployment.
-
The paper considers several continuous-control environments, two RL algorithms (PPO and SAC), and component ablations.
Critical weaknesses
-
Presentation and formatting need improvement. The keyword list appears unusual in the ICLR format. Several figures leave substantial whitespace, and some tables contain awkward line breaks that make their contents difficult to read. The overall layout could be made more compact and consistent.
-
The central mechanisms are largely independent of the SNNs (e.g., Stable and Fast policy decomposition, contribution-aware gradient routing, selective consolidation, and behavioral-drift constraints). GRU or LSTM modules can also replace the recurrent LIF modules. Besides, Appendix C.3 reports that the GRU variant achieves better performance. The method appears to combine several general continual RL mechanisms with an interchangeable SNN implementation with limited novelty and contributions.
Questions for authors
Does SpikeARC improve a conventional GRU/LSTM policy?
Can you compare the performance of vanilla LIF/GRU/LSTM and SpikeARC-LIF/GRU/LSTM in one table?
What is the specific contribution of SNNs to SpikeARC?
Can you include and discuss more recent SNN-RL studies in the related work section?
Can the proposed method be compared against vanilla recurrent policies and relevant SNN-RL baselines?