- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The manuscript focuses on the under-exploration of the row-wise N:M network in the RL area. The motivation is clear but can be improved. The contributions are multi-folds, covering diagnosis and evaluation, stabilization techniques, and time-multiplexing sparsity processing. Experimental results are extensive and convincing, while the generality of the proposed solution can be improved.
Critical strengths
- Clear motivation
- Promising techniques although a bit incremental
- Extensive experiments
Critical weaknesses
- The proposed algorithm is mainly based on incremental innovation and evaluated with TD3.
- The evaluation in certain experimetns is not unified.
Questions for authors
-
While Row-wise N:M sparsity is under-explored in RL, why do we want to use row-wise N:M sparsity in RL? The current described limitation seems to only focus on its less usage, but the rantionale behind this necessity and what benefit RLs could have by using this is not clear.
-
Concerning the observation of smaller initialization, what happens when initialization is adjusted separately for actor and critic?
-
The best return results use wider networks, while the end-to-end timing experiment uses width 256 and a different replay schedule. Can you report time-to-target-return and return-versus-wall-clock curves for the complete proposed configuration against tuned dense alternatives?
-
Can the observed improvements be extended to other RL backbones?
Thank you for the review. Your questions on the motivation, on the initialization, on wall-clock comparisons each led to an addition.
Q1: Why N:M sparsity for RL?
Because its regular structure turns sparsity into shorter training time, not merely fewer parameters. Larger networks are a main route to better returns in current deep RL, and Ma et al. (2025) showed that sparsity helps this scaling. But unstructured sparse networks are executed as masked dense ones and keep the dense cost. In our new experiments:
We strengthened this motivation in the Introduction, Related Work and Section 7.
W1/Q4: Novelty and other backbones
We agree that the paper builds on existing N:M sparsity and RNM-TD3, and we do not claim a fundamentally new RL algorithm. Its contributions are:
New Section 7 extends the static result to SAC with SimBa residual networks; the stabilization of dynamic masks is not tested beyond TD3.
Q2: Separate actor and critic initialization
We ran this experiment (new Appendix K, Table 29): actor and critic initialization multipliers varied independently, six environments, 15 seeds per configuration.
The critic's initialization scale has the stronger effect.
Q3: Return versus wall-clock time against tuned dense networks
W2: Unified evaluation
We agree. In Appendix E.4, all sparse and dense agents now train for two hours on one core with the same batch size (256), buffer (1M) and constant replay ratios (1, 2, 4), and Section 7 reports return and wall-clock time of the same runs. Still not unified: the speed ratios of Section 6 are measured at width 256 only, and Sections 3–5 keep their original settings, because each varies one factor against its own baseline.