- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies parameter-efficient adaptation of cooperative multi-agent policies when mission objectives change in a heterogeneous Earth-observation satellite cluster. It proposes SWORD, which combines SVD-initialized low-rank adaptation with Fisher-information-guided sparse updates to adapt a pretrained HADT policy using HAPPO. The theoretical analysis formulates parameter-efficient adaptation as restricted policy-gradient optimization and provides a conditional local comparison between low-rank and sparse updates. Experiments use a three-satellite Basilisk/BSK-RL environment with four reward configurations that vary the relative importance of target priority and cloud-related objectives, yielding twelve source–target transfer pairs. SWORD reports an average completion rate of 35.45%, compared with approximately 35.99% for full fine-tuning, while updating 7,194 rather than 44,226 parameters. Component ablations suggest a larger contribution from informed sparse selection than from SVD initialization. Reported wall-clock training time is comparable to full fine-tuning.
Critical strengths
-
A useful empirical trade-off between trainable parameter count and adaptation performance. Across twelve transfer directions, SWORD achieves an average completion rate within approximately 0.54 percentage points of full fine-tuning while updating only 16.3% as many parameters. It also outperforms the reported low-rank and sparse baselines on average. This provides encouraging evidence that substantial adaptation capacity can be retained within a restricted parameter budget in this application.
-
A well-motivated application with a controlled transfer setting. Changing mission preferences while keeping the satellite environment and action spaces fixed provides an interpretable setting for studying policy adaptation under reward changes. Evaluating all directed transfers among four task configurations makes the empirical observation more informative than a single selected transfer example. The use of a physics-based satellite simulator also connects the learning problem to concrete resource-management constraints.
Critical weaknesses
-
The algorithm specification and its theoretical interpretation are not sufficiently aligned. In Algorithm 1, the SVD initialization makes equal to the leading rank- component of , while the adapted weight is written as . Without an explicit compensation step, this changes the pretrained policy before adaptation begins. Moreover, Equation (3) defines importance using squared log-policy gradients, whereas Algorithm 1 uses squared gradients of the HAPPO loss; these quantities are generally different. The theoretical analysis introduces another unresolved connection: membership of a LoRA update in a tangent space does not imply that ordinary optimization of its factors implements the orthogonal projected-gradient update used in the analysis. Theorem 1 is a conditional local comparison between low-rank and sparse updates, rather than a guarantee for their combination or its approximation to full fine-tuning. These issues make it difficult to determine precisely which algorithm the reported results validate and how the theory explains that algorithm.
-
The evaluation does not sufficiently isolate the benefit of the combined design or establish practical computational savings. The experiments cover one three-satellite configuration and reward reweightings within the same environment. The LoRA baselines update 81.3% of the parameters, compared with 16.3% for SWORD, without a systematic comparison at matched trainable-parameter budgets. The SVD ablation changes average completion rate by only 0.34 percentage points, but Table 3 does not report uncertainty, so the reliability of this contribution is unclear. Finally, Table 7 reports essentially identical training time for SWORD and full fine-tuning, approximately 0.47 hours. The parameter-count reduction is supported, but the current evidence provides limited support for broader claims about reduced fine-tuning cost or faster adaptation. These limitations matter because the benefit of combining the two adaptation mechanisms is central to the contribution.
Questions for authors
The following questions request clarification or supporting evidence for the current claims, rather than suggesting future extensions.
- What is the exact implemented initialization and sparse-update procedure? Is the leading rank- component subtracted from the frozen base weight so that adding the initialized preserves the source policy? How is initialized, how often is the mask recomputed, and what happens to previously accumulated sparse updates when an entry leaves the mask? Please also identify which layers use each update mechanism and provide a parameter-count breakdown explaining the reported 7,194 trainable parameters.
- Which importance estimator is actually used, and how does the theory correspond to the implemented optimizer? Is selection based on the diagonal score-function Fisher in Equation (3), per-sample squared loss gradients, or a squared batch-averaged loss gradient? What justifies treating the implemented LoRA factor updates as the orthogonal projected-gradient updates in Proposition 3.2?
- How robust is the reported advantage under comparable parameter budgets and explicit uncertainty estimates? Please specify the number of independent training seeds and evaluation episodes, the meaning of the reported “±” values, and the precise definition of completion rate. Can the authors provide uncertainty estimates for Table 3 and comparisons of SWORD, low-rank adaptation, and sparse tuning at a common trainable-parameter budget?
10 comments
Gentle Follow-up on Reviewer 7NcF Feedback
Dear Reviewer 7NcF,
We would like to kindly follow up on our previous response to your comments and suggestions. We would appreciate it if you could let us know whether you have any further questions, concerns, or clarifications regarding our responses and the revisions made to the manuscript.
We would be happy to address any remaining concerns. Thank you very much for your time and consideration.
Kind regards,
Authors
Second Gentle Follow-up
Dear Reviewer 7NcF
As the rebuttal period deadline is approaching, we would greatly appreciate it if you could let us know whether you have any further clarification, questions, or concerns remain. We hope that we can address any additional comments from the reviewer in timely manner.
Thank you.
Kind regards,
Response to Reviewer 7NcF
We thank the reviewer for providing detailed assessment. We have addressed these concerns through several clarifications and revisions:
Weakness-1:
These revisions provide more connections to the implemented algorithm, the theoretical analysis, and the reported experimental results.
Weakness-2:
Question 1:
We apply SVD to extract low rank adaptation weights (A and B) at the initialization phase before adaptation. We decompose the source trained weights (W0) then select only top-r component to calculate A and B matrix. This decomposition may have small error as it is a dense projection from the original W0.
Additionally, Sparse tuning has a sparsity parameter (s) and this value is predefined and we set this value as 0.8 to adapt only 20% of the tuned parameters. Sparse mask is calculated based on Fisher Mutual Information estimation [1]. It approximates the weight importance based on the gradient over mini batch of data sample. In practice, the importance is calculated from the gradients which are accumulated across N mini batches before computing their squared magnitude: FiMI(W)=(N1∑j=1Ngi(Bj))2.
A binary importance mask m∈{0,1}d is then constructed by selecting the parameters with the largest estimated importance value:
We understand that our explanation was not sufficient to cover this important aspect. Now, it has been modified as described in the Section 3.2.
In this work, we use HADT architecture and it consists of several modules. By defining rank and sparsity as r=4, s=0.8 of SWORD , the trainable parameters can be summarised by HADT component as follows and frozen parameters are counted as zero trainable parameters.
Question-2:
The importance estimation is calculated based on Fisher Mutual Information estimation [1]. It approximates the weight importance based on the gradient over mini batch of data sample. In practice, the importance is calculated from the gradients which are accumulated across N mini batches before computing their squared magnitude, as described in Question 1.
Question 3: Our policy adaptation developed based on HARL framework [2] with 5 different random seeds of network parameters and 20 parallel training rollouts. For evaluation, 5 parallel rollouts and 3 episodes are used to ensure policy robustness. The detail of our policy training parameters are presented in Appendix A.1. in the updated version.
We look forward to receiving your feedback on this response. Should you have any further questions or require additional clarification, please do not hesitate to reply. Thank you for your time and consideration.
References:
[1] Zhang, Huan, et al. "Sparse tuning enhances plasticity in PTM-based continual learning." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 40. No. 33. 2026.
[2] Zhong, Yifan, et al. "Heterogeneous-agent reinforcement learning." Journal of Machine Learning Research 25.32 (2024): 1-67.