acceptodds
Under review as a conference paper at ICLR 2027

SWORD: An Efficient Adaptation of Multi-Agent Policy with Sparse and Low Rank Decomposition for Heterogeneous Satellite Cluster

Abstract

This work investigates the adaptability of multi-agent policies to task changes in heterogeneous satellite clusters performing Earth observation missions. Since mission objectives and task requirements may vary across different mission scenarios, a policy trained for an initial task may experience a performance degradation when transferred to a new task, thereby requiring adaptation or fine-tuning. From a machine learning perspective, this problem can be formulated as a domain or task shift. Initially, the policy is trained using an on-policy multi-agent reinforcement learning (MARL) framework and subsequently transferred to a new task. To address the resulting performance degradation, we propose a policy adaptation approach based on Sparse and loW-Rank Decomposition (SWORD), which enables efficient fine-tuning of the previously learned policy by updating a reduced set of parameters. We first provide a theoretical analysis of the applicability of parameter-efficient fine-tuning within the MARL framework and subsequently validate the approach through experimental evaluation. The results demonstrate that SWORD's performance comparable to full-parameter fine-tuning while substantially reducing the number of trainable parameters. Furthermore, SWORD achieves improved performance compared with existing parameter-efficient fine-tuning baselines, demonstrating its potential for efficient policy adaptation in heterogeneous satellite cluster missions.

Primary area:
applications to robotics, autonomy, planning
Keywords:
multi-agent reinforcement learning, policy adaptation, fine tuning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_7NcFOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies parameter-efficient adaptation of cooperative multi-agent policies when mission objectives change in a heterogeneous Earth-observation satellite cluster. It proposes SWORD, which combines SVD-initialized low-rank adaptation with Fisher-information-guided sparse updates to adapt a pretrained HADT policy using HAPPO. The theoretical analysis formulates parameter-efficient adaptation as restricted policy-gradient optimization and provides a conditional local comparison between low-rank and sparse updates. Experiments use a three-satellite Basilisk/BSK-RL environment with four reward configurations that vary the relative importance of target priority and cloud-related objectives, yielding twelve source–target transfer pairs. SWORD reports an average completion rate of 35.45%, compared with approximately 35.99% for full fine-tuning, while updating 7,194 rather than 44,226 parameters. Component ablations suggest a larger contribution from informed sparse selection than from SVD initialization. Reported wall-clock training time is comparable to full fine-tuning.

Critical strengths
  1. A useful empirical trade-off between trainable parameter count and adaptation performance. Across twelve transfer directions, SWORD achieves an average completion rate within approximately 0.54 percentage points of full fine-tuning while updating only 16.3% as many parameters. It also outperforms the reported low-rank and sparse baselines on average. This provides encouraging evidence that substantial adaptation capacity can be retained within a restricted parameter budget in this application.

  2. A well-motivated application with a controlled transfer setting. Changing mission preferences while keeping the satellite environment and action spaces fixed provides an interpretable setting for studying policy adaptation under reward changes. Evaluating all directed transfers among four task configurations makes the empirical observation more informative than a single selected transfer example. The use of a physics-based satellite simulator also connects the learning problem to concrete resource-management constraints.

Critical weaknesses
  1. The algorithm specification and its theoretical interpretation are not sufficiently aligned. In Algorithm 1, the SVD initialization makes BABA equal to the leading rank-rr component of W0W_0, while the adapted weight is written as W0+BA+M⊙SW_0+BA+M\odot S. Without an explicit compensation step, this changes the pretrained policy before adaptation begins. Moreover, Equation (3) defines importance using squared log-policy gradients, whereas Algorithm 1 uses squared gradients of the HAPPO loss; these quantities are generally different. The theoretical analysis introduces another unresolved connection: membership of a LoRA update in a tangent space does not imply that ordinary optimization of its factors implements the orthogonal projected-gradient update used in the analysis. Theorem 1 is a conditional local comparison between low-rank and sparse updates, rather than a guarantee for their combination or its approximation to full fine-tuning. These issues make it difficult to determine precisely which algorithm the reported results validate and how the theory explains that algorithm.

  2. The evaluation does not sufficiently isolate the benefit of the combined design or establish practical computational savings. The experiments cover one three-satellite configuration and reward reweightings within the same environment. The LoRA baselines update 81.3% of the parameters, compared with 16.3% for SWORD, without a systematic comparison at matched trainable-parameter budgets. The SVD ablation changes average completion rate by only 0.34 percentage points, but Table 3 does not report uncertainty, so the reliability of this contribution is unclear. Finally, Table 7 reports essentially identical training time for SWORD and full fine-tuning, approximately 0.47 hours. The parameter-count reduction is supported, but the current evidence provides limited support for broader claims about reduced fine-tuning cost or faster adaptation. These limitations matter because the benefit of combining the two adaptation mechanisms is central to the contribution.

Questions for authors

The following questions request clarification or supporting evidence for the current claims, rather than suggesting future extensions.

  1. What is the exact implemented initialization and sparse-update procedure? Is the leading rank-rr component subtracted from the frozen base weight so that adding the initialized BABA preserves the source policy? How is SS initialized, how often is the mask recomputed, and what happens to previously accumulated sparse updates when an entry leaves the mask? Please also identify which layers use each update mechanism and provide a parameter-count breakdown explaining the reported 7,194 trainable parameters.
  2. Which importance estimator is actually used, and how does the theory correspond to the implemented optimizer? Is selection based on the diagonal score-function Fisher in Equation (3), per-sample squared loss gradients, or a squared batch-averaged loss gradient? What justifies treating the implemented LoRA factor updates as the orthogonal projected-gradient updates in Proposition 3.2?
  3. How robust is the reported advantage under comparable parameter budgets and explicit uncertainty estimates? Please specify the number of independent training seeds and evaluation episodes, the meaning of the reported “±” values, and the precise definition of completion rate. Can the authors provide uncertainty estimates for Table 3 and comparisons of SWORD, low-rank adaptation, and sparse tuning at a common trainable-parameter budget?
10 comments
AuthorsOpenReview ↗

Response to Reviewer 7NcF

We thank the reviewer for providing detailed assessment. We have addressed these concerns through several clarifications and revisions:

Weakness-1:

  • We revised Algorithm 1 to clarify the initialization procedure of the low-rank component and its relationship to the frozen pretrained weight, eliminating ambiguity regarding potential modification of the pretrained policy before adaptation.
  • We clarified the sparse parameter selection procedure and explicitly stated that the reported experiments use the Fisher-information-based importance estimation described in Equation (3), while updating the algorithm description to ensure consistency with the implementation.
  • We revised Section 3.2, Proposition 3.1, Theorem 1, and Appendix A.6 to clearly distinguish the idealized projected-gradient analysis from the practical factorized optimization used in SWORD.

These revisions provide more connections to the implemented algorithm, the theoretical analysis, and the reported experimental results.

Weakness-2:

  • We added a detailed parameter-efficiency analysis that reports the trainable parameter allocation across all SWORD components, providing a clearer understanding of the contribution of the sparse and low-rank mechanisms.
  • We expanded the ablation study to explicitly evaluate the contribution of each adaptation component and added standard deviation reporting to improve the interpretation of our algorithm performance.
  • We do not overstating claims regarding training speed, instead focus on the demonstrated reduction in trainable parameters. The current code implementation contains inefficiency, limiting reductions in runtime. Moreover, our satellite simulator dominates the computational usage compared to training process leading to a small difference in training time.

Question 1:

We apply SVD to extract low rank adaptation weights (AA and BB) at the initialization phase before adaptation. We decompose the source trained weights (W0W_0) then select only top-r component to calculate AA and BB matrix. This decomposition may have small error as it is a dense projection from the original W0W_0.

Additionally, Sparse tuning has a sparsity parameter (s) and this value is predefined and we set this value as 0.8 to adapt only 20% of the tuned parameters. Sparse mask is calculated based on Fisher Mutual Information estimation [1]. It approximates the weight importance based on the gradient over mini batch of data sample. In practice, the importance is calculated from the gradients which are accumulated across NN mini batches before computing their squared magnitude: FiMI(W)=(1N∑j=1Ngi(Bj))2F_i^{MI}(\boldsymbol{W})=\left(\frac{1}{N}\sum_{j=1}^{N}g_i(B_j) \right)^2.

A binary importance mask m∈{0,1}d\mathbf{m}\in\{0,1\}^{d} is then constructed by selecting the parameters with the largest estimated importance value:

We understand that our explanation was not sufficient to cover this important aspect. Now, it has been modified as described in the Section 3.2.

In this work, we use HADT architecture and it consists of several modules. By defining rank and sparsity as r=4, s=0.8r=4,\ s=0.8 of SWORD , the trainable parameters can be summarised by HADT component as follows and frozen parameters are counted as zero trainable parameters.

HADT ComponentSWORD Trainable Parameters
Embedding (frozen)0
Differential Attention (Mixer + λ\lambda)411
MHA 1 (LoRA_A for Wq,WvW_q, W_v)52
MHA 2 (LoRA_A for Wq,WvW_q, W_v)52
Normalization 126
MLP Layer 13,328
MLP Layer 23,290
Normalization 226
Action Output14
Total (sum of component rows)7,199

Question-2:

The importance estimation is calculated based on Fisher Mutual Information estimation [1]. It approximates the weight importance based on the gradient over mini batch of data sample. In practice, the importance is calculated from the gradients which are accumulated across NN mini batches before computing their squared magnitude, as described in Question 1.

Question 3: Our policy adaptation developed based on HARL\textit{HARL} framework [2] with 5 different random seeds of network parameters and 20 parallel training rollouts. For evaluation, 5 parallel rollouts and 3 episodes are used to ensure policy robustness. The detail of our policy training parameters are presented in Appendix A.1. in the updated version.

We look forward to receiving your feedback on this response. Should you have any further questions or require additional clarification, please do not hesitate to reply. Thank you for your time and consideration.


References:

[1] Zhang, Huan, et al. "Sparse tuning enhances plasticity in PTM-based continual learning." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 40. No. 33. 2026.

[2] Zhong, Yifan, et al. "Heterogeneous-agent reinforcement learning." Journal of Machine Learning Research 25.32 (2024): 1-67.

AuthorsOpenReview ↗

Gentle Follow-up on Reviewer 7NcF Feedback

Dear Reviewer 7NcF,

We would like to kindly follow up on our previous response to your comments and suggestions. We would appreciate it if you could let us know whether you have any further questions, concerns, or clarifications regarding our responses and the revisions made to the manuscript.

We would be happy to address any remaining concerns. Thank you very much for your time and consideration.

Kind regards,

Authors

AuthorsOpenReview ↗

Second Gentle Follow-up

Dear Reviewer 7NcF

As the rebuttal period deadline is approaching, we would greatly appreciate it if you could let us know whether you have any further clarification, questions, or concerns remain. We hope that we can address any additional comments from the reviewer in timely manner.

Thank you.

Kind regards,

Reviewer_LLbFOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

The paper studies how to adapt an existing multi-agent policy with fewer trainable parameters after task rewards change in a heterogeneous satellite cluster. Using HADT as the policy architecture and HAPPO as the training framework, the authors combine low-rank adaptation based on the SVD of pretrained weights with Fisher-information-guided sparse updates to form SWORD. The theory compares the local objective improvements of low-rank and sparse updates through gradient projections in restricted parameter spaces. The experiments use a Basilisk/BSK-RL simulation environment with two optical satellites and one SAR satellite. Four tasks are constructed by changing reward weights associated with target priority and cloud coverage, and all 12 directed transfer pairs are evaluated. The authors report that after 160K fine-tuning steps, SWORD achieves an average completion rate of 35.45%, compared with 35.99% for full-parameter fine-tuning, with 7,194 and 44,226 trainable parameters, respectively. Component ablations and training-time comparisons are also provided.

Critical strengths
  1. The application problem is concrete, and the cross-task adaptation setting has a clear practical context. The paper focuses on adapting existing multi-agent policies to new satellite mission preferences by changing reward weights while keeping the environment dynamics and observation and action spaces unchanged. Sensor heterogeneity, power, storage, and downlink constraints provide a concrete setting for the problem. The satellite and orbital parameters in the appendix also help clarify the evaluation environment.

  2. The experiments cover multiple transfer directions and provide preliminary comparisons of parameter-efficient adaptation methods. Table 2 compares all 12 directed transfers among four tasks under a common budget of 160K fine-tuning steps, including Full FT, LoRA SVD, and Sparse FT as baselines. SWORD's average completion rate is 35.45%, close to Full FT at 35.99% and higher than Sparse FT at 33.45% and LoRA SVD at 32.35%. Table 3 also examines the roles of SVD initialization and Fisher-based parameter selection separately. These experiments provide preliminary positive results on the reported completion-rate metric and empirical support for further investigation of parameter-efficient policy adaptation in this setting.

Critical weaknesses
  1. The independent technical contribution of combining low-rank and sparse updates has not been sufficiently substantiated.

    SWORD combines existing LoRA-SVD with Fisher-guided sparse fine-tuning but does not sufficiently explain the complementarity of the two update structures. Based on the shared environment dynamics and observation and action spaces of the source and target tasks, the paper suggests that adapting a source policy may involve modifying only a relatively small portion of its learned parameters. However, low-rank updates are generally dense, so limiting the number of trainable parameters does not establish sparsity in the resulting weight changes. The paper also does not analyze whether the update required for the target task contains components that the low-rank branch cannot represent but that the Fisher-selected sparse branch can supply. Thus, the current motivation is insufficient to explain why the two existing parameterizations need to be combined or what independent advantage this combination offers over using LoRA or sparse fine-tuning alone.

    The theoretical analysis in Section 3.3 also does not establish the effectiveness of this combination. Proposition 3.1 gives a local improvement result for general restricted gradient updates, while Proposition 3.2 describes the respective restricted update forms for LoRA and sparse fine-tuning. Theorem 1 bounds the difference between the local objective improvements of the two updates, assuming small differences in the squared norms of their projected gradients and in their local curvature quadratic forms. This theory does not analyze the joint update BA+M⊙SBA+M\odot S actually used by SWORD, nor does it establish any advantage of this combination over LoRA alone, sparse fine-tuning alone, or full-parameter fine-tuning.

    In addition, Equation (3) in the main text defines per-parameter Fisher importance as the expectation of the squared gradient of the policy log-probability, whereas steps 9–14 of Algorithm 1 in the appendix first compute the HAPPO loss gradient and then define the importance score as F=∥g(b)∥2F=\|g^{(b)}\|_2. These are different statistics. Moreover, under the usual definition of a norm, FF is a scalar and cannot directly be used for the subsequent per-parameter Top-K mask selection in the algorithm. The paper does not explain the per-parameter importance statistic actually used or its relationship to the Fisher definition in the main text, leaving the core sparse parameter-selection mechanism unclear.

    Therefore, the existing method description and theoretical analysis are not yet sufficient to support the necessity of SWORD's combined structure or its independent technical contribution relative to existing methods.

  2. The main experiments do not sufficiently validate target-task adaptation performance, and the core parameter-efficiency results lack verifiable implementation details.

    The paper constructs four tasks by changing reward weights associated with target priority and cloud coverage, but the main transfer comparisons and ablations report only CR (Completion Rate), without explicitly defining how it is computed or comparing the methods' returns under the same target reward function. The distinction between CR and target-task utility directly affects the adaptation conclusion. As Appendix A.3 explains, two policies may receive different returns even if they complete the same number of observation targets, because target priorities or cloud conditions differ. If CR measures only the number of completed targets, then under the paper's experimental setting, with unchanged environment dynamics, a fixed policy without any adaptation to task changes could also maintain the same CR across different reward settings. Thus, SWORD's average CR of 35.45% being close to Full FT's 35.99% in Table 2 is not yet sufficient to establish that the policy has adapted to the changed reward preferences.

    In addition, the paper reports that SWORD updates only 7,194 parameters, or 16.3% of Full FT. However, steps 15–19 of Algorithm 1 in the appendix describe updating both the LoRA factors A,BA,B and additional sparse variables SS, while Table 7 reports far fewer trainable parameters for SWORD than the 35,970 parameters of LoRA alone. The paper does not provide sufficient implementation details to explain this discrepancy. Therefore, the existing experiments neither directly validate adaptation under the target-task reward nor allow the reported parameter-efficiency advantage to be fully verified.

Questions for authors
  1. Complementarity of low-rank and sparse updates and the theoretical basis:

    SWORD combines low-rank and sparse updates, but what structure in the parameter changes required by the target task allows the Fisher-selected sparse updates to complement the low-rank branch? The theory in Section 3.3 mainly compares the individual local improvements of the two restricted updates. How does this analysis support the joint update BA+M⊙SBA+M\odot S used in practice? Could you provide comparisons of purely low-rank, purely sparse, and joint updates with matched trainable-parameter budgets to validate the complementary benefits of the two branches?

  2. The actual adaptation algorithm and trainable-parameter counts:

    Please provide the joint parameterization actually used in the experiments, the parameters to which the mask is applied, the rank, the per-layer freezing rules, and the trainable-parameter counts. Please explain how SWORD's 7,194 parameters and LoRA-SVD's 35,970 parameters are calculated. Algorithm 1 in the appendix describes updating both the low-rank factors A,BA,B and sparse variables SS; why does SWORD nevertheless have fewer parameters? Also, is parameter importance actually computed using the expectation of the squared policy log-probability gradient in Equation (3) of the main text, or the L2 norm of the HAPPO loss gradient in Algorithm 1? If the latter is used, how is this statistic used to compute per-parameter importance and construct the Top-K mask?

  3. The definition of completion rate and target-task utility:

    Please explicitly provide the formula for CR (Completion Rate) and the criteria for determining completion, and clarify whether CR accounts for reward differences arising from target priorities and cloud conditions. Since the four tasks define different optimization objectives primarily by changing reward weights, could you report the average cumulative returns of the methods in Table 2 under the same target reward function and compare the target returns of a source policy deployed directly without adaptation with those after fine-tuning? Please also specify the independent training runs, the number of evaluation trajectories, and how uncertainty in the results is quantified, to establish the difference between SWORD and Full FT in target-task utility.

7 comments
AuthorsOpenReview ↗

Response to Reviewer LLbF

We appreciate the reviewer’s constructive feedback and insightful comments, which have helped us identify areas for improvement in our manuscript. We address each of the concerns and questions raised as described below:

Weakness-1:

  • We modified the theoretical discussion to clarify the combined roles of the two adaptation mechanisms. The low-rank branch captures source task-level policy knowledge, while the Fisher-guided sparse branch enables targeted updates to highly important parameters that may not be efficiently represented by a low-rank update alone.
  • We added studies comparing the complete SWORD method against low-rank-only and sparse-only variants, demonstrating the contribution of each component.
  • We clarified the Fisher-based sparse parameter selection procedure and revised Algorithm 1 to ensure consistency between Equation (3), the implementation, and the reported experiments.

Weakness-2:

  • We added a formal definition of Completion Rate (CR) and clarified its relationship to the target-task objective.
  • To directly evaluate adaptation to the changed reward function, we added rewards metric for all transfer experiments and all adaptation baselines. These results provide direct evidence that SWORD adapts to the target reward rather than merely maintaining completion rate.
  • We expanded the discussion of different-task adaptation to explain why reward-weight changes require policy adaptation despite unchanged environment dynamics.
  • We added a detailed trainable-parameter breakdown showing the contribution of each SWORD component and expanded the implementation details to clarify how the reported parameter counts are computed.
  • Algorithm 1 was revised to clearly describe the SWORD adaptation method, sparse-mask construction, and low-rank initialization procedure, improving reproducibility and verification of the parameter-efficiency results.

Question 1:

We have added additional study of each component of SWORD by comparing LoRA and Sparse Tuning in a comparable number of Parameters provided in the Appendix A.10 of the latest manuscript version. To perform this, both Multi-head Attention (MHA-1 & 2) of the Differential MHA in HADT architecture is finetuned. The rank of LoRA is set as r=8r=8 to obtain 1,024 parameters and the sparsity of Sparse tuning is s=0.88s=0.88 yielding 1,013 parameters of total 44,226 HADT parameters. The results are presented in Table 9 and it has an interesting finding where the Jump Start are comparable, but it has more gaps in after 160K adaptation steps. By using LoRA alone, it has capability to retain source task knowledge as the weight are decomposed with higher magnitude rank. Meanwhile, the Sparse update guided by Fisher importance introduce more plasticity to the target task. Thus, our SWORD algorithm can achieve a better performance.

Question 2:

In this work, we use HADT architecture and it consists of several modules. By defining rank and sparsity as r=4, s=0.8r=4,\ s=0.8 of SWORD , the trainable parameters can be summarised by HADT component as follows and frozen parameters are counted as zero trainable parameters.

HADT ComponentSWORD Trainable Parameters
Embedding (frozen)0
Differential Attention (Mixer + λ\lambda)411
MHA 1 (LoRA_A for Wq,WvW_q, W_v)52
MHA 2 (LoRA_A for Wq,WvW_q, W_v)52
Normalization 126
MLP Layer 13,328
MLP Layer 23,290
Normalization 226
Action Output14
Total (sum of component rows)7,199

The Sparse tuning has a sparsity parameter (s) and this value is predefined and we set this value as 0.8 to adapt only 20% of the tuned parameters. Sparse mask is calculated based on Fisher Mutual Information estimation [1]. It approximates the weight importance based on the gradient over mini batch of data sample.

Question 3: The completion rate (CR) is defined as the ratio of number of imaged targets and total targets, and this CR metric alone cannot reflect the priority/cloud trade-off. We have added and compared the return/reward metrics into our results as shown in Figure 4 and Table2-3. Our policy adaptation developed based on HARL\textit{HARL} framework [2] with 5 different random seeds of network parameters and 20 parallel training rollouts. For evaluation, 5 parallel rollouts and 3 episodes are used to ensure policy robustness. The detail of our policy training parameters are presented in Appendix A.1. in the updated version.


References:

[1] Zhang, Huan, et al. "Sparse tuning enhances plasticity in PTM-based continual learning." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 40. No. 33. 2026.

[2] Zhong, Yifan, et al. "Heterogeneous-agent reinforcement learning." Journal of Machine Learning Research 25.32 (2024): 1-67.

The definition of CR and its limitations have been clarified, and the newly added target-task return results address the concern in my original review about evaluating adaptation performance solely through completion rate. For example, SWORD achieves a similar average reward to Full FT (36.67 versus 37.92) while reducing the number of trainable parameters from 44,226 to 7,199. Compared with LoRA-SVD, its average reward increases from 31.28 to 36.67 while using fewer trainable parameters. Compared with Sparse FT, the average reward increases from 34.20 to 36.67, a gain of 2.47, while the number of trainable parameters decreases from 8,783 to 7,199. These results demonstrate a favorable trade-off between parameter count and performance, although the return improvement over Sparse FT is relatively modest. Whether this is sufficient to substantiate the independent contribution of the combined design still requires further comparative evidence to clarify the additional benefit of the combination itself. The revised theoretical analysis also considers the joint update space.

Two core issues still require clarification.

1. The additional benefit of the combined update has not yet been sufficiently isolated.

Table 9 compares LoRA-only and sparse-only under approximately matched parameter budgets, but does not provide results for the complete SWORD method under a configuration comparable to these two. If the authors intend to use SWORD in Table 2 as the reference, that configuration has 7,199 trainable parameters and involves multiple modules, whereas the two single-branch configurations in Table 9 have only 1,024 and 1,013 parameters, respectively, and fine-tune only the two MHA modules. Therefore, this cross-table comparison does not yet fulfill the original request for a three-way comparison under comparable parameter budgets, nor does it sufficiently distinguish the effects of the combined structure, parameter budget, and module selection.

The ablations on SVD and importance-based selection in Table 3 are informative, but cannot directly replace a comparison between single-branch and joint updates. Could you provide further comparisons or existing evidence to establish the additional benefit of the combination itself over either branch alone?

2. The joint update formula, the variables actually optimized, and the parameter counts still need to be reconciled.

Eq. (7) defines both A,BA,B as trainable factors and includes an additional sparse update term. However, the listed entries in Table 5 sum to exactly 7,199, with the MHA entries labeled only as LoRA_A and no separate entry for BB. Please clarify whether BB is trained and, if so, which entry includes its parameters. If the current counts omit trainable parameters, please provide a corrected breakdown corresponding to the reported experiments and clarify whether this affects the parameter-efficiency comparisons.

Furthermore, each LoRA_A entry in the same table has 128 trainable parameters under the LoRA configuration and 26 under SWORD. Please explain the masking operation corresponding to these counts and how A,BA,B and the sparse update term each participate in optimization. Please also relate these details to Eq. (7) and Algorithm 1 so that the consistency between the method used in the experiments and the described joint update can be verified.

The added results have alleviated some important concerns, but the evidence for the contribution of the combination and the consistency between the method and the experimental configuration still require further confirmation. I therefore provisionally maintain my rating of 2 (weak rejection).

We would like to appreciate reviewer's timely response and feedback and thank you for providing further clarifications to the independent contribution of the combined method (LoRA and Sparse), which are significantly important aspects of our manuscript. These are our response to the latest concern:

Issue-1:

A further study of each component of SWORD has been added by comparing LoRA, Sparse Tuning and SWORD under isolated and comparable number of Parameters. To perform this, only dual MHA part of Differential MHA within HADT policy architecture is fine-tuned, while the others are frozen.

A single MHA consists of tuned attention weight Wv,q∈Rdin×doutW_{v,q} \in \mathbb{R}^{d_{in} \times d_{out}}. Each attention has embedding inputs and outputs din=dout=32d_{in}=d_{out}=32. Thus, total number of each attention weight is 32×32=102432 \times 32 = 1024 parameters. The Differential MHA has dual attention and have in total 4 attention weights yielding 4×1,024=40964 \times 1,024 = 4096 parameters. To study a comparable number of parameters, the setting of each algorithms are set as follow:

  1. LoRA rank is set as r=8r=8, where we tune only the A∈Rr×dinA \in \mathbb{R}^{r\times d_{in}} matrix, obtaining: 4×[8×32]=1,0244 \times [8 \times 32] = 1,024 parameters.

  2. Sparse uses sparsity of tuning as s=0.75s=0.75, yielding : 4×[(1−0.75)×32×32]=1,0244 \times[(1-0.75) \times 32 \times 32] = 1,024 parameters.

  3. SWORD implements sparsity and rank as s=0.6s=0.6 and r=20r=20 resulting: 4×[(1−0.6)×20×32]=1,0244 \times [(1-0.6) \times 20 \times 32] = 1,024 parameters of total 44,226 HADT parameters.

Please note that multiplication within a square bracket indicates integer number as the sparse update relies on TopK()TopK() operation which selects only KK number of parameters with higher fisher information values.

We have added the results to the Table 9 and it has an interesting finding where the Jump Start are comparable to LoRA, but it has more gaps in after 160K adaptation steps compared to Sparse fine-tuning. By using LoRA alone, it has capability to retain source task knowledge as the weight are decomposed with higher magnitude rank. Meanwhile, the Sparse update guided by Fisher importance introduce more plasticity to the target task. Thus, our SWORD algorithm can achieve a better performance under an isolated and comparable parameters settings.

Issue-2:

  • Our LoRA module fine-tunes only A-matrix and freezes B-matrix. This partial tuning produces lower number of tuned parameters and better result in compared with full adapter tuning (both A and B matrix) as it has been studied in [1], where the orthogonal down projection matrix (or B-matrix) is fixed across sequence of tasks. We follow a similar scheme in our adaptation algorithms: LoRA-SVD, LoRA-FFT, and SWORD. We have included this frozen LoRA_B to the Table 5. Also, we have added an explanation to describe this partial tuning in Section 3.2 and Section 4.
  • As shown in Figure 2, HADT consists of several modules or layers including Embedding, Differential (Diff.) MHA, and Action (Act.) layers. Our SWORD algorithm slices this transformer structure by updating only some part of the entire modules. The embedding layer is keep frozen as this layer maintains the learned observation token to embedding feature space transformation, which should be general representation across tasks. For each attention weight Wv,q∈Rdin×doutW_{v,q} \in \mathbb{R}^{d_{in} \times d_{out}} in MHA of Diff. MHA, it has a number of inputs din=dout=32d_{in}=d_{out}=32. Thus, total number of each attention weight: 32×32=102432 \times 32 = 1024. SWORD algorithm adapts and fine-tunes only A∈Rr×din\boldsymbol{A} \in \mathbb{R}^{r \times d_{in}} matrix with low-rank r=4r=4 yielding: 4×32=1284 \times 32 = 128 trainable parameters for each attention weight. SWORD jointly updates this with sparsity s=0.8s=0.8, yielding a lower number of trainable parameters: [(1−0.8)×128]=26[(1-0.8)\times128]=26, where multiplication within square brackets indicates integer number result as the sparse update relies on TopK()TopK() operation which selects only KK number of parameters with higher fisher information values.
  • We have ammended Equation 6, 7, and 8 to precisely represent joint weight adaptation of SWORD. Now, the joint adaptation has been modified as: ΔWSWORD=M⊙ΔWLoRA =M⊙αBA\Delta W_{SWORD}=M \odot \Delta W_{LoRA} \, = M \odot \alpha B A
  • We have modified Algorithm 1 to be aligned with the ammended Equation 8.

Please let us know if any additional explanations or revisions are needed. We would be pleased to address any remaining issues. Thank you for your time and consideration.


References:

[1] He, J., Duan, Z., & Zhu, F. (2025, June). CL-LoRA: Continual low-rank adaptation for rehearsal-free class-incremental learning. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 30534-30544). IEEE.

Reviewer_rx2yOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies adaptation of a pretrained multi-agent satellite-tasking policy when mission reward weights change. SWORD combines SVD-initialized low-rank adaptation with Fisher-guided sparse parameter updates, using a HADT policy trained with HAPPO. The evaluation considers all 12 directed transfers among four reward configurations in a three-satellite Basilisk/BSK-RL environment. The paper also presents a local analysis of restricted policy-gradient updates.

Critical strengths
  1. The problem is practically motivated and the controlled task design is useful. Varying priority/cloud-coverage weights while retaining the same environment isolates adaptation to mission objectives, and evaluating all 12 source-target pairs is more informative than evaluating a single favorable transfer (Table 1, Table 2).

  2. The reported parameter/performance tradeoff is promising: SWORD uses 16.3% of the full fine-tuning parameters with a 0.54 percentage-point reduction in average completion rate, and improves over the reported sparse and low-rank baselines. The Fisher-versus-random-selection ablation also gives some evidence for informed sparse selection (Tables 2-3).

Critical weaknesses
  1. The main evaluation does not yet establish adaptation to the changed task objective. The four tasks differ in their reward weights on target priority and cloud coverage, but the SWORD-versus-baseline transfer results report completion rate (Figure 4 and Tables 2-3); the reward curves in Figure 3 concern source-task policies. The paper needs to explain how completion rate captures the target-dependent tradeoff, or provide target-task returns for the existing transfers. This is necessary to assess whether the parameter savings preserve the intended adaptation behavior.

  2. The theoretical analysis does not establish guarantees for the implemented factor and combined updates. For W = W0 + BA and G = grad_W J, ordinary factor-gradient ascent induces delta W = alpha(G A^T A + B B^T G) + O(alpha^2), whereas Proposition 3.2 assumes the orthogonal projected-gradient step alpha P_L(G). Appendix A.6 proves tangent-space membership rather than this equality, and Theorem 1 compares idealized sparse and low-rank updates under bounds on the differences in squared projected-gradient norms and projected curvature quadratic forms rather than analyzing their combination in SWORD. The theoretical claims should therefore be restricted to the idealized projected update or connected explicitly to Algorithm 1.

Questions for authors
  1. How is completion rate defined, and how does it reflect the priority/cloud tradeoff of each target task? Can the existing SWORD and baseline transfers, including the policy before adaptation, be compared using target-task return to establish that the intended adaptation is preserved?

  2. What connects the factor updates used in Algorithm 1 to the orthogonal projected-gradient update in Equation (8)? Please clarify which conclusions apply to the combined SWORD algorithm. If the analysis concerns an idealized optimizer, can its scope be narrowed accordingly?

  3. Please clarify the implementation used for the reported results: does the sparse mask use the policy-score Fisher estimate in Equation (3) or the HAPPO-loss-gradient quantity in Algorithm 1? Also, is the SVD component subtracted from the frozen base weight at initialization? As written, Algorithm 1 adds a nonzero top-r component to W0 before adaptation.

8 comments
AuthorsOpenReview ↗

Response to Reviewer rx2y

Thank you for pointing out critical weaknesses and questions to our works and these are our response:

Weakness-1: We have made revision to our Figure 4 and Table 2-3. In the current version, we provide both Rewards and Completion Rate (CR) to evaluate our proposed algorithm performance. Also, we provide explanation about how to calculate the CR. Basically, it is a ratio between number of imaged targets and total targets in a EO mission, and we agree that CR alone cannot fully represent the task change objective evaluation.

Weakness-2: We have made major updates to our theoretical analysis and Algorithm 1. It is derived from the combined update of LoRA and Sparse yielding more aligned SWORD adaptation analysis. We provide Proposition 1 and Theorem 1 with its proof in the Appendix. The Algorithm 1 has been modified to be aligned with the new equations.

Question 1: The completion rate (CR) is defined as the ratio of number of imaged targets and total targets, and this CR metric alone cannot reflect the priority/cloud trade-off. We have added and compared the return/reward metrics into our results as shown in Figure 4 and Table2-3.

Question 2: Thank you for questioning this matter. The Equation (8) has no connection to the Algorithm 1 in the previous version as it has been derived individually between LoRA and Sparse. In the updated version we have modified our theoretical analysis and now it is connected to the SWORD update in the Algorithm 1.

Question 3: Firstly, the sparse mask used in our work is adopted from the Fisher Mutual Information estimation [1]. It approximates the weight importance based on the gradient over mini batch of data sample. In practice, the importance is calculated from the gradients which are accumulated across NN mini batches before computing their squared magnitude: FiMI(W)=(1N∑j=1Ngi(Bj))2F_i^{MI}(\boldsymbol{W})=\left(\frac{1}{N}\sum_{j=1}^{N}g_i(B_j) \right)^2

A binary importance mask m∈{0,1}d\mathbf{m}\in\{0,1\}^{d} is then constructed by selecting the parameters with the largest estimated importance value: \begin{equation} M_i = \begin{cases} 1, & F_i^{\mathrm{MI}}\in \operatorname{TopK}(\boldsymbol{F}^{\mathrm{MI}},K),\\ 0, & \text{otherwise}, \end{cases} \end{equation}

We understand that our explanation was not sufficient to cover this important aspect. Now, it has been modified as described in the Section 3.2.

Secondly, we apply SVD to extract low rank adaptation weights (AA and BB) at the initialization phase before adaptation. Instead of subtracting the weights, we decompose the source trained weights (W0W_0) then select only top-r component to calculate A and B matrix. This decomposition may have small error as it is a dense projection from the original W0W_0.

We are looking forward to hearing from you about this response. If you still have any further concerns or queries, please do let us know.

Thank you.


References:

[1] Zhang, Huan, et al. "Sparse tuning enhances plasticity in PTM-based continual learning." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 40. No. 33. 2026.

AuthorsOpenReview ↗

Gentle Follow-up on Reviewer rx2y Feedback

Dear Reviewer rx2y,

We would like to kindly follow up on our previous response to your comments and suggestions. We would appreciate it if you could let us know whether you have any further questions, concerns, or clarifications regarding our responses and the revisions made to the manuscript.

We would be happy to address any remaining concerns. Thank you very much for your time and consideration.

Kind regards,

Authors

AuthorsOpenReview ↗

Second Gentle Follow-up

Dear Reviewer rx2y

As the rebuttal period deadline is approaching, we would greatly appreciate it if you could let us know whether you have any further clarification, questions, or concerns remain. We hope that we can address any additional comments from the reviewer in timely manner.

Thank you.

Kind regards,

Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.