acceptodds
Withdrawn from ICLR 2027

Whose Turn to Adapt? Dynamically Ranked Sparse Low-Rank Adaptation for Onboard Satellite Tasking

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Earth-observation satellite clusters increasingly rely on heterogeneous multi-agent reinforcement learning policies to schedule tasks and allocate resources, yet these policies are trained offline and flown frozen, leaving them pursuing outdated goals whenever mid-mission priority shifts outpace ground-station uplink win- dows. Adapting avoid this delays,low rank adapter (LoRA) finetune small portion of parameter locally reducing computational burden. However, exisitng rank al- locators measure importance from parameter on a fixed calibration set, treating as a single network in isolation and enforcing sparsity by masking, still carrying full-sized tensors that are infeasible for resource-constrained satellites. We pro- pose DRS-LORA, an onboard Dynamic Ranking based Sparse LoRA method that prunes rank directions by the singular value decomposition of each adapter’s out- put drift to train and fientune extremely few paramter when performance degrades when new tasks arrives. Across satellite agents, the budget is split by each agent’s contribution to the HAPPO advantage decomposition, a signal the policy update already computes. Within each agent, rank directions are ordered by the singular value decomposition of the output drift the adapter induces, so truncation gives the minimum output error for that rank. On a simulated four-satellite heterogeneous cluster with adaptation onboard through Jetson Orin Nano, DRS-LORA takesd only 13.7% of the joint policy trainable parameter during transfer learning on new tasks, retaining 92.1% of the full fine-tuning target completion rate.

Primary area:
applications to robotics, autonomy, planning
Keywords:
finetuning, low rank adaptation, MARL, onboard optimization

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_vmaxOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies adaptation of deployed heterogeneous multi-agent satellite tasking policies when mission reward priorities change. It proposes DRS-LoRA on top of HAPPO. A shared adapter rank budget is allocated across agents using the mean squared agent-specific advantage signal. Within each adapted layer, singular value decomposition of the adapter-induced output drift ranks directions for retention; adapters are reprojected and resized periodically, with fresh directions added when capacity is available. A volatility-based coefficient also scales parameter updates. Adapters are merged into the base policy after adaptation. Evaluation uses a simulated four-satellite constellation in Basilisk/BSK-RL, with policy inference and adaptation performed on a Jetson Orin Nano. Across ten reward-weight transfers, Table 1 reports an average completion rate of 37.4% using 13.7% trainable parameters, compared with 40.6% for full fine-tuning. This corresponds to 92.1% of the reference completion rate. The appendix establishes optimal reconstruction of the current batch's single-layer output drift under a rank constraint; it does not establish a corresponding bound on policy return.

Critical strengths
  1. A concrete and useful adaptation problem. Updating satellite tasking policies between ground-contact opportunities is a meaningful motivation for parameter-efficient reinforcement learning. The combination of a heterogeneous constellation simulator and execution on an edge device gives the work a practical evaluation setting.
  2. A promising empirical trade-off with a principled local compression rule. In Table 1, DRS-LoRA at 13.7% trainable parameters achieves 37.4% average completion, versus 40.6% for full fine-tuning and 35.7% for the Fisher-ranked adapter at the same parameter percentage. Its reported mean exceeds all listed onboard baselines on each transfer at sparsity 0.5. The output-drift projection has a valid batchwise reconstruction interpretation, and physically shrinking adapter tensors can reduce their gradient and optimizer-state storage. These observations support further investigation, although statistical reliability and total device savings remain insufficiently established.
Critical weaknesses
  1. The experimental evidence does not yet establish reliable adaptation to the specified target objective. For the Balanced-to-(1.0, 0.0) transfer, Table 1 gives DRS-LoRA completion of 37.3 +/- 4.0, whereas Table 2 gives 38.0 +/- 4.0; the Balanced average changes from 37.3 to 37.5. Using Table 1's per-target values and Table 2's zero-shot values gives aggregate gap recovery of approximately 81.4%, rather than the reported 82.7%. The two tables also use differently named LoRA baselines without explaining their relationship. More fundamentally, completion rate does not directly measure the reward-weighted mission objective: capturing more low-priority or low-utility targets can increase completion without improving target return. Equation (1), the appendix reward in Equation (19), and the resource-randomization description need reconciliation. Missing seed counts, a definition of the reported variability, and essential training settings prevent a reliable assessment of the 1.7 percentage-point gain over Fisher ranking. These issues materially weaken the main empirical claim.
  2. The incremental contribution and resource-constrained feasibility are not sufficiently isolated. E2-LoRA already develops output-drift spectral ordering, adapter transformation, and rank truncation, while LoRASA already applies low-rank adapters to multi-agent policies. The potentially distinctive parts here are online reallocation across agents, repeated truncation during policy training, and the pacing rule, but the paper lacks component ablations and a same-budget ordinary LoRA or directly adapted E2-LoRA comparison. It also does not establish why the squared reweighted HAPPO signal predicts the marginal value of additional rank. Although device figures are reported, there is no complete comparative measurement of peak training memory, adaptation time, energy, or SVD/reallocation overhead. Initial rank-16 adapters and rebuilding tensors may have higher peak costs than the final parameter percentage suggests. Unspecified handling of zero ranks, rank caps, newly added directions, Adam state after reparameterization, and the Algorithm 1 merge scale further obscures the implemented method. These gaps prevent attributing the reported gains and verifying operation within the claimed budget.
Questions for authors
  1. Which DRS-LoRA results are authoritative for the Balanced source: Table 1 or Table 2? Please explain the 37.3 versus 38.0 discrepancy, identify any differing settings, and reconcile the gap-recovery calculation. Are non-uniform LoRA in Table 1 and Vanilla LoRA in Table 2 different implementations? What are the seed count and meaning of +/-?
  2. What exact reward and source/target sampling distributions were used in the experiments? How do Equation (1), Equation (19), and the resource-randomization description correspond? Are weighted target returns and their priority/cloud components available alongside completion rate, and do they preserve the reported method ordering?
  3. How is the allocation advantage estimated, normalized, and ordered within HAPPO? How are zero-energy denominators, zero ranks, rank caps, and unused rounded budget handled? Are all fresh directions trained? Please reconcile the retained-set update and merge in Algorithm 1 with Equations (8)-(9), and specify how Adam moments are handled after reprojection.
  4. What evidence separates the effects of cross-agent allocation, dynamic output-drift ranking, and trust-adaptive pacing? Are results available for equal-budget fixed-rank LoRA, uniform allocation, and removal of the pacing or reranking component, or for a directly adapted E2-LoRA baseline? Please also provide the network dimensions and full optimization/reallocation settings.
  5. What are the measured peak training memory, wall-clock adaptation time, and energy per adaptation window on the Jetson, including initialization, SVD, and tensor rebuilding? How is centralized critic or cross-agent information obtained in the intended deployment? Given identical network shapes after merging, what explains the reported 10.4% inference-latency improvement over Full FT?
Reviewer_86vgOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a parameter-efficient fine-tuning method (named DRS-LoRA) for multi-agent reinforcement learning (MARL) policies in

Earth-observation satellite constellations. The goal is to adapt a pre-trained HAPPO joint policy onboard when mission reward weights change in orbit.

To this end, DRS-LoRA empoloys cross-agent rank allocation, dynamic output-drift ranking, truncation and gradient pacing. The method is evaluated on a simulated 4-satellite constellation in BSK-RL across 10 reward-weight transfer scenarios.

Critical strengths

The paper addresses a practical and critical problem for resource-constrained spacecraft which is parameter-efficient fine-tuning for onboard MARL policies under tasking drift.

Ranking LoRA directions by activation output drift rather than static weight magnitude accounts for the input covariance of visited states. In the experimental analyses, this approach outperforms magnitude and diagonal-Fisher pruning.

Critical weaknesses

At each episode, the adapter matrices are overwritten utilizing an SVD basis rotation. In this case, Bi,lB^{i,l} and Ai,lA^{i,l} are rotated into a completely new coordinate basis at each τ\tau episode. Then, the existing first and second moment buffers (mt,vtm_t, v_t) in the Adam optimizer no longer align with the new coordinates. Moreover, if Bi,lB^{i,l} is padded with zeros up to ri+r_i^+, then the results do not explain why memory is saved if ri+r_i^+ columns are still instantiated, nor how Adam moments are transformed or reset after Eq. (8).

Regarding the experimental analyses, the authors claim onboard execution and fine-tuning on a Jetson Orin Nano. However, it is later stated that DRS-LoRA selects actions in 11.2ms which is 10.4% improvement comparing with Full FT. Furthermore, it is explained that LoRA weights are merged into W0i,lW_0^{i,l} at inference time so the deployed forward pass is identical in cost to the unadapted policy. Therefore, Full FT has no adapter overhead on the forward pass during training. Thus, it is impossible for an unmerged LoRA forward pass to be faster at action selection than Full FT.

Additionally, the authors explicitly state that; the memory and power savings are inferred from the method's structure rather than measured on the device.

In order to explore the effect of core components such as the cross-agent rank allocation, the trust-adaptive gradient pacing cic_i, the reallocation interval τ\tau, or the energy threshold ρ\rho, ablation studies should be provided.

Finally, mathematical results have incremental and limited novely, since Lemma 6.1 and Proposition 1 are immediate consequences of the classical Eckart–Young–Mirsky theorem.

Questions for authors

What is the performance of DRS-LoRA for the cases where uniform ranks across all 4 satellites are fixed instead of using Eq. (4), or if you remove trust-adaptive pacing.

How are the Adam first and second moment buffers updated when A and B are overwritten by Eq. (8) at each episode? Are they reset to zero, rotated, or left unchanged in the wrong basis?

How do you support the claims on inference time and footprint (i.e. 11.2 ms action selection, 10.4% faster than Full FT, 538 MB RAM) when memory and power savings were inferred rather than measured on the device? Why would action selection (forward pass) be faster than Full FT?

Reviewer_p4X9OpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studied onboard adaptation of frozen HAPPO policy when the mission's reward weight changes.

The proposed DRS-LORA attach LORA adapter to each agent and allocate a shared rank budget across agent in proportion to the mean square HAPPO advantage. For every τ\tau episodes, the algorithm adjust the adapter based on leading SVD direction of the output drift, and scales Adam steps using volatility of the allocation signal.

On a simulated four satellite environment, authors shows the DRS-LORA with 13.7% trainable reaches 92% of full fine tune's completion rate.

Critical strengths

according to eq (8). the rank reallocation is an exact, deployable re-parameterization: fresh directions are output-neutral, only the provably minimal drift tail is lost on the recorded batch (Lemma 6.1), deleted directions free their optimizer state, and merging adds no inference cost.

Critical weaknesses
  • My biggest concern is the feasibility of onboard LORA approach. The setting does not establish a need for onboard training and the most direct baseline is missing.
    • First, If earth have windows to update the new reward w_1 and w_2, why they just also upload the new policy. So the application of this paper only apply to the situation "earth only have windows to update reward, but no bandwidth to update policy".
    • Second, mathematically, only the ration of w_1 and w_2 matters, then if p = w_1/w_2, why we just add p as a conditional parameter for the predefined policy, instead of training it on board? I feel this approach is more easy to implement. Author should at least compare these two approaches for their pro and corns.
Questions for authors
  • [Clarification] How is A^i\hat A^i in Eq. (3) computed ? If it is HAPPO's advantage, then Ei\mathcal E_i differs across agents only through that order and those ratios and need not reflect agent ii's contribution.

  • [Suggestion] Please compare with HAPPO policy trained on the ground condition on different weight (weakness)

  • [Clarification] Please also state how the new ω\omega reaches the spacecraft; how many episodes, and how much orbital time in flight, DRS-LoRA needs to reach its Table 1 CR

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.