- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
ASPECT studies parameter-efficient reinforcement learning for multi-agent LLM workflows. It constructs frozen, task-aligned low-rank bases from calibration gradients and trains a private core matrix for each agent using GRPO. Experiments cover mathematical reasoning and code generation across three model scales and three workflows, reporting improved average peak validation accuracy with substantially fewer trainable parameters than standard LoRA sharing strategies.
Critical strengths
-
The method addresses a practical trade-off between parameter sharing and agent-specific adaptation. Its compact private cores substantially reduce trainable parameter counts while allowing independent agent updates.
-
The evaluation covers multiple model scales and workflow structures. The reported results provide encouraging evidence that this parameterization can be useful for multi-agent reinforcement learning.
Critical weaknesses
-
The methodological contribution beyond existing work is limited. As acknowledged in Section 3.3, LoRA-XS already introduces core-only adaptation between frozen factors, and LoRA-SB provides task-aligned initialization through an estimated update and SVD. ASPECT mainly extends this construction by assigning separate cores to agents. The GRPO objective remains unchanged, and gradient isolation follows from using disjoint trainable parameters. Although the application is useful, I do not find the additional methodological insight sufficiently substantial for acceptance.
-
The experiments do not sufficiently isolate the contribution of the proposed sharing design. The main baselines train complete LoRA factors, so comparisons change both the adapter parameterization and the sharing strategy. Comparisons using the same calibrated frozen bases with workflow-shared, role-shared, and agent-specific cores would more directly test the central design choice. The TSC ablation also changes both the bases and core initialization, limiting attribution. Consequently, the results establish a promising practical configuration more clearly than they establish the specific benefits of the proposed multi-agent design.
Questions for authors
-
What technical contribution distinguishes ASPECT from applying LoRA-SB with separate agent cores beyond the extension to multi-agent workflows?
-
Under the same calibrated bases and comparable tuning budgets, how do workflow-shared and role-shared cores compare with agent-specific cores? Can the effects of basis selection and core initialization be separated?
3 comments
Thank you for the detailed response and additional experiments. The new scalability results and controlled ablations address some of my concerns and strengthen the empirical evidence. I have therefore raised my rating.
Thank you for revisiting the paper and raising your rating after considering the additional experiments. We appreciate your acknowledgment that the resource results and ablations strengthen the empirical evidence.
We would like to briefly clarify the design insight behind those results. Building on LoRA-XS and LoRA-SB, ASPECT addresses what to share across agents without tying their trainable updates. Each layer reuses one frozen task-calibrated basis pair, while each agent learns only its own compact core under the same GRPO objective (Sections 3.3 and 4). The experiments in Sections 5.6–5.7 support this choice by examining core sharing, basis selection, and initialization separately.
The practical value is supported by the single-A100 (80 GiB GPU memory) test in Section 5.9: ASPECT completes the 32-agent test where AS encounters OOM. The newly added B.7, Table 9 also shows that ASPECT has the highest mean validation accuracy at step 100 and reaches 96.34% of its step-200 mean using half the updates.
Together, these results connect the compact parameterization to effective early learning and practical memory savings. Thank you for prompting us to make this connection more explicit and test the design more directly.
Thank you for the thoughtful review. We address your questions on the design and ablations below.
Q1: What does ASPECT add beyond applying LoRA-SB separately to each agent?
A1: WS and RS save parameters by tying trainable updates; AS keeps updates independent but trains a full pair of LoRA factors per agent. Building on LoRA-SB calibration, ASPECT shares one frozen task-calibrated basis pair per layer and trains only a private r×r core per agent, without changing the GRPO objective (§§4.1–4.3). It uses over 90% fewer trainable parameters than the baselines (Table 2).
On AgentsNet maximal matching (Qwen3-4B, rank 64, one A100 80 GiB per run), both methods train at 16 agents, but ASPECT uses fewer resources (§5.9, Table 3).
ASPECT cuts resident tensor memory by 59.9% and synchronization time by 25.5%.
We then increased the number of agents to 32. AS failed with a GPU out-of-memory error during weight synchronization, while ASPECT trained successfully.
In the main experiments, ASPECT often approaches its later-stage training success level within about 100 updates, indicating faster convergence and higher sample efficiency (Appendix B.1, Figures 10–11). Across 18 settings, ASPECT reaches 96.34% of its step-200 mean validation accuracy by step 100, the highest ratio among the four methods.
Across 18 settings, ASPECT has the highest mean validation accuracy under all three rules (Appendix B.1, Table 5).
Q2: How do the three core-sharing schemes compare with the same calibrated bases and training budget?
A2: On Qwen3-1.7B Math Orch-Workers, we use the same frozen task bases, initial core values, and 200-update budget. Private cores retain competitive accuracy with lower late-stage training–rollout mismatch (§5.6, Figure 7).
Q3: Can basis choice and core initialization be separated?
A3: With task-calibrated bases fixed, calibrated core initialization raises peak validation accuracy from 59.49% to 62.46%. With R0=0 fixed, task-calibrated bases reach 59.49%, compared with 44.19% for pretrained-weight bases (§5.7, Figure 8).
The nonzero variants use a calibrated core for the task basis and a random core for the pretrained-weight basis. Both ablations support the same conclusion: task calibration and core initialization each contribute to ASPECT’s performance.