- Rating
- 4(+1)
- Confidence
- 3
- Informativeness
- 3(+1)
- Presentation
- 2
- Soundness
- 3(+1)
Read review
Paper summary
This paper addresses cross-task transfer and interference in multi-task RL post-training. The authors propose Task-Vector-Guided Reinforcement Learning (TVG-RL). Starting from a common initialization, brief task-specific warm-ups produce task vectors. Their geometry is decomposed into shared, task-private, and complementary components, which are used to project and rescale subsequent multi-task RL gradients. The method trains a single shared policy without requiring fully trained experts or a final merging stage. In the five-task Qwen2.5-7B setting, TVG-RL achieves 79.00% KK accuracy, improving over mixRL’s 44.75% by 34.25 percentage points. Its HH reward reaches 16.21, compared with 5.35 for the strongest merging baseline. In the four-task Qwen3-1.7B setting, TVG-RL achieves the highest mean score among evaluated generalist methods on each of the five reported benchmarks, including held-out GPQA-D. The reported online overhead is approximately 3.1%–3.8%, excluding warm-up training and basis construction.
Critical strengths
The paper makes a clear and differentiated contribution by turning task vectors from post-hoc merging tools into online optimization priors for multi-task RL. Brief specialization reveals task structure that guides shared-policy learning, avoiding fully trained experts and a final merge.
Why this matters for the overall assessment: This change in training paradigm has independent research value and would be the primary basis for acceptance if the controlled comparisons and ablations hold.
Critical weaknesses
-
-
The key design choices in the shared/private decomposition are not sufficiently validated. The subspace definition, selected tensors, low-dimensional representation, and scaling coefficients may determine the outcome, yet their stability and tuning cost are not established.
-
Why this matters for the overall assessment: If the method works only under carefully selected decomposition settings, the contribution would look more like task-specific engineering, substantially weakening the case for acceptance.
-
-
-
The efficiency and scalability claims are incomplete. Warm-up and basis construction are excluded from the reported online overhead, and the evaluation covers only two model scales and a limit1. ed task set.
-
Why this matters for the overall assessment: The paper claims efficient online coordination relative to expert training and merging, but end-to-end cost and scaling with the number of tasks are not established; this directly affects its practical significance and acceptance strength.
-
Questions for authors
-
Including task-specific warm-ups, vector decomposition, and basis construction, what are the total FLOPs, GPU time, and memory requirements relative to mixRL and to independently training and merging experts?
-
Could the authors provide sensitivity analyses for the shared/private decomposition, including the selected layers, subspace rank, projection coefficients, and task sampling weights?
Summary of Our Rebuttal
We sincerely thank the AC and all reviewers for their time and constructive feedback. We would like to summary the rebuttal.
We are encouraged that Reviewer a1eu recognizes the research value of introducing task vectors as priors for multi-task RL post-training and the distributed projection algorithm design. We also appreciate that Reviewer rb5H and Reviewer high find the underlying idea natural and promising, and recognize the value of using short task-specific warm-ups to guide joint training.
During the rebuttal, we have made the following additions and clarifications:
Added controlled experiments with random task-vector bases and different component weights, together with comparisons against PCGrad and CARE-RL, to further validate the contribution of task-derived geometric structure.
Added mixRL result on Qwen3-1.7B, providing further evidence of the effectiveness and consistency of TVG-RL.
Clarified the implementation and role of norm restoration, refined the theoretical analysis to clarify the scope of our propositions, and provided a comprehensive computational cost analysis.
We sincerely appreciate the reviewers' thoughtful comments and the opportunity to strengthen our work through this process. We will incorporate these additional results and clarifications into the revised manuscript.