- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Bucket-Level Multi-Objective Optimization (Bucket-Level MOO) to address gradient conflicts and negative interference in multilingual LLM fine-tuning. The main idea is to apply existing gradient-based MOO methods, including MGDA, CAGrad, and PCGrad, independently within parameter buckets already used by distributed training systems. This avoids materializing full-model gradients for each language objective and substantially reduces memory overhead.
The paper also introduces bucketwise Pareto stationarity and establishes theoretical descent and convergence guarantees for the bucket-level MGDA and CAGrad variants.
Experiments are conducted on four LLMs ranging from 4B to 8B parameters, covering multilingual instruction fine-tuning and machine translation. The evaluation includes high- and low-resource languages, as well as unseen languages for zero-shot cross-lingual generalization. The results show consistent improvements over standard SFT and language-balanced baselines, while achieving performance comparable to global MOO with approximately 29% lower peak GPU memory usage.
Critical strengths
1. Broad multilingual evaluation with encouraging results.
A notable strength of this paper is its relatively broad multilingual evaluation, covering languages from high-resource to low-resource settings. The experiments also assess zero-shot generalization to unseen languages, which provides useful evidence that the proposed approach benefits multilingual capabilities beyond the languages directly involved in fine-tuning. The consistent improvements across multiple backbones, evaluation benchmarks, and language settings make the empirical results convincing.
2. Practical and memory-efficient integration of MOO into distributed LLM training.
The proposed framework provides a practical way to apply existing MOO algorithms to LLM fine-tuning without materializing full gradients for individual language objectives. The reported reduction of approximately 29% in peak GPU memory compared with global MOO, while maintaining comparable performance, is a meaningful contribution. I also appreciate that the framework can accommodate multiple existing MOO solvers without requiring architectural modifications.
Critical weaknesses
Lack of direct comparisons with closely related methods.
My primary concern is the absence of experimental comparisons with closely related prior work, particularly GradVac, Recon, and RP-MGDA. These methods have explored gradient conflict mitigation, localized optimization, or partition-based multi-objective optimization. Although the proposed framework demonstrates clear improvements over Vanilla SFT and its balanced-data counterpart, it remains unclear how much additional benefit it provides over existing conflict-resolution approaches.
I believe a direct comparison with at least one of these methods, ideally GradVac given its relevance to multilingual gradient optimization, is necessary to better establish the novelty and practical advantages of Bucket-Level MOO. Without such a comparison, it is difficult to assess whether the reported improvements primarily arise from the proposed bucket-level framework or from the underlying MOO techniques themselves. This missing evidence substantially affects my confidence in the paper's contribution relative to prior work.
Questions for authors
Q. Comparison with closely related methods (Major concern).
Could the authors provide a direct experimental comparison with at least one closely related method, such as GradVac, Recon, or RP-MGDA, under comparable training and evaluation settings? In particular, GradVac appears highly relevant because it also addresses gradient conflicts in multilingual training.
I consider this comparison important for assessing the contribution of Bucket-Level MOO beyond existing gradient conflict mitigation techniques. A quantitative comparison covering both multilingual performance and computational efficiency would be especially valuable. The results of such a comparison could materially affect my final assessment of the paper.
3 comments
Response to Reviewer WFHd (2/2)
Where the improvements come from.
- Quality comes from conflict-aware combination. Every conflict-aware rule, GradVac included, exceeds both SFT baselines in all 16 cells of Rebuttal Table 1; no rule dominates. The two GradVac granularities differ by at most 0.7 points, with no consistent sign. The GradVac runs are single-seed. The five-seed standard deviation of the averages, –, covers only Bucket-Level MGDA and Vanilla SFT on Meta-Llama-3-8B, LIMA–s1, two GPUs, four objectives per GPU, where MGDA gains seen and unseen ().
- Cost, on the measured rows. On the measured MGDA pair, bucketing matches Global's step time within 0.03 s and uses 9.2 GB less (Rebuttal Table 2), and GradVac-500M matches PCGrad-500M. Layer-aligned GradVac and GradVac-Global were not timed, and GradVac-Global quality is unreported; Table 3 varies the partition for MGDA, CAGrad, and PCGrad on Qwen3. The streaming Gram exchange, used at and with buckets, avoids a naive all-gather estimated at about 128 GB per GPU ( bytes ). Bucketing accounts for that measured 9.2 GB on the Llama-3-8B MGDA pair and for 29% peak memory on Qwen3 (Table 4). Measured Global MGDA already runs at 68.2 GB on the 80 GB GPU.
Recon and RP-MGDA.
- Recon [2] turns the layers with the most frequent conflicts into task-specific layers, replicating those layers and their optimizer states eight times for . This is orthogonal to gradient combination, so it can be combined with Bucket-Level MOO. Its per-layer cosine statistic is already in our layer-aligned Gram matrices.
- RP-MGDA [3] refines Pareto stationarity from the objective–parameter graph. In a fully shared LLM that graph essentially collapses to one block, up to language-exclusive input-embedding rows, so RP-MGDA is close to global MGDA. That solution is the Global configuration in Table 3, which Bucket-Level MOO matches in quality at 29% lower peak memory on Qwen3 (Table 4).
We will add GradVac to Tables 1, 2, and 5, and add this discussion of Recon and RP-MGDA to the related work.
References
[1] Wang, Zirui, Yulia Tsvetkov, Orhan Firat, and Yuan Cao. "Gradient vaccine: Investigating and improving multi-task optimization in massively multilingual models." In Proceedings of ICLR (2021).
[2] Shi, Guangyuan, Qimai Li, Wenlong Zhang, Jiaxin Chen, and Xiao-Ming Wu. "Recon: Reducing conflicting gradients from the root for multi-task learning." In Proceedings of ICLR (2023).
[3] Hu, Zeou, and Yaoliang Yu. "Leveraging variable sparsity to refine Pareto stationarity in multi-objective optimization." In Proceedings of ICLR (2025).
Comment by Authors
Dear Reviewer WFHd,
Thank you again for your thoughtful review and constructive feedback. We have improved our work to address your comments.
In particular, we have added direct comparisons with GradVac on all four backbones and both datasets, under the same training and evaluation setting, at both the default system buckets and a layer-aligned partition. These comparisons cover multilingual performance and computational cost. GradVac, like the other conflict-aware rules, improves over both fine-tuning baselines, while its runtime and memory stay close to the other bucket-level solvers.
We have also discussed Recon and RP-MGDA. Recon is an architectural method and can be combined with Bucket-Level MOO. In a densely shared model, RP-MGDA reduces to global MGDA, which our global setting already matches in quality at lower peak memory.
We hope these revisions adequately address your concerns. We would be grateful if you could reply and consider your final assessment. Please let us know if any questions remain or if further clarification would be helpful.
Response to Reviewer WFHd (1/2)
Thank you for the thoughtful reviews and constructive feedback. We sincerely appreciate the time and effort you devoted to evaluating our work. We address your points below.
We have added GradVac [1] in exactly the same setting as our main experiments: the same ZeRO-2 pipeline, data, hyperparameters, and evaluation on all four backbones and both LIMA–s1 and WMT24++. GradVac's update depends on the gradients only through their pairwise inner products and norms, so it runs exactly inside our Gram exchange, without materializing any per-language gradient. We use β=0.01 as in [1], initialize all targets at 0, and vaccinate all language pairs (our corpus is parallel, so the high-/low-resource split of [1] does not apply). We evaluate both the 500M system buckets and layer-aligned buckets, the latter being closest to the original per-layer application.
Rebuttal Table 1: Comparison with GradVac. Averages over the four LIMA–s1 benchmarks and the four WMT24++ language pairs (higher is better).
Rebuttal Table 2: Measured step time on Meta-Llama-3-8B (LIMA–s1, 8× H100 80GB, ZeRO-2).