acceptodds
Under review as a conference paper at ICLR 2027

Multilingual Fine-Tuning via Localized Gradient Conflict Resolution

Abstract

Cross-lingual versatility has become a defining capability of modern Large Language Models (LLMs). However, fine-tuning these models often induces *negative interference*, where gains in one language come at another's expense. To address this, we cast multilingual fine-tuning as a multi-objective optimization (MOO) problem. Specifically, we introduce **Bucket-Level MOO**, a scalable distributed framework that applies gradient-based MOO algorithms locally, within the parameter buckets of distributed training. This enables conflict-aware updates without ever materializing a full gradient vector for any objective, and improves multilingual performance over standard fine-tuning by up to 4.3 points. Theoretically, we define *bucketwise Pareto stationarity*, show it is necessary for Pareto optimality under any fixed partition, and prove descent and convergence for representative variants. Our work brings standard MOO within reach for multilingual LLMs at a far lower cost. More broadly, Bucket-Level MOO is agnostic to both the objectives and the update rule, readily extending to other multi-task learning at scale.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Large Language Models, Optimization, Multi-Task Learning

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_WFHdOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Bucket-Level Multi-Objective Optimization (Bucket-Level MOO) to address gradient conflicts and negative interference in multilingual LLM fine-tuning. The main idea is to apply existing gradient-based MOO methods, including MGDA, CAGrad, and PCGrad, independently within parameter buckets already used by distributed training systems. This avoids materializing full-model gradients for each language objective and substantially reduces memory overhead.

The paper also introduces bucketwise Pareto stationarity and establishes theoretical descent and convergence guarantees for the bucket-level MGDA and CAGrad variants.

Experiments are conducted on four LLMs ranging from 4B to 8B parameters, covering multilingual instruction fine-tuning and machine translation. The evaluation includes high- and low-resource languages, as well as unseen languages for zero-shot cross-lingual generalization. The results show consistent improvements over standard SFT and language-balanced baselines, while achieving performance comparable to global MOO with approximately 29% lower peak GPU memory usage.

Critical strengths

1. Broad multilingual evaluation with encouraging results.

A notable strength of this paper is its relatively broad multilingual evaluation, covering languages from high-resource to low-resource settings. The experiments also assess zero-shot generalization to unseen languages, which provides useful evidence that the proposed approach benefits multilingual capabilities beyond the languages directly involved in fine-tuning. The consistent improvements across multiple backbones, evaluation benchmarks, and language settings make the empirical results convincing.

2. Practical and memory-efficient integration of MOO into distributed LLM training.

The proposed framework provides a practical way to apply existing MOO algorithms to LLM fine-tuning without materializing full gradients for individual language objectives. The reported reduction of approximately 29% in peak GPU memory compared with global MOO, while maintaining comparable performance, is a meaningful contribution. I also appreciate that the framework can accommodate multiple existing MOO solvers without requiring architectural modifications.

Critical weaknesses

Lack of direct comparisons with closely related methods.

My primary concern is the absence of experimental comparisons with closely related prior work, particularly GradVac, Recon, and RP-MGDA. These methods have explored gradient conflict mitigation, localized optimization, or partition-based multi-objective optimization. Although the proposed framework demonstrates clear improvements over Vanilla SFT and its balanced-data counterpart, it remains unclear how much additional benefit it provides over existing conflict-resolution approaches.

I believe a direct comparison with at least one of these methods, ideally GradVac given its relevance to multilingual gradient optimization, is necessary to better establish the novelty and practical advantages of Bucket-Level MOO. Without such a comparison, it is difficult to assess whether the reported improvements primarily arise from the proposed bucket-level framework or from the underlying MOO techniques themselves. This missing evidence substantially affects my confidence in the paper's contribution relative to prior work.

Questions for authors

Q. Comparison with closely related methods (Major concern).

Could the authors provide a direct experimental comparison with at least one closely related method, such as GradVac, Recon, or RP-MGDA, under comparable training and evaluation settings? In particular, GradVac appears highly relevant because it also addresses gradient conflicts in multilingual training.

I consider this comparison important for assessing the contribution of Bucket-Level MOO beyond existing gradient conflict mitigation techniques. A quantitative comparison covering both multilingual performance and computational efficiency would be especially valuable. The results of such a comparison could materially affect my final assessment of the paper.

3 comments
AuthorsOpenReview ↗

Response to Reviewer WFHd (1/2)

Thank you for the thoughtful reviews and constructive feedback. We sincerely appreciate the time and effort you devoted to evaluating our work. We address your points below.

Weakness 1 + Question 1: Comparison with closely related methods, e.g., GradVac.

We have added GradVac [1] in exactly the same setting as our main experiments: the same ZeRO-2 pipeline, data, hyperparameters, and evaluation on all four backbones and both LIMA–s1 and WMT24++. GradVac's update depends on the gradients only through their pairwise inner products and norms, so it runs exactly inside our Gram exchange, without materializing any per-language gradient. We use β=0.01\beta=0.01 as in [1], initialize all targets at 0, and vaccinate all language pairs (our corpus is parallel, so the high-/low-resource split of [1] does not apply). We evaluate both the 500M system buckets and layer-aligned buckets, the latter being closest to the original per-layer application.


Rebuttal Table 1: Comparison with GradVac. Averages over the four LIMA–s1 benchmarks and the four WMT24++ language pairs (higher is better).

MethodLIMA–s1 SeenLIMA–s1 UnseenWMT24++ X→EnWMT24++ En→X
Meta-Llama-3-8B
V. SFT44.050.782.481.6
V. SFT + bal-loader45.351.382.081.9
GradVac (500M)45.652.285.182.3
GradVac (Layer-Aligned)46.352.085.482.3
MGDA46.852.885.882.5
CAGrad46.252.485.282.2
PCGrad46.952.785.182.4
Llama-3.1-8B
V. SFT44.751.882.781.9
V. SFT + bal-loader46.152.682.481.6
GradVac (500M)47.553.685.482.8
GradVac (Layer-Aligned)47.254.285.382.8
MGDA47.254.285.882.7
CAGrad47.354.385.682.9
PCGrad46.954.185.482.6
Qwen3-4B-Base
V. SFT62.068.279.978.4
V. SFT + bal-loader62.268.180.178.5
GradVac (500M)63.770.683.479.8
GradVac (Layer-Aligned)63.570.783.580.1
MGDA63.670.884.279.0
CAGrad64.070.783.579.5
PCGrad64.070.083.580.0
Qwen3-8B-Base
V. SFT67.673.083.482.8
V. SFT + bal-loader68.173.483.182.8
GradVac (500M)69.674.786.184.2
GradVac (Layer-Aligned)69.274.886.384.3
MGDA69.374.886.784.1
CAGrad69.574.886.384.1
PCGrad69.274.386.284.2

Rebuttal Table 2: Measured step time on Meta-Llama-3-8B (LIMA–s1, 8× H100 80GB, ZeRO-2).

MethodPartition#BucketsStep (s)vs. V. SFTPeak nvidia-smi (GB)
V. SFT––5.07–59.0
MGDA500M (default)176.00+18%59.0
MGDAGlobal16.03+19%68.2
PCGrad500M (default)176.14+21%60.9
GradVac500M176.16+21%60.9
AuthorsOpenReview ↗

Response to Reviewer WFHd (2/2)

Where the improvements come from.

  • Quality comes from conflict-aware combination. Every conflict-aware rule, GradVac included, exceeds both SFT baselines in all 16 cells of Rebuttal Table 1; no rule dominates. The two GradVac granularities differ by at most 0.7 points, with no consistent sign. The GradVac runs are single-seed. The five-seed standard deviation of the averages, 0.10.1–0.60.6, covers only Bucket-Level MGDA and Vanilla SFT on Meta-Llama-3-8B, LIMA–s1, two GPUs, four objectives per GPU, where MGDA gains +2.6+2.6 seen and +1.8+1.8 unseen (p=0.0002p=0.0002).
  • Cost, on the measured rows. On the measured MGDA pair, bucketing matches Global's step time within 0.03 s and uses 9.2 GB less (Rebuttal Table 2), and GradVac-500M matches PCGrad-500M. Layer-aligned GradVac and GradVac-Global were not timed, and GradVac-Global quality is unreported; Table 3 varies the partition for MGDA, CAGrad, and PCGrad on Qwen3. The streaming Gram exchange, used at K=1K=1 and with buckets, avoids a naive all-gather estimated at about 128 GB per GPU (T×∣θ∣×2T\times|\boldsymbol\theta|\times 2 bytes =8×8B×2=8\times8\text{B}\times2). Bucketing accounts for that measured 9.2 GB on the Llama-3-8B MGDA pair and for 29% peak memory on Qwen3 (Table 4). Measured Global MGDA already runs at 68.2 GB on the 80 GB GPU.

Recon and RP-MGDA.

  • Recon [2] turns the layers with the most frequent conflicts into task-specific layers, replicating those layers and their optimizer states eight times for T=8T=8. This is orthogonal to gradient combination, so it can be combined with Bucket-Level MOO. Its per-layer cosine statistic is already in our layer-aligned Gram matrices.
  • RP-MGDA [3] refines Pareto stationarity from the objective–parameter graph. In a fully shared LLM that graph essentially collapses to one block, up to language-exclusive input-embedding rows, so RP-MGDA is close to global MGDA. That solution is the Global configuration in Table 3, which Bucket-Level MOO matches in quality at 29% lower peak memory on Qwen3 (Table 4).

We will add GradVac to Tables 1, 2, and 5, and add this discussion of Recon and RP-MGDA to the related work.

References

[1] Wang, Zirui, Yulia Tsvetkov, Orhan Firat, and Yuan Cao. "Gradient vaccine: Investigating and improving multi-task optimization in massively multilingual models." In Proceedings of ICLR (2021).

[2] Shi, Guangyuan, Qimai Li, Wenlong Zhang, Jiaxin Chen, and Xiao-Ming Wu. "Recon: Reducing conflicting gradients from the root for multi-task learning." In Proceedings of ICLR (2023).

[3] Hu, Zeou, and Yaoliang Yu. "Leveraging variable sparsity to refine Pareto stationarity in multi-objective optimization." In Proceedings of ICLR (2025).

AuthorsOpenReview ↗

Comment by Authors

Dear Reviewer WFHd,

Thank you again for your thoughtful review and constructive feedback. We have improved our work to address your comments.

In particular, we have added direct comparisons with GradVac on all four backbones and both datasets, under the same training and evaluation setting, at both the default system buckets and a layer-aligned partition. These comparisons cover multilingual performance and computational cost. GradVac, like the other conflict-aware rules, improves over both fine-tuning baselines, while its runtime and memory stay close to the other bucket-level solvers.

We have also discussed Recon and RP-MGDA. Recon is an architectural method and can be combined with Bucket-Level MOO. In a densely shared model, RP-MGDA reduces to global MGDA, which our global setting already matches in quality at lower peak memory.

We hope these revisions adequately address your concerns. We would be grateful if you could reply and consider your final assessment. Please let us know if any questions remain or if further clarification would be helpful.

Reviewer_9p1uOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes bucket-Level Multi Objective Optimization (MOO). It is a distributed protocol that applies existing MOO methods to gradient buckets for multilingual finetuning.The main contribution of the paper is a streaming implementation that prevents materializing full gradients for every objective. The paper provides theoretical convergence analysis for two variants. It evaluates multilingual instruction tuning and translation across four models.

Critical strengths

1- This is a well motivated problem and there is real practical advantage in resolving conflicts without having to materialize large LLM gradients in MOO.

2- The results show big memory improvements compared to a global MOO.

3- The experimental results (including unseen languages and forgetting) look promising.

Critical weaknesses

1- Localized gradient manipulation and layer-based multilingual gradient behavior have been studied before. Gradient Vaccine already analyzes variation across layers and evaluates layer-wise PCGrad. Recon finds certaion layers have frequent conflicts. So the main contribution of the paper is the system contribution (the distributed bucket implementation), rather than the general discovery or treatment of localized conflicts that lines 89-90 suggest.

2- Since the main contribution is a system contribution, the runtime evaluation is not broad enough. Runtime is reported for only one model and setting. There is 17–33% overhead relative to SFT. The paper does not compare runtime with global MOO. It also does not provide communication vs solving runtime analysis. Although the memory result is encouraging, the overall efficiency advantage is not fully established.

3- Though it is cited, the paper does not compare with GradVac. I understand that the PCGrad configuration is technically a localized baseline. However, a matched comparison with GradVac would help establish whether the proposed implementation provides a better quality/memory/runtime advantage over a related multilingual method.

Questions for authors

1- Why does the introduction motivate localized conflict resolution as necessary (lines 73-79)? The ablations show that global MOO achieves similar quality to a local resolution. The provided results show localization is good for memory use. They don't show that local resolution is necessary for optimization performance.

2- Have the authors studied the more practical case of 10s of languages? What I mean is what happens if there are more objectives than workers? Does the method stay efficient? That analysis could be very insightful.

3- I think there is more investigation that the authors could add to explain why some of the results happen . Examining actual update–gradient alignment and checking whether improvements concentrate in certain languages or in layers with greater initial conflict would help the readers gain deeper understanding.

4- Lines 97-98 the authors state that bucket-level MOO significantly improves performance. For such claim the paper needs statistical significance results. Have the authors conducted such experiments? Why In Section 4, the paper calls changes up to 1.1–1.3 points “barely moving”, while Section 3.2 (and the introduction) treats similar improvement over baselines as significant and big?

5 comments
AuthorsOpenReview ↗

Response to Reviewer 9p1u (1/4)

We are grateful for your insightful comments and the care you put into reviewing our submission. Your feedback is highly appreciated. Our responses to the raised concerns are as follows.

Weakness 1: Distinguishing contributions from prior work.

We agree. Layer-wise variation of multilingual gradient similarity was analyzed by GradVac [1], and Recon [2] showed that certain layers conflict more frequently than others. We do not claim to discover localized conflicts or to be the first to treat them locally. We will rewrite lines 89–90 to state that we confirm this heterogeneity at the 4B–8B scale (Figure 1), with credit to [1, 2].

Our contributions are (i) the protocol that runs such rules inside sharded training, and (ii) Theorem 2.2b plus Gram-only execution. Bucket-Level MGDA inherits Hu & Yu's partition-wise guarantees on system buckets [3]; what we add is that Bucket-Level CAGrad's fixed points remain Pareto stationary under αt=1/T\alpha_t=1/T (Theorem 2.2b), with both rules run from Gram matrices alone (Appendix D). We remove BPS from the contribution list, as in the response to Reviewer vAcK. The protocol hosts any rule expressible through the bucket Gram matrix, avoiding the ~128 GB-per-GPU naive gather at 8B/T=8T=8; GradVac is one (Weakness 3).

Weakness 2: Runtime efficiency.

Rebuttal Table 1 is the measured 8-GPU record on Meta-Llama-3-8B. Bucketing versus Global is the MGDA pair, 6.00 vs. 6.03 s. The +21% cells are (6.14−5.07)/5.07(6.14-5.07)/5.07 and (6.16−5.07)/5.07(6.16-5.07)/5.07.


Rebuttal Table 1: Measured step time on Meta-Llama-3-8B (LIMA–s1, 8× H100 80GB, ZeRO-2).

MethodPartition#BucketsStep (s)vs. V. SFTPeak nvidia-smi (GB)
V. SFT––5.07–59.0
MGDA500M (default)176.00+18%59.0
MGDAGlobal16.03+19%68.2
PCGrad500M (default)176.14+21%60.9
GradVac500M176.16+21%60.9
  • Measured pair. Global MGDA is 0.03 s slower than 500M (6.03 vs. 6.00) and 9.2 GB higher in nvidia-smi (68.2 vs. 59.0). PCGrad and GradVac were timed only at 500M. Broadcast, inner products, solve, and all-reduce were not profiled, and no second backbone was timed.
  • Table 5 stays on H200. Appendix F and Table 5 are eight H200s, rounded to 0.1 h: V. SFT 0.6, CAGrad 0.7, MGDA 0.8, PCGrad 0.8, i.e. (0.7−0.6)/0.6=17(0.7-0.6)/0.6=17 and (0.8−0.6)/0.6=33(0.8-0.6)/0.6=33. One bin is six minutes. From 6.00 s to 6.16 s is about 49 s over the 306-step budget, inside that bin. CAGrad's 0.7 h was not remeasured, so that gap is not attributed to rounding. We will not replace Table 5.
  • H100 peaks and the two-GPU run. The nvidia-smi gaps here (9.2 GB for Global; 1.9 GB for PCGrad and GradVac versus MGDA) include cache, and no live trace exists, so Table 4 (Qwen3, H200) is unchanged. Two GPUs with four objectives each cost 7.4% wall-clock (3.57 vs. 3.33 h).
AuthorsOpenReview ↗

Response to Reviewer 9p1u (2/4)

Weakness 3: Comparison with GradVac.

We added GradVac [1] on all four backbones and both datasets, with the same ZeRO-2 pipeline, data, hyperparameters, and evaluation as the main experiments. Its update depends on the gradients only through pairwise inner products and norms, so it runs inside the Gram exchange. We use β=0.01\beta=0.01 as in [1], initialize targets at 0, and vaccinate all pairs (the corpus is parallel, so the high-/low-resource split of [1] does not apply), on both 500M and layer-aligned buckets.


Rebuttal Table 2: Comparison with GradVac. Averages over the four LIMA–s1 benchmarks and the four WMT24++ language pairs (higher is better).

MethodLIMA–s1 SeenLIMA–s1 UnseenWMT24++ X→EnWMT24++ En→X
Meta-Llama-3-8B
V. SFT44.050.782.481.6
V. SFT + bal-loader45.351.382.081.9
GradVac (500M)45.652.285.182.3
GradVac (Layer-Aligned)46.352.085.482.3
MGDA46.852.885.882.5
CAGrad46.252.485.282.2
PCGrad46.952.785.182.4
Llama-3.1-8B
V. SFT44.751.882.781.9
V. SFT + bal-loader46.152.682.481.6
GradVac (500M)47.553.685.482.8
GradVac (Layer-Aligned)47.254.285.382.8
MGDA47.254.285.882.7
CAGrad47.354.385.682.9
PCGrad46.954.185.482.6
Qwen3-4B-Base
V. SFT62.068.279.978.4
V. SFT + bal-loader62.268.180.178.5
GradVac (500M)63.770.683.479.8
GradVac (Layer-Aligned)63.570.783.580.1
MGDA63.670.884.279.0
CAGrad64.070.783.579.5
PCGrad64.070.083.580.0
Qwen3-8B-Base
V. SFT67.673.083.482.8
V. SFT + bal-loader68.173.483.182.8
GradVac (500M)69.674.786.184.2
GradVac (Layer-Aligned)69.274.886.384.3
MGDA69.374.886.784.1
CAGrad69.574.886.384.1
PCGrad69.274.386.284.2
  • Quality. Both GradVac variants exceed both SFT baselines in all 16 backbone–split cells, as do our three solvers. Differences among the four rules are mostly within one point, next to the Llama seed σ=0.1\sigma=0.1–0.60.6 (Question 4). No rule dominates, so the gain is the conflict-aware combination, and the framework is solver-agnostic.
  • Granularity. Per-layer GradVac and GradVac on 500M buckets differ by at most 0.7 points, with no consistent sign, mirroring Table 3.
  • Runtime. The measured GradVac point is 500M: 6.16 s/step and 60.9 GB nvidia-smi, against 6.14 s for PCGrad and 59.0 GB for Vanilla SFT (Rebuttal Table 1). GradVac Global was an estimate and is withdrawn. A direct GradVac implementation at this scale would still need every per-language gradient at once.

We will add GradVac to Tables 1 and 2.

Question 1: Why does the introduction motivate localized conflict resolution as necessary (lines 73–79)?

"Necessary" overstates the evidence. Instead of supporting the claim that localized resolution is better than global resolution, we argue that Bucket-Level MOO is an efficient and effective alternative to the conventional one. We will revise lines 73–79 to that claim.

  • Adequate for quality. Table 3: the spread across six partitions is at most 1.3 points, and bucketing matches Global MOO. Figure 1 shows that conflicts concentrate in some layers. The appendix records PS⊆BPS\mathrm{PS}\subseteq\mathrm{BPS}, and BPS does not imply PS when buckets use different weights. The evidence that per-bucket decisions lose no quality is Table 3.
  • Cheaper in memory. Table 4: the 500M default cuts nvidia-smi peak by 29% (16.1 GB on Qwen3-4B, 24.6 GB on Qwen3-8B). The 128 GB figure in Weakness 1 is the naive gather, which neither bucketing nor streaming Global MOO uses.
AuthorsOpenReview ↗

Response to Reviewer 9p1u (3/4)

Question 2 + Question 4: What happens if there are more objectives than workers? How statistically significant are the results?

More objectives than workers. One objective per GPU is a convenience, not a requirement. With WW GPUs and T=mWT=mW objectives, each GPU owns mm objectives and processes them as mm micro-steps of a single optimizer step, as in gradient accumulation. After each of the first m−1m-1 micro-steps, the bucket gradients are copied to pinned host memory and the GPU buffers are reused. At the last micro-step, the full T×TT\times T Gram matrix of each bucket is assembled from the local, stashed, and exchanged bucket gradients; the same solver as in the TT-GPU case returns the TT coefficients; and the combined update is all-reduced. The update is therefore identical to the one-objective-per-GPU step.

Regarding efficiency as TT grows, the method's costs scale as follows:

  • GPU memory: bounded by one extra bucket, independent of TT.
  • Host memory: the stash grows as (m−1)∣θ∣(m-1)|\boldsymbol\theta| per GPU.
  • Communication: each GPU receives (T−m)∣θ∣(T-m)|\boldsymbol\theta| elements per step, linear in TT and identical to Global MOO.
  • Solver: O(T3)O(T^3) scalar work per bucket, negligible even for T=64T=64 (≈2.6×105\approx2.6\times10^5 operations).

We ran eight languages on two GPUs (four objectives per GPU) for Meta-Llama-3-8B on LIMA–s1, with five seeds.


Rebuttal Table 3: Lower bound on 8-GPU step time from the measured T=8T=8 MGDA exchange. For T>8T>8, step =5.07+(T/8)×0.93=5.07+(T/8)\times 0.93 s excludes host$\leftrightarrow$device stash traffic; the stash column (T/8−1)×16(T/8-1)\times 16 GB is memory, not that traffic. Rows with T>8T>8 are not measurements.

TTmmVolume vs. T=8T=8Step (s)OverheadHost stash
811×1\times (7∣θ∣7\lvert\boldsymbol\theta\rvert)6.00+18%0
1622×2\times6.93+37%≈16\approx 16 GB
3244×4\times8.79+73%≈48\approx 48 GB
6488×8\times (56∣θ∣56\lvert\boldsymbol\theta\rvert)12.51+147%≈112\approx 112 GB

Under this lower bound the exchange alone exceeds the 5.07 s Vanilla SFT step between T=32T=32 (+73%) and T=64T=64 (+147%). Stash traffic adds further time on top. The O(T3)O(T^3) solve stays negligible beside moving 56∣θ∣56|\boldsymbol\theta| elements.

Statistical significance. The five-seed study compares MGDA with Vanilla SFT only, on Meta-Llama-3-8B, two GPUs, four objectives per GPU.


Rebuttal Table 4: Five-seed results on Meta-Llama-3-8B after LIMA–s1 fine-tuning with eight objectives on two GPUs (mean ± standard deviation over seeds). The last row gives the difference and the two-sided Welch tt-test pp-value.

Method8 Seen Languages5 Unseen Languages
BELEARC-EPolyMathMMLUAvg.BELEARC-EPolyMathMMLUAvg.
V. SFT53.4 ± 0.561.7 ± 0.319.1 ± 1.143.1 ± 0.944.3 ± 0.660.8 ± 0.171.5 ± 0.221.7 ± 0.248.8 ± 0.250.7 ± 0.1
MGDA56.6 ± 0.463.4 ± 0.422.4 ± 0.745.2 ± 0.646.9 ± 0.363.1 ± 0.672.4 ± 0.324.8 ± 0.549.8 ± 0.652.5 ± 0.3
Δ\Delta (pp)+3.2 (<0.001)+1.7 (<0.001)+3.3 (<0.001)+2.1 (0.003)+2.6 (<0.001)+2.3 (<0.001)+0.9 (0.002)+3.1 (<0.001)+1.0 (0.013)+1.8 (<0.001)

With four objectives per GPU, Bucket-Level MGDA uses nearly the same peak GPU memory as Vanilla SFT (+0.5 GB) and adds 7.4% wall-clock time.

Statistical significance. The improvement is significant on both aggregates: +2.6+2.6 points on seen languages (95% CI [1.8,3.4][1.8, 3.4], p=0.0002p=0.0002) and +1.8+1.8 points on unseen languages (95% CI [1.4,2.2][1.4, 2.2], p=0.0002p=0.0002). Every MGDA seed exceeds every Vanilla SFT seed on both averages. Per benchmark, the gains are significant on all four benchmarks in both splits, including MMLU (+2.1+2.1, p=0.003p=0.003 seen; +1.0+1.0, p=0.013p=0.013 unseen).

"Barely moving" versus "significant". We agree that the wording was inconsistent, and we thank the reviewer for pointing it out. The two statements concern different comparisons:

  • Section 4 compares six alternative partitions of the same method. Their 1.1–1.3-point spread is what seed variation alone produces: with σ=0.3\sigma=0.3–0.60.6, the expected range of six runs is about 2.5σ≈0.82.5\sigma\approx0.8–1.51.5 points. Moreover, no partition is consistently best across solvers, splits, and backbones.
  • Section 3.2 compares Bucket-Level MOO with the baselines. Here the effect is consistent in sign. In Table 1, every Bucket-Level variant beats the balanced-loader control on all four backbones and both splits. The seed study above confirms that this effect is significant.

In the revision, we will use "significantly" only where it is backed by a test, replace "barely moves" with the quantitative statement above, and add the seed study.

Reviewer_Sg7YOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Bucket-Level MOO applies existing MOO solvers within parameter buckets for multilingual fine-tuning. It reports improvements over SFT and similar scores to Global MOO with lower memory usage. The contribution centers on a memory-efficient distributed implementation.

Critical strengths

The default bucket configuration reduces peak memory by approximately 29% relative to Global MOO while achieving similar evaluation scores, offering practical value.

Experiments cover multiple backbones and three MOO algorithms, with a language-balanced sampling baseline that helps separate optimization benefits from sampling effects.

Critical weaknesses

Table 3 reports similar scores for partitioned and Global MOO, leaving the claimed necessity of local resolution insufficiently supported. Given prior local gradient-adjustment methods, the conceptual advance beyond memory efficiency remains unclear, limiting the strength of the novelty claim.

The paper does not report variability across runs or confidence intervals for key performance differences. This limits confidence in small gains and in the claims of comparable quality and insensitivity to partition granularity.

The evaluation lacks matched Global MOO timings and measurements of communication costs as the number of objectives grows. Table 5 reports only one 8B configuration (SFT: 0.6 h; bucket methods: 0.7–0.8 h), leaving the time–memory trade-off and broader scalability unclear.

Questions for authors

What evidence distinguishes improved optimization from the memory benefits of bucketing? How should the similar local and global results in Table 3 be reconciled with the claimed necessity of local resolution?

What is the principal contribution beyond existing local gradient-adjustment methods, and which experiments directly establish it?

The appendix acknowledges that bucketwise Pareto stationarity does not imply Pareto stationarity. Does bucket-level MGDA stop updating while a common descent direction still exists globally, and if so, how does this interact with the reported comparisons against AdamW?

6 comments
AuthorsOpenReview ↗

Response to Reviewer Sg7Y (1/2)

Thank you for your detailed and helpful reviews. We value the effort you have invested in providing such meaningful feedback. We respond to your concerns point by point below.

Weakness 1.1 + Question 1 + Question 2: Concerns about the necessity of localized conflict resolution and the novelty.

Instead of supporting the claim that localized resolution is better than global resolution, we argue that Bucket-Level MOO is an efficient and effective alternative for the convetional one. The supported claim is a memory saving at the measured MGDA step times. Section 4 and the Conclusion state that bucketing matches Global MOO quality (Table 3). The layer-wise heterogeneity of negative interference (Figure 1) does not imply such superiority. Rather, because conflicts concentrate in specific layers, it motivates that localized resolution is adequate: a decision made per bucket already has the information it needs. Each claim is tested against its own control:

  • Quality comes from conflict-aware combination. Against the balanced-loader control (same data, per-step language balance, and pipeline, but uniform weights), every Bucket-Level variant gains +0.8+0.8 to +1.8+1.8 points (seen) and +0.9+0.9 to +2.7+2.7 points (unseen) on all four backbones, and Global MOO gains the same (Table 3). The new five-seed study (Weakness 2) confirms a significant gain over Vanilla SFT.
  • Bucketing is adequate and cheaper. Against Global MOO (same solver and exchange, one model-sized bucket), bucketing gives the same quality (Table 3) and runtime (Rebuttal Table 2 below in Weakness 3), with 29% lower peak memory on Qwen3 (Table 4) and 9.2 GB lower on Meta-Llama-3-8B.

Contribution. Existing local methods, such as GradVac [1] and Recon [2], specify what to do with conflicting per-layer gradients and assume that per-objective gradients are available. They were developed on models where this holds. Our contribution is how to obtain the required statistics and apply such rules inside sharded training without ever materializing a per-objective gradient, for any rule expressible through the Gram matrix, together with a theory for system-dictated partitions (BPS as the exact fixed-point certificate of Bucket-Level MGDA, with descent and convergence guarantees).

Weakness 2: Statistical significance test.

We ran five seeds of Vanilla SFT and Bucket-Level MGDA on Meta-Llama-3-8B with LIMA–s1, using two GPUs (four objectives per GPU) due to compute constraints; all hyperparameters match the paper.


Rebuttal Table 1: Five-seed results on Meta-Llama-3-8B after LIMA–s1 fine-tuning (mean ± standard deviation over seeds). The last row gives the difference and the two-sided Welch tt-test pp-value.

Method8 Seen Languages5 Unseen Languages
BELEARC-EPolyMathMMLUAvg.BELEARC-EPolyMathMMLUAvg.
V. SFT53.4 ± 0.561.7 ± 0.319.1 ± 1.143.1 ± 0.944.3 ± 0.660.8 ± 0.171.5 ± 0.221.7 ± 0.248.8 ± 0.250.7 ± 0.1
MGDA56.6 ± 0.463.4 ± 0.422.4 ± 0.745.2 ± 0.646.9 ± 0.363.1 ± 0.672.4 ± 0.324.8 ± 0.549.8 ± 0.652.5 ± 0.3
Δ\Delta (pp)+3.2 (<0.001)+1.7 (<0.001)+3.3 (<0.001)+2.1 (0.003)+2.6 (<0.001)+2.3 (<0.001)+0.9 (0.002)+3.1 (<0.001)+1.0 (0.013)+1.8 (<0.001)

That gap is +2.6+2.6 seen (95% CI [1.8,3.4][1.8, 3.4], p=0.0002p=0.0002) and +1.8+1.8 unseen ([1.4,2.2][1.4, 2.2], p=0.0002p=0.0002); every MGDA seed beats every Vanilla SFT seed on both averages. Per-benchmark pp-values are in the table.

As above, the Table 3 spreads are 0.30.3–1.31.3 points (at most 1.31.3 on Qwen3-4B and 1.11.1 on Qwen3-8B). The Llama σ=0.1\sigma=0.1–0.60.6 is not used to interpret those spreads or the GradVac differences. GradVac 45.645.6 vs. the balanced loader's 45.345.3, and English$\to$X cells such as Meta-Llama-3-8B CAGrad 82.282.2 vs. 81.981.9 (Table 2), are single-run margins. Failing to detect a difference does not show equivalence. "Same quality" will mean a difference inside a reported 95% CI.

AuthorsOpenReview ↗

Response to Reviewer Sg7Y (2/2)

Weakness 3: Runtime and communication costs of Global MOO.

Computing costs (runtime, VRAM) are provided in Rebuttal Table 2 where Global MGDA as a representative for Global ones.


Rebuttal Table 2: Measured step time on Meta-Llama-3-8B (LIMA–s1, 8× H100 80GB, ZeRO-2).

MethodPartition#BucketsStep (s)vs. V. SFTPeak nvidia-smi (GB)
V. SFT––5.07–59.0
MGDA500M (default)176.00+18%59.0
MGDAGlobal16.03+19%68.2
PCGrad500M (default)176.14+21%60.9
GradVac500M176.16+21%60.9

One overhead. Computing the Gram matrix is the exchange itself. Appendix D's added traffic is the broadcasts, (T−1)∣θ(k)∣(T-1)|\boldsymbol\theta_{(k)}| per bucket, the same total as Global MOO, plus a scalar O(T3)O(T^3) solve. That reading is inferred from Appendix D and the measured pair (17 solves vs. one). Broadcast, inner products, solve, and all-reduce were not profiled.

Both runs use T=8T=8. Eight GPUs: one objective each, 7∣θ∣7|\boldsymbol\theta| received, MGDA +18+18 per step (6.006.00 vs. 5.075.07 s). Two GPUs: m=4m=4 micro-steps and (T−m)∣θ∣=4∣θ∣(T-m)|\boldsymbol\theta|=4|\boldsymbol\theta| received, wall-clock +7.4+7.4 (3.573.57 vs. 3.333.33 h). The smaller percentage reflects that extra local compute. We have not timed T∈2,4,8T\in2,4,8.

Hardware and peaks. This table is 8×H100 80GB. Table 5 stays on the paper's eight H200s (Appendix F). The Llama nvidia-smi saving is 13.513.5 (9.2/68.29.2/68.2), beside 2929 on Qwen3 (Table 4). Appendix D names which buffers become model-sized; it does not predict a rise of three bf16 copies. On Qwen3-4B the allocated gap is 35.84−25.97=9.8735.84-25.97=9.87 GB, about one bf16 copy of a 4B model (≈8\approx 8 GB, half of the 1616 GB 8B copy). The nvidia-smi gap is larger (55.255.2 vs. 39.139.1) because that counter includes reserved cache. We will revise the Appendix D sentence that this layout "accounts for" the Table 4 footprint to that statement. No allocated trace exists for these H100 runs or for Qwen3-8B (nvidia-smi gap 24.624.6 GB, 85.985.9 vs. 61.361.3). PCGrad and GradVac at 60.960.9 GB vs. MGDA at 59.059.0 GB is an H100 nvidia-smi gap; lines 392–393 remain the Table 4 caption, where the three Qwen3 solvers share one footprint. GPU memory of the exchange is one bucket at any TT. The host stash is (m−1)∣θ∣(m-1)|\boldsymbol\theta| per GPU.

Question 3: Does bucket-level MGDA stop updating while a common descent direction still exists globally, and if so, how does this interact with the reported comparisons against AdamW?

In principle, yes: the generic effect is a shorter raw step, and a stall is the extreme. At θ=(x,y)\boldsymbol\theta=(x,y) with ∇L1=(1,−1)\nabla\mathcal L_1=(1,-1) and ∇L2=(−1,2)\nabla\mathcal L_2=(-1,2), each bucket has a vanishing convex combination, so Bucket-Level MGDA returns 0\mathbf 0. Globally, α(1,−1)+(1−α)(−1,2)=(2α−1,2−3α)\alpha(1,-1)+(1-\alpha)(-1,2)=(2\alpha-1,2-3\alpha) never vanishes, and d=(3,2)/13\mathbf d=(3,2)/13 has ⟨∇Lt,d⟩=1/13>0\langle\nabla\mathcal L_t,\mathbf d\rangle=1/13>0 for both objectives. With M=∑kM(k)M=\sum_kM^{(k)},

(d2BK)2=∑kmin⁡α∈Δα⊤M(k)α≤min⁡α∈Δα⊤Mα=(d2Glob)2.(\mathbf{d}^{\mathrm{BK}}_{2})^{2}=\sum_{k}\min_{\boldsymbol{\alpha}\in\Delta}\boldsymbol{\alpha}^{\top}M^{(k)}\boldsymbol{\alpha}\leq\min_{\boldsymbol{\alpha}\in\Delta}\boldsymbol{\alpha}^{\top}M\boldsymbol{\alpha}=(\mathbf{d}^{\mathrm{Glob}}_{2})^{2}.

A nonzero bucket step remains common descent (Theorem 2.1b).

AdamW. All methods share AdamW, the schedule, and 306 steps, scored at that endpoint; MOO replaces only the vector AdamW receives. Appendix D limits Theorems 2.1 and 2.2 to θ←θ−ηd\boldsymbol\theta\leftarrow\boldsymbol\theta-\eta\mathbf d and states that AdamW absorbs a constant factor (1/T1/T from ZeRO-2; CAGrad's 1/(1+c2)1/(1+c^2)). Per-coordinate second-moment normalization therefore largely removes a uniform shrinkage, so the raw BPS certificate does not describe the trained models. A direction that differs from the global one is not restored.

Single-run controls. On Qwen3-4B, bucketed MGDA is at least Global MGDA on all five partitions and both splits; on Qwen3-8B the deficit is at most 0.30.3 (Table 3). It is within 0.70.7 of Bucket-Level CAGrad on every average in Tables 1 and 2. These are Weakness 2's single-run gaps.

(d2BK)2/(d2Glob)2(\mathbf{d}^{\mathrm{BK}}_{2})^{2}/(\mathbf{d}^{\mathrm{Glob}}_{2})^{2} is fixed by the Gram blocks already exchanged. It is not logged; we will report its median and 5th percentile over steps once a run is re-logged.

References

[1] Wang, Zirui, Yulia Tsvetkov, Orhan Firat, and Yuan Cao. "Gradient vaccine: Investigating and improving multi-task optimization in massively multilingual models." In Proceedings of ICLR (2021).

[2] Shi, Guangyuan, Qimai Li, Wenlong Zhang, Jiaxin Chen, and Xiao-Ming Wu. "Recon: Reducing conflicting gradients from the root for multi-task learning." In Proceedings of ICLR (2023).

Thank you for the rebuttal and additional experiments. The five-seed evaluation and runtime/memory comparisons partially address concerns about statistical reliability and efficiency, but key issues remain. First, methodological novelty is limited: Bucket-Level MOO appears to be an efficient distributed implementation of existing MOO algorithms, and the rebuttal does not sufficiently establish a substantial advance beyond existing local gradient-adjustment techniques. Second, scalability analysis is incomplete: communication overhead is not separately profiled, scaling with more objectives is untested, and the frequency and practical impact of bucket-level MGDA stalling are not quantified. Overall, the rebuttal strengthens the evaluation, but my core concerns remain insufficiently addressed. Unless these are more convincingly resolved, I am inclined to maintain my rating.

Reviewer_vAcKOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies negative interference in multilingual LLM fine-tuning from a multi-objective optimization (MOO) perspective. It proposes Bucket-Level MOO, a distributed protocol that applies existing gradient-conflict methods such as MGDA, CAGrad, and PCGrad independently to parameter buckets as they become available during backpropagation. This avoids materializing a complete gradient for every language and substantially reduces the memory cost of MOO. The paper further analyzes bucketwise Pareto stationarity and provides descent/convergence results for the MGDA and CAGrad variants. Experiments across four LLM backbones, multilingual instruction tuning, machine translation, seen and unseen languages, and multiple partition granularities show consistent improvements over standard SFT while approaching its memory footprint.

Critical strengths
  • Bucket-Level MOO provides a practical way to make gradient-based MOO feasible for large-scale distributed fine-tuning.
  • The empirical evaluation is broad and consistently supports the usefulness of conflict-aware optimization.
Critical weaknesses
  1. The methodological and theoretical novelty relative to prior localized/partitioned gradient optimization is not sufficiently established. GradVac [1] already studies gradient conflicts in massively multilingual models and applies gradient manipulation independently across model components or layers. More importantly, [2] define Pareto stationarity with respect to an arbitrary parameter partition, which appears closely related to the paper's definition of Bucketwise Pareto Stationarity (BPS), and they also establish that Pareto-optimal solutions satisfy the corresponding partition-wise stationarity condition. Although the distributed bucket execution proposed here is different and practically useful, the paper should more clearly distinguish its optimization and theoretical contributions from these prior works. Direct experimental comparisons with closely related localized multilingual optimization methods are also missing.

  2. The experiments do not clearly support the claim that localized conflict resolution itself is important, and the scalability claim is currently demonstrated only for a small number of objectives. The paper motivates Bucket-Level MOO by showing that gradient conflicts are localized across layers. However, the partition ablation shows that performance is almost invariant from layer-aligned or small buckets all the way to Global MOO. This suggests that the primary advantage of bucketing may be memory-efficient execution rather than better optimization from resolving conflicts locally. In addition, the current implementation assigns one language objective per worker, communication volume still grows with the number of objectives, and the solver has cubic scalar complexity in the number of objectives.

Reference:

[1] Zirui Wang, Yulia Tsvetkov, Orhan Firat, and Yuan Cao. Gradient vaccine: Investigating and improving multi-task optimization in massively multilingual models. ICLR 2021.

[2] Zeou Hu and Yaoliang Yu. Leveraging variable sparsity to refine pareto stationarity in multi-objective optimization. ICLR 2025.

Questions for authors
  1. Can the authors provide direct comparisons with localized multilingual gradient methods such as GradVac, ideally under the same LLM fine-tuning setting?
  2. Since BPS is weaker than standard Pareto stationarity and permits different objective weights in different buckets, can Bucket-MGDA converge to a BPS point that is not Pareto stationary even when a global common descent direction still exists? How significant is this issue in practice?
5 comments
AuthorsOpenReview ↗

Response to Reviewer vAcK (1/4)

Thank you for your detailed and helpful reviews. We value the effort you have invested in providing such meaningful feedback. We respond to your concerns point by point below.

Weakness 1: Distinguishing contributions from prior work.

The BPS definition, its necessity, and Theorem 2.1 reproduce Hu & Yu's partition-wise, per-block MGDA analysis. The additions are system-dictated partitions, Theorem 2.2b, and Gram-only execution.

Relation to GradVac [1]. GradVac vaccinates per layer or per matrix on models of at most a few hundred million parameters, where per-language gradients fit on one device. It does not form those gradients once they are sharded. A naive gather of the T=8T=8 full bf16 gradients of an 8B model is about 128 GB per GPU (T×∣θ∣×2T\times|\boldsymbol\theta|\times 2 bytes =8×8B×2=8\times 8\text{B}\times 2). Appendix D's Gram exchange avoids that gather for every Gram-based rule, Global MOO included. GradVac is such a rule and is provided in our new experiments to response your Question 1 below.

Relation to Hu & Yu [2]. We agree that the partition-wise first-order condition is closely related. Theorem A.4 (BPS is necessary for Pareto optimality) is a supporting lemma rather than a contribution of our work; it follows the same block-coordinate argument as their partition-wise result, and we will cite [2] there explicitly. Beyond this shared lemma, the two works use partitions in opposite directions:

  • Hu & Yu derive the partition from the objective–variable dependency graph so that the condition becomes stronger than Pareto stationarity (PO⊆RPS⊆PS\mathrm{PO}\subseteq\mathrm{RPS}\subseteq\mathrm{PS}), trimming non-optimal stationary points. In a densely shared LLM, every language loss depends on every parameter, so their refined partition collapses to a single block, RPS reduces to PS, and RP-MGDA reduces to global MGDA, i.e., the Global column of our Table 3.
  • Our partition is not chosen for optimality; it is dictated by the distributed system (its communication buckets). The resulting condition is weaker (PS⊆BPS\mathrm{PS}\subseteq\mathrm{BPS}), and our theory characterizes what a solver forced to act bucket by bucket still guarantees: BPS is the exact fixed-point certificate of Bucket-Level MGDA (Theorem 2.1a); the concatenated update remains a common descent direction for the whole model, with ⟨∇Lt,d⟩≥d2\langle\nabla\mathcal L_t,\mathbf d\rangle\geq\mathbf d^2 for every tt (Theorem 2.1b); it converges under an explicit step size (Theorem 2.1c); and the fixed points of Bucket-Level CAGrad remain Pareto stationary, not merely BPS (Theorem 2.2b).

Overall, to the best of our knowledge, our work is the first to run gradient-based MOO rules inside the sharded buckets of ZeRO-style training of multi-billion-parameter LLMs. It does so with peak memory close to standard SFT (Table 4) and runtime equal to Global MOO (Rebuttal Table 1 below), thereby bridging conventional MOO methods and modern LLM training. We will rewrite the contribution statement to separate this protocol-level contribution from the underlying solvers.

Weakness 2.1: Concerns about the necessity of localized conflict resolution.

Instead of supporting the claim that localized resolution is better than global resolution, we argue that Bucket-Level MOO is an efficient and effective alternative for the convetional one. The paper reports that bucketing matches Global MOO quality (Section 4, Conclusion). The layer-wise heterogeneity of negative interference (Figure 1) does not imply such superiority. Rather, because conflicts concentrate in specific layers, it motivates that localized resolution is adequate: a decision made per bucket already has the information it needs. Table 3 confirms this. Across layer-aligned, 100M–1B, and global partitions, quality varies by at most 1.3 points, within the 0.8–1.5 points expected from seed variation alone (seed standard deviation 0.3–0.6 in a new five-seed study of Bucket-Level MGDA and Vanilla SFT on Meta-Llama-3-8B), and all 72 configurations exceed the balanced-loader baseline by at least 0.8 points.

Given adequacy, the case for localization is cost. Global resolution treats the whole model as a single bucket, so its gradient and reduction buffers are model-sized rather than bounded by the bucket size (Appendix D). This raises peak memory by 29% (16.1 GB on Qwen3-4B and 24.6 GB on Qwen3-8B; Table 4) and by 9.2 GB on Meta-Llama-3-8B at the same runtime (Rebuttal Table 1). We will clarify lines 73–79 and the first contribution accordingly: localized resolution is adequate for quality and necessary for scalability.

AuthorsOpenReview ↗

Response to Reviewer vAcK (2/4)


Rebuttal Table 1: Measured training cost on Meta-Llama-3-8B (LIMA–s1, 8× H100 80GB, ZeRO-2). Step time is the median over steps 4–10 of the slowest rank; memory is the per-GPU peak.

MethodPartition#BucketsStep (s)vs. V. SFTPeak nvidia-smi (GB)
V. SFT––5.07–59.0
MGDA500M (default)176.00+18%59.0
MGDAGlobal16.03+19%68.2
PCGrad500M (default)176.14+21%60.9
GradVac500M176.16+21%60.9

Weakness 2.2: Communication complexity in terms of the number of objectives.

We address the three points in turn.

One objective per worker is not required. With WW GPUs and T=mWT=mW objectives, each GPU owns mm objectives and processes them as mm micro-steps of one optimizer step, as in gradient accumulation. The bucket gradients of the first m−1m-1 micro-steps are stashed in pinned host memory; at the last micro-step, the full T×TT\times T Gram matrix of each bucket is assembled and solved exactly as in the TT-GPU case. Running T=8T=8 languages on only two GPUs, Bucket-Level MGDA improves over Vanilla SFT by +2.6+2.6 (seen) and +1.8+1.8 (unseen) points over five seeds (p=0.0002p=0.0002; Rebuttal Table 2), with 7.4% more wall-clock time (3.57 vs. 3.33 h).

Communication volume. We agree that it grows linearly with TT. Per optimizer step, each GPU receives (T−m)∣θ∣(T-m)|\boldsymbol\theta| gradient elements to form the Gram matrices, in addition to the closing all-reduce that ZeRO-2 already performs for Vanilla SFT. This volume is identical to that of Global MOO and is inherent to every rule that needs all pairwise inner products (including GradVac and PCGrad); bucketing does not increase it. Appendix D (Distributed Execution, paragraph "Cost") already states that "Communication volume, by contrast, is not reduced"; we will make this limitation explicit in the main text. Three orthogonal reductions are possible: forwarding gradients around a ring halves the volume; per-bucket solving and reduction can overlap with the backward computation of the next bucket, whereas Global MOO must finish the whole backward pass before it can weight and reduce any part of the gradient; and inner products could be estimated from low-dimensional sketches of each bucket.

Solver cost. The O(T3)O(T^3) scalar work per bucket is negligible in practice. For example, even for T=64T=64, T3≈2.6×105T^3\approx2.6\times10^5 scalar operations per bucket are negligible compared with moving a bucket of 500M500M elements. Rebuttal Table 1 provides costs where Global MGDA as a representative method for Global ones. The only bucket/Global pair is MGDA: 6.00 s (500M, 17 buckets) vs. 6.03 s (Global).


Rebuttal Table 2: Five-seed results on Meta-Llama-3-8B after LIMA–s1 fine-tuning with eight objectives on two GPUs (mean ± standard deviation over seeds). The last row gives the difference and the two-sided Welch tt-test pp-value.

Method8 Seen Languages5 Unseen Languages
BELEARC-EPolyMathMMLUAvg.BELEARC-EPolyMathMMLUAvg.
V. SFT53.4 ± 0.561.7 ± 0.319.1 ± 1.143.1 ± 0.944.3 ± 0.660.8 ± 0.171.5 ± 0.221.7 ± 0.248.8 ± 0.250.7 ± 0.1
MGDA56.6 ± 0.463.4 ± 0.422.4 ± 0.745.2 ± 0.646.9 ± 0.363.1 ± 0.672.4 ± 0.324.8 ± 0.549.8 ± 0.652.5 ± 0.3
Δ\Delta (pp)+3.2 (<0.001)+1.7 (<0.001)+3.3 (<0.001)+2.1 (0.003)+2.6 (<0.001)+2.3 (<0.001)+0.9 (0.002)+3.1 (<0.001)+1.0 (0.013)+1.8 (<0.001)
AuthorsOpenReview ↗

Response to Reviewer vAcK (3/4)

Question 1: Adding the baseline GradVac.

We added GradVac [1] under exactly the same setting as our main experiments: the same ZeRO-2 pipeline, data, hyperparameters, and evaluation on all four backbones and both LIMA–s1 and WMT24++. Since GradVac's update depends on the gradients only through their pairwise inner products and norms, it runs exactly inside our Gram exchange, without materializing any per-language gradient. We use β=0.01\beta=0.01 (the value selected in [1]), initialize all similarity targets at 0, and vaccinate all language pairs, because our corpus is parallel and the high-/low-resource split of [1] does not apply. We evaluate two granularities: the 500M system buckets and layer-aligned buckets (one decoder layer per bucket), the latter being closest to the original per-layer application.


Rebuttal Table 3: Comparison with GradVac. Averages over the four LIMA–s1 benchmarks and the four WMT24++ language pairs (higher is better).

MethodLIMA–s1 SeenLIMA–s1 UnseenWMT24++ X→EnWMT24++ En→X
Meta-Llama-3-8B
V. SFT44.050.782.481.6
V. SFT + bal-loader45.351.382.081.9
GradVac (500M)45.652.285.182.3
GradVac (Layer-Aligned)46.352.085.482.3
MGDA46.852.885.882.5
CAGrad46.252.485.282.2
PCGrad46.952.785.182.4
Llama-3.1-8B
V. SFT44.751.882.781.9
V. SFT + bal-loader46.152.682.481.6
GradVac (500M)47.553.685.482.8
GradVac (Layer-Aligned)47.254.285.382.8
MGDA47.254.285.882.7
CAGrad47.354.385.682.9
PCGrad46.954.185.482.6
Qwen3-4B-Base
V. SFT62.068.279.978.4
V. SFT + bal-loader62.268.180.178.5
GradVac (500M)63.770.683.479.8
GradVac (Layer-Aligned)63.570.783.580.1
MGDA63.670.884.279.0
CAGrad64.070.783.579.5
PCGrad64.070.083.580.0
Qwen3-8B-Base
V. SFT67.673.083.482.8
V. SFT + bal-loader68.173.483.182.8
GradVac (500M)69.674.786.184.2
GradVac (Layer-Aligned)69.274.886.384.3
MGDA69.374.886.784.1
CAGrad69.574.886.384.1
PCGrad69.274.386.284.2

Rebuttal Table 3 supports three conclusions:

  • All conflict-aware rules help, GradVac included. Both GradVac variants exceed both SFT baselines in all 16 backbone–split cells, as do our three solvers. The differences among the four rules are small, mostly within one point, and no single rule dominates.
  • The original per-layer granularity brings no advantage. Per-layer GradVac and GradVac on 500M system buckets differ by at most 0.7 points, with no consistent sign, mirroring the granularity invariance of Table 3. Localized rules can therefore run directly on the buckets that distributed training already produces.
  • GradVac at this scale uses the same exchange. On the measured 500M row it matches PCGrad (6.16 vs. 6.14 s/step) and stays near Vanilla SFT in memory (Rebuttal Table 1). Extrapolating that measured step over the 306-step budget gives 306×6.16306\times 6.16 s ≈0.53\approx 0.53 h. Global GradVac was an estimate and is omitted from Rebuttal Table 1.

We will add GradVac to Tables 1 and 2 as well its computing costs (VRAM, runtime) to the manuscript.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of the Discussion Period

Dear Reviewers, Area Chairs, and SACs,

We sincerely thank the reviewers for their thoughtful and constructive feedback. We are encouraged by their recognition of Bucket-Level MOO as a practical way to make gradient-based MOO feasible in distributed LLM fine-tuning (vAcK, Sg7Y, 9p1u, WFHd), of its memory advantage over Global MOO (vAcK, Sg7Y, 9p1u, WFHd), and of the breadth of our multilingual evaluation across backbones, benchmarks, and seen and unseen languages (vAcK, 9p1u, WFHd).

The discussion period has substantially strengthened the paper. In particular, we addressed the following key points:

  • Comparison with closely related methods (vAcK, 9p1u, WFHd). We added GradVac [1] on four backbones, both datasets, and two granularities. Every conflict-aware rule beats both SFT baselines in all 16 cells; none dominates, and per-layer GradVac is within 0.7 of 500M buckets. Recon [2] (architectural, hence composable) and RP-MGDA [3] (its partition collapses to one block in a densely shared LLM) are not direct competitors.

  • Significance, timing, and scaling (Sg7Y, 9p1u, vAcK). Five seeds give +2.6 seen (95% CI [1.8, 3.4]) and +1.8 unseen (95% CI [1.4, 2.2]) over Vanilla SFT (Welch p = 0.0002). Matched timings save 9.2 GB over Global MOO for 0.02 s/step. Thirty-two languages on two GPUs give +2.69, leading on 31 of 32.

  • Relation to prior work (vAcK, Sg7Y, 9p1u). Accounting for GradVac [1], Recon [2], and Hu & Yu [3] keeps the contribution distinct: the Gram protocol on system buckets, with common descent and BPS convergence for MGDA [4] and Pareto-stationary fixed points for CAGrad [5].

  • BPS stall (vAcK, Sg7Y). We explain the stall in theory. A counterexample and the step-length bound give a BPS point that is not Pareto stationary, at which the step vanishes; every nonzero step remains common descent.

Remaining concerns are: 1) step times are aggregate, and 2) the stall is unquantified (Sg7Y). Firstly, the communication volume is (T−1)|θ| per worker. One bucket and seventeen take 6.03 s and 6.00 s against 5.07 s for Vanilla SFT, so the added time is the Gram exchange while communication volume is negligible. Secondly, Bucket-Level MOO quality matches Global MOO, so the stall is not operative in our experiment. We will acknowledge this lack of empirical measurements about BPS stall in the revised manuscript, similarly as Reviewer 9p1u suggested treating gain and partition uncertainty as a limitation.

Overall, the reviewers' feedback will help us further consolidate the manuscript by integrating the additional evidence and analyses developed during the discussion, with changes highlighted in blue.

Thank you again for the time and effort devoted to reviewing our work.

Authors of Submission 27925

References

[1] Wang, Zirui, Yulia Tsvetkov, Orhan Firat, and Yuan Cao. "Gradient vaccine: Investigating and improving multi-task optimization in massively multilingual models." In Proceedings of ICLR (2021).

[2] Shi, Guangyuan, Qimai Li, Wenlong Zhang, Jiaxin Chen, and Xiao-Ming Wu. "Recon: Reducing conflicting gradients from the root for multi-task learning." In Proceedings of ICLR (2023).

[3] Hu, Zeou, and Yaoliang Yu. "Leveraging variable sparsity to refine Pareto stationarity in multi-objective optimization." In Proceedings of ICLR (2025).

[4] Désidéri, Jean-Antoine. "Multiple-gradient descent algorithm (MGDA) for multiobjective optimization." Comptes Rendus Mathematique, 350(5–6):313–318 (2012).

[5] Liu, Bo, Xingchao Liu, Xiaojie Jin, Peter Stone, and Qiang Liu. "Conflict-averse gradient descent for multi-task learning." In Advances in Neural Information Processing Systems (2021).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.