- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes DN-MOPD to balance domain feedback in MOPD by rescaling existing advantages with a clipped ratio of pooled to domain standard deviation. Experiments on Qwen3.5 family models demonstrate the performance of DN-MOPD.
Critical strengths
- The method is motivated by empirical observations of uneven feedback scales across domains. This method is intuitive, simple, and cost-effective, which need no additional teacher calls.
Critical weaknesses
- The method's novelty is limited. From my point of view, The idea of reweighting domain signals to balance MOPD is not new, such as Open-MOPD and DeepSeek-V4 [1]. Although the paper discusses how its scaling rule differs from Open-MOPD, it does not provide a direct empirical comparison with closely related weighting methods. A matched comparison, especially Open-MOPD and DeepSeek-V4, is necessary.
- The empirical validation remains limited. 1, the paper discloses the origin and intended role of the clipping bounds [0.25, 4], but the selection procedure for these endpoints and sensitivity to alternative bounds is unclear. 2, the main experiments only focus on the Qwen3.5 model family, and the performance of DN-MOPD on other architectures is unclear.
[1] Xu, Anyi, et al. "Deepseek-v4: Towards highly efficient million-token context intelligence." arXiv preprint arXiv:2606.19348 (2026).
Questions for authors
- How does DN-MOPD differ from related domain-weighting approaches, such as Open-MOPD and DeepSeek-V4, and how does it perform against a comparable weighting baseline under the same experimental setup?
- How sensitive is DN-MOPD to the clipping bounds across different MOPD configurations?
5 comments
W2 and Q2, clipping bounds
"the selection procedure for these endpoints and sensitivity to alternative bounds"
Selection: the same pair comes from the earlier Qwen3 development instrument and was carried unchanged to all three Qwen3.5 sizes (App. A.4); it was not tuned per size, and no run before submission used any other pair. The two bounds are reciprocals, so the rule can amplify or shrink any one domain's weight by at most a factor of four. Neither alternative floor does detectably better, as the sensitivity reported next shows. Before training these runs, we registered the three-point ladder reported next—floors of 0.10, 0.25 and 0.50 with the upper bound held at 4—together with the reason for each new floor, so the alternative floors were not chosen after seeing the results. Because which bound binds depends on the pool, we report sensitivity by pool. Across the three main Qwen3.5 expert pools at all three sizes and both caps (seed 42), no recorded batch reaches the upper bound of 4; only the floor binds, on IF. In those pools every measured factor stays below the upper bound, so no larger value of that bound would change their results. The upper bound is reached in one of the pools whose IF teacher we replaced: on the pool with the tightest IF feedback, DN-MOPD's measured IF factor reaches that bound and is clipped there. The multiplier the rule applies to IF on that pool is therefore the bound itself. That is the rule doing what it is for: raising the weight of a domain whose feedback is unusually concentrated. How a clipped factor is to be treated was registered before any of these pools was read, so this is a case the registration provided for rather than one we found afterwards. No other replaced-teacher pool approaches either bound, and on Granite 4.2 3B every factor is near one. A floor of 0.10 shows no detectable difference from 0.25 at any size or cap; a floor of 0.50 lowers Total by 1.27 points at 2B at 16K and by 1.41 to 1.96 points at every size at 8K (App. D.6 of the revision).
W2, second item: other architectures
"the performance of DN-MOPD on other architectures is unclear"
DN-MOPD has no architecture-specific component: it rescales the per-token feedback already computed by label-routed OPD, using a scale that can be measured from one rollout batch on any architecture. Beyond the three Qwen3.5 sizes and the earlier Qwen3-4B construction, we trained a new expert pool on another family, Granite 4.2 3B, with three GRPO experts trained for 400 rollout iterations each (1,600 optimizer updates each). Before any student was trained on this pool, we registered a rule that maps the first-batch spread ratios to a prediction. The ratios of this pool are close to balanced (1.01, 0.98 and 1.18), for which the rule predicts no detectable difference in Total between DN-MOPD and Label, and neither difference is detectable (+0.14 points at 16K, −0.09 at 8K; one seed). With two further seeds, the three-seed mean difference remains not detectable at either cap (App. E.3 of the revision). Per domain on this pool the three-seed means at 16K are +1.64 for mathematics, with the interval above zero, and −1.25 for code and −0.38 for IF, neither detectable (Table F.6); on seed 42 alone IF is 2.07 points above Label and code 2.47 below (descriptive). A second family has been started: the three experts of an OLMo-3-7B-Instruct pool are trained. That read was registered as a single read and has been taken: the sealed route-2a read is in App. F.5, where the new mathematics expert is 3.38 points above the O7 base on Math at 8K at 97.5% (the code and IF rows are carried from the first read at 95%). The students on this pool are a separate registered read, not part of this one.
Thank you for noting that the method is motivated by empirical observations of uneven feedback scales across domains, and that it is intuitive, simple and cost-effective. This reply is split into three comments: W1/Q1 here, then the clipping bounds, then other architectures. The main comparisons below are also in the table of our comment to all reviewers.
W1 and Q1
DN-MOPD's weighting rule appears in neither report. Open-MOPD sets per-domain loss weights from a running estimate of each domain's mean reward magnitude, and reports that inverting this rule, so that domains with smaller rewards receive larger weights, creates an unstable feedback loop; DeepSeek-V4 uses importance weights for which its report gives no estimation rule. DN-MOPD weights each domain by the inverse spread of its feedback, measured on the current batch: it turns the dominant domain down, has no per-domain weight to set by hand, and completed training in every run of the main experiments (App. D.1).
Matched comparisons fix experts, initialization, prompts, and update budget. (i) Open-MOPD's allocation rule divides by token share and scales by a clipped relative reward-magnitude factor (App. C.8 of the revision). Its division by token share is stated for a token-mean loss, so that each domain's share of the loss meets a target share. This paper averages within each response first, where that share is the domain's share of the responses in the batch, so the port divides by the response share with the same equal target and keeps the source's own clamp on the magnitude factor. Trained at all three sizes: at 16K this port is behind DN-MOPD in Total at every size, by 1.82 points at 9B, 2.25 at 4B and 2.89 at 2B, each interval wholly above zero, and it is ahead of label routing at no size, the difference being not detectable at 9B and 4B and 0.69 points behind at 2B. At 8K, over all six suites, it is behind DN-MOPD at every size as well: 3.79 points at 9B, 4.60 at 4B and 3.84 at 2B, each interval wholly above zero (App. F.3).
The 95% interval of every row above is in Table F.3 of the revision, and the per-suite rows behind its 16K totals are in Table F.5.
One qualification on the 16K figures. One code cell of the 2B port cannot be graded: the official runner returns an empty result for one sampled response, and the grader treats any such sample as an infrastructure failure. That reproduced identically on three hosts. We therefore removed that suite from all nine runs of this read — three arms at each of the three sizes, not only the one that lost the cell. The 16K Total here is therefore an average over five suites rather than the six the paper defines, and its code domain rests on a single suite with a correspondingly wider interval (intervals and provenance in Table F.3 and App. F.3; per-suite rows in Table F.5). (ii) Hand-set weights. DeepSeek-V4's report gives no estimation rule, so there is no rule of its to port; what these runs test is the family that it belongs to, per-domain weights set by hand. The conditions are matched, as above; the rule is not reproduced, so we report these as a stress test of hand-set weighting rather than as a port of DeepSeek-V4. We trained four such weightings at 4B and 2B: mathematics doubled, IF quartered, and IF raised to 2 and to 4. At 16K none is detectably ahead of DN-MOPD in Total and six of the eight are detectably behind, by up to 3.14 points (App. C.6 and C.9; the run with IF weight 2 is in Table C.16); the weights that raise IF to 4 are behind at every size and both caps (App. C.8). DN-MOPD reaches the level of the best of them in one run.
(iii) A second control built from the two quantities that Open-MOPD uses, token share and remaining gap (a share-and-gap weighting; App. C.8 states how it differs from Open-MOPD's rule), over three seeds: DN-MOPD is ahead at 9B at 16K, where the three-seed mean difference is 0.79 points and all three seed-matched differences favour DN-MOPD, and at every size at 8K, by 0.91 to 1.93 points on average (Table C.14). At 4B all three seed-matched differences also favour DN-MOPD, by 0.62 points on average, and at 2B two of three do; neither three-seed difference is detectable at 16K (Sec. 4 and App. C.8 of the revision).