- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes DUET, a two-stage inference framework in which a capable model generates a reasoning signal and a lightweight model uses that signal to produce the final answer. Both models are trained with GRPO. The capable model receives a reward based on the improvement over the lightweight model's standalone performance, together with an adaptive length penalty; the lightweight model receives the task reward. Experiments use Qwen3-4B and Qwen3-0.6B, several mathematical training datasets, and five reasoning benchmarks. The authors report substantial reductions in large-model output tokens, with ablations examining joint training and the reward components.
Critical strengths
1. A practical approach to reducing expensive-model generation.
Delegating final-answer generation to a lightweight model offers a useful route to reducing inference cost. Table 1 reports a reduction from 6,948 to 1,800 large-model tokens on AMC23, with accuracy changing from 0.90 to 0.88. This is a potentially valuable efficiency trade-off, even though it does not establish unchanged accuracy or an equivalent reduction in end-to-end latency.
2. Evaluation across multiple training distributions and reasoning tasks.
The paper evaluates several mathematical training datasets and five reasoning benchmarks, and includes comparisons with concise-reasoning baselines and component ablations. This breadth is useful for assessing where the method works and where it encounters limitations, provided the reported results and experimental settings are reconciled.
Critical weaknesses
1. The distinctive contribution is not sufficiently isolated or validated.
The two-stage architecture closely resembles the general principle of model delegation: a capable model provides guidance and a smaller model completes the response. The more specific contribution lies in the training objective and its adaptive penalty. Although the paper includes a fixed-decoder comparison and reward ablations, these do not sufficiently establish the distinctive benefit over simpler delegation under matched accuracy, computational budgets, and clearly documented regularization settings. The reported token reductions also do not establish that the models have learned the proposed division between reasoning-intensive computation and lightweight response generation. Without stronger controlled comparisons or analysis of the intermediate signals, the methodological contribution and its explanatory value remain limited.
2. Inconsistent numerical reporting undermines confidence in the experimental results.
For MATH-LightEval training, Tables 9 and 11 report 52.3% accuracy and 2,690 tokens on GPQA, implying an IPT of approximately 0.194. However, Table 5 reports 0.162 at 100 training steps. The entire GPQA triplet in Table 5 is also identical for MATH-LightEval and DAPO despite their different results in Table 2. These discrepancies cannot be explained by ordinary rounding. Additionally, the DeepScaleR GPQA result is reported as 0.48 accuracy in Tables 1, 3, and 4, but as 0.47 in Table 2 and 47.2% in Table 8; the run/checkpoint relationship is not explained.
The key ablation introduces another ambiguity. Table 6 reports 27% accuracy and zero output tokens for the -MU-LPS variant on AMC23, whereas Table 12 reports 77.5% accuracy and 1,394 tokens with KL regularization; the collapsed result corresponds closely to the no-KL condition. Since KL regularization is described as part of the default setup, it is unclear whether this ablation changes an additional factor. This affects the attribution of training collapse to the removal of marginal utility and the penalty schedule.
These inconsistencies affect evidence supporting both efficiency and component-level contributions. Until the authors reconcile them, I cannot confidently assess the empirical claims or support acceptance in the paper's current form.
Questions for authors
-
What evidence distinguishes DUET's contribution from existing large-small model collaboration or delegation approaches? Could controlled comparisons establish the additional benefit of the proposed marginal-utility objective and adaptive length penalty beyond simpler alternatives?
-
Could the authors provide accurate, internally consistent numerical results and reconcile the discrepancies across Tables 1-6 and 8-12? Please identify the corresponding runs, checkpoints, and KL configurations, and provide the underlying evaluation records needed to verify accuracy, token counts, and IPT.