- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies how to train language models and stopping controllers to balance answer quality and sampling cost in adaptive inference. EXACT-G uses dynamic programming to average over executions on a sampled response bank. EXACT-G computes response-level credits for training the generator under a fixed stopping policy. EXACT-C freezes the generator and trains the stopping controller. Experiments on mathematical reasoning tasks with Qwen3 models show the effectiveness of the proposed methods.
Critical strengths
- The dynamic programming formulation is interesting.
- The theoretical analysis provides a clear justification for averaging execution randomness and explicitly distinguishes conditional exactness from uncertainty due to response sampling.
Critical weaknesses
- The improvements in experiments are moderate. For example, Table 3 shows that EXACT-C achieves comparable accuracy and slightly lower mean utility than Order-MC-16, while requiring substantially more CPU time (389.8 versus 225.3 seconds).
- The ablation studies do not sufficiently isolate the contribution of exact compilation.
- Experiments are limited to one model family (Qwen) and one task (math reasoning).
Questions for authors
-
Why choose ? Could the authors provide an ablation study or sensitivity analysis on this parameter?
-
Compared with GRPO, EXACT-G changes both the training objective and gradient estimation. Could the authors provide an ablation study to distinguish the effects of these two changes?
-
Could the authors provide statistical significance tests for the reported improvements in Table 2 and Table 3, and so on?
-
Could the authors provide more results on other model families or other tasks?
-
What if EXACT-G and EXACT-C are trained jointly? Are there any difficulties with joint training, and could it lead to further improvements?