- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
EMIT studies the reliability of curriculum decisions made from small groups of RLVR rollouts. The paper gives a probability-based account of how often a controller repeats the same action, tests it on frozen server outputs, and traces the behavior of a released hint-selection implementation. It also runs a five-seed comparison of the released adaptive controller, a tuned static hint schedule, and ordinary GRPO using a matched completion-token budget.
Critical strengths
- I like the decision-level perspective. Curriculum papers usually report the final benchmark score without showing how the examples were selected. Here the two-repeat action consistency criterion makes the instability near a decision threshold explicit. The released-selector audit is especially telling: changing the extractor takes maximum-hint decisions on Math-7B from 400 of 400 prompts to 319 of 400, using the same frozen outputs.
- I appreciate the comparison against a tuned static schedule. The five paired runs and common token cap give a much clearer picture of the controller's value than a comparison against GRPO alone.
Critical weaknesses
- The main question for me is what to do with the diagnosis. The reliability-based policy that would sample more near uncertain boundaries or pool decisions is presented as a design template. I would have liked to see this rule executed, at least by replaying stored rollouts. The current results establish that a controller can make unstable decisions, but I am left unsure how much better its decisions become when EMIT is used to change the sampling procedure.
- The end-to-end numbers also temper my enthusiasm for adaptive control. On MATH500, released SEELE scores 65.0 and the development-selected static schedule scores 64.7. The paired interval for the 0.3-point difference runs from -0.7 to 1.3. Both curricula comfortably beat plain GRPO, yet the adaptive part appears to add very little on this training setup. I see a useful diagnostic study here, with a less clear algorithmic payoff.
- There is a related ambiguity in the extractor experiment. A tolerant grader can recover legitimate answers that were missed by a strict parser, and it can also accept extraneous numbers or malformed reasoning. The change in hint distribution is compelling evidence of sensitivity, but I would like to know how many changed labels are actually correct.
Questions for authors
- Has the team inspected the strict-versus-tolerant disagreements for a sample of prompts? In particular, how many are genuine answers missed by the original extractor?
- Could the proposed reliability-aware sampling or pooling rule be run on the frozen rollout groups, with the resulting action stability reported alongside the extra number of samples it uses?
- There seems to be a white box covering part of Figure 3. Could the authors check and fix this?
6 comments
Official Response to Reviewer kwoB (Part 2/4)
About Sampling, Resource Costs, and Pooling
Could the proposed reliability-aware sampling or pooling rule be run on the frozen rollout groups, with the resulting action stability reported alongside the extra number of samples it uses?
Yes. We executed sampling and pooling through the unchanged released five-action fit/regularize/invert/snap map (Appendix K). Appendix L separately evaluates the four-round state lifecycle.
Each of Qwen2.5-Math-7B (Math), Qwen2.5-3B (Qwen), and Llama-3.2-3B (Llama) supplies 400 prompts, five hint levels, and 16 stored responses/level, graded strict/tolerant. A/B repeats use disjoint groups of eight. Targeted sampling starts with four responses/level and adds four where Eq. (5) is closed (tau=0.5, kappa=0.9, lambda=1.5).
The random control receives exactly the same total top-up count per repeat and condition. We integrate fixed-count random allocation exactly: execute each prompt's 32 four/eight-sample masks through the released map, then weight the resulting actions. For N=2,000 cells and m top-ups, a particular five-level mask with k top-ups has weight C(N-5,m-k)/C(N,m). This preserves dependence between level allocations and avoids fitting at an averaged rate vector.
| Model / verifier | Targeted agreement % | Expected random % (upper bound) | Difference pp; nominal 95% lower bound | Samples/prompt/repeat |
|---|---|---|---|---|
| Math / strict | 100.00 | <=100.00 | Agreement ceiling | 21.490 |
| Math / tolerant | 75.00 | <=72.369 | >=2.631; L95>0 | 26.650 |
| Qwen / strict | 92.00 | <=89.688 | >=2.312; L95>0 | 24.150 |
| Qwen / tolerant | 85.25 | <=82.620 | >=2.630; L95>0 | 24.865 |
| Llama / strict | 100.00 | <=100.00 | Agreement ceiling | 20.460 |
| Llama / tolerant | 97.25 | <=94.821 | >=2.429; L95>0 | 22.235 |
L95 denotes the nominal 95% interval's lower endpoint. Intervals use 2,000 paired bootstrap resamples of 400 prompt clusters, retaining all levels and recomputing matched top-up counts. They concern the conditional expected-random baseline. The earlier realized-random contrast is separate: Math/tolerant +4.00 pp [0, 8.25].
Uniform8 uses 40 responses/prompt/repeat. On Math/tolerant, targeted uses 26.65 with 75.00% agreement, compared with Uniform8's 73.25% (original Figure 3, C=1-flip) and Uniform4's 70.25%. Mean selector completion tokens per prompt/repeat are <=19,000.033 for targeted, >=21,000.036 for expected-random, and >=30,000.030 for Uniform8. These are selector costs; all 80 cached outcomes/prompt also support holdout evaluation. Sample counts, selector completion tokens, and total training costs are distinct resource measures.
We also executed cell-based pooling: return hint 4 if any final level remains unresolved. Math/tolerant results are:
| Rule | Agreement % | Max-hint % | Entropy bits | Both repeats nonpooled % |
|---|---|---|---|---|
| Targeted | 75.00 | 57.75 | 1.681 | 100.00 |
| Targeted+pool4 | 90.00 | 65.375 | 1.504 | 84.75 |
| Constant4 | 100.00 | 100.00 | 0 | 0 |
Pooling increases agreement while retaining action diversity and 84.75% both-repeat nonpooled coverage. Strict Math/Llama select maximum hint throughout, so their perfect agreement must be read alongside that concentration.
The 20-response pilot comparison gives +4.750 pp agreement and an independent score gain >=5.087e-3 for 6.65 extra responses on Math/tolerant, both with L95>0. Part 4 reports independent objective checks; the full pilot, per-prompt matched-budget, and histogram-controlled pooling comparisons appear in Parts 4-7 of our response to Reviewer 6e6B.
Official Response to Reviewer kwoB (Part 3/4)
About the Adaptive Increment
The original Table 2 reports the following across five paired Qwen2.5-3B runs (mean +/- sample SD, percentage points):
| Arm | MATH500 |
|---|---|
| No-curriculum GRPO | 45.8 +/- 1.4 |
| Development-selected static (4 to 4) | 64.7 +/- 0.9 |
| Released SEELE | 65.0 +/- 0.7 |
The prespecified SEELE-minus-static effect is +0.30 pp, paired 95% CI [-0.70, +1.30], with paired d_z=0.40 (Section 7; Appendix H.4). Both curricula have large observed gains over GRPO. The static control isolates the additional adaptive comparison, and the paired interval reports its precision.
The comparator was selected from all 15 monotone schedules on 200 development groups and retrained from the base model for each confirmatory initialization. Test benchmarks were not used for selection. The shared resource axis is committed completion tokens; tuning and calibration overhead are accounted for separately in Table 7.
The additional response experiments execute selective sampling and pooling, with agreement, independent local objectives, and costs measured together. The original training comparison remains the estimate of downstream utility.
Official Response to Reviewer kwoB (Part 1/4)
Thank you for these concrete questions.
About Answer Recovery and the Verifier Guard
We examined all 30 retained disagreements. AI-assisted answer attribution identified 29 explicit final declarations matching the reference, one incidental-number acceptance, and no unresolved cases.
The rubric counts an unambiguous final declaration matching the supplied reference as answer-level recovery. Appendix I (p. 19) reports the 25 boxed-present and five unboxed cases as branch exposure; the additional attribution is:
Examples include a final 4π matching reference 4π, and 72 matching 72 outside the shipped strict format. The audit follows final-answer exact-match criteria. The incidental-number case credits reference 2 from the denominator of C=3/2 without a declared answer 2.
Executed repair (E15). On the identical 30 tails, we disabled unanchored numeric fallback while preserving answer-tag/boxed priority, normalization, and exception behavior. Strict and current-tolerant replay each reproduce the retained labels in 30/30 cases; the guard matches the intended branch intervention in 30/30. These cases were selected as strict-wrong/tolerant-right, so strict zero is by construction.
The guard retains 26/29 explicit recoveries and rejects the incidental-number case. The cost is three genuine numeric recoveries. All four numeric-accepting cases were included, making the repair's trade-off explicit on identical responses.
Branch assignment and answer attribution are separate. Current-tail branches are five answer-tag, 21 boxed, and four numeric. For 20 records the full-response path is attributable; for 10, earlier tags absent from the retained tail could change parser priority. Those 10 are not unresolved answer judgments.
The audit uses 1,200-character response tails and 300-character question prefixes: all 30 cases are at hint 2, with 22 distinct prefixes. The attribution method is AI-assisted final-answer exact match. These retained cases and the 400-prompt selector sweep are evaluated as distinct datasets.
Appendix I reports sensitivity to the disputed-stratum false-positive fraction f parametrically through a=t(1−fd). The strict 400/400 maximum-hint result is independent of tolerant labels; Table 8's 319/400 is a ten-pass replay, distinct from Figure 4's single-map histogram. The audit identifies legitimate answer declarations missed by the strict format and a specific incidental-number failure that the guard removes.