- Rating
- 2
- Confidence
- 3(+1)
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper separates the roles of trajectory scaffolds and teacher contexts in OPSD and introduces OASIS, which distills on short outcome-verified scaffolds using self-generated contexts. Experiments across three Qwen3 sizes show gains over OPSD without written reference solutions, at increased rollout-generation cost.
Critical strengths
-
Separating trajectory scaffolds from teacher contexts provides a useful diagnostic perspective on OPSD. Comparisons at fixed prefixes show how privileged teacher advantages vary with trajectory outcomes, informing the choice of where to apply distillation supervision.
-
OASIS removes the need for written reference solutions while improving reasoning accuracy across three Qwen3 sizes. These results support its practical value in settings with outcome verification but no reference solutions.
Critical weaknesses
OASIS combines additional sampling, verified-scaffold selection, and self-generated teacher contexts, but their contributions are not sufficiently separated. A control using the same selected scaffolds and training budget without the additional context is needed to establish its added value beyond successful-trajectory selection.
Questions for authors
Could you add an ablation without the self-generated teacher context, keeping the selected scaffolds and training budget unchanged? This would clarify whether the context contributes beyond successful-trajectory selection.
5 comments
Thank you for adding the teacher-context ablation. My remaining concern is whether OASIS offers enough benefit to justify its sampling cost at scale. In the reported 1.7B setting, OASIS generates about 16 times as many rollout tokens as standard OPSD for a 0.59 percentage-point gain in average accuracy. Although this ratio does not measure total training cost, it shows substantial additional generation overhead for a modest accuracy gain. The reported results do not establish that this trade-off is worthwhile.
I would find it informative to compare OASIS with SFT on the same verified trajectories and report the total compute for both methods, including the cost of generating and filtering those trajectories. The results show that trajectory selection matters, but it remains unclear how much distillation adds once the correct trajectories have already been selected, or whether that added benefit would justify the cost for larger models.
Distillation Benefits and Computational Efficiency
Thank you for your thoughtful feedback and continued engagement. We have conducted additional experiments to address your remaining concerns.
1. Distillation improves performance beyond trajectory selection
Since OASIS selects the shortest verified trajectory for supervision, we investigate whether its gains come from trajectory selection alone or from the distillation objective. We compare standard SFT on written gold solutions with rejection-sampling SFT* , which uses the exact same shortest verified trajectories as OASIS. For both SFT* and OASIS, we generate 8 rollouts per problem and apply identical filtering and training steps.
Table 1. SFT and OASIS comparison on Qwen3-1.7B.
| Method | Training supervision | Thinking AIME24 Avg@12 | Output tokens | Non-thinking AIME24 Avg@12 | Non-thinking HMMT25 Avg@12 |
|---|---|---|---|---|---|
| SFT | Written gold solutions | 48.40 | 729 | 11.67 | 5.83 |
| SFT* | Hard labels on verified trajectories | 48.61 | 610 | 11.39 | 5.83 |
| OASIS | Teacher distributions on the same trajectories | 54.61 | 610 | 22.78 | 10.28 |
SFT* performs almost identically to gold-solution SFT. This is consistent with the observation in Zhao et al. (2026) that SFT on concise reasoning traces can limit test-time generation length and reasoning performance. Therefore, SFT* is unlikely to provide substantial gains from these concise self-generated trajectories, consistent with findings in prior work. However, OASIS improves over SFT* by 6.00 points in thinking mode, around 11 points on AIME24 and 4.45 points on HMMT25 in non-thinking mode. Both methods generate the same average number of tokens for student during training. This shows that the improvement comes from token-level distillation.
These results are reported in Appendix A.4, Table 15, pages 21–23 of the revised paper.
2. Practical Rollout Cost, Training Efficiency, and Performance Gains on Smaller Models
OASIS generates more candidate tokens than OPSD, but as you also noted, these tokens are used to identify verified trajectories. Only the selected trajectories are used for student training. The additional generation also does not translate directly into the same increase in wall-clock time because rollouts are generated in parallel batches.
Table 2. Rollout generation cost and non-thinking AIME24 performance on Qwen3-1.7B.
| Method | Rollouts () | Generated tokens | Supervised tokens | Latency per batch (s) | AIME24 Avg@12 |
|---|---|---|---|---|---|
| SFT | — | — | — | 0.20 | 11.67 |
| OPSD | 1 | 2.76M | 2.76M | 3.71 | 18.61 |
| OASIS | 4 | 22.8M | 2.01M | 4.17 | 21.94 |
| OASIS | 8 | 43.5M | 1.98M | 5.70 | 22.78 |
Table 3: Comparison under a non-thinking evaluation on Qwen3-1.7B (Avg@12, %). The teacher operates with thinking mode enabled, while the student operates with thinking mode disabled. Best result per column in bold.
| Method | AIME24 | HMMT25 | Average |
|---|---|---|---|
| OPSD | 18.61 | 8.33 | 13.47 |
| OASIS (ours) | 22.78 | 10.28 | 16.53 |
On a single NVIDIA H100, increasing the rollout budget from 1 to 8 increases batch latency by only 1.54× (3.71s to 5.70s), despite the much larger generated-token count. Reducing the budget from 8 to 4 nearly halves the generated tokens and lowers latency to 4.17s, while retaining similar accuracy (21.94% compared with 22.78% on non-thinking AIME24). The 4-rollout setting also achieves 52.77% on thinking-mode AIME24. We also notice that although the average gap between OPSD and OASIS is only 0.59% in thinking mode for 1.7b model, it jumps to 3.06% (13.47% vs. 16.53%) on the same model when thinking is turned off during evaluation
There is also a difference in how the rollouts are used. GRPO generates eight rollouts per problem and uses all of them for policy optimization. In our setup, GRPO requires 500 training steps, whereas OASIS requires only 100. OASIS uses only the selected verified trajectory for student distillation. The other candidates are discarded after selection, so they do not need to be stored across iterations or used for student backpropagation.
Ablation of Self-Generated Teacher Context and Component Contributions
Thanks for your thoughtful feedback.
We conducted the requested ablation by removing the self-generated teacher context while keeping the selected verified scaffolds, training budget, and hyperparameters unchanged. We evaluated both settings on Qwen3-4B over three independent runs.
Table 1: Ablation of self-generated teacher context in OASIS
All values are percentages averaged over three independent runs. Negative Δ indicates lower performance without the context.
These results show that OASIS retains most of its Avg@12 performance without self-generated teacher context, suggesting that verified-scaffold selection accounts for the majority of the overall performance gains. Nevertheless, the context provides additional benefits in majority-vote accuracy (Maj@12), with improvements of 1.1, 3.3, and 9.9 percentage points on AIME24, AIME25, and HMMT25, respectively. Thus, while verified scaffolds are the primary contributor to OASIS's performance, self-generated teacher context provides complementary benefits, particularly in improving the consistency of correct reasoning trajectories.
So, The ablation shows that removing the teacher context has limited impact on overall performance. We further analyzed the contribution of individual components. We found that the main performance gains come from using verified scaffolds (Table 8, Page 19-20). We also examined the effect of the teacher's thinking mode and found that it affects OPSD but has no noticeable impact on OASIS (Table 10-11, Page 20-21).