- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes Socratic-Zero, a framework for bootstrapping mathematical reasoning from seed data through three agents; A Solver (Qwen3-8B-Base) is trained with DPO on its own trajectories. A frozen Teacher (Qwen3-235B-A22B) verifies answers and rewrites failed problems into new targeted ones. A Generator (Qwen3-32B) distills the Teacher’s problem rewriting through SFT, with each example weighted by how close the Solver’s success rate on the new problem is to 0.5. The main experiment Starts from a 1,500-example SFT model and 100 seed problems on Math domain, and the proposed method report a +20.2-point average gain on seven math benchmarks, outperforming prior self-play methods. The paper also reports transfer gains on science QA, deep research, SWE-bench, and multimodal chart and code tasks.
Critical strengths
- Timely and relevant problem. Self-improvement and synthetic curriculum generation are active directions (e.g., Absolute Zero, R-Zero, SPICE), so a framework that reduces dependence on annotated data is timely and important topic.
- A practical idea for amortizing an expensive curriculum designer. Distilling a large Teacher’s failure-driven problem refinement into a smaller, trainable Generator is a sensible engineering idea. Weighted supervised finetuning on the distillation data by how close each problem is to the Solver’s capability boundary is a simple, interpretable way to target useful difficulty.
- Broad experimental coverage. The paper examines many components on ablation, including verifier choice, Teacher scale, seed source, Teacher solution injection, DPO vs. GRPO, and reward shape. This breadth would make the work informative to practitioners.
Critical weaknesses
-
The empirical evidence is not reliable enough to support the main claims. For example, the reported base accuracies of Qwen3-8B-Base (48.8 on MATH-500, 63.4 on GSM8K) are far below commonly reported values for this model; both benchmarks are near saturation for recent models. This inflates the reported gains and suggests a non-standard evaluation setup. Despite this ambiguity on the evaluation setup, author used reported number for other baselines and provide "approximate estimates" for methods whose per-benchmark breakdown is not explicitly reported, without clear explanation on which baseline numbers were actually reproduced under the same base model, SFT initialization and evaluation protocol, and which were estimated. 1.2. Insufficient statistical support despite small differences. Some table shows small differences, e.g. in 0.4 gain over the second best line on table 4, the differences are reported without variance. 1.3 I might misunderstood, but In Table5, the generator, which is trained to imitate the Teacher, ends up outperforming the teacher. This is counter-intuitive result of distillation but the paper didn't analyze it.
-
Overstated framing and limited novelty relative to prior self-play work. For example, the title highlighted the method as "data-free", yet the method requires a cold-start SFT on 1,500 high-quality MATH problem-solution pairs, 100 carefully curated seeds. Also, the introduction criticizes reliance on curated data and on “human-curated knowledge boundaries”, but knowledge boundary moves from human annotation to the 235B-scale Teacher and the method does not show self-improvement beyond it. Although the method is presented as "paradigm-shifting", co-evolutionary task generation and solving is already established (Absolute Zero, R-Zero, SPICE, SPIRAL). The remaining components are standard: preference optimization on self-generated trajectories with teacher-provided positives, which closely resembles RL with teacher-guided or context distillation, and SFT distillation of a teacher’s outputs with a smooth difficulty weight. The work reads as a variant of existing self-play with added frontier-teacher distillation.
-
The true self-play setting underperforms. Although the paper positions itself as self-play method, it relies on large-scale Teacher model. And when the Teacher matches the Solver in scale (Qwen3-8B, Table 7a), the gain is +10.7, below Absolute Zero (+16.7) and R-Zero (+18.8). As those baselines operate without a much larger external model, this result raises a concern about whether the method fits for true self-play frame.
-
Table 2 is uninterpretable, as it has no baselines. It is also unclear whether these are math-trained models evaluated out of domain or separate models trained with domain-specific Socratic-Zero loops; the base models and tasks differ (e.g., Qwen2.5-7B-Instruct for deep research, a VL model for charts). Gains such as +36 on NQ from a math curriculum would need substantial explanation.
[minor]
- Figure 1(b) uses GPT- and Gemini-style logos for the Teacher and Generator, while the paper uses open-weight Qwen models. This is misleading, because whether a component can be fine-tuned is central to the method.
- Figure 2 occupies a lot of space but conveys little concrete information. The geometric shapes and bookshelf icons do not relate to the math setting. A concrete example of a failure, its refinement and its utility weight would be more informative.
- Calling a fixed, non-optimized Teacher an “agent” in a “co-evolution” framework is a stretch. Only the Solver and Generator evolve.
Questions for authors
- Which rows in Table 1 were reproduced by author under the matched same base model, SFT initialization, and which were estimated?
- Why are the base-model accuracies on MATH-500 and GSM8K so much lower than published figures for Qwen3-8B-Base? What prompt, answer extraction and generation length were used, and do the gains persist under a standard evaluation?
- Since the core difference from other self-play is the usage of Teacher model, was a standard distillation baseline evaluated, e.g., SFT on Teacher-generated problems and solutions from the same 235B model at matched data volume?
- Regarding the verification, is V(q, y) an external checker given to the Teacher, or the Teacher’s own judgment? For Teacher-generated problems, how is the correctness of new problems and answers ensured?
- Section 4.2.1 says human annotators judged validity. If humans were involved, please describe, the number of problems, the protocol and inter-annotator agreement. Also, as the domain is mainly about complicated math problem, please describe how the author make sure their expertise on math domain.
- In table2, was each model trained on math and evaluated out of domain, or trained with a domain-specific loop? What are the seeds, Teacher and verifier for non-math domains, and how do appropriate baselines (e.g., the same amount of Teacher distillation) perform?
- Can authors report variance across seeds for all main tables, and include a compute-matched comparison that accounts for Teacher inference cost?
- The claim that Absolute Zero and R-Zero “lack effective quality control” should be made precise. Both include filtering mechanisms, such as executor-based validation and difficulty and repetition filtering, so please specify which aspect is missing and how Socratic-Zero addresses it.
9 comments
Response to Reviewer qMVd (Part 2 of 3)
Q3. About Domain-Specific Training and Baselines. Table 2 reports separate domain-specific Teacher–Student training. We apologize that this scope was unclear. Qwen3-235B-A22B teaches the three text domains; Qwen3-VL-235B-A22B teaches the two multimodal domains. Students use DPO with rule-based verification, including execution feedback for software engineering. Generator training and continuation are evaluated in the mathematical experiments. The four Instruct-based settings start directly from seeds, without the 1,500-pair SFT.
| Domain | Student model | Seed initialization | Generated training examples |
|---|---|---|---|
| Science reasoning | Qwen3-8B-Base | 100 SciBench problems | 5K |
| Deep research | Qwen2.5-7B-Instruct | 500 DeepDive tasks | 30K |
| Software engineering | Qwen3-4B-Instruct-2507 | 128 SWE-smith tasks, one per repository | 36K |
| Multimodal chart reasoning | Qwen2.5-VL-7B-Instruct | 100 ChartQAPro examples | 5K |
| Multimodal coding | Qwen2.5-VL-7B-Instruct | 430 examples from DesignBench’s screenshot-to-code subset | 10K |
Seeds initialize the curriculum; generated examples are the training data. In SWE, we compare both data source and training objective using Qwen3-4B-Instruct-2507 and the same SWE-bench Verified harness. Direct SWE-smith training uses its full ~50K task pool, which also supplies our 128 seeds, one per repository. Synthesized-data SFT uses the same 36K examples as our method. Both controls match our Solver training budget; R-Zero, SPIRAL and Absolute Zero are reproduced at 36K examples.
| Method | Training task pool | SWE-bench Verified (%) |
|---|---|---|
| Base | — | 21.6 |
| R-Zero | 36K | 21.3 |
| SPIRAL | 36K | 24.3 |
| Absolute Zero | 36K | 24.9 |
| Direct training on SWE-smith | ~50K | 25.3 |
| SFT on Socratic-Zero data | 36K | 27.4 |
| Socratic-Zero | 36K | 30.0 |
Our gains over SWE-smith and synthesized-data SFT are 4.7 and 2.6 points. We clarified settings in Table 2 (p. 6, lines 291–295), added the SWE comparison to Table 9 (p. 10, lines 486–497), and documented models, seeds, data and verification in Appendix S (p. 27, Table 20; control protocols: lines 1449–1456).
Q4. About Repeated Runs, Compute, and the All-8B System. We report three independent runs for the main Solver (56.1 ± 0.2%) and each 47K continuation branch (Teacher: 62.8 ± 0.3%; Generator: 67.5 ± 0.2%), with mean ± standard deviation across runs. Continuation starts from a shared checkpoint. The separate 10K-problem data-utility evaluation uses five downstream Student training runs: 39.61 ± 0.17% for Socratic-Generator-32B versus 38.79 ± 0.23% for Claude-4.1-Opus. These results appear in Table 1 (p. 6, line 276), Tables 5 and 7 (pp. 8–9), and below Table 4 (p. 7).
Table 13 estimates total compute in equivalent A100 GPU hours, including Teacher inference. All our configurations use an 8B Solver; Teacher-only rows report bootstrap, while Generator rows include Generator training and guided continuation. The breakdown is Solver training + Teacher inference (+ Generator training).
| Method | Estimated total cost (A100h) | Average accuracy (%) |
|---|---|---|
| Absolute Zero | ~384 | 52.6 |
| SPIRAL | ~672 | 47.0 |
| R-Zero | ~576 | 54.7 |
| SPICE | >1000 | 53.8 (estimate) |
| Socratic-Zero (8B Teacher) | ~98 = 96 + 2 | 47.6 |
| Socratic-Zero (235B Teacher) | ~112 = 96 + 16 | 56.1 |
| Socratic-Zero (8B Teacher, 8B Generator) | ~382 = 320 + 2 + 60 | 59.3 |
| Socratic-Zero (235B Teacher, 32B Generator) | ~492 = 332 + 16 + 144 | 67.5 |
The complete all-8B system achieves 59.3%, exceeding Absolute Zero by 6.7 points at comparable estimated compute (382 vs. 384 A100h) and R-Zero by 4.6 points at lower compute (382 vs. 576 A100h). The 47.6% row isolates 8B-Teacher bootstrap; its compute is about one-quarter of Absolute Zero's and one-sixth of R-Zero's. We apologize for the typo in Table 7a: the correct bootstrap score and gain are 47.6% and +11.7 points over 35.9%. We corrected that entry (p. 9, line 441) and highlighted the complete-system comparison in the main text (p. 10, lines 514–521), with cost accounting in Appendix A.4 (p. 14, lines 736–741; Table 13). SPICE follows the estimated-score protocol in Q1.
Response to Reviewer qMVd (Part 3 of 3)
Q5. About Quality-Control Mechanisms. Thank you for the suggestion to make the comparison more precise. Absolute Zero uses execution-based validation, including program-integrity and determinism checks; R-Zero uses Solver answer consistency, difficulty filtering, repetition penalties and format checks. Our contribution connects the content of a failed Solver trajectory to targeted mathematical refinement and reference validation: the Teacher uses the parent problem, reference and failure analysis to construct a revised problem–solution pair, which passes consistency filtering, MathRule checking and failure-triggered review. The Generator learns this refinement policy through difficulty-weighted training. We made this distinction explicit in the Introduction (p. 1, lines 43–50) and Related Work (p. 2, lines 94–99); Q2 and Appendix I (pp. 20–21) describe the validation procedure.
Q6. About the Generator's Curriculum Advantage. The Generator learns through reward-weighted distillation: Teacher refinements provide training examples, while the Solver's success rate supplies their utility reward. WSFT prioritizes problems near 50% success, specializing the Generator in curricula that challenge the current Solver. This follows the reward-guided intuition of offline reinforcement learning and is implemented through weighted SFT. The fixed Teacher provides refinement examples; the trainable Generator learns which refinements are most useful for the Solver, explaining its advantage in curriculum utility.
Both continuation branches start from the same Qwen3-8B Solver checkpoint at 56.1% and use DPO at matched cumulative data scales. Qwen3-235B-A22B supplies the Teacher branch; Socratic-Generator-32B supplies the Generator branch.
| Cumulative examples | Teacher-guided Solver (%) | Generator-guided Solver (%) | Difference (pp) |
|---|---|---|---|
| ~8K, shared starting point | 56.1 | 56.1 | — |
| ~16K | 59.2 | 59.5 | +0.3 |
| ~28K | 62.3 | 64.1 | +1.8 |
| ~47K | 62.8 ± 0.3 | 67.5 ± 0.2 | +4.7 |
The 47K statistics cover three independent runs per branch. Table 4 separately evaluates generated-data utility using the same DeepSeek-R1-Distill-Llama-8B Student: 36.04% for our Generator versus 35.20% for the Teacher. The weighted objective is defined in Section 3.3 (p. 5, lines 221–246), trainability in Appendix N.1 (p. 24, lines 1246–1251), and continuation roles and statistics in Tables 5 and 7 (pp. 8–9).
Q7. About Terminology and Presentation. “Data-free” describes automatic curriculum expansion from a small seed set, without additional externally curated problem–solution training data. The 1,500-example SFT primarily aligns instruction following for the Base Solver; it is optional when the starting model already has that ability. Table 2's deep research, SWE, chart reasoning and multimodal coding experiments start directly from Instruct models and small seed curricula. This follows the zero-style self-play convention: related methods also specify prompts and initialization, such as R-Zero's Solver/Challenger templates (Appendix A.2) and Absolute Zero's prompted buffer initialization (Section 3.3.1).
Within this loop, “co-evolution” refers to the Solver and its learned curriculum policy: the Solver and Generator are optimized, while the Teacher remains fixed during bootstrap. “Autonomous” describes the automatic generation–verification–training cycle driven by Solver feedback. Our contribution is learning and transferring a failure-conditioned refinement policy, supported by the direct-distillation, all-8B and Generator–Teacher comparisons above. We clarified these definitions and initialization settings in the Introduction (p. 2, lines 69–75), Section 3.1 (p. 3, lines 151–161), Appendix O (p. 25, lines 1335–1338), and Appendix S (p. 27, lines 1426–1430).
For the figures, the icons were schematic illustrations of roles. We will label Figure 1 with the actual Qwen3-235B-A22B Teacher, Qwen3-8B-Base Solver and Qwen3-32B Generator, clearly distinguishing fixed and trainable components. Figure 2 will include a concrete mathematical example linking a failed Solver attempt, targeted problem refinement and the success rate used for its utility weight. These changes make the same learning loop explicit in both terminology and illustration.
Response to Reviewer qMVd (Part 1 of 3)
Thank you for the careful review. We address the concerns below; key manuscript revisions are marked in blue.
Q1. About Baselines and Evaluation Settings. All trained baselines in Table 1 except SPICE were reproduced across all seven mathematical benchmarks: Static Augmentation, Distillation (memath_math), DPO (Polaris), DPO (MATH-full), LLM2LLM, Absolute Zero, SPIRAL, and R-Zero. They share our Qwen3-8B-Base checkpoint, 1,500-pair MATH Level-5 SFT initialization, verifier, and zero-shot Mean@32 evaluation; the Base Model and Initial SFT rows were also evaluated locally. SPICE's training-document collection was unavailable, so its scores are estimates based on published results and locally evaluated reference checkpoints. Its approximately 82K training instances are estimated from its reported configuration; other baseline data scales follow their original papers.
The base-score discrepancy comes from evaluation settings. We use the Appendix B.1 zero-shot prompt with a chat template, reasoning steps and a boxed answer; temperature 0.7, top-p 0.9, and at most 8,192 new tokens. Mean@32 averages correctness over 32 independent responses per problem and then over problems. MathRule extracts and normalizes answers against benchmark references, with model checks for ambiguous cases. The official Qwen3 report uses four-shot chain-of-thought prompting. With that setup and greedy decoding, our GSM8K score is 89.74%, matching the official 89.84% within 0.10 points. Our reported 35.9% → 56.1% gain uses the same zero-shot Mean@32 protocol throughout.
The requested direct-distillation baseline was already evaluated; we apologize that it was not presented prominently enough. The first three conditions below share the 235B Teacher, 8B Solver, 100 seeds, SFT initialization and approximately 7.9K generated examples. Direct distillation uses Teacher problem–solution pairs generated in advance; adaptive SFT uses verified successful Solver trajectories; the full method uses the adaptive curriculum with DPO.
The full method exceeds direct distillation by 17.0 points at matched data volume. We clarified sources and evaluation in Section 4.1 (p. 5, lines 252–263), Table 1 (p. 6, lines 270–276), and Appendix H (p. 20, lines 1032–1044), and brought the distillation comparison into Table 8 (p. 9, lines 463–478; protocol: Appendix Q, p. 26, lines 1387–1393).
Q2. About Verification and Human Calibration. MathRule is the primary checker in V(q, y): it extracts and normalizes the Solver's final answer against the curriculum reference, which is implicit in this notation. During bootstrap, the Teacher supplies supplementary semantic judgments when rule-based extraction is insufficient or ambiguous. The 100 seed problem–solution pairs provide initial references. For a refined problem, the Teacher constructs its reference from the parent problem and solution, the failed Solver trajectory, and the refinement context. We sample three independent Teacher solutions and retain pairs with majority-consistent final answers. If all eight Solver attempts fail, the Teacher re-solves and reviews the problem for mathematical errors, ambiguity and missing information; failed pairs are excluded from Solver and Generator training. Rule/model disagreements also trigger checks. During continuation, the Generator takes over reference construction and model-side verification, with MathRule still primary.
We apologize for the unclear human-evaluation description. Human review calibrated the judge's prompt: ten evaluators with mathematics-competition experience blindly reviewed 1,000 model-accepted problems in total and cross-checked model-flagged cases. Each problem received three judgments, resolved by majority vote, checking mathematical validity, clarity and answer definiteness against references. After prompt calibration, model and human judgments showed good agreement; the calibrated model judge performed the full Table 3 evaluation. Section 4.2.1 now states this correctly (p. 7, line 364). The verification and filtering procedures are clarified in Section 3.1 (p. 4, lines 167–175) and Appendix I (pp. 20–21, lines 1072–1119).