acceptodds
Under review as a conference paper at ICLR 2027

OmniEdu: Open Foundation Models for Learning and Teaching

Abstract

Educational foundation models must do more than produce correct answers: they must understand where a problem sits in a curriculum, diagnose why a learner is struggling, and choose an appropriate instructional response. Existing educational language models often specialize in either subject problem solving or tutoring, while their training mixtures are commonly organized by source or task and do not explicitly balance these capabilities. We present OmniEdu, an open family of foundation models for K–12 learning and teaching, trained with a capability-oriented instruction-tuning corpus. The corpus combines more than 100 educational resources and general instruction sources and organizes supervision around four complementary capabilities: subject competence, curriculum grounding, diagnostic reasoning, and pedagogical action and scaffolding. A multi-stage pipeline performs deterministic cleaning, semantic auditing and rewriting, task-specific quality scoring, token-budgeted diversity selection, and pedagogical instruction assignment, yielding 69,999 examples and 15.96M supervised response tokens, including 60,951 education-specific examples. We fine-tune 4B, 9B, and 27B models and evaluate them on curriculum-grounding, K–12 problem-solving, and pedagogical-tutoring benchmarks, with auxiliary tests of general capability. Across model scales, education-oriented tuning consistently improves all three educational capability groups. In particular, OmniEdu-27B reaches 63.12% EM / 76.69% F1 on K12-Bench, 85.89% on MathFish, 86.95% on EDUMATH, 78.74% on MathTutorBench's Scaffold setting, and the best Teaching average of 3.02 on LongTutor among the evaluated models. These results show that carefully curated, capability-balanced supervision can turn a general language model into a stronger educational system that not only solves problems, but also understands curriculum structure and supports effective teaching interactions.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Educational Model, LLM, Data Curation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_UGREOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper describes the process for finetuning 4B, 9B, and 27B Qwen family models for education-related tasks, including curriculum grounding, K-12 problem solving, and pedagogical tutoring.

Critical strengths

Open models are usually a helpful resource for the community, and the authors show that their model outperforms other open models in this domain on relevant benchmarks (e.g. Table 1, Table 2).

Critical weaknesses

The data curation and filtering process could have been more strongly justified by some experiments motivating the choices being made or illustrating the choices’ empirical benefits. Some parts of the process are pretty sensible on a surface-level, e.g. sampling for diversity. However, the LM-assisted step of auditing and rewriting training data raises questions about the reliability of the LM in actually substantively improving the data rather than potentially harming quality and performance.

The “benchmarks” paragraph of the paper is a bit dense and could be revised/rewritten so that it doesn’t resemble a laundry list of benchmarks too much.

The paper shows solid performance numbers and producing a strong open model can facilitate future research, but I would have been interested in seeing more scientific contributions from a paper like this. Essentially this paper describes domain-specific model finetuning, and that finetuning yields domain-specific gains, which is a phenomenon that is already known by the research community. By scientific contributions, I’m looking for findings that target the “why” and link model design and training decisions to improvements in performance, so that future model developers in this domain have concrete findings or recommendations to learn from. In particular, it would have been good to emphasize more the unique nature of developing a model for education and potential takeaways or challenges of this specific area. Right now, the paper reads in a way where the domain itself is kind of a secondary concern (e.g. the paper narrative focuses more on generic model development concerns like, “data is varied and messy” and “our performance numbers on benchmarks are high!”). Currently, the main potential recommendation of this paper for other researchers seems to be “curate your data like we do” which is a bit vague. However, I have not carefully looked at the paper appendix, so if there are some concrete nuggets in there, it could be good to reference those during discussion/rebuttal.

Questions for authors

None

4 comments
AuthorsOpenReview ↗

Authors' Response [1/n]

Thank you very much for your recognition and further valuable suggestions. We will clarify the following points regarding these issues and will eventually include them in the main text or appendix.

Response to Weakness 1

Thank you for this valuable suggestion. We provide additional controlled experiments and human evaluations to validate our data curation decisions, particularly the effectiveness of LLM-assisted auditing, rewriting, and quality filtering.

(1) Controlled Evaluation of Data Curation. We conduct controlled SFT experiments using Qwen3.5-4B-Base with matched supervised token budgets, training hyperparameters, and evaluation protocols:

  • A (Full Pipeline): Our complete capability-oriented data construction pipeline.
  • B (Vanilla Edu-SFT): Educational QA data with basic cleaning and decontamination, natural source proportions, and a unified general instruction.
  • C (w/o Capability Allocation): The same quality-filtered candidate pool and selection procedures as A, but with budgets allocated according to the original capability distribution.

Table 1. Controlled ablation study results

ModelK12 EM (%)K12 F1 (%)MathFish (%)EDUMATH (%)GAOKAO (%)EXAMS-V (%)MDK12 (%)MT-S (%)MT-SH (%)TutorBench (%)LT-E (%)LT-T
A · Full Pipeline54.2571.7583.1968.4090.4557.6246.3675.7984.7746.6765.882.29
B · Vanilla Edu-SFT42.0370.0981.3156.8477.8748.5441.4455.5860.1633.1320.392.11
C · w/o Capability Allocation48.2870.8882.4365.7988.7952.8344.1672.2180.0844.3161.102.24

As shown in Tables 1, A outperforms both B and C across all 12 metrics. Compared with B, A improves K12-Bench EM by 12.22 points, GAOKAO-Bench by 12.58 points, and MathTutor-S by 20.21 points. Compared with C, A further gains 5.97 points on K12-Bench EM, 4.79 on EXAMS-V, and 4.78 on LongTutor-E. These results demonstrate benefits beyond conventional educational SFT and provide direct evidence for the effectiveness of capability-aware data allocation.

(2) Human Evaluation of Semantic Auditing and Rewriting.

As described in Section 3.3, we employ Qwen3.5-122B-A10B-FP8 for capability-specific semantic auditing and targeted rewriting. To evaluate rewriting quality, we randomly sample 1,000 original–rewritten pairs, stratified across four educational capabilities. Two annotators independently examine each pair in randomized order and select the better version, or indicate that both are comparable, based on task-specific quality criteria. Disagreements are resolved through adjudication (inter-annotator agreement: Cohen's κ = 0.92).

Table 4. Human preference between original and rewritten supervision (%).

CapabilityImproved (95% CI)Tie (95% CI)Degraded (95% CI)
Subject Competence83.6 (78.5, 87.7)14.8 (10.9, 19.7)1.6 (0.6, 4.0)
Pedagogical Action & Scaffolding94.0 (90.3, 96.3)5.6 (3.4, 9.2)0.4 (0.1, 2.2)
Curriculum Grounding86.8 (82.0, 90.4)12.4 (8.9, 17.1)0.8 (0.2, 2.9)
Diagnostic Reasoning91.2 (87.0, 94.1)7.6 (4.9, 11.6)1.2 (0.4, 3.5)
Overall88.9 (86.8, 90.7)10.1 (8.4, 12.1)1.0 (0.5, 1.8)

Human evaluation shows that rewriting improves supervision quality in 88.9% of cases, with only 1.0% degraded. The consistent improvements across all four educational capabilities support the effectiveness of targeted rewriting while indicating a low risk of introducing errors.

AuthorsOpenReview ↗

Authors' Response [2/n]

(3) Human Validation of Fine-Grained Quality Scoring.

As described in Section 3.4, we employ GPT-5.6-Terra to score training examples using capability-specific quality rubrics and predefined acceptance thresholds. To validate its filtering decisions, we sample 2,000 scored examples, stratified by capability and filtering outcome. Two annotators independently assess each example against the corresponding quality criteria without access to the LLM scores. Disagreements are resolved through adjudication (inter-annotator agreement: Cohen's κ = 0.84).

Table 5. Human validation of LLM quality-filtering decisions (%).

CapabilityValid among accepted (95% CI)Invalid among rejected (95% CI)
Subject Competence96.4 (93.3, 98.1)88.4 (83.8, 91.8)
Pedagogical Action & Scaffolding96.8 (93.8, 98.4)94.4 (90.8, 96.6)
Curriculum Grounding95.6 (92.3, 97.5)90.8 (86.6, 93.8)
Diagnostic Reasoning94.8 (91.3, 96.9)92.0 (88.0, 94.8)
Overall95.9 (94.5, 97.0)91.4 (89.5, 93.0)

Human evaluation confirms that 95.9% of LLM-accepted examples meet the required quality standards, while 91.4% of rejected examples are independently judged unsuitable, supporting the reliability of our fine-grained quality filtering.

Together, the controlled ablations and human evaluations demonstrate the downstream benefits of our data curation strategy and the reliability of its two LLM-assisted quality-control stages. We will incorporate these analyses into the revised appendix.

Response to Weakness 2

Thank you for this suggestion. Our intention was to explain what each benchmark evaluates, helping readers understand the results beyond the reported scores. We will revise this section to organize the evaluation around three educational capability groups—curriculum grounding, K–12 problem solving, and pedagogical tutoring—highlighting their complementary evaluation objectives. Detailed descriptions of individual benchmarks will be moved to the appendix for reference.

Response to Weakness 3

Thank you for this thoughtful suggestion. We would like to clarify that education-specific considerations are central to our framework. We also provide additional controlled experiments to establish more explicit connections between our design choices and model performance.

(1) Education-Specific Data Design.

Our framework organizes educational supervision around four core capabilities: Subject Competence, Pedagogical Action & Scaffolding, Curriculum Grounding, and Diagnostic Reasoning. These capabilities guide data collection, task-specific quality assessment, semantic auditing and rewriting, and training-budget allocation. Moreover, as discussed in Section 3.6, educational tasks may require conflicting response behaviors (e.g., providing answers versus guiding learners without revealing them), which we address through 19 task-specific pedagogical system instructions. This solution highlights a practical lesson for educational model development.

(2) Dataset, Pipeline, and Model Contributions.

We contribute a 69,999-example capability-oriented dataset, a reproducible education-specific data construction pipeline, and 4B/9B/27B open-weight educational models achieving leading or competitive performance across diverse educational benchmarks. These resources provide a practical foundation for further research and development in educational AI.

(3) Empirical Insights and Generalizable Lessons.

As detailed in our Response to Weakness 1, controlled experiments show that our full pipeline outperforms conventional educational SFT, and that capability-aware allocation brings additional gains under matched training budgets.

These findings highlight the importance of balancing training data across educational capabilities. Moreover, our task-specific system instructions address another challenge of adapting model responses to different teaching tasks, which often require distinct pedagogical behaviors.

We will revise the paper to emphasize these education-specific challenges, empirical findings, and actionable design insights.

AuthorsOpenReview ↗

Follow-up on Our Rebuttal

Dear Reviewer UGRE,

Thank you again for your insightful and constructive suggestions. In response to your concerns, we have added controlled ablation experiments and human evaluations of LLM-assisted auditing, rewriting, and quality filtering to provide stronger evidence for our data curation decisions. We have also clarified the education-specific design considerations and empirical insights of our framework and outlined revisions to improve the presentation of our evaluation benchmarks.

We would greatly appreciate it if you could review our rebuttal and share whether these additions address your concerns. If so, we would also be grateful if you could consider updating your overall rating. We would be happy to address any remaining questions.

Best regards, The Authors

Dear Reviewer UGRE,

Thank you for asking what other educational model developers can learn from this work. Table 1 of our response tests a concrete design choice. With the candidate pool, selection procedure, and token budget held fixed, explicitly allocating tokens across educational capabilities improved K12-Bench EM by 5.97 points and LongTutor-E by 4.78 points. In a separate human assessment of 1,000 pairs, annotators preferred the rewritten supervision in 88.9% of cases. Does the allocation result address your concern about what the paper contributes beyond domain-specific fine-tuning?

Reviewer_YMzROpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces OmniEdu, a family of open educational language models adapted for K–12 learning and teaching. The authors propose organizing educational supervision around four capabilities: subject competence, curriculum grounding, diagnostic reasoning, and pedagogical action and scaffolding.

The training corpus is constructed from educational resources and general instruction sources through deterministic cleaning, LLM-assisted semantic auditing, quality scoring, diversity-aware selection, and pedagogical instruction assignment. The resulting dataset contains approximately 70K instruction examples and 15.96M supervised response tokens.

The authors fine-tune Qwen-based 4B, 9B, and 27B models and evaluate them across curriculum-grounding, K–12 problem-solving, and pedagogical-tutoring benchmarks, with additional general-capability evaluations.

Critical strengths
  1. Broad evaluation across complementary educational capabilities. The paper evaluates more than subject-level answer correctness. Curriculum grounding, problem solving, error diagnosis, and tutoring quality are examined using multiple benchmarks, providing a relatively comprehensive picture of educational model behavior. The improvements are consistent across all three model scales. This breadth is valuable because educational usefulness cannot be adequately characterized by examination accuracy alone.

  2. Detailed documentation of dataset provenance and preprocessing. The paper documents a multi-stage curation process, retains source-level provenance, and provides task-specific quality rubrics and pedagogical instructions. This level of transparency is useful for a data-centric model-development contribution.

Critical weaknesses
  1. The decontamination methodology is insufficiently robust for the degree of overlap between training sources and evaluation tasks. This is relevant because the training mixture contains data from sources closely related to evaluated benchmarks. For example, the training set includes K12-KGraph, MathFish, and LongTutor-derived examples, while the evaluation includes K12-Bench, MathFish, and LongTutor. This does not establish that actual contamination occurred. However, the current procedure does not convincingly exclude overlap at the level of underlying problems, curriculum graph facts, student histories, or generated teaching targets. The particularly large improvements in some curriculum and longitudinal-tutoring metrics make this concern consequential. A stronger decontamination analysis is necessary before interpreting these gains as evidence of generalization.

  2. Several baseline comparisons appear affected by model compatibility, answer parsing, or inference configuration. The paper also acknowledges that some baseline models do not support image inputs and therefore receive only the textual components of multimodal questions. Such models are included in the overall comparisons despite being unable to access information required for successful predictions.

Several proprietary models also produce unexpectedly low results on some benchmarks. For example, GPT-5.6-Sol achieves 24.82% on EXAMS-V and 10.53% on MathTutorBench Scaffold, while smaller education-tuned models score substantially higher. These outcomes may reflect genuine differences in task behavior, but warrant additional validation given the reported inference settings and parsing constraints.

Questions for authors
  1. Can the authors report overlap statistics beyond exact question matching, including semantic near-duplicates and shared source entities, curriculum graph relations, and student-history records? In particular, how are the training and evaluation partitions of K12-KGraph/K12-Bench, MathFish, and LongTutor separated?

  2. Can the authors provide answer-parse success rates, invalid-output rates, and prediction distributions for EXAMS-V and MathTutorBench? What explains the identical 21.95% EXAMS-V results for multiple unrelated educational models?

5 comments
AuthorsOpenReview ↗

Authors' Response [1/n]

Thank you very much for your recognition and further valuable suggestions. We will clarify the following points regarding these issues and will eventually include them in the main text or appendix.

Response to Weakness 1

Thank you for these valuable suggestions. We provide additional details on data separation and conduct further overlap analysis across the relevant benchmarks.

(1) Data Splits and Overlap Analysis.

  • K12-Bench: Among 23,638 evaluation instances, only 311 (1.32%) share assessed knowledge-graph triples with training data. Character-level similarity screening (≥0.9) identifies 364 (1.54%) near-duplicate candidates, of which 5.77% are confirmed as substantive overlaps through manual verification.
  • MathFish: Training uses the official train split and evaluation uses the official dev split, with zero shared problem IDs. Similarity screening (≥0.9) identifies 141 near-duplicate candidates.
  • TutorBench: All 3,492 related training examples originate from TutorChat. Among 1,473 evaluation instances, we find zero direct matches in text, images, or source records, and no near-duplicates with similarity ≥0.9.
  • LongTutor: Portions of LongTutor-Gold and LongTutor-Silver were used for training, and the corresponding instances were excluded from evaluation. Further verification confirmed zero overlapping student IDs or learning trajectories between the training and remaining evaluation data. LongTutor-Synthesis was separately generated from external problem sources (e.g., GSM8K).

(2) Decontamination and Generalization.

To further verify the robustness of our findings, we evaluate model performance after excluding potentially overlapping instances from K12-Bench and MathFish.

Table 1. Evaluation results before and after excluding potentially overlapping test instances

ModelK12-Bench EM (Original / Clean)K12-Bench F1 (Original / Clean)MathFish Acc. (Original / Clean)
OmniEdu-4B54.25% / 54.01%71.75% / 71.58%83.19% / 83.21%
OmniEdu-9B55.46% / 54.92%73.68% / 72.89%83.70% / 83.23%
OmniEdu-27B63.12% / 63.23%76.69% / 75.96%85.89% / 85.31%

As shown in Table 1, performance remains largely stable after excluding potentially overlapping evaluation instances, with changes of at most 0.79 percentage points across all three models and metrics. These results suggest that the reported performance is not primarily driven by the identified training–evaluation overlaps.

We will also clarify the data splits, overlap-detection procedures, and synthetic data provenance in the revised appendix, providing stronger evidence that the observed improvements are not driven by training–evaluation overlap.

AuthorsOpenReview ↗

Authors' Response [2/n]

Response to Weakness 2

Thank you for this observation. We clarify the following four aspects regarding evaluation fairness.

(1) Model Compatibility. As stated in Section 4.1, some educational baselines do not support image inputs. Therefore, they receive only the textual components of multimodal questions, which reflect differences in modality support as well as educational capabilities. We preserve the original image-based setting rather than introducing OCR, which could introduce additional recognition errors. In addition, our primary evidence for training effectiveness comes from paired comparisons between OmniEdu models and their corresponding backbones, where input modalities and evaluation protocols are identical.

(2) Answer Parsing and EXAMS-V. We generally follow benchmark-specific evaluation protocols, applying the same output-format requirements and parsing rules to all compared models. We further analyze parsing success rates, invalid output rates, and prediction distributions, as summarized in Table 1. Most parsing failures involve 823 Arabic-language questions, where models output Arabic option labels instead of the required A–E labels. Even after excluding these instances, GPT-5.6-Sol achieves only 25.73% accuracy, and manual inspection confirms that its low score primarily reflects incorrect predictions rather than parsing failures.

Table 1. EXAMS-V Parsing and Prediction Diagnostics. Confucius3-Math generates lengthy reasoning about missing image inputs, while GLM-5.3 refuses to answer. Both achieve 100% parsing success solely through fallback parsing.

ModelFinal Parse Success (%)Invalid Outputs (%)A (%)B (%)C (%)D (%)E (%)Others/Empty (%)Overall Acc. (%)
Qwen3.5-4B-Base100.000.0046.2017.8014.119.0912.360.4444.69
OmniEdu-4B100.000.0019.8932.2133.9812.090.581.2557.62
Qwen3.5-9B-Base100.000.0023.8521.8929.6914.746.363.4863.00
OmniEdu-9B100.000.0022.7031.1926.8116.410.442.4666.40
Qwen3.8-27B100.000.0025.2424.9927.4116.140.965.2568.65
OmniEdu-27B99.710.2922.7625.6228.0815.571.296.6769.52
Confucius3-Math100.00*0.00*100.000000021.95
MuduoLLM100.000.00100.000000021.95
EduChat-SFT-Qwen2.5-7B100.000.00100.000000021.95
EduChat-R1-Qwen3-8B100.000.000000100.0000.69
EduChat-R1-Qwen3-32B100.000.00100.000000021.95
GPT-5.496.453.5525.0328.0526.6913.870.835.5335.66
GPT-5.6-Sol96.453.5524.1927.9528.4711.990.636.7824.82
Claude-Opus-599.370.6321.4826.9026.8018.251.155.4264.34
GLM-5.3100.00*0.00*46.8831.6712.928.540.000.0022.29
Kimi-K3100.000.0024.5824.3827.7117.501.254.5887.29

(3) MathTutorBench Scaffold. GPT-5.6-Sol achieves a 100% evaluation success rate with 0% invalid outputs, ruling out parsing failures as the cause of its low Scaffold score. We further manually inspect a randomly sampled 20% of its responses and find that 91.2% directly reveal the final answer instead of providing scaffolded guidance, resulting in low pedagogical reward scores and explaining its 10.53% Scaffold win rate.

(4) Inference Configuration. As stated in Appendix A.5 and Table 10, we standardize inference settings wherever possible, using temperature=0 and non-thinking mode. For the few models that cannot disable reasoning, we use the lowest available reasoning effort, while OmniEdu is evaluated with thinking disabled. Thus, these exceptions do not give our models an advantage.

We will incorporate these diagnostic analyses and clarify the evaluation limitations in the revised appendix.

AuthorsOpenReview ↗

Authors' Response [3/n]

Response to Question 1

We have discussed this in Weakness 1.

Response to Question 2

We have discussed the parsing success rates, invalid outputs, and prediction distributions in Weakness 2.

Regarding the identical 21.95% EXAMS-V accuracy obtained by four educational models, we explain the cause below.

(1) Evaluation Setup and Modality Limitation.

In EXAMS-V, both questions and answer options are provided as images. We intentionally preserve the original image-based evaluation setting rather than converting screenshots into text via OCR, as OCR introduces additional recognition errors and confounding factors. Consequently, the four text-only educational models, which do not support image inputs, receive only the general answering instruction without the actual question content. This reflects their input-modality limitations under the original benchmark setting.

(2) Explanation of Identical Scores.

We had identified and verified the cause of these identical scores during evaluation. Specifically, MuduoLLM, EduChat-R1-Qwen3-32B, and EduChat-SFT-Qwen2.5-7B consistently output A for every question, with 0% strict parsing failure. Confucius3-Math instead generates lengthy reasoning about the missing question content, resulting in 100% strict parsing failure. Our fallback parser nevertheless extracts A from the opening word Okay in its response. Thus, all four models ultimately receive the same prediction (A) for every instance, explaining their identical scores.

Reviewer_VSDVOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper constructs an educational instruction-tuning dataset through multi-stage quality filtering, embedding-based diversity selection, supervised-token budgeting, and task-specific pedagogical system instructions. The authors perform full-parameter supervised fine-tuning of existing models at 4B, 9B, and 27B scales. Evaluation covers curriculum understanding, subject problem solving, and pedagogical tutoring, with aggregate improvements over the corresponding backbones. The main contributions are the educational dataset, model resources, and post-training pipeline.

Critical strengths

The problem is practically relevant, and the evaluation covers several important capabilities. The paper evaluates not only answer correctness but also curriculum understanding, learner diagnosis, and instructional guidance. These capabilities are relevant to educational assistants and provide a broader assessment than subject-level question answering alone. Comparisons against the corresponding backbones provide meaningful empirical evidence. Improvements across multiple model scales and benchmarks suggest that the supervision mixture can improve educational task performance, particularly scaffolded responses. The dataset and model resources have potential community value. Organizing educational supervision by intended behavior and allocating budgets by supervised response tokens are useful practical design choices. The released resources could support further research on educational post-training.

Critical weaknesses

The central data-design contribution is not sufficiently isolatedThe paper attributes the improvements to capability-oriented data organization and curation, but the experiments mainly compare the complete fine-tuning pipeline against backbones that have not undergone this tuning. These comparisons do not separate the effects of educational SFT itself, quality filtering, capability allocation, diversity selection, and task-specific system instructions. In particular, the 4B and 9B backbones are labeled as Base models. Their large improvements in pedagogical behavior may partly reflect general instruction alignment rather than the proposed data organization. I do not expect exhaustive ablations of every pipeline stage. However, a simple educational SFT baseline using the same backbone and supervised-token budget, together with a targeted ablation of the central design, would substantially strengthen the contribution. Without these controls, the paper establishes a useful training recipe but does not clearly identify the generalizable empirical insight underlying its effectiveness. 2. Unusual baseline results require clarificationIn Appendix Table 8, four different educational models have identical EXAMS-V scores across the overall metric and every reported subset. For example, all four obtain 21.95% overall and 22.26% on text-only examples. This does not establish an evaluation error, but it warrants an explanation of whether parsing failures, fallback predictions, or model-adaptation issues are involved. The paper also supplies only textual inputs to models without image support. While this limitation is disclosed, such comparisons combine differences in educational capability with differences in available input modalities. Reporting parsing-failure rates and clearly separating matched-input comparisons would make the results more interpretable. 3. Train–test separation is insufficiently documentedThe training mixture includes MathFish, TutorBench-derived data, and LongTutor data, while evaluation uses the corresponding benchmarks. Using legitimate training splits is reasonable, but the description of evaluation decontamination does not provide enough detail to assess the separation. The authors should specify the splits used, the overlap-detection procedure, whether near-duplicates and shared problems or student trajectories were checked, and whether synthetic-data generation accessed test content. This is a concern about the strength of the generalization evidence, not an allegation of data leakage. 4. Claims about pedagogical and diagnostic improvements need qualificationThe gains vary substantially across benchmarks and metrics. For the 27B model, TutorBench overall improves by only 0.84 percentage points. On LongTutor diagnosis, accuracy increases, but F1 decreases from 38.41% to 30.58%. These changes deserve discussion, particularly whether they reflect shifts in class-prediction behavior. Furthermore, improved response quality on tutoring benchmarks does not establish improved student learning. I would not require a classroom study as a condition for acceptance, but the conclusions should remain limited to the educational capabilities measured by the reported evaluations.

Questions for authors

Under matched supervised-token and training budgets, how much additional benefit does the proposed pipeline provide over a simple mixture of high-quality educational data? Can a targeted ablation isolate the effects of capability-oriented allocation or task-specific instructions? Why do several educational models have identical EXAMS-V results across all reported subsets? What are the output-parsing failure rates, and how are invalid responses handled? How are benchmarks sharing sources with the training mixture split and decontaminated? Can the authors provide split information and near-duplicate checking results, including checks for shared student trajectories? How should the increase in LongTutor diagnosis accuracy and decrease in F1 be interpreted? Does this indicate a change in class-prediction behavior?

5 comments
AuthorsOpenReview ↗

Authors' Response [1/n]

Thank you very much for your recognition and further valuable suggestions. We will clarify the following points regarding these issues and will eventually include them in the main text or appendix.

Response to Weakness 1

Thank you for this suggestion. We provide additional experiments to better isolate the contributions of our data design.

(1) Matched-Backbone Comparisons.

We extend our evaluation to five backbone families (7B–32B), comparing Omni-Edu-text with existing educational models fine-tuned from the corresponding backbones. As shown in Tables 1, our models outperform these educational baselines in 48 out of 55 comparisons across curriculum grounding, K–12 problem solving, and pedagogical tutoring, demonstrating the effectiveness of our approach beyond comparisons with the original backbones.

Table 1. Matched-backbone comparisons results

ModelK12 EM (%)K12 F1 (%)MathFish (%)EDUMATH (%)GAOKAO (%)MDK12 (%)MT-S (%)MT-SH (%)TutorBench (%)LT-E (%)LT-T
DeepSeek-R1-Distill-Qwen-14B family
DeepSeek-R1-Distill-Qwen-14B48.1970.1779.3259.9587.4746.7119.1319.5729.875.161.02
Confucius3-Math5.2319.230.003.0084.5139.1327.6820.7034.7223.071.36
+ Omni-Edu-text (ours)51.6871.5081.4376.4387.9647.0724.3520.8035.7138.971.39
Qwen2.5-14B-Instruct family
Qwen2.5-14B-Instruct35.2968.6882.3384.1087.4646.3938.8732.7229.016.021.03
MuduoLLM48.2470.4682.8557.2089.6645.4430.4223.4435.9824.311.28
+ Omni-Edu-text (ours)45.6571.1283.4887.9688.0551.4774.5274.6232.5742.391.48
Qwen2.5-7B family
Qwen2.5-7B30.2563.5123.9441.4473.1836.312.000.3122.684.501.08
EduChat-SFT-Qwen2.5-7B45.2067.5878.2328.0070.7432.5018.2122.6621.4826.551.21
+ Omni-Edu-text (ours)46.3966.8782.9051.9677.9943.7378.7081.0424.9337.371.56
Qwen3-8B family
Qwen3-8B40.4968.7475.1087.1486.0840.8433.0431.8030.036.051.03
EduChat-R1-Qwen3-8B25.2549.6719.4947.0076.0641.2911.587.4223.7133.751.14
+ Omni-Edu-text (ours)40.7469.7083.5188.2486.8246.4774.1767.5833.8939.331.34
Qwen3-32B family
Qwen3-32B46.1071.8671.1591.8988.2048.0251.3042.8134.9722.461.06
EduChat-R1-Qwen3-32B49.4271.2183.7161.2083.8741.8422.2116.8025.9252.031.56
+ Omni-Edu-text (ours)51.1673.4784.3092.7193.6449.2673.2271.5635.5842.361.44
AuthorsOpenReview ↗

Authors' Response [2/n]

(2) Controlled Ablation Study.

Using Qwen3.5-4B-Base, we compare three settings under matched supervised response-token budgets, training hyperparameters, and evaluation protocols:

  • A (Full Pipeline): Our complete capability-oriented data construction pipeline.
  • B (Vanilla Edu-SFT): Educational QA data with basic cleaning and decontamination, natural source proportions, and a unified general instruction.
  • C (w/o Capability Allocation): The same quality-filtered candidate pool and selection procedures as A, but with budgets allocated according to the original capability distribution.

Table 2. Controlled ablation study results

ModelK12 EM (%)K12 F1 (%)MathFish (%)EDUMATH (%)GAOKAO (%)EXAMS-V (%)MDK12 (%)MT-S (%)MT-SH (%)TutorBench (%)LT-E (%)LT-T
A · Full Pipeline54.2571.7583.1968.4090.4557.6246.3675.7984.7746.6765.882.29
B · Vanilla Edu-SFT42.0370.0981.3156.8477.8748.5441.4455.5860.1633.1320.392.11
C · w/o Capability Allocation48.2870.8882.4365.7988.7952.8344.1672.2180.0844.3161.102.24

As shown in Tables 2, A outperforms both B and C across all 12 metrics. Compared with B, A improves K12-Bench EM by 12.22 points, GAOKAO-Bench by 12.58 points, and MathTutor-S by 20.21 points. Compared with C, A further gains 5.97 points on K12-Bench EM, 4.79 on EXAMS-V, and 4.78 on LongTutor-E. These results demonstrate benefits beyond conventional educational SFT and provide direct evidence for the effectiveness of capability-aware data allocation.

(3) Improvements on Post-Trained Models.

Our approach also improves already post-trained models, including Qwen3.8-27B, Qwen2.5-14B-Instruct, and DeepSeek-R1-Distill-Qwen-14B. For example, K12-Bench EM improves from 52.11% to 63.12% on Qwen3.8-27B, MathTutor-S from 38.87% to 74.52% on Qwen2.5-14B-Instruct, and EDUMATH from 59.95% to 76.43% on DeepSeek-R1-Distill-Qwen-14B. These results confirm that the benefits of our educational adaptation extend beyond initial instruction alignment.

We will incorporate these results and corresponding analyses into the revised paper.

Response to Weakness 2

Thank you for this observation. The identical EXAMS-V scores arise from the benchmark's image-based question format and the response behavior of models without visual input support. We clarify the evaluation setup and the resulting predictions below.

(1) Evaluation Setup and Modality Limitation.

In EXAMS-V, both questions and answer options are provided as images. We intentionally preserve the original image-based evaluation setting rather than converting screenshots into text via OCR, as OCR introduces additional recognition errors and confounding factors. Consequently, the four text-only educational models, which do not support image inputs, receive only the general answering instruction without the actual question content. This reflects their input-modality limitations under the original benchmark setting.

(2) Explanation of Identical Scores.

We had identified and verified the cause of these identical scores during evaluation. Specifically, MuduoLLM, EduChat-R1-Qwen3-32B, and EduChat-SFT-Qwen2.5-7B consistently output A for every question, with 0% strict parsing failure. Confucius3-Math instead generates lengthy reasoning about the missing question content and its inability to answer, resulting in 100% strict parsing failure. Our fallback parser nevertheless extracts A from the opening word Okay in its response. Thus, all four models ultimately receive the same prediction (A) for every instance, explaining their identical scores.

We will clarify these evaluation conditions and parsing behaviors in the revised appendix, and explicitly distinguish modality-limited results from evaluations in which models receive the complete question content.

AuthorsOpenReview ↗

Authors' Response [3/n]

Response to Weakness 3

Thank you for these valuable suggestions. We provide additional details on data separation and conduct further overlap analysis across the relevant benchmarks.

(1) Data Splits and Overlap Analysis.

  • K12-Bench: Among 23,638 evaluation instances, only 311 (1.32%) share assessed knowledge-graph triples with training data. Character-level similarity screening (≥0.9) identifies 364 (1.54%) near-duplicate candidates, of which 5.77% are confirmed as substantive overlaps through manual verification.
  • MathFish: Training uses the official train split and evaluation uses the official dev split, with zero shared problem IDs. Similarity screening (≥0.9) identifies 141 near-duplicate candidates.
  • TutorBench: All 3,492 related training examples originate from TutorChat. Among 1,473 evaluation instances, we find zero direct matches in text, images, or source records, and no near-duplicates with similarity ≥0.9.
  • LongTutor: Portions of LongTutor-Gold and LongTutor-Silver were used for training, and the corresponding instances were excluded from evaluation. Further verification confirmed zero overlapping student IDs or learning trajectories between the training and remaining evaluation data. LongTutor-Synthesis was separately generated from external problem sources (e.g., GSM8K).

(2) Decontamination and Generalization.

To further verify the robustness of our findings, we evaluate model performance after excluding potentially overlapping instances from K12-Bench and MathFish.

Table 1. Evaluation results before and after excluding potentially overlapping test instances (%)

ModelK12-Bench EM (Original / Clean)K12-Bench F1 (Original / Clean)MathFish Acc. (Original / Clean)
OmniEdu-4B54.25% / 54.01%71.75% / 71.58%83.19% / 83.21%
OmniEdu-9B55.46% / 54.92%73.68% / 72.89%83.70% / 83.23%
OmniEdu-27B63.12% / 63.23%76.69% / 75.96%85.89% / 85.31%

We will also clarify the data splits, overlap-detection procedures, and synthetic data provenance in the revised appendix, providing stronger evidence that the observed improvements are not driven by training–evaluation overlap.

Response to Weakness 4

Thank you for this valuable suggestion. We provide the following clarifications regarding performance variations and the scope of our conclusions.

(1) Performance Across Model Scales.

While the 27B model shows a modest point improvement on TutorBench, the 4B and 9B models improve by 1.15 and 2.78 points, respectively. More substantial gains appear on LongTutor. These results demonstrate improvements across multiple model scales and pedagogical dimensions.

(2) Diagnostic Prediction Behavior.

For the 27B model, TutorBench UC3-Text improves by 6.66 points, while UC1-Text decreases by 4.01 points.

On LongTutor diagnosis, Accuracy increases from 51.78% to 54.04%, whereas Macro-F1 decreases from 38.41% to 30.58%. Specifically, fine-tuning shifts predictions from Procedural Error toward Conceptual Gap and Transfer Deficit. This change in class-wise precision and recall helps explain why overall Accuracy improves while Macro-F1 declines. Meanwhile, the 27B model achieves substantial improvements in LongTutor evidence use (36.80% to 78.20%) and teaching quality (2.74 to 3.02), demonstrating gains in other important pedagogical dimensions.

(3) Evaluation Scope and Future Work.

We select a comprehensive suite of educational benchmarks to evaluate multiple dimensions of model capabilities. However, we agree that improvements on these evaluations do not necessarily translate into better student learning outcomes in real classrooms, and will carefully scope our conclusions to the educational capabilities measured by our benchmarks.

Since releasing our models, we have also received encouraging qualitative feedback from practicing teachers in the open-source community regarding their educational usefulness. Moving forward, we aim to conduct classroom-based studies with real students and teachers to further validate their practical educational impact.

Response to Question 1~4

We have discussed these in Weakness 1~4.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.