acceptodds
Under review as a conference paper at ICLR 2027

SIMPLE: Verifiable Student-Improvement Feedback for Pretraining-Time Learning by Teaching

Abstract

Learning-by-Teaching (LbT) requires feedback on whether a teacher helps a student, yet raw pretraining text provides no such feedback. Neither token-level proxy gaps nor hint validity alone can provide this feedback signal, while raw student improvement can be inflated by answer leakage. We introduce Student Improvement Mode of Pseudo Learning by Teaching (SIMPLE), a framework that constructs source-grounded teaching events and derives teacher-training feedback from answer-masked likelihood gains in frozen students. SIMPLE measures teaching utility as answer-masked gains in frozen students' answer log-likelihood, and grounding and anti-hacking checks decide which hints may receive credit at all. Coverage correction assigns zero utility to events without a valid candidate, so the metric jointly accounts for valid-hint availability and the usefulness of selected hints. The teacher learns through supervised hint imitation followed by reward-filtered preference refinement that prioritizes more useful hints over valid but less useful alternatives. Across eight benchmark-derived evaluations and two small-model backbones, SIMPLE attains the highest coverage-corrected teaching utility among the evaluated methods.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Learning by Teaching, LLM Pretraining, student improvement, verifiable feedback

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_k7v7OpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper trains a hint-generating teacher using a utility signal defined by the change in a frozen student’s likelihood of the correct answer. Candidate hints are screened for validity, scored for student utility, and used to construct preference data for teacher refinement. The evaluation reports improvements across eight data sources and two main student models, together with selection analyses and exploratory transfer experiments. The target is immediate assistance to a frozen student, rather than persistent learning or student parameter updates. The results support gains for this operational objective.

Critical strengths

SIMPLE makes a useful distinction between hint validity and hint utility. Grounding and anti-hacking checks restrict which candidates receive credit, while student-conditioned likelihood gains distinguish among valid hints. Coverage correction also accounts for failures to produce valid candidates. Together, these choices provide a concrete and reusable training signal for learning by teaching.

The reported gains span eight data sources and both main student models, supporting the practical value of the complete training procedure within its stated objective. The additional selection and exploratory transfer analyses help characterize the signal beyond a single aggregate result. This empirical breadth, combined with the operational clarity of the objective, supports acceptance despite the remaining attribution and generalization questions.

Critical weaknesses

The computational cost could be documented more fully. The paper reports training time, but candidate generation and student-based scoring also contribute to the overall cost. A clearer breakdown would help readers understand the practical requirements of applying SIMPLE and compare it with the simpler training baselines. This is primarily a reporting limitation rather than a concern about the observed gains.

The transfer analyses provide useful additional evidence, although their relationship to the final system could be clearer. Some experiments concern hint selection or a pre-refinement checkpoint, while the main results concern the final SIMPLE teacher. Explicitly separating these settings would make the scope of the cross-student findings easier to interpret.

Questions for authors

Could the authors provide an approximate breakdown of the computation spent on candidate generation, student-based scoring, and teacher training? It would also be helpful to indicate which stage accounts for most of the additional cost relative to the baselines.

Could the authors clarify which cross-student results apply to the final SIMPLE teacher and which apply to the pre-refinement checkpoint or selection procedure? If the final teacher has been evaluated on additional students, a brief summary would help readers understand its transferability.

4 comments
AuthorsOpenReview ↗

Response to Reviewer k7v7

We thank the reviewer for the positive assessment of SIMPLE's operational objective and empirical results, and for identifying two areas where the reporting can be clearer. Below, we provide an approximate cost breakdown for the Qwen SIMPLE training pipeline, including data preparation, and clarify the scope of the cross-student evaluations.

W1/Q1. Computational cost.

Thank you for pointing out this reporting gap. We profiled candidate generation and student scoring separately and combined these estimates with the recorded teacher-training times. The approximate breakdown is:

StageGPU-hours
Candidate generation26.9
Student scoring and validity checks17.6
Teacher training, including Hint-SFT and refinement4.0
Other data-preparation overhead3.3
Total51.9

The total is computed from unrounded values, so it differs slightly from the sum of the displayed entries. Data-preparation costs are estimated by profiling 32 examples per collection stage and extrapolating to the corresponding 5,000-example collections. They are not exact historical timings. The 4.0 GPU-hours correspond to 60.2 minutes of teacher training on four NVIDIA RTX 3090 GPUs, comprising 32.8 minutes for Hint-SFT and 27.4 minutes for refinement.

The initial data preparation is shared with Hint-SFT. Relative to this baseline, the additional refinement-data collection costs approximately 15.1 GPU-hours, compared with 1.8 GPU-hours for refinement training. Data collection is therefore the main source of SIMPLE's additional cost. We will include this breakdown, the hardware configuration, and the estimation procedure in the revision.

W2/Q2. Scope of the cross-student results.

Thank you for highlighting this ambiguity. The labels "pre-refinement SimPO" and "pre-RFT SimPO" were incorrectly retained in the manuscript. The cross-student analyses in Figures 7-8 and Table 5 use the final Qwen3-1.7B SIMPLE checkpoint reported in Table 1, trained through Hint-SFT followed by reward-filtered preference refinement. This checkpoint is distinct from the SimPO baseline in Table 1. The exploratory two-student refinement checkpoint in Appendix L is a separate experiment. We will correct the legacy labels consistently and explicitly identify the teacher checkpoint, selection student(s), and evaluation student for each analysis.

For example, on 300 SQuAD events, the final teacher achieves coverage-corrected masked utility of 0.368 for Qwen3-0.6B, with Qwen3-1.7B selecting the hints. Additional accuracy evaluations using the same teacher and student roles show improvements of 0.20-1.01 percentage points over the no-hint condition across SciQ, ARC-Easy, ARC-Challenge, RACE-Middle, and RACE-High.

We will also distinguish exclusion from hint selection from exclusion from training feedback. Qwen3-0.6B contributed feedback to initial Hint-SFT data construction, although it did not contribute to subsequent refinement feedback or hint selection in these evaluations. These Qwen3-0.6B results therefore support cross-student usefulness, with the qualification that this student was not unseen throughout teacher training.

AuthorsOpenReview ↗

Follow-up on Our Rebuttal Responses

Dear Reviewer k7v7,

I hope this message finds you well. As the discussion period is nearing its end with less than two days remaining, I wanted to ensure we have addressed all your concerns satisfactorily. If there are any additional points or feedback you’d like us to consider, please let us know. Your insights are invaluable to us, and we’re eager to address any remaining issues to improve our work.

Thank you for your time and effort in reviewing our paper.

Thank you for the response, which addressed most of my concerns. I believe this is a good paper, so I will maintain my positive assessment.

AuthorsOpenReview ↗

Thank You for Your Constructive Feedback

Dear Reviewer k7v7,

Thank you for your careful review and continued positive assessment of our work. Your questions helped us provide a more complete account of the computational costs and clarify the checkpoints and student roles in the cross-student evaluations.

We appreciate your constructive guidance and will incorporate these clarifications into the revised manuscript to make the practical requirements and scope of our findings clearer.

Reviewer_4kxYOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how language models can be trained to generate more useful hints by measuring their effects on frozen student models. The authors propose SIMPLE, which constructs source-grounded teaching events, measures hint utility using answer-masked student likelihood improvements, and incorporates grounding, anti-leakage, and coverage constraints.

The teacher is first optimized through supervised imitation of selected hints and subsequently refined using utility-filtered preference pairs. Experiments with two small-model backbones demonstrate consistent improvements over several training baselines across eight evaluation sources. Additional analyses examine selection strategies, reward hacking, candidate-search budgets, and transfer to held-out students. The focus is on log-likelihood improvement which need not translate into increased answer accuracy.

Critical strengths
  1. A practically motivated formulation of student-feedback optimization.

The paper identifies an important distinction between whether a hint is valid and whether it actually helps a student. This is relevant to synthetic-data generation, teacher–student optimization, and training small language models.

  1. Consistent results supported by targeted diagnostic experiments.

The paper reports improvements over Hint-SFT on all eight evaluation sources for both Qwen3-1.7B and Gemma-3-1B-IT. The shared-candidate-pool experiments strengthen the analysis of selection rules by controlling for differences in the generated candidates.

The reward-hacking ablation further demonstrates that masking and hard-pass validity filtering address distinct failure modes. Accounting for events without usable hints is also important when comparing methods with different coverage.

The held-out-student analyses, while not conclusive for the final training method, provide encouraging preliminary evidence that selected hints can have positive utility beyond the student used for selection.

Critical weaknesses
  1. The demonstrated teaching gains are predominantly likelihood-based.

The main evidence supports improved reference-answer likelihood under a defined hint-selection protocol. The authors appropriately acknowledge that these gains do not consistently translate to higher answer accuracy.

I would therefore interpret the contribution as a method for optimizing immediate student-conditioned hint utility, rather than evidence that the students acquire durable new capabilities.

This limitation does not invalidate the proposed method, but the claims should remain aligned with the actual experimental evidence.

  1. The qualitative usefulness and validity of generated hints require stronger independent evidence.

Some hints in Appendix K appear largely templated and consist of source keywords rather than clear explanations or meaningful guidance. It is difficult to assess their pedagogical value from the qualitative examples alone.

Furthermore, the zero-leakage findings are based on the recorded rule-based checks used by the validity mechanism. These results establish consistency with those checks but do not independently rule out semantic leakage or other forms of reward exploitation.

An evaluation using independent human judgments or a separate semantic verifier would strengthen the evidence.

Questions for authors

Question 1 : Have the authors conducted any blinded evaluation of whether SIMPLE-generated hints are more informative or useful than Hint-SFT hints, independently of the student-likelihood reward?

Question 2 : Can the authors provide evidence that the validity constraints detect indirect or paraphrased answer leakage, beyond the rule-based checks currently reported?

Question 3 : Do the authors intend SIMPLE primarily as a method for optimizing immediate hint utility, or as a mechanism expected to improve downstream student learning after additional training? Clarifying this distinction would help align the conclusions with the evaluation.

3 comments
AuthorsOpenReview ↗

Response to Reviewer 4kxY

Thank you for the constructive review. We provide additional task-based evidence of immediate hint usefulness, examine the scope of the validity checks, and clarify the intended learning setting.

W1 / Q1. Answer accuracy and method-blinded assessment of hint usefulness.

We added a method-blinded, task-based evaluation to assess usefulness beyond the likelihood-gain reward. We compared No hint, Hint-SFT, and SIMPLE on common original multiple-choice events: SciQ, RACE-Middle, RACE-High, ARC-Easy, and ARC-Challenge.

The answering model received the question and, when available, an answer-masked hint. During final answer prediction, the answering model was not given method identities, correct-option labels, or reward values. Hint selection and answer masking followed the original evaluation procedure. Candidate answer texts were scored identically in randomized order, without marking the correct answer. Ties were resolved by that order, and correctness was computed offline against separately stored labels. Calls across conditions were randomized; identical inputs shared a cached prediction. Events without a usable hint received exactly the No hint input.

For each teacher backbone, we evaluated both the selection student and a frozen Qwen3-0.6B student excluded from hint selection. Both received the same selected hints. The prediction was the candidate answer with the highest length-normalized token log probability. Thus, the reported values are unweighted mean accuracy (%) across the five datasets.

Table R1 Mean Answer Accuracy (%) of Selection and Held-Out Students.

Teacher backboneStudentNo hintHint-SFTSIMPLE
Qwen3-1.7BSelection37.1738.6039.14
Qwen3-1.7BHeld-out34.1033.4934.66
Gemma-3-1B-ITSelection34.9939.5841.99
Gemma-3-1B-ITHeld-out34.1032.9734.72

SIMPLE achieves higher mean answer accuracy than both Hint-SFT and No hint for both teacher backbones and both student roles. Relative to Hint-SFT, the gains are 0.54 and 2.41 percentage points for the selection students, and 1.17 and 1.75 points for the held-out student, for Qwen and Gemma teachers, respectively. The held-out gains over No hint are modest (0.56–0.62 points), providing evidence of usefulness beyond the student used to select hints.

These results provide method-blinded, task-based evidence of immediate hint usefulness. Here, “held-out” refers to exclusion from hint selection: Qwen3-0.6B contributed feedback to initial Hint-SFT data construction but not subsequent refinement.

W2 / Q2. Answer-equivalent disclosures missed by exact/fuzzy matching.

We conducted a paired evaluation showing that the complete validity constraints can reject explicitly announced answer-equivalent disclosures missed by exact/fuzzy matching.

We used 60 distinct QA events with real Qwen3-1.7B hints that passed the current complete gate: 30 from SIMPLE and 30 from Hint-SFT. Keeping the original source cues, we constructed direct and answer-equivalent disclosures. The equivalent expressions covered number forms, chemical notation, contextual synonyms, and spacing variants—for example, NaCl instead of sodium chloride. The gate implementation and thresholds remained fixed.

Table R2 Detection of Explicit Answer Disclosure.

Constructed conditionExact/fuzzy component: flaggedComplete gate: rejected
Direct answer disclosure120/120120/120
Answer-equivalent disclosure3/6060/60

The direct-disclosure row includes two wordings per event; the answer-equivalent row covers variants that explicitly announced the response. The complete gate blocked all 60 answer-equivalent disclosures, including 57 not flagged by the exact/fuzzy component. These additional rejections were attributable to the existing answer-disclosure phrase constraints, providing experimentally demonstrated protection beyond literal answer matching.

Q3. Intended scope: immediate hint utility or downstream student learning?

SIMPLE is primarily intended to be a method for optimizing immediate hint utility. It improves teachers’ ability to provide useful inference-time hints to frozen students. Only the teacher’s parameters are updated; student feedback measures the immediate change in reference-answer likelihood when a hint is provided.

The additional accuracy results likewise evaluate immediate assistance, rather than learning retained through student parameter updates. Whether SIMPLE-generated hints can serve as effective training materials for downstream student learning remains an open empirical question. We will make this distinction explicit in the introduction and conclusions.

AuthorsOpenReview ↗

Follow-up on Our Rebuttal Responses

Dear Reviewer 4kxY,

I hope this message finds you well. As the discussion period is nearing its end with less than two days remaining, I wanted to ensure we have addressed all your concerns satisfactorily. If there are any additional points or feedback you’d like us to consider, please let us know. Your insights are invaluable to us, and we’re eager to address any remaining issues to improve our work.

Thank you for your time and effort in reviewing our paper.

Reviewer_JK6zOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors study how to train a teacher language model to generate useful hints for frozen students. They propose SIMPLE, which measures hint usefulness through answer-masked changes in the student's reference-answer log-likelihood. Rule-based checks determine which hints can receive credit, and events without a valid hint contribute zero to the reported utility. The teacher is trained through hint imitation followed by preference refinement. Experiments on two small backbones and eight QA sources show higher coverage-corrected utility than the evaluated baselines.

Critical strengths
  1. The paper studies an interesting problem in learning from student feedback. Separating hint validity, student utility, and coverage makes the evaluation easier to interpret and exposes different ways the reported improvement can be inflated.

  2. The evaluation on seven sources outside the training source provides useful evidence of transfer within QA.

Critical weaknesses
  1. My main concern is whether the gain comes from the preference construction or from extra training. Final SIMPLE adds 600 refinement steps on top of Hint-SFT (Table 2), but there is no Hint-SFT baseline trained with the same compute. LM-Matched does not address this, since it uses neither the hint data nor the full budget. Since the main claim rests on the gain over Hint-SFT, a compute-matched Hint-SFT baseline is needed.

  2. It is not clear what this objective provides in practice, for either the student or the teacher. For the student, Table 1 reports likelihood-based utility, and it is unclear whether these gains lead to higher accuracy. The same student also selects and scores the hints, which favors methods trained on its feedback, and the accuracy and held-out results (Figure 9, Section 3.4) do not use the Table 1 teachers. For the teacher, the learning-by-teaching framing suggests that the teacher itself should improve, but only validation perplexity is reported. Please report student accuracy before and after the hint for each Table 1 teacher on the multiple-choice sources (SciQ, RACE, ARC), for both the selection student and a held-out student. Please also show whether training to teach improves the teacher on its own tasks.

Questions for authors
  • Clarifications:
  1. How was the pre-refinement SimPO teacher trained? Most analyses (Figures 4 and 6 to 9) use this teacher rather than final SIMPLE, so it is unclear whether these findings hold for the proposed method.
  2. Faithfulness is only described in one sentence in Appendix E.1, and the stricter threshold used by the faithful selection variants is not reported. Please define both in the main text.
  • Requests that could change my assessment:
  1. The related work discusses Montessori-Instruct and HiLL, which also use student feedback to train a teacher or a hinter. Please compare with any method that applies to this setting, or explain concretely what prevents a direct comparison.
  2. The students are kept frozen, but they are small models that could be fine-tuned directly. The teacher also sees the source, which could instead be given to the student. In what practical setting is improving the teacher's hints preferable to these options? A discussion, or a comparison with direct student fine-tuning on the same data, would make the motivation clearer.
  • Suggestions:
  1. Training uses only SQuAD, and the hints in Table 12 are mostly lists of source keywords. It would help to discuss how the approach extends beyond the evaluated QA settings, for example to math, where a hint could be a direction, a step, or a concept.
  2. The baselines and selection rules are used before they are defined, so a short description before Section 3.2 would help. The faithful selection variants could also be shown more compactly, as an analysis of the trade-off between grounding, utility, and coverage.
7 comments
AuthorsOpenReview ↗

Response to Reviewer JK6z

Thank you for your constructive suggestions. Following are point-to-point responses.

W1. Does SIMPLE outperform Hint-SFT at the same number of training steps?

We agree that LM-Matched does not isolate the effect of continued training on hint data. We therefore added a step-matched Hint-SFT baseline. For each backbone, we resumed training from the Hint-SFT checkpoint for 600 steps with the same Hint-SFT objective, matching the number of steps in SIMPLE’s preference refinement.

As shown in Table R1, the additional Hint-SFT steps changed mean coverage-corrected teaching utility from 1.396 to 1.388 for Qwen3-1.7B and from 3.814 to 3.859 for Gemma-3-1B-IT. SIMPLE achieved 1.591 and 4.307, exceeding the step-matched baseline by 0.203 and 0.448, respectively. SIMPLE also outperformed this baseline on all eight evaluation sources for both backbones. These results show that continued Hint-SFT for the same number of steps did not reproduce SIMPLE’s gains.

Table R1 Coverage-corrected teaching utility of SIMPLE, Hint-SFT, and step-matched Hint-SFT.

ModelMethodAdvQASQuADSQuAD2SciQRACE-MRACE-HARC-EARC-CAvg.
Qwen3-1.7BHint-SFT1.3501.6012.1773.8940.3230.0021.0010.8181.396
Qwen3-1.7Bstep-matched Hint-SFT1.4111.5792.1083.9520.2930.0430.9530.7621.388
Qwen3-1.7BSIMPLE1.5851.9702.4384.0650.6670.1121.0510.8431.591
Gemma-3-1B-ITHint-SFT4.7113.9564.4704.3172.4001.3785.4793.8053.814
Gemma-3-1B-ITstep-matched Hint-SFT4.6074.0574.6253.8652.8241.4405.5343.9183.859
Gemma-3-1B-ITSIMPLE5.0754.6245.2094.5553.0041.5056.1344.3514.307

W2a. Student accuracy with the Table 1 teachers.

We measured student accuracy on five multiple-choice datasets with and without hints from each teacher in Table 1. Tables R2 and R3 report results for SciQ, RACE-Middle, RACE-High, ARC-Easy, and ARC-Challenge. For each dataset, we evaluated both the student that selected the hints and a held-out student that did not participate in hint selection.

Frozen student model used for accuracy evaluation

modelSelection studentHeld-out student
Qwen teachersQwen3-1.7B(Qwen/Qwen3-1.7B)Qwen3-0.6B(Qwen/Qwen3-0.6B)
Gemma teachersGemma-3-1B-IT(google/gemma-3-1b-it)Qwen3-0.6B(Qwen/Qwen3-0.6B)

The selection student participates in hint selection and answers each question using the selected hint. The held-out student answers each question using the same selected hint without participating in hint selection. Both tables use Qwen3-0.6B as the held-out student.

Compared with the corresponding no-hint baseline, SIMPLE improved student accuracy on all five datasets for both teacher backbones and both student roles. With Qwen3-1.7B teachers, the gains ranged from 1.01 to 3.25 percentage points for the selection student and from 0.20 to 1.01 percentage points for the held-out student. With Gemma-3-1B-IT teachers, the gains ranged from 1.41 to 11.66 percentage points for the selection student and from 0.10 to 1.21 percentage points for the held-out student. SIMPLE achieved the highest accuracy among the Table 1 teachers in all evaluation settings.

Table R2 Student accuracy (%) without hints and with hints from the Qwen3-1.7B teachers in Table 1.

DatasetStudentNo hintBaseLM-onlyLM-MatchedHint-SFTHint-SFT+ARSimPOSIMPLE
SciQSelection47.1647.4647.2647.1650.3049.4949.7050.41
SciQHeld-out46.9646.7546.9647.1646.0445.8446.0447.26
ARC-ESelection39.2039.3038.9038.9040.9040.9040.8041.20
ARC-EHeld-out34.0034.1034.1034.0034.1034.1033.8034.20
ARC-CSelection42.7142.2141.4141.9143.6243.2243.5243.72
ARC-CHeld-out36.1836.1836.1836.3836.4836.9836.4837.19
RACE-MSelection30.3331.3332.3331.8330.8331.5331.2332.53
RACE-MHeld-out28.8328.7328.7329.1326.4327.1326.8329.33
RACE-HSelection26.4327.7326.8326.5327.3327.3327.0327.83
RACE-HHeld-out24.5224.5224.1224.7224.4225.2325.0325.33

Table R3 Student accuracy (%) without hints and with hints from the Gemma-3-1B-IT teachers in Table 1.

DatasetStudentNo hintBaseLM-onlyLM-MatchedHint-SFTHint-SFT+ARSimPOSIMPLE
SciQSelection45.7445.7445.5445.7453.1447.6753.2554.26
SciQHeld-out46.9646.8646.9646.8645.7446.1546.5547.06
ARC-ESelection40.4040.2040.1040.4046.1045.8046.2049.00
ARC-EHeld-out34.0034.2034.0033.9032.3033.8032.9034.40
ARC-CSelection34.2734.0734.3734.2740.6039.8040.3045.93
ARC-CHeld-out36.1836.1835.9836.2834.8737.2935.2837.39
RACE-MSelection30.2329.8330.2330.0333.1331.6333.1335.04
RACE-MHeld-out28.8329.0329.3328.7327.7328.1329.2329.43
RACE-HSelection24.3224.1224.3224.1224.9225.6325.4325.73
RACE-HHeld-out24.5224.3224.6224.7224.2224.2224.5225.33

W2b. Does training to teach improve the teacher’s own task performance?

We agree that the learning-by-teaching framing should make the scope of teacher improvement explicit. We have not evaluated the teacher’s standalone QA performance, and our current results therefore do not establish an improvement in its own QA accuracy. Table 2 reports a decrease in Qwen3-1.7B’s language-model validation perplexity from 12.98 with Hint-SFT to 12.83 after SIMPLE refinement. This result concerns language modeling and does not demonstrate improved standalone QA or general reasoning ability. Our claim about teacher improvement is limited to its ability to provide useful, source-grounded hints to frozen students, assessed through coverage-corrected teaching utility under the reported evaluation protocol. We will revise the introduction and related statements to make this scope explicit and acknowledge the absence of standalone teacher QA evaluation as a limitation. Whether improved teaching utility transfers to the teacher’s own QA performance remains an open question for future evaluation.

Q1. Clarification of the “pre-refinement SimPO” checkpoint.

Thank you for pointing out this ambiguity. The labels “pre-refinement SimPO” and “pre-RFT SimPO” were incorrectly retained in the manuscript; both refer to the final Qwen3-1.7B SIMPLE checkpoint reported in Table 1. This checkpoint was trained on SQuAD through 1,000 steps of Hint-SFT followed by 600 steps of reward-filtered, SimPO-style preference refinement. Refinement prioritizes valid but lower-utility negatives while retaining the language-modeling objective and a small hint-imitation anchor, as described in Section 2.3, Table 2, and Appendix E.2. This checkpoint is distinct from the SimPO baseline in Table 1. Accordingly, the analyses in Figures 4 and 6–9 evaluate final SIMPLE, including its preference-refinement stage. We will replace the legacy labels throughout the main text, figure captions, and appendix, explicitly distinguish SIMPLE from the SimPO baseline, and clarify the candidate pools and student-selection settings used in these analyses.

Q2. Faithfulness definition and thresholds.

Faithfulness is the larger of two token-set Jaccard overlaps: the hint with the source and the hint with the evidence, after removing the reference answer from each text. Ordinary hard-pass selection uses 0.10 and the faithful variants use 0.30. The cascade falls back to ordinary reward-best if no candidate meets the stricter threshold. We will add this definition and both thresholds to the main text.

Q3. Comparison with Montessori-Instruct and HiLL.

We added a comparison with a QA adaptation of Montessori-Instruct that preserves its central feedback mechanism. For each candidate hint, we temporarily update the student on the corresponding hinted QA example, measure the resulting change in loss on a separate reference set, and restore the student and optimizer before scoring the next candidate. The adapted baseline and SIMPLE use the same Qwen3-1.7B backbone, Hint-SFT initialization, and SQuAD training source. We evaluate both on the same examples using 12 hint candidates per question, the same frozen student, and identical selection and scoring rules.

DatasetHint-SFTAdapted Montessori-InstructSIMPLE
SciQ3.8943.9014.065
RACE-M0.3230.3390.667
RACE-H0.0020.0030.112
ARC-E1.0010.9951.051
ARC-C0.8180.7760.843

The metric is coverage-corrected answer-masked teaching utility. The Montessori-Instruct column averages three teacher DPO runs that share one pool of student influence scores. SIMPLE exceeds Adapted Montessori-Instruct on each of the five datasets.

HiLL jointly updates a reasoner and a hinter using failed reasoning trajectories and verifier-scored retries. Applying that training loop directly would change the student during training, whereas our setting fixes the student and uses its answer-likelihood response to source-grounded hints as feedback. Freezing HiLL’s reasoner would remove its joint-update mechanism. We will add the Montessori-Instruct comparison and explain this difference in the revised manuscript.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response: new experiments and revisions to the paper

We thank all three reviewers for their careful and constructive feedback. We are encouraged that the reviewers found the problem interesting and practically relevant (JK6z, 4kxY), and recognized the effectiveness of our SIMPLE and the breadth of its empirical support (4kxY, k7v7).

The table summarizes the additional experiments and clarifications addressing the main concerns. Full details are in our individual replies. We are working on a revised PDF of the paper that will incorporate the rebuttal results.

Concern (reviewer)New experiment or clarificationResult
Gains from extra training? (JK6z)Continued Hint-SFT for the same 600 additional steps as SIMPLE’s refinement.SIMPLE remains better on all eight sources for both backbones. Additional Hint-SFT steps do not reproduce its gains.
Answer accuracy and blinded usefulness (JK6z, 4kxY)Five-dataset accuracy evaluation. Method-blinded comparison of SIMPLE, Hint-SFT, and No hint.SIMPLE ranks first among Table 1 teachers in all 20 dataset–backbone–student-role settings, and exceeds No hint.
Usefulness beyond the selection student (JK6z, 4kxY, k7v7)Evaluated the same selected hints with Qwen3-0.6B, excluded from hint selection.Mean accuracy improves by 1.17–1.75 percentage points over Hint-SFT and 0.56–0.62 points over No hint.
Do analyses evaluate final SIMPLE? (JK6z, k7v7)Clarified the incorrect legacy “pre-refinement SimPO” labels.Figures 4 and 6–9 already use final SIMPLE, including preference refinement. We will correct the labels and distinguish the separate checkpoints.
Related student-feedback methods (JK6z)Added QA-adapted Montessori-Instruct under a shared evaluation setup. Clarified HiLL’s training setting.SIMPLE achieves higher utility on all five datasets. HiLL’s joint updates differ from our frozen-student setting.
Leakage missed by literal matching (4kxY)Fixed-gate stress test of explicitly announced answer-equivalent disclosures.Rejected 60/60 cases, including 57 missed by exact/fuzzy matching. In these tests, the full checks caught answer leaks that answer matching alone missed.
Full computational cost (k7v7)Tested whether the existing checks catch hints that directly reveal the answer in different words or forms.Estimated total: 51.9 GPU-hours. Additional data collection costs 15.1 GPU-hours, versus 1.8 for refinement training, identifying the main additional cost.
Learning scope and practical motivation (JK6z, 4kxY)Clarified inference-time assistance to frozen students with limited input budgets.Evidence supports immediate hint usefulness within QA. Durable student learning and improved standalone teacher QA remain open questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.