acceptodds
Under review as a conference paper at ICLR 2027

The Assistance Dilemma: Learning to Teach via Multi-Turn Reinforcement Learning

Abstract

Large language models (LLMs) trained to answer questions are natively poor at teaching, and Reinforcement Learning (RL) training with a simulated student is a promising approach to improve the pedagogical quality of LLMs. Existing RL-trained tutors reward the student's success on the tutored problem with the tutor's words still in context; thus, the reward is easiest to raise by telling students answers, and a tuned penalty is needed to reduce telling. This work studies this simulation-based training approach for LLM tutors and how the learning sciences-inspired approach can let LLMs learn to teach more effectively. We introduce a masked near-transfer post-test: the student is tested on an unseen variant of the tutored problem with the tutor's utterances masked, so the reward can rise only through the student's own turns. This prevents cognitive offloading by the student and allows the continuous penalty to be replaced by two binary reward gates (factual correctness of tutor response, no solution handover). A leave-one-out ablation shows that the learning-gain reward on its own does not separate teaching from telling, while binary reward gates suppress solution handover, the near-transfer post-test improves out-of-domain transfer, and removing tutor utterance masking coincides with a rise in thinking tokens. Using these reward designs, we develop Eduardo, a recipe for training LLM tutors, and use it to train 4B, 9B, 14B and 27B models from two distinct LLM architectures. Our post-trained Eduardo-27B model matches Gemini-3.1-Pro on MathTutorBench and Claude-Opus-4.8 on TutorMoments at 2.4-6.2x fewer thinking tokens (a requirement for interactive tutoring), demonstrates an increase in push for justification teacher moves without it being named in the reward, and trains out support fading (e.g., assigning independent work), whose payoff lies beyond a single-problem dialog episode. We open-source our training environment, an 8,671-problem near-transfer dataset, and trained models for further development.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLMs, tutoring, reinforcement learning, alignment, multi-turn RL, AI for education

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_VaXyOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Eduardo trains language-model tutors with multi-turn RL against a frozen student. Its learning signal compares a measured cold solve rate with performance on a near-transfer problem after tutoring, retaining only the student's turns in the post-test context. Separate factual-correctness and no-handover gates, plus an efficiency discount, shape the trajectory reward. The paper constructs 8,671 source/variant pairs and trains 4B–27B tutors across two architectural families. Evaluation combines held-out simulated gains with MathDial/Eedi responses, MathTutorBench, and TutorMoments using different evaluation models. The results show useful pedagogical-score improvements and lower thinking-token use, while the 4B leave-one-out study suggests that the gates, transfer test, and masking influence different behaviors. The paper also reports limitations, including leakage remaining substantial for smaller tutors and loss of support-fading behavior.

Critical strengths
  1. A concrete, inspectable objective addressing an important reward-design failure. Sections 3.3–3.6 replace same-problem/full-dialog evaluation with masked near transfer and explicitly separate gain, behavioral gates, and efficiency. The no-gates ablation is especially informative: leakage rises from 31% to 61% while measured gain stays similar. This demonstrates why optimizing the student proxy alone is not enough and makes the paper valuable even where the proxy fails.
  2. Substantial external evaluation and candid negative findings. The work spans model sizes/families and uses evaluation instruments distinct from the training judge, including an independent simulated student in TutorMoments. It reports cases without improvement, base-model truncation issues, continued leakage at small sizes, and the decline of support fading. The strong MathDial results and external rubric improvements support a useful open-model training recipe, without needing to interpret every simulator score as human learning.
Critical weaknesses
  1. Masking tutor turns does not guarantee that telling cannot improve the reward. Sections 3.2–3.4 and the abstract describe student-notes-only evaluation as preventing cognitive offloading. However, a student can repeat a tutor-provided method or derivation in its own turns; that material survives the mask and can help on a near-transfer variant. Masking establishes a channel restriction, not that surviving content was independently constructed. The separate leakage gate is therefore doing essential work, and similar average gains with/without the gates do not identify the causal effect of telling. The paper itself notes copying-related behavior in Section 5.2. The empirical objective remains useful, but “prevents” and “telling no longer raises the gain” should be weakened unless supported by an explicit mediation or transcript audit.
  2. Mechanistic conclusions are stronger than the ablation design supports. The leave-one-out study uses one seed, one size, and one checkpoint per arm, which the limitations acknowledge. The no-masking arm also has substantial cap overruns/truncation, so its longer thinking and lower scores cannot cleanly identify a pedagogical mechanism. Both gates are removed together, and every arm shares a token-efficiency term. Moreover, the fixed gate reward is −0.2 (Appendix B), while valid negative gain can in principle be lower; this is a finite penalty, not a hard feasibility constraint. These details do not invalidate the recipe but qualify the claimed necessity and independence of its components. Existing bootstrap intervals over problems should not be interpreted as variation across training runs.
Questions for authors
  1. Clarification: How should the no-offloading claim be interpreted when students repeat tutor-supplied reasoning in their own turns? Do existing transcripts/audits distinguish reproduction from independent construction?
  2. Clarification: Was each ablation checkpoint selected by the same validation rule, and how much of the no-masking deficit remains in completed, non-overrun dialogs according to existing logs?
  3. Clarification: Since a gate failure receives −0.2 whereas a gate-passing negative gain can be smaller, is the intended mechanism a soft penalty rather than a strict constraint? What existing reward statistics characterize this case?
  4. Existing-evidence clarification: Were the gates calibrated against any human annotations, and are gate disagreement/error rates available? Additional seeds would strengthen future claims but are not demanded during rebuttal.
3 comments

Thank you for your review. We agree with your precision points and will update our phrasing accordingly.

1. Channel Masking vs. Anti-Dictation Gates (Q1 & W1)

  • Revising wording: You are right that masking tutor turns imposes a channel restriction (removing direct access to tutor utterances), whereas discouraging tutor-supplied reasoning from being echoed into student turns relies on the binary gates. We have revised the Abstract and will revise Sections 3.2–3.5 to replace "prevents cognitive offloading" and "telling no longer raises the gain" with wording that accurately reflects this joint mechanism.
  • Auditing dictation: We anticipated "answer-echo laundering" early in development (App. A), so Iquality\mathbb{I}_{\text{quality}} rejects dictated speech (App. C.2) and Ileakage\mathbb{I}_{\text{leakage}} rejects stating methods/formulas before a student attempt (App. C.1). With an independent Gemini-3.1-Pro judge, pedagogy scores reach 4.50/5 on the 494 held-out dialogs (Table 1). Additionally, we ran a dictation audit on 200 TutorMoments dialogs using a Gemini-3.1-Pro judge, which flagged 2 of 200 dialogs, with 0 cases of solution dictation upon inspection (both read problem text or a basic definition aloud: "Can you say 34.04 out loud?"; "What number is two steps left from zero?").

2. Ablation Checkpoint Selection & w/o Masking Overruns (Q2 & W2)

  • Uniform selection: Yes, all four 4B ablation arms (Eduardo-4B, w/o Gates, w/o Masking, w/o Near-Transfer) were selected by the same rule under the identical protocol in Table 4 and Appendix A.
  • Isolating from multi-turn truncation: Conditioning on non-truncated environment rollouts would introduce survivorship bias toward easier problems where w/o Masking stays under budget. Instead note that MathDial (n=500n=500) and Eedi (n=500n=500) evaluate all items at a single mid-conversation turn, where multi-turn sequence truncation cannot accumulate. Even there, w/o Masking trails full Eduardo-4B substantially: MathDial RM 7.92→6.487.92 \to 6.48 (Win 78.078.0\\% \to 65.0\\%) and Eedi RM 8.83→7.488.83 \to 7.48 (Win 72.472.4\\% \to 59.0\\%), while thinking tokens increase 3.5–5.7x.

3. Fixed Penalty vs. Hard Constraint (Q3)

  • You are right: rpenalty=−0.2r_{\text{penalty}} = -0.2 is a fixed penalty, not a strict feasibility constraint, and we will correct the wording in Sections 3.2-3.5.
  • Prior statistics (Table 5): Because negative gains are unnormalized (Δ=ppost(xtgt)−ppre(xtgt)\Delta = p_{\text{post}}(x_{\text{tgt}}) - p_{\text{pre}}(x_{\text{tgt}})), a gate-passing dialog only scores below −0.2-0.2 if tutoring drops the target solve rate by >0.20>0.20. In our filtered training set (Table 5, n=8238n=8 238), the median target prior is ppre(xtgt)=0.188p_{\text{pre}}(x_{\text{tgt}}) = 0.188 (3/163/16), so on at least half of training items, Δ(τ)<−0.20\Delta(\tau) < -0.20 cannot occur even if ppostp_{\text{post}} drops to 00. At the mean target prior (0.2800.280), it requires dropping below 0.080.08. We kept the penalty flat so identical violations are not punished more harshly on easy (high pprep_{\text{pre}} solution rate) items and to limit sensitivity to occasional judge false positives.

4. Gate Design & Human Alignment (Q4)

  • Rather than calibrating on a static crowd-annotated dataset, our gate prompts operationalize Wood’s (1994) funneling/focusing boundary (App. C), refined via qualitative inspection of reward-hacking transcripts (App. A) and stabilized during RL with 3-vote majority sampling per criterion (Qwen3.6-27B).
  • To check that behavior generalizes beyond Qwen3.6-27B, we evaluate with an independent judge (Gemini-3.1-Pro), Ped-RM, and in our new Section 6 Teacher Study with 7 experienced STEM teachers, who judged Eduardo-27B lower on solution handover (2 vs. 5 rounds) and over-helping (2 vs. 6 rounds) than its base model, with few factual errors (1 vs. 2 rounds).

Thank you for your response. After reconsideration, I have decided to keep my score unchanged.

Thank you for engaging. Please see our global comment, which takes a step back to lay out the novel contributions (and the model ranks in the top 3 on an independent leaderboard on which the model is not trained directly, well above every open-weight model on the leaderboard).

Reviewer_ktfNOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes COMET, an online reinforcement-learning framework for dialogue tutoring that co-evolves a Tutor and a Student Simulator. The framework alternates between Tutor optimization and simulator adaptation on Tutor-induced histories, using ZPD-inspired rewards, reference-based behavioral constraints, and judge-anchored evaluation. Experiments on MathDial and CoMTA report improvements in tutoring outcomes and student-response assessment, including gains when evaluated with an independent frozen student simulator.

Critical strengths

S1. Meaningful research motivation.

The paper addresses an important challenge in AI-assisted education: training LLMs to guide students toward independent problem-solving rather than simply providing answers. The distinction between teaching and answer-giving is practically relevant and provides a clear motivation for the proposed approach.

S2. Thoughtful reward design and extensive evaluation.

The proposed masked near-transfer post-test and binary reward gates offer an interesting approach to improving pedagogical reinforcement learning. The paper also evaluates multiple model sizes and includes comparisons with existing tutoring models, strong proprietary LLMs, and ablation studies.

Critical weaknesses

W1. Limited evidence of effectiveness for real human students.

The proposed method relies on a frozen LLM to simulate students during training. However, LLMs and real students differ substantially in their reasoning processes, cognitive abilities, and learning behaviors. A teacher model optimized to improve an LLM student's performance may not necessarily be effective for human learners. Although the authors acknowledge this limitation, it remains a fundamental concern for a method intended to improve educational outcomes. Evaluations involving other LLM-based student simulators do not fully address this issue.

W2. The near-transfer evaluation may not reliably measure genuine learning.

The proposed post-test removes the teacher's responses but retains the student's previous dialogue, which may contain reusable solution steps or formulas. Furthermore, near-transfer questions are generated mainly by changing numbers or surface-level scenarios while preserving the original problem-solving structure. Consequently, improved performance may reflect the reuse of contextual information rather than genuine understanding or knowledge acquisition. Without further analysis separating these effects, the reported improvements provide limited evidence of actual learning gains.

Questions for authors

Q1. Given the substantial differences between LLM-based student simulators and real human learners, what evidence supports the transferability of the learned tutoring strategies to human students? Could the authors provide additional validation beyond LLM-based simulations?

Q2. Since the masked post-test retains student-generated dialogue, how do the authors distinguish genuine learning from the reuse of formulas or solution steps already present in the context? Could the authors evaluate performance after removing explicit solution-related information from the retained student turns?

Q3. The near-transfer questions largely preserve the original problem-solving structure. Could the authors evaluate the proposed method on more challenging transfer tasks that require applying learned concepts to problems with different structures, rather than primarily changing numerical values or surface-level descriptions?

Q4. The ablation results show similar simulated learning gains despite substantial differences in answer leakage and pedagogical quality. How can the authors establish that the proposed near-transfer reward reliably measures genuine knowledge acquisition rather than contextual information reuse? Could the authors provide additional experiments to distinguish these effects?

2 comments

Thank you for your review and questions on human transferability and how our masked near-transfer objective operates.

(Note: the first paragraph of the Paper Summary appears to have been swapped with a different submission, whereas the remaining summary and questions are relevant to our paper, so we address all of those points below.)

1. Evidence Beyond LLM Simulators (W1 & Q1)
We added a blind, paired user study with 7 experienced STEM teachers role-playing authentic classroom misconceptions (Section 6, detailed in our Global Comment). Even under zero-shot shift to Computer Science and Physics (10/11 rounds) and short 5-word median replies, we observe directional preference for Eduardo-27B over its base (8/11 rounds, +0.93+0.93 on [−3,+3][-3, +3], 95% CrI [−0.13,1.93][-0.13, 1.93], P(μ>0)=0.96P(\mu>0)=0.96), lower teacher-judged over-helping (2 vs. 6) and solution handover (2 vs. 5), qualitative alignment with Move G ("asking about your thinking"), and 3.7x fewer tokens per dialog (p=0.002p=0.002).

2. Contextual Reuse vs. Tutor Learning & Ablation Saturation (W2, Q2 & Q4)
Here it helps to separate (a) how the frozen student computes the training signal from (b) what updates in the tutor’s weights:

  • Why student turns remain in context (Q2): Because the student LLM (πS\pi_S) has frozen weights (Sec. 3.1), its state KSK_S exists only in context through the reasoning it generates during the dialog. Removing student-authored steps/formulas before the post-test would leave a frozen LLM with an empty mathematical context, reverting it to pprep_{\text{pre}}.
  • Why Δtransfer\Delta_{\text{transfer}} saturates across ablations (Q4): You point out that Δtransfer\Delta_{\text{transfer}} stays around +0.23+0.23 to +0.25+0.25 despite 6161\\% leakage in w/o Gates. We agree, and this is our central finding (Sec. 5.2): post-test gain on a frozen student alone cannot distinguish student-constructed reasoning from tutor-dictated or leaked steps echoed by the student.
  • How the gates penalize dictation and handover: Masking blocks direct reading of tutor turns, while Condition 3 penalizes indirect copying into student turns: Ileakage\mathbb{I}_{\text{leakage}} rejects stating answers, key formulas, or solution paths before a student attempt (App. C.1), and Iquality\mathbb{I}_{\text{quality}} explicitly rejects dictated speech (App. C.2). We ran two independent Gemini-3.1-Pro evaluations to check whether dictation still occurs:
    1. Held-out whole-dialog pedagogy scores (which penalize revealing intermediate steps or funneling; Table 10) reach 4.50/5 for Eduardo-27B (vs. 4.204.20 base).
    2. A dedicated dictation audit across 200 TutorMoments dialogs flagged only 2 of 200 dialogs, and manual inspection found 0 dialogs with solution dictation (both merely read problem text or a definition: "Can you say 34.04 out loud?"; "What number is two steps left from zero?").
  • Why masked near-transfer is still essential alongside the gates: Importantly, the gates alone do not produce good teaching either: in Table 2, w/o Near-Transfer and w/o Masking keep both binary gates active, yet MathDial win rate drops 22.0 pp (78.078.0\\% \to 56.0\\%) without near-transfer and also drops 13.0 pp (78.078.0\\% \to 65.0\\%) without masking (drop on Eedi is 13.413.4 pp). Thus, while we never interpret Δtransfer\Delta_{\text{transfer}} itself as human learning (Sec. 5.1), optimizing masked near-transfer jointly with anti-handover/anti-dictation gates is what pressures the tutor’s weights to elicit transferable student reasoning (doubling prompting for justification, 9.09.0\\% \to 21.8\\%), which we then evaluate on OOD benchmarks and human teachers.

3. Beyond Near-Transfer (Q3)
We did not test student transfer to structurally different problems. Here are reasons why near-transfer is the right first step and how we tested generalizability:

  • During training: Within a single-problem dialog (T≤22T \le 22 turns) on an 8B frozen student, isomorphic near-transfer preserves matched difficulty (r≥0.65r \ge 0.65, Table 5) for a tractable RL signal while discouraging verbatim answer copying (lifting MathDial win rate from 56.056.0\\% without near-transfer to 78.078.0\\%).
  • At evaluation: We test whether the trained tutor generalizes beyond isomorphic rewrites: both to unseen problem structures in MathDial, Eedi, MathTutorBench, and TutorMoments, and zero-shot to Physics and Computer Science in Section 6. Extending multi-episode training to far-transfer tasks (which could also preserve multi-problem support fading, Move H) is an important direction for future work.
AuthorsOpenReview ↗

Reminder 2

Dear reviewer, as the discussion closes soon, we would like to send you a reminder to submit any final acknowledgements.

Reviewer_pcknOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents a way to train a LLM to be a better tutor, using a frozen LLM as a student as the environment in which to perform training. The method involves a. evaluating on a held out target problem that is isomorphic to one that is tutored, but involves new numbers or a different surface story. b. masking the tutor utterances in the history at student evaluation time c. Changing the reward structure so that the model is penalized for providing incorrect information, is penalized for providing the solution, and is penalized for longer thinking traces/tokens.

The results show that they outperform several prior tutoring LLM baselines on when their model is evaluated for pedagogical performance mid turn on several datasets (mathDial and Eedi), as well as some other experiments.

Critical strengths

The results show that they outperform several prior tutoring LLM baselines on when their model is evaluated for pedagogical performance mid turn on several datasets (mathDial and Eedi), as well as some other experiments.

Critical weaknesses

The presented technical advances are limited. (c) is a simple specification of the reward, and (a) is a straightforward thing to do (evaluate on a held out problem). Masking tutor utterances in the history at student evaluation time is more novel, but the ablation Table 2 suggests that on a number of metrics removing masking yields very similar results. It is harder to assess this since standard errors are not presented.

I am also concerned that the model is being trained against a frozen LLM model that is likely not very good, and the evaluations are in terms of pedagogical patterns, not real student outcomes. While it is hard to run student studies, it would be much more compelling if these simple changes led to improvements in real student learning vs a “best” prior baseline.

Questions for authors

Please add standard errors to all tables including:

Table 1: Can you show stderr for the RM? It would help to understand if these differences are significant.

On Eedi looks like Qwen base models at 14B and 27B are similar to Eduardo-14B/27B

On the training example, Eduardo 14B seems worse than Eduardo 4B and 9B. Why?

on Table 3, for out-of-domain generalization on TutorMoments dialogs, can you also show standard errors so we can evaluate the significance of the differences?

2 comments

Thank you for your review and specific questions.

1. Evaluation Beyond LLM Simulators: Teacher Study (New Section 6)
Before running trials with human learners, we evaluated Eduardo-27B against Qwen3.8-27B in a blind, paired study with 7 experienced STEM teachers role-playing classroom misconceptions (summarized in our Global Comment). Despite zero-shot shift to Physics and Computer Science (10/11 rounds) and short 5-word median replies, results provide directional support for human transfer: teachers preferred Eduardo-27B in 8/11 rounds (+0.93+0.93 on [−3,+3][-3, +3], 95% CrI [−0.13,1.93][-0.13, 1.93], P(>0)=0.96P(>0)=0.96), judged it lower on over-helping (2 vs. 6) and solution handover (2 vs. 5), noted its "asking about your thinking" behavior, and our model used 3.7x fewer tokens per conversation (p=0.002p=0.002).

2. Contribution & Effect of Removing Masking (Table 2)

  • Conceptual contribution: Testing on an unseen variant is intuitive, yet our ablation shows unseen-problem evaluation alone does not stop answer-telling: removing the binary gates raises leakage (31% →\to 61%) without measurably changing Δtransfer\Delta_{\text{transfer}} (+0.23+0.23 vs. +0.25+0.25), indicating that handover remains largely unpenalized without explicit cost. Moreover, without naming any pedagogical move in the reward or prompt, optimizing masked near-transfer yields approximately the same move profile as an expert-defined rubric (e.g., more than doubling prompting for justification, 9.0% →\to 21.8%; Fig. 2).
  • Impact of w/o Masking in Table 2: While Δtransfer\Delta_{\text{transfer}} saturates across arms, removing masking substantially degrades out-of-domain performance and efficiency: MathDial RM drops 7.92→6.487.92 \to 6.48 (Win 78.078.0\\% \to 65.0\\%), Eedi RM drops 8.83→7.488.83 \to 7.48 (Win 72.472.4\\% \to 59.0\\%, lowest of all ablations), and thinking tokens jump 3.5–5.7x (456→1597456 \to 1597 on MathDial; 184→1041184 \to 1041 on Eedi).

3. Statistical Significance & Uncertainty
For the training environment, Appendix F (Table 7) reports 10,000-resample 95% bootstrap CIs (e.g., Eduardo-27B Δtransfer=+0.29 [0.25,0.32]\Delta_{\text{transfer}} = +0.29\ [0.25, 0.32] vs. base +0.22 [0.18,0.25]+0.22\ [0.18, 0.25]). On MathDial (n=500n=500), binomial standard errors on Win% (the binary Ped-RM win rate against human teachers) are ±1.4\pm 1.4\\%–2.22.2\\%. Gains are statistically significant across the Qwen3.5/3.8 family: at 4B (30.230.2\\% \pm 2.1\\% \to 78.0\\% \pm 1.9\\%), 9B (38.038.0\\% \pm 2.2\\% \to 82.2\\% \pm 1.7\\%), and 27B (82.482.4\\% \pm 1.7\\% \to 89.2\\% \pm 1.4\\%, p=0.002p=0.002; +11+11–1515 pp above Gemini 3.1 Pro/3.6 Flash). The 14B model stays at parity (70.270.2\\% \to 71.4\\%, Sec. 5.1). On TutorMoments, Bernoulli-bounded SEs are ≤0.05\le 0.05, separating Eduardo-27B from its base on Scaffolding (0.60±0.05→0.96±0.020.60 \pm 0.05 \to 0.96 \pm 0.02) and Avoids Over-Help (0.52±0.05→0.76±0.040.52 \pm 0.05 \to 0.76 \pm 0.04).

4. Eedi dataset at 14B/27B & Eduardo-14B vs. 4B/9B

  • Eedi: The Eedi dataset consists of simpler, shorter school-level diagnostic items, where Qwen3.8-27B base is already near ceiling (84.384.3\\% win rate, 9.999.99 RM). Eduardo-27B maintains parity (82.382.3\\% win, 3.313.31 vs. 3.273.27 Ped.) while cutting thinking tokens by 51% (258→127258 \to 127). Where base models are not near ceiling (4B and 9B), Eduardo improves Eedi win rates by +33.7 pp and +21.3 pp.
  • 14B architecture: Qwen3-14B is an earlier-generation dense Transformer whose base weights default to significant handover (98% leakage, 1.241.24 Ped., vs. 1717–2323\\% base leakage for newer Qwen3.5-4B/9B Gated DeltaNet models). In our setting with zero per-model tuning, RL stays at parity on MathDial/Eedi while cutting 14B environment leakage from 9898\\% \to 32\\%, lifting environment Ped. (1.24→3.171.24 \to 3.17), and improving MathTutorBench (Fig. 3).
AuthorsOpenReview ↗

Reminder 2

Dear reviewer, as the discussion closes soon, we would like to send you a reminder to submit any final acknowledgements.

Comments on the paper

3 comments
AuthorsOpenReview ↗

Global Comment (to AC and all Reviewers)

We thank the reviewers for their constructive feedback. Two themes emerged across reviews: (1) whether tutoring policies trained against a frozen LLM transfer to human interactions, and (2) how masked near-transfer and binary reward gates jointly separate teaching from telling.

1. User Study with Teachers (Added as Section 6)
As an essential step prior to evaluating experimental RL policies with real students, we conducted a blind, paired onsite study with 7 experienced upper-secondary teachers (3 physics, 3 computer science, 1 math; all 2+ years of experience, two with 6-10 years, two 10+ years) role-playing authentic classroom misconceptions (following the LearnLM setup; LearnLM Team et al., 2024). It was conducted with institutional ethics board approval. Across 11 paired rounds, teachers conversed with Eduardo-27B and its Qwen3.8-27B base model in randomized order under the identical tutor system prompt. This tests three simultaneous distribution shifts: (i) zero-shot domain transfer (10/11 rounds in CS and Physics rather than Math), (ii) student brevity (median 5-word replies vs. our verbose training student), and (iii) human expert judgment:

  • Directional Preference: Teachers preferred Eduardo-27B in 8 of 11 rounds and across 5 of 7 teachers (6 of 8 wins rated "better" or "much better", vs. all 3 base wins "slightly better"). A Bayesian hierarchical model estimates a mean preference of +0.93+0.93 on [−3,+3][-3, +3] (95% CrI [−0.13,1.93][-0.13, 1.93], P(μ>0)=0.96P(\mu>0)=0.96), given the sample size (teacher-level BF01=1.04\mathrm{BF}_{01}=1.04), we report this as directional evidence.
  • Consistent Efficiency Gains: Across all 10 logged rounds, the base model generated 5x more tokens per reply, wrote 2.4x longer visible messages, and asked twice as many questions at once (exact sign test p=0.002p=0.002). Eduardo-27B performed more turns (12.3 vs. 8.6) while it generated 3.7x fewer total tokens per conversation (p=0.002p=0.002).
  • Behavioral Alignment: Teachers more often judged the base model to over-help (6 vs. 2 rounds) and hand over solutions (5 vs. 2 rounds). One noted that the base "recommends a lot in advance and takes away the thinking" while describing Eduardo as proceeding step-by-step and asking "about your thinking" (mirroring Move G in Sec. 5.4). They also surfaced an important boundary case: with a disengaged student, Eduardo kept asking guiding questions where a worked example could be better.

2. Summary of Revisions
Alongside a new Section 6 with expert human teachers, in the response we report a new Gemini-3.1-Pro dictation audit on 200 TutorMoments dialogs (0/2000/200 solution dictations), highlight uncertainty bounds from our tables and Appendix F, and we propose to refine wording in the Abstract and Sections 3.2–3.5 and 5.2 to distinguish channel masking from soft gate penalties.

3. Broader Impact
While outcome-based RL excels at training LLMs to solve problems, applying it naively to multi-turn interaction trains models to take over the student's work and act as a homework solver in tutoring (not like a teacher). EDUARDO provides an open RL recipe (4B–27B across two architectures) and an 8,671-pair near-transfer dataset showing how to pedagogically align models to build user capability instead (e.g. through guiding questions) and matching frontier proprietary tutors on several independent datasets while using 2.4–6.2x fewer thinking tokens, which matters for low-latency real-time deployments in education.

AuthorsOpenReview ↗

Summary of Impact (TutorMoments Leaderboard)

Open-Weights Eduardo-27B Ranks Top 3 of 21 on Full Independent TutorMoments benchmark (Ai2, N=520N=520)

We believe a core strength of this work is rigorous evaluation from multiple independent perspectives, spanning distinct model judges (Ped-RM, Gemini-3.1-Pro, Claude Opus 4.6), unseen student distributions (real student turns in Eedi, independent Gemini and Claude students, and human STEM teachers in Physics/CS), and four external benchmarks (MathDial, Eedi, MathTutorBench, and the latest TutorMoments), none of which overlap with our training setup.

Confirming this at full scale, we ran all 520 TutorMoments scenarios (previously we used a subset in Section 5.3) using the exact official leaderboard setup (Claude Opus 4.6 student and judge; Claude Opus 4.8 for move taxonomy classification):

  • Our 27B Open Model in the Top 3 Against Frontier Models: Under the plain prompt (testing default tutoring scenario without disclosing the evaluation rubric, same prompt as Table 3) Eduardo-27B scores 0.6080.608 overall and ranks 3rd of 21 tutors, placing in the top 3 on every metric: Appropriate Scaffolding (0.9040.904, #3), Appropriate Rigor (0.3120.312, #3), and Avoids Over-Scaffolding (0.6730.673, #2), while supplying answers in just 1.31.3\\% of actions (vs. 11.411.4\\% for human tutors). Despite its compact 27B size, it trails only Claude Sonnet 5.5 (0.6670.667) and GPT-6 Astra (0.6340.634), outperforming 18 frontier and open models including GPT-6 Sol (0.5230.523), Claude Opus 5.5 (0.5120.512), Gemini 4 Argon (0.5120.512), Claude Opus 4.8 (0.4110.411), DeepSeek V4 Pro 0813 (0.3270.327), and GPT 5.5 (0.1580.158). It scores well above every open-weight model on the leaderboard.
  • Why it matters: Real educational deployments require low latency, data privacy, and strong out-of-the-box tutoring without engineered rubric prompts. Our recipe is fully open, and it puts a 27B model alongside the strongest proprietary tutors on exactly this default-behavior measure. Importantly, the scaffolding-rigor trade-off that TutorMoments measures is never trained for directly: our reward scores only the simulated student’s near-transfer learning gain, gated on factual correctness and no solution handover, and names no tutoring move (Sec. 5.4).

Dear Reviewers,

We posted our responses a couple of days ago. Could you please go through them and respond to them?

The discussion period ends soon (Nov 18).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.