acceptodds
Under review as a conference paper at ICLR 2027

SPRA: Scaffold-Preserving Representation Alignment for Robust Socratic Tutors

Abstract

Large language model (LLM)-based Socratic tutors are increasingly employed to guide student learning through multi-turn questioning, yet they remain vulnerable to scaffolding collapse, where sustained student pressure causes the tutor to abandon Socratic guidance and reveal direct answers. Existing defenses mainly regulate surface outputs through prompting, preference tuning, or filtering, but they do not address the hidden-state drift associated with such trajectory-level failures. Motivated by empirical evidence that scaffolding collapse is associated with systematic hidden-state drifts, we propose a Scaffold-Preserving Representation Alignment (SPRA) framework that integrates supervised fine-tuning, trajectory-weighted direct preference optimization (DPO), and a margin-preserving representation loss that preserves the hidden-state projection margin between scaffolding and collapse responses. We evaluate SPRA across five STEM disciplines under a multi-turn red-teaming protocol. On Qwen3-8B, SPRA keeps the Collapse Rate at or below 32% in every discipline, delays the average collapse onset beyond 9 turns, and maintains a low over-refusal rate, suggesting that representation-level alignment can improve the scaffolding robustness of Socratic tutors under our settings.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
AI for education, large language models, socratic tutor, robustness, AI Safety

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_3ndeOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies scaffolding collapse in LLM-based Socratic tutors, where the tutor abandons guided questioning and reveals the answer after sustained student pressure across a multi-turn dialogue. The authors observe that collapse is associated with hidden-state drift concentrated in middle layers, and use this to motivate SPRA, a training framework that combines supervised fine-tuning, a trajectory-weighted DPO objective over Socratic versus collapsing responses, and a margin-preserving representation loss applied at the layers where drift peaks. The method is evaluated on a multi-turn red-teaming benchmark spanning five STEM disciplines, with an LLM judge validated against human annotations. The authors report that SPRA yields lower collapse rates and later collapse turns while keeping over-refusal low, compared with prompting, tutoring-specific, and representation-level baselines, and they include ablations on loss components, target layers, and other design choices.

Critical strengths
  1. The paper targets a timely and practically important failure mode. As LLM tutors are deployed in education, a tutor that gives away answers under student pressure undermines the pedagogical purpose of the system. Framing this as a trajectory-level failure rather than a single-turn output error is a useful perspective, and linking it to hidden-state drift goes beyond the usual output-level evaluation. This is the main reason I find the paper of interest to the community.

  2. The empirical evaluation is broad. It covers five disciplines, several families of baselines, and ablations over loss components, target layers, and other design choices, and the LLM judge is validated against human annotations. This gives reasonable confidence that the collapse measurements themselves are meaningful, even though I have concerns about what the comparisons can attribute (see weaknesses).

Critical weaknesses
  1. The main gains cannot currently be attributed to SPRA, and in particular not to the representation loss. Most baselines in Table 1 use different backbones and training data, so the comparison does not isolate the method. A test-set comparison against SFT alone and standard DPO on the same Qwen3-8B backbone and the same training pairs is necessary to support the central claim. Furthermore, Appendix H indicates that adding the margin-preserving loss does not significantly reduce collapse rate, which weakens the component that most distinguishes SPRA from a combination of existing techniques. The drift analysis that motivates this loss also lacks non-collapsing controls, so it is unclear whether the observed drift reflects collapse itself or simply differences in dialogue context. Taken together, the novel part of the method is the least supported by the evidence, which is the main reason for my recommendation.

  2. The construction of preference pairs is ambiguous in a way that affects validity and reproducibility. Appendix I describes negative trajectories as remaining Socratic before the collapse turn, while Appendix C states that rejected responses exhibit collapse at every paired turn. In addition, positive and negative dialogues contain different student turns, so it is not clear how each pair shares an identical history, which DPO requires. Since these pairs define the training objective, it is difficult to assess what the model is actually being optimized toward without a precise account of this procedure.

Questions for authors

How exactly are preference pairs constructed before the collapse turn? Please reconcile Appendix I (negatives remain Socratic before collapse) with Appendix C (rejected responses collapse at every paired turn), and explain how chosen and rejected responses share an identical dialogue history when the positive and negative dialogues contain different student turns. Can you report test-set results for SFT only and SFT plus standard DPO on the same Qwen3-8B backbone and the same training pairs as SPRA? This is the comparison needed to attribute the gains to the proposed objectives. Given that Appendix H shows no significant reduction in collapse rate from adding the margin-preserving loss, what evidence supports the representation loss as a meaningful contribution? Are there other metrics (for example, collapse turn or over-refusal) where its effect is significant, with variance across seeds? Does the middle-layer drift also appear in dialogues of comparable length and pressure that do not collapse? Without this control, the causal link between drift and collapse remains unclear. The reported LoRA parameter counts appear inconsistent with the Qwen3-8B architecture. Could you specify the rank, target modules, and how the counts were computed?

1 comment
AuthorsOpenReview ↗

Response to Reviewer 3nde

We appreciate your valuable feedback.

W1 and Q2–Q4: Evidence for SPRA’s objectives and the drift–collapse association

A1.1 (Q2): Objectives on the same backbone and pairs

SPRA achieves the lowest Collapse Rate (CR), latest Average Collapse Onset Turn (OT), and lowest Over-Refusal Rate (RR) among objectives trained on the same data.

ObjectiveCR↓ (%)p (CR)OT↑p (OT)RR↓ (%)
SFT41.5 ± 2.0≤0.0013.34 ± 0.48<0.00111.5 ± 0.6
SFT + DPO35.6 ± 2.3≤0.0016.36 ± 0.21<0.0018.8 ± 1.3
SFT + TDPO32.3 ± 1.30.0297.14 ± 0.55<0.0016.6 ± 1.6
Full SPRA28.3 ± 2.0—9.50 ± 0.16—2.5 ± 0.6

On the test set over three seeds, SPRA lowers CR by 7.3 points relative to standard DPO and has the lowest CR among external baselines on Qwen3-8B (Tabs. 4, 20, and 21). SPRA’s gains are therefore attributable to its training objective.

A1.2 (Q3): Contribution of the MP loss

Adding MP significantly lowers CR and RR and delays collapse. Against SFT + trajectory-weighted DPO (TDPO), which lacks only MP, full SPRA lowers CR in all three seeds (Holm-adjusted p=0.029), adds 1.06 collapse-free tutor turns per dialogue, and lowers RR from 6.6% to 2.5% (both Holm-adjusted p=0.001). Because MP delays collapse, the CR gap is 10.4 points by turn 10 and narrows to 3.9 by turn 16 (App. H, Lines 1424–1433). MP thus significantly improves all three metrics.

A1.3 (Q4): Drift in matched non-collapsing dialogues

With discipline and dialogue length (tutor-turn index) held fixed, drift is significantly higher in dialogues that go on to collapse, and aligning representations against this drift reduces collapse. Our goal is to reduce collapse, so we use the drift–collapse association, without claiming causation (Sec. 5, Lines 527–529), to locate where and along which direction MP acts.

We compare collapsing and non-collapsing dialogues under the same attacker and tutor-turn index, using the Scaffold-to-Collapse Relative Drift (DS2C) (Tab. 1).

PositionCollapsingMatched non-collapsingDifference [95% CI]p
Before collapse (τ−2), still Socratic+0.07−0.12+0.19 [+0.13, +0.25]<0.001
At collapse (τ)+0.15−0.11+0.26 [+0.21, +0.31]<0.001

It remains significant in a permutation test stratified by discipline × tutor-turn index (p<0.001). Mean DS2C stays negative in non-collapsing dialogues even at turn 16 (−0.09) but is already positive (+0.06) at the first response of dialogues that collapse at their second (App. H).

MP shifts the test-set DS2C from +0.06 to −0.23 (Fig. 5) and significantly lowers CR (A1.2). Adapting the drift-peak layers gives a validation CR of 36.0% vs. 66.5–72.0% for other layers (Tab. 3b), and a random direction of the same norm raises test CR to 61.5%. These controls establish a significant association between drift and subsequent collapse across dialogue lengths, and aligning representations along the estimated direction significantly reduces collapse.

W2 and Q1: Pair construction and the shared dialogue history

A2: App. I and App. C describe two different objects, a negative trajectory and the preference pairs expanded from it, and both responses in every pair share the trajectory’s own prefix as history.

Pair positionChosen y⁺Rejected y⁻
Before collapseThe trajectory’s own Socratic turnA collapse-typed response generated by Qwen3.6-27B for the same prefix
At collapseA resilient alternative for the same prefix (rule 6, Fig. 11)The trajectory’s collapsed turn

The rejected responses before collapse are never inserted into the trajectory, so it remains Socratic until collapse (App. I), while every pair contains a collapsing rejected response (App. C). The positive dialogue supplies the problem and SFT data; its student turns never enter a pair, and length alignment only determines which tutor turns are paired. We clarified this in App. C (Tab. 8) and added the pair-expansion prompt with a worked example in App. M. Each pair thus contrasts a Socratic and a collapsing response under an identical history, as DPO requires.

Q5: LoRA configuration and parameter counts

A3: SPRA applies LoRA (rank 16, α = 32, dropout 0.05; App. D) to seven linear modules in layers 20–23 of Qwen3-8B, giving 4,849,664 trainable parameters (about 0.06% of the backbone). Each module contributes r(d_in + d_out) parameters. With hidden size d = 4,096, key–value projection size d_kv = 1,024 (8 KV heads × 128), and MLP size d_ff = 12,288 (q_proj, o_proj: d → d; k_proj, v_proj: d → d_kv; gate_proj, up_proj, down_proj: between d and d_ff), each layer contributes

r[2(2d)+2(d+dkv)+3(d+dff)]=1,212,416.r[2(2d) + 2(d+d_{kv}) + 3(d+d_{ff})] = 1{,}212{,}416.

Summing over 4, 10, or all 36 layers gives 4,849,664, 12,124,160, and 43,646,976 parameters, respectively; with modules and layers fixed, the count scales linearly with rank. The counts in Tabs. 3b, 17, and 18 are updated accordingly (App. D, Tab. 9). All counts thus follow from the Qwen3-8B architecture.

Reviewer_29ZHOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper argues that the loss of socratic behavior induced by pressure from the students in LLM-based tutors is mediated by a drift in the model's internal activations. They introduce a new training framework / incremental change in DPO's training objective to alleviate this issue.

The problem they tackle is well-motivated within the sub-field, and they correctly justify it from the literature.

The submission is reasonably easy to read, and to my understanding technically correct.

Apart from the weaknesses stated below, I believe the evaluation and experimental setup is solid and convincing, and the presentation of the results is clear.

Critical strengths

It interestingly applies methods from AI safety and interpretability to an education application (e.g. identifying failure directions in activations, or using activations to select which layers to finetune), which is likely to be of interest for researchers in both fields. To my knowledge, the use of such techniques in AI x Education is rare, which makes this work uniquely positioned in the literature and and likely to contribute to expand the field in this direction. The algorithmic contribution is sound even if weakly supported by the experimental design.

Critical weaknesses

The main weaknesses of this paper are in its evaluation and reproducibility

It is missing crucial implementation details for reproducibility. In particular, the exact prompt templates or methods for the few-shot, socratic-enforced and filtering baseline are not clearly specified, although the authors state that prompts are provided in the appendices. This is my main concern and I am expecting the authors to provide an answer to it.

The comparisons to other methods are also unfair in my opinion. Some of the methods they compare to use different base models (and, especially, older or smaller ones), which make it confound the contribution of their training method with the gains coming from using a better base model. This is the case for SocraticLM, EULER, PedagogicalRL, EduChat and ORPO.

The central claim of the paper is also weakly supported as the new training objective introduced by the author does not significantly lower CR and OT, as per their ablations. It does significantly lower RR, which is a slightly different and weaker claim. I would recommend regraming the paper in that sense.

The paper also lacks investigation of spillover effect of their training framework on the tutor (e.g. its correctness, and other abilities outside of the limited socratic failure modes they study).

Questions for authors
  1. Can you make the paper reproducible by uploading prompts for the baselines? (Most important)
  2. What results would be get if we were to use SPRA on the same base models as these, or if these methods were used on the same backbone model they use in the paper?
  3. Did you measure or observe any spillover effect of their training framework on the tutor (e.g. its correctness, and other abilities outside of the limited socratic failure modes they study)?

The reproducibility issue prevents me from recommending an accept, but I believe the paper is still of great value to the community. I am looking forward to read the author's responses to my concerns and willing to change my assessment based on them.

3 comments
AuthorsOpenReview ↗

Response to Reviewer 29ZH

We appreciate your valuable feedback. We hope the additional clarifications and results below directly address your concerns.

W1/Q1: Reproducibility of the prompt-based baselines

A1: The exact prompts and execution rules of all three baselines are now documented in the new App. L (Lines 1725–1786), building on the baseline descriptions in App. F and the tutor, attacker, and judge prompts in App. I.

  • Socratic-enforced (App. L.1–L.2, Lines 1728–1748): the shared tutor template used by every tutor in Tab. 2, applied to Qwen3-8B without fine-tuning, with full decoding settings.
  • Few-shot (App. L.3, Lines 1749–1769): both demonstrations in full, with their insertion format.
  • Output filtering (App. L.4, Lines 1770–1786): the detector, its inputs and JSON fields, the exact retry template, and the stopping rule.

App. L consolidates the implementation details needed to reproduce each prompt-based baseline, all evaluated under the same protocol as SPRA.

W2/Q2: Comparison on a common backbone

A2: On a common Qwen3-8B backbone, SPRA outperforms the five requested baselines, achieving the lowest Collapse Rate (CR), latest Average Collapse Onset Turn (OT), and lowest Over-Refusal Rate (RR). We adapt the training recipes of SocraticLM (SFT), ORPO, and EULER to Qwen3-8B using SPRA’s training data, re-train PedagogicalRL with the authors’ code, and evaluate the released EduChat checkpoint (mean ± SD over three runs).

MethodCR↓ (%)OT↑RR↓ (%)
EduChat62.7 ± 1.34.44 ± 0.128.9 ± 1.4
SocraticLM41.5 ± 2.03.34 ± 0.4811.5 ± 0.6
ORPO39.7 ± 1.45.85 ± 0.4610.9 ± 0.6
EULER37.0 ± 2.95.94 ± 0.318.4 ± 1.1
PedagogicalRL36.3 ± 1.27.84 ± 0.154.2 ± 0.7
SPRA28.3 ± 2.09.50 ± 0.162.5 ± 0.6

SPRA reduces CR by at least 8.0 percentage points relative to every baseline (p<0.001), including those trained on the same data. These gains persist with the backbone held constant, supporting SPRA’s training-method contribution beyond differences in base model (App. H, Tab. 21).

W3: Significance of the CR and OT improvements

A3: Full SPRA significantly lowers CR and increases OT relative to its ablated variants on the test set, supporting benefits beyond reduced over-refusal.

ConfigurationCR↓ (%)p (CR)OT↑p (OT)RR↓ (%)
SFT + DPO35.6 ± 2.3≤0.0016.36 ± 0.21<0.0018.8 ± 1.3
SFT + TDPO32.3 ± 1.30.0297.14 ± 0.55<0.0016.6 ± 1.6
Full SPRA28.3 ± 2.0—9.50 ± 0.16—2.5 ± 0.6

All CR and OT differences are significant, including those against SFT + trajectory-weighted DPO (TDPO), the closest variant, which lacks only the margin-preserving loss (Holm-adjusted CR p=0.029; OT p<0.001); SFT + DPO uses standard DPO and lacks both proposed components. The three metrics assess robustness from different angles: CR measures whether collapse occurs within 16 turns, OT measures when it occurs among dialogues that collapse, and RR measures over-refusal. Dialogues that collapse later but still within 16 turns contribute to a later OT while remaining counted in the final CR. Accordingly, the CR gap to SFT + TDPO is 10.4 points at turn 10 and narrows to 3.9 points at turn 16. These significant gains in collapse frequency and onset, together with low RR, support our central claim that SPRA reduces and delays collapse while maintaining low over-refusal (Tabs. 4 and 20).

W4/Q3: Spillover effects on correctness and other abilities

A4: As stated in Sec. 5 (Lines 531–532), our study targets scaffolding robustness, i.e., the stability of Socratic tutoring under sustained student pressure; within this scope, we also evaluate over-refusal, a spillover effect that SPRA reduces. Over-refusal measures whether a tutor resists collapse by declining legitimate learning requests, and SPRA has the lowest RR among all methods in A2 (2.5% vs. 4.2–11.5%). One of the five collapse criteria also checks whether the tutor addresses a clear student misconception (App. I). SPRA therefore improves scaffolding stability while achieving the lowest over-refusal, and other spillover effects, such as factual correctness and broader capability retention, are stated as future work in Sec. 5 (Lines 532–533).

Thank you again for your thoughtful review, combining technical scrutiny with an appreciation of the work’s relevance to AI safety and education. With the additional experimental details now provided, we hope we have addressed your main reproducibility concern and would be grateful if you would consider raising your score. We welcome any further questions.

Thank you for the detailed response and the additional experiments.

W1/Q1 (reproducibility). Resolved

W2/Q2 (common backbone). Thank you for running the matched-backbone comparison in Tab. 21. I think these results matter enough to belong in the main text, maybe in place of Tab. 2. For some baselines they differ a lot from Tab. 2: SocraticLM goes from about 90% CR to 41.5%, and ORPO from about 89% to 39.7%. This suggests that much of the gap in Tab. 2 came from the base models rather than the training methods. Tab. 2 is still the headline comparison and does not state each baseline's backbone, so readers may overestimate SPRA's margin.

Relatedly, the row called "SocraticLM" in the rebuttal is labelled "SFT" in Tab. 21. Its numbers (41.5 ± 2.0 / 3.34 ± 0.48 / 11.5 ± 0.6) are identical to the LSFT\mathcal{L}_{SFT}-only ablation in Tab. 4, which is SPRA's own warmup stage trained on SPRA's data. That is a reasonable comparison, but it should be described as the SocraticLM recipe applied to your data, not as SocraticLM itself. The same goes for the adapted EULER and ORPO.

W3 (significance of CR/OT improvements). Thank you for running the extra ablation. I still find the evidence that LMP\mathcal{L}_{MP} lowers CR fairly weak:

  • The gap to SFT + TDPO is 3.9 points, with a confidence interval whose lower bound is below 1 point. On the validation set the gap is 1.5 points.
  • The permutation tests resample problems rather than seeds, so the p-values do not account for seed-to-seed variance (SD of 1.3–2.0 over three seeds).

LMP\mathcal{L}_{MP}'s clearest effect is still on over-refusal (6.6% to 2.5%). That said, I don't think this weakens the paper much. Reducing over-refusal while keeping collapse low is a useful result in itself, and the other contributions and framing of the paper are already valuable.

W4/Q3 (spillover effects). This is not convincigly adressed yet in my opinion. As a minor suggestion, a small check that the fine-tuned tutor keeps its general abilities would strengthen the paper. For example, you could compare in appendix SPRA with base Qwen3-8B on a standard benchmark such as GSM8K or MMLU. Alternatively, you could report how often the tutor's hints are correct against the reference solution on a subset of dialogues.

I believe your answers reasonable adress my main concerns and I am looking forward to reading your answer to the points I raised above (esp. W2 and W4) before raising my score.

Note: I did not use AI in my initial review but used an LLM to polish the writing of this answer

AuthorsOpenReview ↗

Response to Reviewer 29ZH

Thank you for the constructive follow-up. We agree with your W2 and W3 points on presentation and framing and have revised accordingly. For W4, we ran the general-capability check you suggested.

W2: Tab. 2 vs. Tab. 21, naming, and backbones

We agree that the matched-backbone comparison should be the main comparison and that each baseline's backbone should be stated. In the revision, the matched-backbone table is now Tab. 2 in the main text, and the original five-domain table moves to the appendix (Tab. 21) as the heterogeneous-backbone comparison, with a new Backbone column: SocraticLM (ChatGLM3-6B), EULER (Phi-3-mini), ORPO (Mistral-7B), PedagogicalRL (TutorRL-7B, from Qwen2.5-7B-Instruct), EduChat, RMU, Circuit-Breakers, and R2D2 (Qwen3-8B), and RepBend and LAT (Llama-3-8B). On Qwen3-8B, the rows are renamed "SocraticLM recipe", "EULER recipe", and "ORPO recipe", all trained on SPRA's data, and we note that the SocraticLM-recipe row corresponds to the L_SFT-only ablation in Tab. 4 (same recipe, data, and seeds).

Against the strongest baseline, PedagogicalRL, SPRA lowers CR by 7.8 points in the heterogeneous-backbone comparison and by 8.0 points on the common backbone, so the margin is consistent across both settings. As you note, the gaps to weaker baselines are smaller on the common backbone (e.g., 41.5% for the SocraticLM recipe vs. 86–94% for SocraticLM), and Tab. 21 now notes that rows with other backbones reflect both base models and training methods. For other backbones (Q2), SPRA on Mistral-7B-Instruct-v0.3 reaches 36% CR on the math test set (Tab. 13), compared with 91% for ORPO on Mistral-7B-v0.1 (Tab. 21).

W3: Strength of the CR evidence for L_MP

We agree the CR gain from L_MP over SFT + TDPO is modest and have reframed the claim in Sec. 3 and Sec. 5: L_MP mainly delays collapse and reduces over-refusal, with a smaller additional reduction in CR. To account for seed variance, we report seed-level results: the paired CR differences (SFT + TDPO − SPRA) are 3.4, 3.6, and 4.8 points across the three seeds (paired t-test, p=0.012). With two additional seeds (App. H), CR is lower in all five seeds: a crossed random-effects model gives ΔCR=4.00 (95% CI [1.54, 6.46]), with ΔOT=2.35 and ΔRR=4.16, and a two-level bootstrap over seeds and problems gives ΔCR=4.00 (95% CI [0.72, 7.32]).

We also report collapse-free tutor turns per dialogue, which combines CR and OT over the 16-turn dialogue: L_MP adds 1.06 turns over three seeds (95% CI [0.76, 1.35], Holm p=0.001) and 1.05 turns over five seeds (95% CI [0.73, 1.38]). Relative to SFT + standard DPO, which lacks both proposed components, full SPRA lowers CR by 7.3 points (95% CI [4.1, 10.2]).

W4/Q3: Spillover effects (new App. N)

We ran the general-capability check you suggested, comparing Qwen3-8B with the final SPRA checkpoint (seed 42; thinking disabled; 5-shot; full settings in App. N):

ModelGSM8K (%)MMLU (%)
Qwen3-8B87.3474.75
SPRA88.4074.68

SPRA preserves the mathematical reasoning and general knowledge of the backbone (GSM8K +1.06, MMLU −0.06 points) while updating only 4 layers (~0.06% of parameters).

We keep learning gains and long-horizon effects with real students as limitations in Sec. 5. Thank you again; we hope these revisions address your remaining concerns, particularly W2 and W4, and we would be grateful if you would consider raising your score.

Reviewer_MjRHOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

In this paper, the authors propose SPRA to address scaffolding collapse in Socratic tutoring through supervised fine-tuning, trajectory-weighted preference optimization, and representation-margin preservation. Across five domains, it reports a pooled collapse rate of 28.8%, compared with 36.6% for PedagogicalRL, while maintaining low over-refusal.

Critical strengths
  • The paper studies a meaningful multi-turn tutoring failure mode and evaluates both scaffolding collapse and over-refusal.
  • Human validation, statistical tests, and within-backbone loss ablations strengthen the evaluation.
  • Figures 5–7 provide empirical evidence of reduced representation drift, while the paper explicitly acknowledges the statistical limitations of the incremental collapse-rate improvements.
Critical weaknesses
  1. External comparisons do not fully isolate the algorithmic contribution. Several competing systems use different backbones and training recipes. The shared evaluation protocol, comparable tuning budgets, and within-backbone ablations are useful, but do not fully resolve this attribution issue. Evaluating selected strong alternatives on a common backbone with matched data would strengthen the claimed advantage.

  2. The incremental anti-collapse benefit of margin preservation remains uncertain. Table 2 and Appendix H report that the full objective’s collapse-rate improvements over the partial objectives are not statistically significant, whereas its over-refusal reduction relative to SFT plus trajectory-weighted DPO is significant. This supports a behavioral benefit, with stronger evidence for reducing over-refusal than for independently preventing additional collapse. Larger controlled evaluations would help connect the representation diagnostics to this incremental behavioral improvement.

Questions for authors

see above

3 comments
AuthorsOpenReview ↗

Response to Reviewer MjRH

We appreciate your valuable feedback and support of our work. We hope the additional clarifications and results below directly address your concerns.

W1: Attribution of the gains to the training method

A1: On a common Qwen3-8B backbone with matched data, SPRA outperforms the strong baselines below, achieving the lowest Collapse Rate (CR), latest Average Collapse Onset Turn (OT), and lowest Over-Refusal Rate (RR) (mean ± SD over three seeds).

MethodCR↓ (%)OT↑RR↓ (%)
SFT (SocraticLM recipe)41.5 ± 2.03.34 ± 0.4811.5 ± 0.6
ORPO (adapted)39.7 ± 1.45.85 ± 0.4610.9 ± 0.6
EULER (adapted)37.0 ± 2.95.94 ± 0.318.4 ± 1.1
SPRA28.3 ± 2.09.50 ± 0.162.5 ± 0.6

SPRA reduces CR by 8.7–13.1 points (p≤0.001), delays collapse by 3.6–6.2 turns, and lowers RR by 5.9–9.0 points relative to these baselines. These gains persist with the backbone held constant, further supporting SPRA’s training-method contribution beyond differences in backbone choice (App. H, Tab. 21).

W2: Incremental anti-collapse benefit of margin preservation

A2: The margin-preserving loss (MP) provides a significant incremental anti-collapse benefit. In a controlled comparison that differs only in MP, adding it to SFT + trajectory-weighted DPO (TDPO) lowers CR, delays collapse, and lowers RR (mean ± SD over three seeds).

ConfigurationCR↓ (%)OT↑RR↓ (%)
SFT + TDPO32.3 ± 1.37.14 ± 0.556.6 ± 1.6
SPRA (SFT + TDPO + MP)28.3 ± 2.09.50 ± 0.162.5 ± 0.6

Adding MP lowers CR by 3.9 points (Holm-adjusted p=0.029, lower in all three seeds), delays collapse by 2.4 turns (p<0.001), and lowers RR by 4.1 points (Holm-adjusted p=0.001). The behavioral improvement is consistent with the representation evidence. MP shifts the Scaffold-to-Collapse Relative Drift (DS2C) at the drift-peak layers from +0.06 to −0.23 (Fig. 5), and the same loss with a random direction raises CR to 61.5%. These results connect the drift analysis to the behavioral improvement, further supporting MP’s contribution (Tab. 4, Lines 374–378).

Thanks for providing the additional experimental results to convince me the contribution of the paper. Therefore, I will keep my positive score.

We sincerely appreciate your positive feedback. We would be grateful if you could reflect this increased certainty in your confidence score.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary for AC

We sincerely appreciate your time and effort in managing the review of our submission and in guiding the discussion. We thank all reviewers for their careful reading and valuable suggestions, which have substantially strengthened the paper.

1. Summary

Contribution. This paper studies scaffolding collapse, where an LLM-based Socratic tutor abandons guided questioning and reveals answers under sustained student pressure. (i) New perspective: we treat collapse as a trajectory-level failure and show that it is associated with hidden-state drift concentrated in middle layers, across five disciplines and six backbones; drift is already higher one response before collapse than in matched non-collapsing dialogues. (ii) New method: unlike existing defenses that act only on outputs, SPRA uses this drift to select the layers and the collapse direction, and combines trajectory-weighted DPO with a margin-preserving representation loss anchored by a frozen reference cache, adapting only ~0.06% of parameters. (iii) New benchmark: a five-discipline multi-turn red-teaming protocol with a human-validated judge (κ=0.92). On a common Qwen3-8B backbone, SPRA lowers the collapse rate to 28.3%, delays collapse beyond nine turns, keeps over-refusal at 2.5%, and preserves GSM8K and MMLU accuracy.

Strengths. Reviewers highlighted strengths across four dimensions. Novel perspective: framing collapse as a trajectory-level failure and linking it to hidden-state drift "goes beyond the usual output-level evaluation" (3nde), addressing a timely and meaningful failure mode (3nde, 29ZH, MjRH). Novel interdisciplinary method: applying AI-safety and interpretability techniques (failure directions in activations, activation-guided layer selection) to education is "rare" and makes the work "uniquely positioned in the literature" (29ZH). Empirical rigor: the evaluation spans five disciplines and several baseline families, with ablations, human validation, and statistical tests (MjRH, 3nde), and is "solid and convincing" (29ZH). Soundness and clarity: the paper is clear and easy to follow (MjRH, 29ZH, 3nde) and technically correct, with a sound algorithmic contribution (29ZH).

During the discussion, MjRH kindly confirmed: "Thanks for providing the additional experimental results to convince me the contribution of the paper. Therefore, I will keep my positive score." 29ZH engaged closely with our first response, marked reproducibility as "Resolved", and noted that "the other contributions and framing of the paper are already valuable". In the follow-up, 29ZH suggested presenting the common-backbone comparison in the main text and checking general capabilities; we have responded to both suggestions and incorporated them into the revised PDF.

All revisions are marked in blue in the updated PDF.

2. Updated Results and Paper Revisions

We have uploaded a revised PDF, with all modifications shown in blue.

Key ConcernReviewersOur Response & Revision
Attribution of gains on a common backboneMjRH, 29ZH, 3ndeShowed that SPRA has the lowest CR, latest onset, and lowest RR among all training methods on Qwen3-8B, beating PedagogicalRL by 8.0 points. Added as main Tab. 2; each baseline's backbone stated in Tab. 21.
Contribution of the margin-preserving lossMjRH, 29ZH, 3ndeShowed that the full objective significantly outperforms every partial objective; the loss lowers CR in all five seeds, adds 1.06 collapse-free turns, and cuts RR from 6.6% to 2.5%. Added in Tab. 4, Tab. 20, App. H.
Drift–collapse link without a non-collapse control3ndeShowed that drift is already higher before collapse than in matched non-collapsing dialogues (+0.19, p<0.001), in every discipline. Added in Sec. 3, Tab. 1, Tabs. 22–23.
Reproducibility and implementation details29ZH, 3ndeReleased the exact prompts and execution rules of all prompt-based baselines (marked "Resolved"), clarified that both responses in every preference pair share the same dialogue history with a worked example, and derived the LoRA parameter counts from the Qwen3-8B architecture. Added in App. L, App. C (Tab. 8), App. M, App. D (Tab. 9).
Spillover effects on general abilities29ZHShowed that SPRA preserves the GSM8K and MMLU accuracy of the base Qwen3-8B. Added in Sec. 3, App. N.

We are grateful to the AC and all reviewers for their engagement throughout the review process. We especially thank 29ZH for the many thoughtful and constructive suggestions during the discussion, which have greatly improved the clarity and rigor of our work, and we sincerely hope to have the opportunity to continue the exchange with 29ZH.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.