- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies scaffolding collapse in LLM-based Socratic tutors, where the tutor abandons guided questioning and reveals the answer after sustained student pressure across a multi-turn dialogue. The authors observe that collapse is associated with hidden-state drift concentrated in middle layers, and use this to motivate SPRA, a training framework that combines supervised fine-tuning, a trajectory-weighted DPO objective over Socratic versus collapsing responses, and a margin-preserving representation loss applied at the layers where drift peaks. The method is evaluated on a multi-turn red-teaming benchmark spanning five STEM disciplines, with an LLM judge validated against human annotations. The authors report that SPRA yields lower collapse rates and later collapse turns while keeping over-refusal low, compared with prompting, tutoring-specific, and representation-level baselines, and they include ablations on loss components, target layers, and other design choices.
Critical strengths
-
The paper targets a timely and practically important failure mode. As LLM tutors are deployed in education, a tutor that gives away answers under student pressure undermines the pedagogical purpose of the system. Framing this as a trajectory-level failure rather than a single-turn output error is a useful perspective, and linking it to hidden-state drift goes beyond the usual output-level evaluation. This is the main reason I find the paper of interest to the community.
-
The empirical evaluation is broad. It covers five disciplines, several families of baselines, and ablations over loss components, target layers, and other design choices, and the LLM judge is validated against human annotations. This gives reasonable confidence that the collapse measurements themselves are meaningful, even though I have concerns about what the comparisons can attribute (see weaknesses).
Critical weaknesses
-
The main gains cannot currently be attributed to SPRA, and in particular not to the representation loss. Most baselines in Table 1 use different backbones and training data, so the comparison does not isolate the method. A test-set comparison against SFT alone and standard DPO on the same Qwen3-8B backbone and the same training pairs is necessary to support the central claim. Furthermore, Appendix H indicates that adding the margin-preserving loss does not significantly reduce collapse rate, which weakens the component that most distinguishes SPRA from a combination of existing techniques. The drift analysis that motivates this loss also lacks non-collapsing controls, so it is unclear whether the observed drift reflects collapse itself or simply differences in dialogue context. Taken together, the novel part of the method is the least supported by the evidence, which is the main reason for my recommendation.
-
The construction of preference pairs is ambiguous in a way that affects validity and reproducibility. Appendix I describes negative trajectories as remaining Socratic before the collapse turn, while Appendix C states that rejected responses exhibit collapse at every paired turn. In addition, positive and negative dialogues contain different student turns, so it is not clear how each pair shares an identical history, which DPO requires. Since these pairs define the training objective, it is difficult to assess what the model is actually being optimized toward without a precise account of this procedure.
Questions for authors
How exactly are preference pairs constructed before the collapse turn? Please reconcile Appendix I (negatives remain Socratic before collapse) with Appendix C (rejected responses collapse at every paired turn), and explain how chosen and rejected responses share an identical dialogue history when the positive and negative dialogues contain different student turns. Can you report test-set results for SFT only and SFT plus standard DPO on the same Qwen3-8B backbone and the same training pairs as SPRA? This is the comparison needed to attribute the gains to the proposed objectives. Given that Appendix H shows no significant reduction in collapse rate from adding the margin-preserving loss, what evidence supports the representation loss as a meaningful contribution? Are there other metrics (for example, collapse turn or over-refusal) where its effect is significant, with variance across seeds? Does the middle-layer drift also appear in dialogues of comparable length and pressure that do not collapse? Without this control, the causal link between drift and collapse remains unclear. The reported LoRA parameter counts appear inconsistent with the Qwen3-8B architecture. Could you specify the rank, target modules, and how the counts were computed?
Response to Reviewer 3nde
We appreciate your valuable feedback.
A1.1 (Q2): Objectives on the same backbone and pairs
SPRA achieves the lowest Collapse Rate (CR), latest Average Collapse Onset Turn (OT), and lowest Over-Refusal Rate (RR) among objectives trained on the same data.
On the test set over three seeds, SPRA lowers CR by 7.3 points relative to standard DPO and has the lowest CR among external baselines on Qwen3-8B (Tabs. 4, 20, and 21). SPRA’s gains are therefore attributable to its training objective.
A1.2 (Q3): Contribution of the MP loss
Adding MP significantly lowers CR and RR and delays collapse. Against SFT + trajectory-weighted DPO (TDPO), which lacks only MP, full SPRA lowers CR in all three seeds (Holm-adjusted p=0.029), adds 1.06 collapse-free tutor turns per dialogue, and lowers RR from 6.6% to 2.5% (both Holm-adjusted p=0.001). Because MP delays collapse, the CR gap is 10.4 points by turn 10 and narrows to 3.9 by turn 16 (App. H, Lines 1424–1433). MP thus significantly improves all three metrics.
A1.3 (Q4): Drift in matched non-collapsing dialogues
With discipline and dialogue length (tutor-turn index) held fixed, drift is significantly higher in dialogues that go on to collapse, and aligning representations against this drift reduces collapse. Our goal is to reduce collapse, so we use the drift–collapse association, without claiming causation (Sec. 5, Lines 527–529), to locate where and along which direction MP acts.
We compare collapsing and non-collapsing dialogues under the same attacker and tutor-turn index, using the Scaffold-to-Collapse Relative Drift (DS2C) (Tab. 1).
It remains significant in a permutation test stratified by discipline × tutor-turn index (p<0.001). Mean DS2C stays negative in non-collapsing dialogues even at turn 16 (−0.09) but is already positive (+0.06) at the first response of dialogues that collapse at their second (App. H).
MP shifts the test-set DS2C from +0.06 to −0.23 (Fig. 5) and significantly lowers CR (A1.2). Adapting the drift-peak layers gives a validation CR of 36.0% vs. 66.5–72.0% for other layers (Tab. 3b), and a random direction of the same norm raises test CR to 61.5%. These controls establish a significant association between drift and subsequent collapse across dialogue lengths, and aligning representations along the estimated direction significantly reduces collapse.
A2: App. I and App. C describe two different objects, a negative trajectory and the preference pairs expanded from it, and both responses in every pair share the trajectory’s own prefix as history.
The rejected responses before collapse are never inserted into the trajectory, so it remains Socratic until collapse (App. I), while every pair contains a collapsing rejected response (App. C). The positive dialogue supplies the problem and SFT data; its student turns never enter a pair, and length alignment only determines which tutor turns are paired. We clarified this in App. C (Tab. 8) and added the pair-expansion prompt with a worked example in App. M. Each pair thus contrasts a Socratic and a collapsing response under an identical history, as DPO requires.
A3: SPRA applies LoRA (rank 16, α = 32, dropout 0.05; App. D) to seven linear modules in layers 20–23 of Qwen3-8B, giving 4,849,664 trainable parameters (about 0.06% of the backbone). Each module contributes r(d_in + d_out) parameters. With hidden size d = 4,096, key–value projection size d_kv = 1,024 (8 KV heads × 128), and MLP size d_ff = 12,288 (q_proj, o_proj: d → d; k_proj, v_proj: d → d_kv; gate_proj, up_proj, down_proj: between d and d_ff), each layer contributes
r[2(2d)+2(d+dkv)+3(d+dff)]=1,212,416.
Summing over 4, 10, or all 36 layers gives 4,849,664, 12,124,160, and 43,646,976 parameters, respectively; with modules and layers fixed, the count scales linearly with rank. The counts in Tabs. 3b, 17, and 18 are updated accordingly (App. D, Tab. 9). All counts thus follow from the Qwen3-8B architecture.