- Rating
- 4(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The authors focus on developing models that teach humans more efficiently rather than performing tasks themselves, specifically considering counselor training as a real-world application. To do so, they first created a dataset for this task, with different scenarios simulating practitioner-client interactions, annotated with common errors and the feedback a junior practitioner would receive. They then trained models using SFT on this dataset to align them with the structure and knowledge of the task, and finally fine-tuned them using GRPO, based on the improvement of a student LLM in solving the task after receiving the generated feedback. They analyze the performance improvements achieved by the trained model on this task, evaluating how well it identifies errors and provides feedback. Finally, they also tested the feedback provided by the trained model with Master's students and showed that it helped improve their performance over time.
Critical strengths
-
The paper is generally well written and well motivated. It is easy to read and follow, without being filled with unnecessary details that are not particularly informative for the reader. It is also very well structured, and the reader can, in a sense, anticipate what comes next, which greatly helps with understanding the paper without feeling lost.
-
The authors use the downstream helpfulness of the model's generated output as a training signal. This is interesting and perfectly aligned with their objective. I appreciate this elegant solution.
-
I think the experimental setup is very well designed, also covering a real-world use case in which the model is used to train Master's students, with positive results.
Critical weaknesses
-
Sometimes, important details needed to properly follow the paper are missing from the main text. They are present in the appendix, but, for instance, discussing the results related to VGR without explaining what VGR is (even at a very high level) makes it difficult to follow (similar examples apply to MITI, PSC, and WA). The same issue also occurs in some parts of the method section (e.g., Prec, R_loc, etc.).
-
The discussion of the results is rather basic. The authors mostly describe what is already presented in the figures and tables without providing deeper insights into what those results mean. For instance, why is the improvement in Task 1 much more significant than in Task 2?
-
Figure 4 is not particularly informative. It is more visually appealing than useful. Statistical measures (e.g., Cohen's kappa) would be much more useful and give the reader a clearer picture of the results.
Questions for authors
See the questions above.
-
If you observe a strong correlation between human raters and the model you used in Section 4.3 to evaluate your models' answers, why not use its preferences to directly optimize the models as well?
-
How well would a model trained to perform the counselor task itself perform? Would it perform better than a student receiving feedback or a model receiving feedback? Is giving feedback more difficult than performing the task itself?
4 comments
Response to Reviewer E32H (2/2)
Questions
-
Q1. Thank you for raising this question.
-
Our central claim is that the supervisor both produces professionally sound feedback and helps novices improve through that feedback. We evaluate this claim from two complementary perspectives. The expert perspective assesses whether feedback is professional, objective, comprehensive, clear, and actionable, as examined by the expert and model evaluations in Section 4.3. The novice perspective asks whether learners can use the feedback to improve their judgments and responses, as examined by task performance before and after feedback and by downstream counseling performance.
-
The agreement between human and model ratings noted by the reviewer supports the expert perspective on feedback quality. Yet an explanation that experts understand and endorse may rely on knowledge a novice has not acquired, or may not address the novice's particular misunderstanding. Highly rated feedback may therefore leave a novice unsure how to judge the case or revise a response. Section 4.3 provides evidence for important aspects of feedback quality, but rating agreement alone does not establish its practical benefit to novices.
-
We chose NGR to incorporate the novice perspective directly into the supervisor's optimization objective. We keep the simulated novice model fixed, compare its ethical-violation localization and classification performance before and after reading explanatory feedback, and use the resulting gain as a reward in GRPO. This provides a direct signal for optimizing explanations that novices can understand and use, in line with our claim about instructional guidance.
-
-
Q2. Thank you for raising this question.
-
We additionally evaluated the ETHICSCAFF-trained Qwen3-8B supervisor acting directly as a counselor, using the same six counseling quality metrics as in Table 4. It achieves the highest scores among the compared conditions on EFT-TFS, HTAIS, PSC, and TES, exceeding the trained counselor model PsyDT on those four metrics. This indicates that the trained supervisor also performs well when counseling directly.
-
However, the novice receiving its feedback scores higher than the supervisor counseling directly on two dimensions: 7.22 versus 6.24 on MITI (Motivational Interviewing Treatment Integrity), and 6.06 versus 5.16 on WAI (Working Alliance Inventory). Providing feedback and counseling directly have different objectives and output requirements, and the supervisor shows different strengths in these two roles. A teacher need not earn the highest exam score to explain concepts clearly and help a student learn. This is why we assess a supervisor's teaching value by whether its feedback helps learners improve their performance. These results alone do not establish which of the two tasks is intrinsically harder.
-
Table 4: Performance Comparison Across Counseling Quality Metrics Under Different Counselor Conditions and Feedback Settings. All metrics are higher-is-better.
| Condition | Method | EFT-TFS | HTAIS | MITI | PSC | TES | WAI |
|---|---|---|---|---|---|---|---|
| No Feedback | Novice Counselor | 2.21 | 4.79 | 5.62 | 5.98 | 7.65 | 4.68 |
| No Feedback | Normal Counselor (PsyDT) | 3.01 | 5.34 | 7.35 | 7.14 | 8.73 | 6.00 |
| With Feedback | Baseline Feedback | 2.78 | 5.21 | 6.16 | 6.31 | 7.80 | 5.05 |
| With Feedback | Our Feedback | 2.97 | 5.35 | 7.22 | 7.01 | 8.54 | 6.06 |
| Supervisor Trained on ETHICSCAFF | Qwen3-8B Full | 4.13 | 5.40 | 6.24 | 7.90 | 9.58 | 5.16 |
| Improvement over Novice | Baseline Feedback | +0.57 | +0.42 | +0.54 | +0.33 | +0.15 | +0.37 |
| Improvement over Novice | Our Feedback | +0.76 | +0.56 | +1.60 | +1.03 | +0.89 | +1.38 |
We hope these clarifications address the reviewer's concerns, and we would be grateful if they could be considered in the final assessment.
I thank the authors for the additional clarification, and most of my concerns have been addressed. I will increase my score to 4.
Further Insights on Teaching-Oriented AI Supervision
Thank you for your positive reassessment and thoughtful feedback! Your questions helped sharpen two insights, which we briefly highlight below.
- Finding & Insight 1: The largest feedback gains occur precisely where the supervisor lacks a direct-task advantage.
-
A notable, counterintuitive finding is that the dimensions on which the supervisor helps novices improve the most are not those on which it has the strongest direct-task advantage. When acting directly as a counselor, the supervisor achieves the highest scores among the compared conditions on EFT-TFS, HTAIS, PSC, and TES. On MITI and WAI, however, the novice receiving its feedback scores 7.22 and 6.06, exceeding the supervisor’s direct scores of 6.24 and 5.16. In this sense, the novice surpasses the supervisor in specific dimensions, not independently, with the supervisor’s guidance. These are also the two metrics with the largest gains from our feedback relative to no feedback: +1.60 and +1.38, respectively. In other words, the dimensions on which the supervisor is outperformed in direct counseling are precisely those on which its feedback yields the greatest novice improvement.
-
MITI concerns collaborative interviewing, while WAI concerns agreement on therapeutic goals and tasks and the counselor–client bond. One possible explanation is that producing a strong response and diagnosing, explaining, and guiding the revision of another’s response draw on partially different capabilities. For example, the supervisor may identify overly directive responses or failures to consider the client’s perspective and suggest adjustments that the novice incorporates into its own response. Effective feedback does not perform the task on the learner’s behalf; it enables the learner to perform it better, even, in some dimensions, better than the supervisor itself.
-
This finding suggests that instructional effectiveness cannot be inferred solely from a model’s own task performance. Instructional models require an independent assessment of how much they help learners improve, beyond how well they perform themselves. This motivates NGR: by directly optimizing for performance gains in a frozen simulated novice after feedback, we target not merely professionally sound feedback, but feedback that learners can understand and use.
-
- Finding & Insight 2: Bridging principles and practice: contextualization as a core supervisory capability
-
The challenge in ethics instruction is not merely helping novices know the rules, but helping them understand how those rules apply to a specific interaction. “Respect client autonomy” is a general principle; recognizing when a suggestion becomes a decision made on the client’s behalf requires interpreting the wording, surrounding dialogue, and the client’s expressed preferences. A supervisor’s key role is therefore to bridge abstract principles and concrete practice: explaining what a principle means in the present context and translating it into targeted guidance for revision.
-
ETHICSCAFF scaffolds this bridge through structured feedback format that connects problematic utterances with relevant ethical principles, potential consequences, and revision directions. NGR uses gains in a frozen simulated novice’s violation localization and classification after reading the feedback as an optimization signal. The objective thus goes beyond producing professionally sound explanations of principles to asking whether novices can use those explanations in case-specific judgments. Contextualization is therefore not merely a way of presenting structured feedback, but an instructional capability that connects clinical principles with novice action.
-
In the four-week human study, 36 counseling-psychology master’s students were assigned to three conditions: no feedback, feedback from a base model, or feedback from our supervisor. The group receiving our feedback showed the largest cumulative increase in self-reported counseling self-efficacy, with continued gains in later weeks. These findings provide complementary evidence from actual trainees for the value of this bridging approach: its benefits are reflected not only in simulated novices’ task performance, but also in trainees’ confidence in counseling practice. The implication is that supervisors need not only clinical principles, but also the ability to translate those general ones into context-specific guidance that novices can understand and use.
-
Response to Reviewer E32H (1/2)
We sincerely thank you for your appreciation of our solution as "elegant." Your suggestions help us clarify the method and strengthen the evaluation.
Weaknesses
W1. Key terms and notation are insufficiently explained in the main text.
W2. Why does Task 1 (classification) improve more than Task 2 (localization)?
We hypothesize that the two tasks benefit differently from the training data. The data are generated around 15 mistake categories, which may help the model learn the boundaries between dialogue cues and error categories relatively quickly. Localization, however, requires identifying an average of 2.74 problematic utterances in dialogues averaging 12.08 turns. The same type of mistake can appear in different wording, and its ethical meaning depends on the surrounding context; localization therefore also requires selecting the right utterances and interpreting them in context.
In Table 3, the model before our training already achieves 91.04% recall on localization, suggesting that it usually finds candidate problematic utterances. Further improvement depends on reducing false positives and identifying the exact problematic utterances. We therefore hypothesize that learning category boundaries contributes to the larger gain in classification, while contextual ambiguity and the already high baseline recall constrain the gain in localization.
W2. Deeper discussions
W3. Thank you for suggesting agreement statistics.