- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces MDPSA, a dataset of 8,098 generated multi-turn dialogues built from PENGUIN seeds across seven sensitive domains. It combines 58 response strategies with simulated emotional updates and persona-based jailbreak queries. Experiments cover four Llama/Qwen models and SFT, DPO, PPO, and GRPO. All methods improve the proposed composite score, with DPO performing best. Additional validation includes persona-augmented HarmBench, a second GLM evaluator, and a qualitative human audit.
Critical strengths
The dataset brings vulnerability neglect and persona-induced safety violations into the same conversational setting. This is a useful training focus because models need to recognise individual risk while maintaining their safety boundaries. Multiple candidate responses provide supervision beyond a binary safe/unsafe label.
The improvements in composite scores are consistent across four models and four training methods. On persona-augmented HarmBench, Qwen-7B’s reported jailbreak rate also falls from 13.5% to 6%. These results provide initial evidence that the dataset teaches useful behaviour.
Critical weaknesses
The evaluation does not sufficiently separate improved safety from improved response style. Equation 6 assigns detected cold refusals 0.01, below every response classified as neither refusal nor jailbreak. In Table 5, Llama-8B’s composite score improves substantially while its reported jailbreak rate rises from 0% to 1.5%. Refusing harmful HarmBench requests also does not establish over-refusal. Furthermore, risk sensitivity and personalization can reward the same recognition of user-specific danger. The paper provides no inter-dimension correlations or component ablations to establish what each dimension contributes. Agreement between two GLM judges and an audit of their rationales do not resolve these concerns about what the score measures.
The incremental contribution of the dynamic dataset design remains unclear. PENGUIN already establishes personalized safety, and CURATe evaluates safety-critical personal context in multi-turn dialogue. The experiments compare training algorithms using MDPSA, without a matched comparison against existing safety data or a static-persona/fixed-emotion version of the dataset. Appendix A also restricts emotional updates to becoming more positive. The results therefore do not isolate the benefit of modelling changing emotional states or establish coverage of escalating conversational risk.
Questions for authors
These requests concern the central claims and could materially change my assessment.
Q1: Can you evaluate harmful compliance, appropriate refusal, and supportive redirection separately using independent human judgments? How should the Llama-8B jailbreak increase in Table 5 be interpreted?
Q2: Can you report inter-dimension correlations and ablations establishing the distinct contribution of risk sensitivity, empathy, and personalization? Where these dimensions guide training, individual and leave-one-out variants should be compared under matched budgets and fixed, independent evaluation criteria.
Q3: Do the gains persist against a matched static-persona or fixed-emotion training baseline? What evidence supports the benefit of emotional dynamics when updates are restricted to increasing positivity?
Q4: Is the train/test split grouped by original PENGUIN seed, with all related trajectories and candidate responses kept together?
7 comments
2. Inter-Dimension Correlations and Strategy-Level Isolation
Table 2 reports Pearson correlations across candidate responses and the offline Top-1 reward-ranking ablation, both computed over the 91,096 test candidates (11,387 turns 8). The full 300,088-candidate pool gives essentially the same coefficients (0.809 / 0.845 / 0.854, with – ranked first 98.08% and – 0.99%), so the conclusions do not depend on which pool is used. The ablation re-selects the top-scoring candidate per turn with one dimension removed, which isolates how each dimension gates the trajectory that enters the dataset.
Table 2: Inter-dimension Pearson correlations and Top-1 reward-ranking ablations over the 91,096 test candidates.
| Pool or ablation variant | Risk vs. Empathy | Risk vs. Pers. | Empathy vs. Pers. | Ranked – (%) | Ranked – (%) | Ranked jailbreak (%) | Ranked refusal (%) |
|---|---|---|---|---|---|---|---|
| Full test pool () | 0.807 | 0.845 | 0.854 | n/a | n/a | n/a | n/a |
| Positive strategies (–) | 0.525 | 0.630 | 0.616 | n/a | n/a | n/a | n/a |
| Negative strategies (–) | 0.246 | 0.505 | 0.119 | n/a | n/a | n/a | n/a |
| Negative and universal (–) | 0.555 | 0.713 | 0.604 | n/a | n/a | n/a | n/a |
| Full reward (Risk + Emp + Pers) | n/a | n/a | n/a | 98.18 | 0.90 | 0.84 | 0.08 |
| Ranking without Risk | n/a | n/a | n/a | 98.12 | 0.75 | 1.06 | 0.07 |
| Ranking without Empathy | n/a | n/a | n/a | 96.75 | 2.14 | 0.99 | 0.11 |
| Ranking without Personalisation | n/a | n/a | n/a | 98.42 | 0.90 | 0.61 | 0.08 |
| Ranking on Risk Sensitivity only | n/a | n/a | n/a | 95.01 | 4.02 | 0.69 | 0.27 |
The pooled coefficients are high because the candidate pool is stratified by strategy, and positive strategies (–) score high on all three dimensions at once, so the pooled value is dominated by strategy identity rather than by redundancy among the dimensions. Within individual strategy subsets the correlations fall well below that figure, to 0.25–0.63, and to 0.119 between Empathy and Personalisation inside –. Two further points bear on the reviewer's question about what each dimension contributes. Preference pairs are constructed directly from strategy pools (– versus –) rather than from scalar cutoffs, so negative strategies are strictly restricted to and a polite tone cannot override safety during preference training; and in the ranking ablation the dimensions gate distinct failure channels, with removing Risk Sensitivity raising jailbreak selection from 0.84% to 1.06% and removing Empathy more than doubling negative-strategy selection from 0.90% to 2.14%. The three dimensions are therefore correlated but not interchangeable: removing any one of them lets a different kind of bad response through.
3. Independent Human Audit on Boundary Cases
In an independent blinded audit of 100 boundary cases across four strata of Llama-3-8B outputs, human annotators achieved 85.0% three-way agreement with the automated labels (Cohen's ; Table 4 gives the per-stratum breakdown). Agreement is 86.7% and 90.0% in the two strata that govern training-pool membership and 100% in S4, where all 20 high-scoring responses on adversarial turns were confirmed as genuine safe redirections. The sample is deliberately non-random, drawn from strata where a rubric score could plausibly be inflated by tone, so it characterises behaviour on hard cases rather than average accuracy.
4. Paper Revisions
In the revised manuscript, we will:
- Formally define the composite reward as an interaction-quality metric in Section 4, reporting continuous dimensions, cold refusals and hard safety violations separately.
- Incorporate the decoupled metrics, the HarmBench counts and the correlation and ranking analyses (Tables 1 and 2) into Section 5.2 and Appendix C.
Response to Weakness 2: Incremental Contribution of the Dynamic Dataset Design
We thank the reviewer for these critical questions regarding the role of dynamic personal context, comparisons to related resources and emotional dynamics. Below, we clarify the distinction from prior benchmarks, present conversational depth evidence, and explain the operational mechanics of our dataset.
1. Distinction from PENGUIN and CURATe
- PENGUIN is a single-turn static benchmark focused on evaluating initial persona safety.
- CURATe is a multi-turn dialogue safety benchmark designed for evaluation, without multi-candidate preference data or training supervision.
- MDPSA is a multi-turn training resource with 8 contrastive candidate responses per turn, 58 fine-grained interaction strategies and structured multi-turn trajectories, which supports offline preference optimisation (DPO, PPO, GRPO) over evolving context.
2. Empirical Verification of Multi-Turn Contextual Alignment
To examine whether multi-turn data provides measurable benefits over single-turn modelling, Table 3 stratifies performance by dialogue depth on the 11,387-turn test partition (Turn 1: 2,433 turns; Turns 2–3: 4,188; Turns 4–6: 4,766). Turn 1 is the closest available single-turn proxy, since only the seed query and initial profile are visible.
Table 3: Composite performance stratified by conversational turn depth on the MDPSA test set (11,387 turns).
| Turn depth bin | Turn count () | Share (%) | Llama-3-8B Naive | Llama-3-8B DPO | Qwen-2.5-7B Naive | Qwen-2.5-7B DPO |
|---|---|---|---|---|---|---|
| Turn 1 (single-turn proxy) | 2,433 | 21.37% | 0.7210 | 0.7561 | 0.5235 | 0.7594 |
| Turns 2–3 | 4,188 | 36.78% | 0.5501 | 0.7746 | 0.6555 | 0.7741 |
| Turns 4–6 (deep context) | 4,766 | 41.85% | 0.4621 | 0.7710 | 0.6562 | 0.7664 |
| All turns | 11,387 | 100.0% | 0.5498 | 0.7692 | 0.6276 | 0.7677 |
The baselines fail in opposite directions: Llama-3-8B drops by 0.2589 from Turn 1 to Turns 4–6, while Qwen-2.5-7B starts from 0.5235 at Turn 1 and only partially recovers to 0.6555/0.6562. MDPSA-aligned models hold a depth-invariant 0.76–0.77 band across every bin. Following the turn-position diagnostics of Ivry and Watanabe [1], this confirms that single-turn capability does not automatically transfer to accumulated context and that MDPSA alignment achieves depth-invariant stability; we will also include a matched single-turn training control in the revised manuscript to further isolate turn-level training effects.
3. Information Flow and Coverage of Escalating Risk
- Simulator latent state vs. model input. During SFT/DPO training and testing, target models receive only the initial static profile (including baseline emotion ) and the dialogue history . The updated state is strictly an internal latent variable of the offline data-generation pipeline (GLM-4.7), where it conditions trajectory and candidate synthesis as well as subsequent query generation, so the target model is never trained on or conditioned on explicit labels.
- Coverage of escalating risk. Monotonic positive emotional updates simulate supportive counselling de-escalation, in which positive interventions (–) stabilise distress, while acute crises and risk escalation are introduced through the adversarial query generator (Prompt 14), which injects high-risk prompts in 24.0% of turns and 66.0% of dialogues. Table 7 of the manuscript, Turn 3, illustrates how acute stress triggers a high-risk inquiry about leftover prescription painkillers, testing boundary adherence under escalating pressure. Section 4.2 and Appendix A will present the two mechanisms as complementary.
4. Paper Revisions
In the revised manuscript, we will:
- Include the depth-stability analysis (Table 3) in Section 5.2 as a depth diagnostic, with an explicit statement that a matched single-turn control is the remaining causal test.
- Clarify in Section 3.1 that operates strictly within the offline dataset-construction pipeline, rather than serving as an explicit training label for target models.
- State in Section 4.2 and Appendix A that supportive de-escalation () and adversarial risk escalation () are complementary simulator mechanisms.
Response to Weakness 1: Separating Safety from Communicative Style
We thank the reviewer for highlighting these methodological concerns regarding metric decoupling, correlation among dimensions and evaluation interpretability. Below, we provide decoupled evaluations, report inter-dimension correlations, and clarify our HarmBench findings.
1. Decoupled Safety Metrics and a Benign Control
We agree that the composite reward in Eq. (6) reflects overall interaction quality and cannot stand in for an isolated certificate of safety. Table 1 reports the three continuous dimensions together with the discrete detector rates (e1 cold refusal and e2 jailbreak) on the 11,387-turn MDPSA test set, and the out-of-domain results on the 200 persona-augmented HarmBench behaviours.
Table 1: Decoupled evaluation dimensions and detector rates on the MDPSA test set (11,387 turns) and HarmBench (200 behaviours).
Benign refusalcolumn of Table 1 is the same e1 detector restricted to the 8,589 non-adversarial MDPSA test turns; there cold refusals fall from 6.35% to 0.00% for Llama-8B. Because these requests are safe and the correct behaviour is to help, this is where over-refusal can actually be assessed, and it is eliminated without any increase in hard violations on those turns.