acceptodds
Under review as a conference paper at ICLR 2027

Empathetic yet Guarded: Dynamic Personalized Safety Alignment in Multi-Turn Conversations

Abstract

While current safety alignment methods have significantly improved the general harmlessness of Large Language Models (LLMs), they predominantly enforce uniform safety boundaries and overlook the complex, individualized contexts of human interactions. This uniform approach often leads to critical failures. For example, models may provide advice that is universally benign but harmful to specific vulnerable users. Alternatively, they may excessively cater to user personas, risking a bypass of general safety rules. To bridge this critical gap, we introduce MDPSA, a novel Multi-Turn Dialogue Dataset for Personalized Safety Alignment. Our dataset spans seven high-risk domains and comprises 58 fine-grained response strategies, uniquely capturing the dynamic evolution of user emotional states and complex, context-driven jailbreak scenarios. To validate its effectiveness, we conduct comprehensive experiments across various post-training and alignment paradigms. Empirical results demonstrate that while standard models struggle heavily with personalized safety, training on MDPSA effectively empowers them to resolve the tension between general safety and individual adaptability, achieving significant performance gains. Our work formally establishes a robust data foundation and empirical baseline for building safe, empathetic, and context-aware LLMs.

Primary area:
datasets and benchmarks
Keywords:
Personalized Safety Alignment, Individual safety, General Safety, PSA, Multi-Turn Dialogue

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_a2ctOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces MDPSA, a dataset of 8,098 generated multi-turn dialogues built from PENGUIN seeds across seven sensitive domains. It combines 58 response strategies with simulated emotional updates and persona-based jailbreak queries. Experiments cover four Llama/Qwen models and SFT, DPO, PPO, and GRPO. All methods improve the proposed composite score, with DPO performing best. Additional validation includes persona-augmented HarmBench, a second GLM evaluator, and a qualitative human audit.

Critical strengths

The dataset brings vulnerability neglect and persona-induced safety violations into the same conversational setting. This is a useful training focus because models need to recognise individual risk while maintaining their safety boundaries. Multiple candidate responses provide supervision beyond a binary safe/unsafe label.

The improvements in composite scores are consistent across four models and four training methods. On persona-augmented HarmBench, Qwen-7B’s reported jailbreak rate also falls from 13.5% to 6%. These results provide initial evidence that the dataset teaches useful behaviour.

Critical weaknesses

The evaluation does not sufficiently separate improved safety from improved response style. Equation 6 assigns detected cold refusals 0.01, below every response classified as neither refusal nor jailbreak. In Table 5, Llama-8B’s composite score improves substantially while its reported jailbreak rate rises from 0% to 1.5%. Refusing harmful HarmBench requests also does not establish over-refusal. Furthermore, risk sensitivity and personalization can reward the same recognition of user-specific danger. The paper provides no inter-dimension correlations or component ablations to establish what each dimension contributes. Agreement between two GLM judges and an audit of their rationales do not resolve these concerns about what the score measures.

The incremental contribution of the dynamic dataset design remains unclear. PENGUIN already establishes personalized safety, and CURATe evaluates safety-critical personal context in multi-turn dialogue. The experiments compare training algorithms using MDPSA, without a matched comparison against existing safety data or a static-persona/fixed-emotion version of the dataset. Appendix A also restricts emotional updates to becoming more positive. The results therefore do not isolate the benefit of modelling changing emotional states or establish coverage of escalating conversational risk.

Questions for authors

These requests concern the central claims and could materially change my assessment.

Q1: Can you evaluate harmful compliance, appropriate refusal, and supportive redirection separately using independent human judgments? How should the Llama-8B jailbreak increase in Table 5 be interpreted?

Q2: Can you report inter-dimension correlations and ablations establishing the distinct contribution of risk sensitivity, empathy, and personalization? Where these dimensions guide training, individual and leave-one-out variants should be compared under matched budgets and fixed, independent evaluation criteria.

Q3: Do the gains persist against a matched static-persona or fixed-emotion training baseline? What evidence supports the benefit of emotional dynamics when updates are restricted to increasing positivity?

Q4: Is the train/test split grouped by original PENGUIN seed, with all related trajectories and candidate responses kept together?

7 comments

Response to Weakness 1: Separating Safety from Communicative Style

We thank the reviewer for highlighting these methodological concerns regarding metric decoupling, correlation among dimensions and evaluation interpretability. Below, we provide decoupled evaluations, report inter-dimension correlations, and clarify our HarmBench findings.

1. Decoupled Safety Metrics and a Benign Control

We agree that the composite reward in Eq. (6) reflects overall interaction quality and cannot stand in for an isolated certificate of safety. Table 1 reports the three continuous dimensions together with the discrete detector rates (e1e_1 cold refusal and e2e_2 jailbreak) on the 11,387-turn MDPSA test set, and the out-of-domain results on the 200 persona-augmented HarmBench behaviours.

Table 1: Decoupled evaluation dimensions and detector rates on the MDPSA test set (11,387 turns) and HarmBench (200 behaviours).

BenchmarkModelMethodRisk Sens. ↑\uparrowEmpathy ↑\uparrowPers. Match ↑\uparrowJailbreak / e2e_2 (%) ↓\downarrowCold refusal / e1e_1 (%) ↓\downarrowBenign refusal (%) ↓\downarrow
MDPSA test setLlama-3-8BNaive2.802.852.580.0112.736.35
MDPSA test setLlama-3-8BDPO3.444.223.880.480.000.00
MDPSA test setQwen-2.5-7BNaive3.043.422.951.390.000.00
MDPSA test setQwen-2.5-7BDPO3.574.113.830.410.000.00
HarmBench (200)Llama-3-8BNaive0.420.200.150.00 (0/200)89.50 (179/200)n/a
HarmBench (200)Llama-3-8BDPO3.814.204.591.50 (3/200)0.00 (0/200)n/a
HarmBench (200)Qwen-2.5-7BNaive3.543.113.1613.50 (27/200)0.00 (0/200)n/a
HarmBench (200)Qwen-2.5-7BDPO3.823.714.166.00 (12/200)0.00 (0/200)n/a
  • HarmBench refusals are not over-refusal. We agree with the reviewer's terminology and state the distinction explicitly. Over-refusal in the safety literature denotes a false refusal of a benign request; declining a harmful HarmBench request is desirable and is not over-refusal. Our e1e_1 detector (Appendix B, Prompt 15) counts only cold refusals, a bare rejection that offers no explanation, empathy or alternative. A response that declines the harmful request while offering empathetic support and lawful alternatives is a Safe Redirection. By construction such a response is classified as false by both binary detectors (e1=Falsee_1 = \text{False}, e2=Falsee_2 = \text{False}, the latter under e2e_2's "Refusal with Legal Alternative" clause), so it is not penalised at 0.01 and instead enters the e4e_4 rubric, where it earns a composite reward in the 0.80–1.00 range. Neither detector assigns it a zero score; the binary flags and the reward are different quantities.
  • What the Llama-8B shift means. Because Naive Llama-8B already sits at a 0.00% (0/200) floor on e2e _ {2} by defaulting to bare rejection on 89.50% of prompts, the goal on this model is to resolve Vulnerability Neglect (Failure Mode 1 in Figure 1) without compromising its universal safety boundary. After DPO training, the model still safely declines 98.50% (197/200) of HarmBench requests: the slight difference in e2e _ {2} flags (0/200 vs. 3/200, 1.50%) is statistically indistinguishable from zero (two-sided Fisher exact p=0.248p = 0.248), while Risk Sensitivity improves substantially from 0.42 to 3.81 (Empathy 0.20 →\to 4.20, Personalisation 0.15 →\to 4.59) and cold refusals drop from 89.50% to 0.00%. Thus, Llama-8B preserves its universal safety boundary while shifting from blunt rejection to context-aware safe redirection.
  • Genuine over-refusal is measured on benign turns. The Benign refusal column of Table 1 is the same e1e_1 detector restricted to the 8,589 non-adversarial MDPSA test turns; there cold refusals fall from 6.35% to 0.00% for Llama-8B. Because these requests are safe and the correct behaviour is to help, this is where over-refusal can actually be assessed, and it is eliminated without any increase in hard violations on those turns.
  • Qwen-2.5-7B. DPO cuts the HarmBench jailbreak rate by more than half, from 27/200 (13.50%) to 12/200 (6.00%), two-sided Fisher exact p=0.017p = 0.017. This is the one cell where the out-of-domain evidence supports a genuine hard-boundary improvement.

2. Inter-Dimension Correlations and Strategy-Level Isolation

Table 2 reports Pearson correlations across candidate responses and the offline Top-1 reward-ranking ablation, both computed over the 91,096 test candidates (11,387 turns ×\times 8). The full 300,088-candidate pool gives essentially the same coefficients (0.809 / 0.845 / 0.854, with s1s_1–s4s_4 ranked first 98.08% and s5s_5–s8s_8 0.99%), so the conclusions do not depend on which pool is used. The ablation re-selects the top-scoring candidate per turn with one dimension removed, which isolates how each dimension gates the trajectory that enters the dataset.

Table 2: Inter-dimension Pearson correlations and Top-1 reward-ranking ablations over the 91,096 test candidates.

Pool or ablation variantRisk vs. EmpathyRisk vs. Pers.Empathy vs. Pers.Ranked s1s_1–s4s_4 (%)Ranked s5s_5–s8s_8 (%)Ranked s10s_{10} jailbreak (%)Ranked s9s_9 refusal (%)
Full test pool (N=91,096N=91{,}096)0.8070.8450.854n/an/an/an/a
Positive strategies (s1s_1–s4s_4)0.5250.6300.616n/an/an/an/a
Negative strategies (s5s_5–s8s_8)0.2460.5050.119n/an/an/an/a
Negative and universal (s5s_5–s10s_{10})0.5550.7130.604n/an/an/an/a
Full reward (Risk + Emp + Pers)n/an/an/a98.180.900.840.08
Ranking without Riskn/an/an/a98.120.751.060.07
Ranking without Empathyn/an/an/a96.752.140.990.11
Ranking without Personalisationn/an/an/a98.420.900.610.08
Ranking on Risk Sensitivity onlyn/an/an/a95.014.020.690.27

The pooled coefficients are high because the candidate pool is stratified by strategy, and positive strategies (s1s_1–s4s_4) score high on all three dimensions at once, so the pooled value is dominated by strategy identity rather than by redundancy among the dimensions. Within individual strategy subsets the correlations fall well below that figure, to 0.25–0.63, and to 0.119 between Empathy and Personalisation inside s5s_5–s8s_8. Two further points bear on the reviewer's question about what each dimension contributes. Preference pairs are constructed directly from strategy pools (s1s_1–s4∈Rt+s_4 \in \mathcal R_t^+ versus s5s_5–s10∈Rt−s_{10} \in \mathcal R_t^-) rather than from scalar cutoffs, so negative strategies are strictly restricted to Rt−\mathcal R_t^- and a polite tone cannot override safety during preference training; and in the ranking ablation the dimensions gate distinct failure channels, with removing Risk Sensitivity raising jailbreak selection from 0.84% to 1.06% and removing Empathy more than doubling negative-strategy selection from 0.90% to 2.14%. The three dimensions are therefore correlated but not interchangeable: removing any one of them lets a different kind of bad response through.

3. Independent Human Audit on Boundary Cases

In an independent blinded audit of 100 boundary cases across four strata of Llama-3-8B outputs, human annotators achieved 85.0% three-way agreement with the automated labels (Cohen's κ=0.525\kappa = 0.525; Table 4 gives the per-stratum breakdown). Agreement is 86.7% and 90.0% in the two strata that govern training-pool membership and 100% in S4, where all 20 high-scoring responses on adversarial turns were confirmed as genuine safe redirections. The sample is deliberately non-random, drawn from strata where a rubric score could plausibly be inflated by tone, so it characterises behaviour on hard cases rather than average accuracy.

4. Paper Revisions

In the revised manuscript, we will:

  • Formally define the composite reward as an interaction-quality metric in Section 4, reporting continuous dimensions, cold refusals and hard safety violations separately.
  • Incorporate the decoupled metrics, the HarmBench counts and the correlation and ranking analyses (Tables 1 and 2) into Section 5.2 and Appendix C.

Response to Weakness 2: Incremental Contribution of the Dynamic Dataset Design

We thank the reviewer for these critical questions regarding the role of dynamic personal context, comparisons to related resources and emotional dynamics. Below, we clarify the distinction from prior benchmarks, present conversational depth evidence, and explain the operational mechanics of our dataset.

1. Distinction from PENGUIN and CURATe

  • PENGUIN is a single-turn static benchmark focused on evaluating initial persona safety.
  • CURATe is a multi-turn dialogue safety benchmark designed for evaluation, without multi-candidate preference data or training supervision.
  • MDPSA is a multi-turn training resource with 8 contrastive candidate responses per turn, 58 fine-grained interaction strategies and structured multi-turn trajectories, which supports offline preference optimisation (DPO, PPO, GRPO) over evolving context.

2. Empirical Verification of Multi-Turn Contextual Alignment

To examine whether multi-turn data provides measurable benefits over single-turn modelling, Table 3 stratifies performance by dialogue depth on the 11,387-turn test partition (Turn 1: 2,433 turns; Turns 2–3: 4,188; Turns 4–6: 4,766). Turn 1 is the closest available single-turn proxy, since only the seed query and initial profile are visible.

Table 3: Composite performance stratified by conversational turn depth on the MDPSA test set (11,387 turns).

Turn depth binTurn count (NN)Share (%)Llama-3-8B NaiveLlama-3-8B DPOQwen-2.5-7B NaiveQwen-2.5-7B DPO
Turn 1 (single-turn proxy)2,43321.37%0.72100.75610.52350.7594
Turns 2–34,18836.78%0.55010.77460.65550.7741
Turns 4–6 (deep context)4,76641.85%0.46210.77100.65620.7664
All turns11,387100.0%0.54980.76920.62760.7677

The baselines fail in opposite directions: Llama-3-8B drops by 0.2589 from Turn 1 to Turns 4–6, while Qwen-2.5-7B starts from 0.5235 at Turn 1 and only partially recovers to 0.6555/0.6562. MDPSA-aligned models hold a depth-invariant 0.76–0.77 band across every bin. Following the turn-position diagnostics of Ivry and Watanabe [1], this confirms that single-turn capability does not automatically transfer to accumulated context and that MDPSA alignment achieves depth-invariant stability; we will also include a matched single-turn training control in the revised manuscript to further isolate turn-level training effects.

3. Information Flow and Coverage of Escalating Risk

  • Simulator latent state vs. model input. During SFT/DPO training and testing, target models receive only the initial static profile U0\mathcal{U}_0 (including baseline emotion E0E_0) and the dialogue history Ht\mathcal{H}_t. The updated state EtE_t is strictly an internal latent variable of the offline data-generation pipeline (GLM-4.7), where it conditions trajectory and candidate synthesis as well as subsequent query generation, so the target model is never trained on or conditioned on explicit EtE_t labels.
  • Coverage of escalating risk. Monotonic positive emotional updates simulate supportive counselling de-escalation, in which positive interventions (s1s_1–s4s_4) stabilise distress, while acute crises and risk escalation are introduced through the adversarial query generator IjailI_{\text{jail}} (Prompt 14), which injects high-risk prompts in 24.0% of turns and 66.0% of dialogues. Table 7 of the manuscript, Turn 3, illustrates how acute stress triggers a high-risk inquiry about leftover prescription painkillers, testing boundary adherence under escalating pressure. Section 4.2 and Appendix A will present the two mechanisms as complementary.

4. Paper Revisions

In the revised manuscript, we will:

  • Include the depth-stability analysis (Table 3) in Section 5.2 as a depth diagnostic, with an explicit statement that a matched single-turn control is the remaining causal test.
  • Clarify in Section 3.1 that EtE_t operates strictly within the offline dataset-construction pipeline, rather than serving as an explicit training label for target models.
  • State in Section 4.2 and Appendix A that supportive de-escalation (EtE_t) and adversarial risk escalation (IjailI_{\text{jail}}) are complementary simulator mechanisms.
Reviewer_hiVHOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper constructs multi-turn dialogue and preference learning data and utilizes then to train LLMs to follow general safety rules and appropriately respond to user-specific vulnerabilities.

Critical strengths

Constructs a multi-turn dataset that addresses two conflicting failure modes in personalized safety and shows consistent performance improvements across various training methods.

Critical weaknesses
  1. There is no comparison with single-turn training, so the independent contribution and necessity of multi-turn data are not sufficiently verified. In addition, the user's emotional state is designed to be updated only in a positive direction, which may not fully capture the diverse emotional shifts that occur in real-world interactions.
  2. It is not sufficiently explained whether the current reward design reflects real-world personalized safety issues. An inappropriate response that doesn't sufficiently consider user-specific risks could receive a higher reward than a safe refusal if its empathy and personalization scores are high.
Questions for authors
  1. Is there any additional evidence or analysis showing the benefits of multi-turn dataset over single-turn training?
  2. Would the reported performance improvements remain consistent even if using evaluators from other model families other than the GLM family?
6 comments

Response to weakness 1: Multi-Turn Depth Diagnostics and Emotional Shifts

We thank the reviewer for raising these constructive points regarding the role of multi-turn modeling and the coverage of emotional transitions. Below, we present conversational-depth diagnostics on the MDPSA test set, clarify the role of emotional state updates, and explain how negative emotional shifts and escalating user risk are represented in the dialogue trajectories.

1. Conversational Depth Diagnostics

To examine how behaviour changes across dialogue depth, Table 1 breaks down model performance across dialogue depths on the 11,387-turn test partition (2,433 complete sessions). Turn 1 is the closest available single-turn proxy, since only the user profile and the initial query are visible.

Table 1: Composite performance stratified by conversational turn depth on the MDPSA test set (11,387 turns).

Turn depth binTurn count (NN)Share (%)Llama-3-8B NaiveLlama-3-8B DPOQwen-2.5-7B NaiveQwen-2.5-7B DPO
Turn 12,43321.37%0.72100.75610.52350.7594
Turns 2–34,18836.78%0.55010.77460.65550.7741
Turns 4–64,76641.85%0.46210.77100.65620.7664
All turns11,387100.0%0.54980.76920.62760.7677

As shown in Table 1, the two naive baselines exhibit two opposite pathologies, which a single "degradation" narrative would miss.

  • Late-turn fatigue (Llama-3-8B). At Turn 1 the naive model is already reasonably aligned (0.7210), trailing DPO by only 0.0351. Under accumulated context (Turns 4–6, 41.85% of all test turns) it falls to 0.4621, a drop of 0.2589.
  • Early-turn coldness (Qwen-2.5-7B). At Turn 1, where only the seed query and initial profile are visible, the naive model scores 0.5235, reflecting terse or blunt single-turn behaviour. It recovers to 0.6555/0.6562 once the user restates the problem, which is above the naive Llama baseline at the same depths (0.5501/0.4621) but below the 0.76–0.77 band DPO sustains.
  • Depth-invariant stability under MDPSA. DPO alignment removes both pathologies at once. Llama-3-8B stays within 0.7561–0.7746 and Qwen-2.5-7B within 0.7594–0.7741 across every bin, a uniform 0.76–0.77 band with no depth-dependent drift in either direction.

This is a diagnostic analysis rather than a causal single-versus-multi-turn ablation. What it establishes is that single-turn capability, as approximated at Turn 1, does not transfer to longer contexts, and that MDPSA alignment is depth-invariant. Single-turn benchmarks such as PENGUIN assess initial persona awareness but cannot observe how behaviour evolves as context accumulates.

2. Emotional State Updates and the Representation of Escalating Risk

  • Simulator latent state vs. model inputs. The updated emotional state EtE_t is strictly an internal latent variable of the offline data-generation pipeline (GLM-4.7), where it conditions trajectory and candidate synthesis as well as subsequent query generation. During SFT, DPO and inference, target models never receive explicit EtE_t updates as input prompts or supervision targets. They receive only the initial static profile U0\mathcal{U}_0 (including baseline emotion E0E_0) and the dialogue history Ht\mathcal{H}_t, and must infer evolving psychological states from the conversation.
  • Design goal of monotonic transitions. In psychological counselling and safety de-escalation, effective intervention strategies (s1s_1–s4s_4) aim to gradually stabilise the user and reduce acute distress. The positive emotional update was designed to model successful de-escalation trajectories.
  • Why monotonicity does not constrain the model. Because EtE_t is never observed by the target model, the direction of the simulator-side update affects only the distribution of next-turn queries; it does not restrict the model's behaviour space or the set of trajectories it is trained to handle. The monotonicity is a data-generation design choice, not a model-level limitation, and Appendix A of the revised manuscript states this explicitly.
  • Coverage of escalating distress and adversarial shifts. Negative emotional shifts, acute crises and risk escalations are introduced through the adversarial query generator IjailI_{\text{jail}} (Prompt 14), which injects high-risk, desperate requests in 24.0% of turns and 66.0% of complete dialogues. Table 7 of the manuscript, Turn 3, gives a typical example: the user's extreme exhaustion and financial stress trigger a risky inquiry about leftover prescription painkillers. The dataset therefore actively models conversational conflict, desperation and boundary testing, and Section 4.2 and Appendix A will present EtE_t and IjailI_{\text{jail}} as the two complementary mechanisms of the simulator.

3. Paper Revisions

In the revised manuscript, we will:

  • Include the turn-depth stability analysis (Table 1) in Section 5.2 as a depth diagnostic, with the scope statement above.
  • Clarify in Section 3.1 that EtE_t operates strictly within the offline dataset-construction pipeline, rather than serving as an explicit training label for target models.
  • State in Section 4.2 and Appendix A that supportive de-escalation (EtE_t) and adversarial risk escalation (IjailI_{\text{jail}}) are complementary simulator mechanisms.

Response to weakness 2: Reward Design and Personalized Safety Reflectiveness

We thank the reviewer for raising this important concern regarding whether the reward function accurately reflects personalised safety trade-offs. Our framework distinguishes cold refusals from supportive redirections, and its training supervision is separated from the continuous scalar so that inappropriate, surface-empathetic responses cannot be learned as positive targets.

1. Cold Refusals (0.01) versus Safe Redirections

The fixed score of 0.01 in Eq. (6) applies strictly to cold refusals (e1=Truee_1 = \text{True}), which are bare rejections offering no constructive help, empathy or alternatives, such as "I cannot assist with this request". In contrast, the appropriate response in personalised counselling is a Safe Redirection: a reply that explicitly declines the user's hazardous or harmful premise while offering empathetic support and a constructive, lawful alternative. Under our rubric:

  • A safe redirection is classified as false by both binary detectors (e1=Falsee_1 = \text{False} and e2=Falsee_2 = \text{False}).
  • It then enters the multidimensional rubric e4e_4, where high risk sensitivity, supportive empathy and personalisation match yield composite rewards between 0.80 and 1.00 (Table 7 of the manuscript, Turn 4, receives 0.8000).
  • An appropriate safe refusal therefore receives a far higher reward than an inappropriate or risk-insensitive response, and the two are never in competition within Eq. (6).

2. Training-Level Separation Between Strategy Pools

To ensure that surface empathy cannot compensate for poor safety during post-training, MDPSA separates the construction of preference pairs from the continuous scalar sum.

  • Pool separation in SFT and DPO. As defined in Section 4.2, Step 2, positive training targets are drawn exclusively from the positive strategy pool Rt+\mathcal{R}_t^+ (s1s_1–s4s_4), whereas candidates generated under negative strategies (s5s_5–s8s_8, such as Toxic Positivity or Dismissiveness) are assigned to the non-preferred pool Rt−\mathcal{R}_t^- by construction rather than by thresholding the scalar.
  • Consequence. Negative strategies therefore have a 0% probability of being selected as positive SFT or DPO targets, whatever reward they obtain. Even a superficially empathetic inappropriate response, such as Turn 3 of Table 7 of the manuscript, which endorses medication misuse in an encouraging tone and scores 0.267, serves strictly as a rejected candidate in the preference pairs (Rt+,Rt−)(\mathcal{R}_t^+, \mathcal{R}_t^-). The reward value and the training label are two different objects, and a high style score never converts a negative-strategy response into a positive target.

3. Empirical Validation of the Scalar Reward and Boundary Census

The separation above holds by construction, but the reviewer's concern also applies to the scalar formula taken on its own, which governs trajectory construction and reward-weighted optimisation in PPO/GRPO. We therefore tested Eq. (6) in isolation by asking, at each test turn, which of the eight candidates receives the highest scalar score, arg⁡max⁡k∈{1…8}R(rt(k))\arg\max_{k \in \{1 \dots 8\}} R(r_t^{(k)}), over all 91,096 test candidates (11,387 turns ×\times 8). Separately, we ran a full census of the 11,387 Llama-3-8B DPO test outputs for the specific boundary pattern in which high style scores are combined with minimal risk credit, defined as Risk=1\text{Risk} = 1 with Empathy≥4\text{Empathy} \ge 4 and Personalization≥4\text{Personalization} \ge 4.

Table 2: Top-1 scalar reward ranking over 91,096 test candidates and the boundary-pattern census over 11,387 test turns.

AnalysisCondition or strategy groupShare / frequency
Top-1 scalar reward ranking, arg⁡max⁡kR(rt(k))\arg\max_k R(r_t^{(k)})Positive strategies (s1s_1–s4s_4) ranked first98.18%
Top-1 scalar reward ranking, arg⁡max⁡kR(rt(k))\arg\max_k R(r_t^{(k)})Negative strategies (s5s_5–s8s_8) ranked first0.90%
Top-1 scalar reward ranking, arg⁡max⁡kR(rt(k))\arg\max_k R(r_t^{(k)})Jailbreak responses (s10s_{10}) ranked first0.84%
Top-1 scalar reward ranking, arg⁡max⁡kR(rt(k))\arg\max_k R(r_t^{(k)})Cold refusals (s9s_9) ranked first0.08%
Boundary-pattern census, 11,387 turnsRisk =1= 1, Empathy ≥4\ge 4, Personalization ≥4\ge 40.69% (79 turns)

As shown in Table 2, even when candidates are ranked purely by the scalar in Eq. (6), a positive strategy (s1s_1–s4s_4) receives the highest score in 98.18% of turns, while a negative strategy (s5s_5–s8s_8) does so in 0.90% of turns. These are reward-ranking shares and not training-sampling rates: the responses that reach Rt+\mathcal{R}_t^+ are fixed by strategy identity, so the 0.90% figure describes how often the scalar formula alone would prefer a negative strategy, and those turns remain excluded from positive supervision regardless.

Reviewer_g27oOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces MDPSA, a synthetic multi-turn dialogue dataset for personalized safety alignment. Starting from PENGUIN seeds, the authors construct 8098 dialogues across seven sensitive domains, incorporating user profiles, probabilistic emotional-state updates, and persona-conditioned adversarial queries. The dataset uses 58 response strategies and provides eight candidate responses per turn, together with evaluations of risk sensitivity, emotional empathy, and personalization.

The authors evaluate SFT, DPO, PPO, and GRPO on four Llama/Qwen model configurations. Training improves the proposed composite score across these configurations, with DPO achieving the strongest results. Additional validation includes evaluation on 200 persona-augmented HarmBench behaviors, comparison between two GLM evaluators, and a qualitative human audit of 100 evaluation instances. The intended contribution is a dataset and empirical foundation for adapting responses to individual vulnerabilities while maintaining general safety boundaries.

Critical strengths

1. The paper addresses an important problem through a potentially useful training resource.

The distinction between ignoring a user's vulnerability and over-accommodating a risky persona is well motivated. Figure 1 makes the practical importance of context-sensitive safety clear. The dataset also represents a useful range of response failures, including dismissiveness, toxic positivity, and inappropriate attribution, alongside explicit harmful assistance.

The multi-candidate structure is a meaningful feature: retaining alternative responses, strategy labels, and scores can support several training objectives and enable analysis of preferences between different response behaviors. This gives the work potential value as a dataset contribution.

2. The experiments show consistent preliminary gains and include useful attempts at external validation.

Table 1 reports improvements across all four evaluated model configurations and all four training methods. The average composite score increases from 0.58 for the naive models to 0.7641 with DPO. The external evaluation provides a particularly encouraging result for Qwen-7B: its detected jailbreak rate on persona-augmented HarmBench decreases from 13.5% to 6% (Table 5).

The authors also make constructive efforts to examine evaluator reliability through a second GLM evaluator and a human audit. These checks strengthen the submission, even though their current form leaves the measurement questions below unresolved.

Critical weaknesses

1. The evaluation does not yet establish that the reported gains correspond to improved contextual safety.

The main difficulty is interpreting the composite reward as evidence of safety. Equation (6) assigns 0 to detected jailbreaks, 0.01 to detected refusals, and at least 0.2 to all remaining responses. Appendix B clarifies that the refusal detector counts only cold refusals without useful support or alternatives (supportive refusals are exempt). This distinction is reasonable for assessing response quality. Nevertheless, the scoring rule makes the validity of the safety detector crucial: if a harmful response escapes that detector, empathy and personalization can compensate for poor risk sensitivity. For illustration, an unflagged response scored 1 for risk sensitivity and 5 for both remaining dimensions would receive (11/15, roughly 0.733). This is a consequence of the written rule, rather than a claim that this exact example was observed.

The published safety-detection prompt raises a related concern. Appendix B’s explicitly includes severe medical hazards, but also excludes "Bad advice that doesn't violate laws" and states that safety boundaries require specific harmful illegal content. These instructions need reconciliation with the paper's central motivation: advice may be lawful yet dangerous for a particular vulnerable person. Moreover, the published template shows only a response-content field, without explicit persona or history fields. Please clarify what contextual information the detector actually receives.

Table 7 provides a concrete case requiring adjudication. At turn 3, the response appears to endorse the user's proposed use of leftover prescription painkillers for energy, yet receives reward 0.27 and the dialogue continues. Its assignment to a negative strategy is understandable, but the nonzero reward appears difficult to reconcile with the appendix's stated prescription-drug-abuse boundary. Clarifying this example would help establish how the safety labels operate in practice.

The external results further demonstrate why safety and response quality should be reported separately. For Llama-8B in Table 5, the composite score increases from 0.0582 to 0.8402, while the detected jailbreak rate changes from 0% to 1.5% and cold refusals fall from 89.5% to 0%. Three detected violations among 200 examples do not establish a statistically reliable safety regression. However, the large composite increase does not by itself demonstrate improved general safety. Likewise, this HarmBench experiment establishes reduced cold refusal on harmful requests. Evaluating reduced refusal of benign requests requires corresponding benign controls.

Finally, the evaluator-validation results do not fully resolve these concerns. Table 6's overall difference of 0.042 compares aggregate means, rather than judgments on individual examples. It partly reflects cancellation: personalization increases by 0.271 between judges, while risk sensitivity and empathy decrease. Preserved aggregate rankings are useful evidence, but do not establish accuracy on safety-boundary cases. The 100-instance human audit examines whether judge rationales are logical and consistent with the rubric. It does not report independent blinded response ratings or quantitative safety-classification errors.

How to strengthen the paper:

Report contextual harmfulness separately from empathy, personalization, and cold refusal. Validate the safety judgments against independent human annotations of raw responses and their relevant context, including examples of lawful but personally dangerous advice. Report disagreement/error counts and uncertainty for the main safety comparisons. A focused audit of representative and boundary cases would directly test whether the improvements persist beyond the preferred response style.

As optional context for this validation, the authors may find Schwinn et al., A Coin Flip for Safety: LLM Judges Fail to Reliably Measure Adversarial Robustness (ICML 2026) useful for its analysis of safety-judge reliability under distribution shifts. Ivry and Watanabe, LALM-as-a-Judge: Benchmarking Large Audio-Language Models for Safety Evaluation in Multi-Turn Spoken Dialogues (ICML 2026) provides complementary diagnostics of sensitivity, severity ordering, and turn-position effects. The latter concerns spoken dialogues, so its relevance here is methodological. These are optional pointers for the evaluation discussion.

2. The experiments do not isolate the contribution of dynamic personalization.

The central distinguishing feature of MDPSA is its evolving multi-turn personal context. However, Table 1 compares different training methods on MDPSA against naive models. These comparisons establish that training on the dataset improves the reported score, but do not isolate the benefit of emotional-state changes or temporal contextual adaptation. A matched static-persona/no-emotion-update control, or a suitable comparison with the PENGUIN seed resource, is missing. Consequently, the results leave open whether the gains primarily come from dynamic context use, generally empathetic supervision, or other properties of the generated preferences.

The implemented emotional dynamics are also narrower than the main framing suggests. Appendix A's emotional-state update prompt explicitly requires the next state to be more positive than the current state (p. 22, lines 1007-1020). Thus, the specified transition mechanism does not cover worsening distress or reversals. The update frequencies in Table 4 verify the presence of the prescribed transitions, but do not establish that trained models respond appropriately to changing safety-relevant states.

The information available to the target model also matters. Section 3.1 describes persona information as gradually embedded in the conversation, while Equation (4) supplies the generator with the full current structured context. The evaluation protocol should clarify whether the target models receive the full profile and updated emotional state explicitly, or must infer relevant information from what has been disclosed in the dialogue.

How to strengthen the paper: A focused comparison on one model with DPO would be informative: full MDPSA versus a matched variant with fixed emotional states, using comparable training data volume and optimization. Complement this with a small set of matched test cases that hold the request fixed while changing a safety-relevant vulnerability or preceding history, and examine whether responses change appropriately. Changes to irrelevant persona attributes could serve as controls. Report outcomes separately for the two claimed failure modes and for turns where relevant context changes.

These diagnostics can be conducted within the paper's offline setting. The claims about emotional dynamics should also be scoped to the improving states actually simulated, unless broader transitions are supported by additional evidence.

Questions for authors

1. Clarification of safety labels and evaluator inputs:

What exact inputs do App. B (e_2) and the multidimensional evaluator receive, including the user's query, persona, and conversation history? How are lawful but personally harmful responses adjudicated? In particular, how should the nonzero reward at turn 3 of Table 7 be interpreted relative to the stated prescription-drug-abuse boundary?

2. Additional evidence on evaluator validity:

Can you provide independently assigned human safety labels for a representative subset of raw responses, with annotators blinded to model identity and judge rationales, and report disagreement/error counts? Do the main safety improvements remain when evaluated using those labels separately from empathy and personalization?

3. Additional evidence on the dynamic contribution:

Can you report a matched static/no-emotion-update comparison for one model and training method, together with a small history- or persona-sensitive evaluation? Please also specify whether target models receive the full structured profile and updated emotional state directly during testing.

4. Clarification of splitting and reproducibility:

Does the 70:30 split keep all turns, candidate responses, and any trajectories derived from the same PENGUIN seed within a single partition? Please identify where reviewers can inspect the dataset, fixed splits, complete strategy prompts, exact checkpoints, and training/evaluation configurations. These are necessary to assess and reproduce the resource contribution.

11 comments

Response to weakness 1: Composite Reward and Safety Evaluation

We thank the reviewer for highlighting this critical methodological distinction. We agree that the composite reward in Eq. (6) serves as an overall response-quality metric balancing risk sensitivity, emotional empathy, and personalization match, rather than an isolated certificate of safety. When a high-risk response escapes hard detection, stylistic features can indeed artificially inflate the final score. We address this directly by decoupling the evaluation metrics, auditing boundary behaviours with blinded human annotators, clarifying the two-tier detector mechanics, and refining our claims.

Notation note. All turn-level statistics in this letter refer to the 11,387-turn test partition (2,433 test dialogues), which contains 2,798 adversarial turns (24.6%) and 8,589 benign-context turns. This matches the dataset-wide adversarial distribution reported in Table 4 of the manuscript (24.0% of turns; 66.0% of dialogues).

1. Decoupled Evaluation Dimensions and Benign Controls

To evaluate communicative quality separately from safety constraints, Table 1 reports the three continuous rating dimensions alongside the discrete detector rates across all 11,387 MDPSA test turns (2,798 adversarial turns, 24.6%, and 8,589 benign-context turns). The three safety columns are read together rather than in isolation. e1e_1 Cold Refusal is the rate of bare rejections that offer no help, empathy, or alternative (Appendix B, Prompt 15); e2e_2 Jailbreak flags hazardous content; and Benign Refusal is the same e1e_1 detector restricted to the 8,589 non-adversarial turns, which isolates false refusals on requests that should be answered. Because a refusal, however warm, contains no hazardous content for e2e_2 to flag, a low e2e_2 rate is interpretable only alongside the corresponding e1e_1 rate.

Table 1: Decoupled evaluation dimensions and detector rates on the MDPSA test set (11,387 turns).

ModelMethodRisk Sens. ↑\uparrowEmpathy ↑\uparrowPers. Match ↑\uparrowe2e_2 Jailbreak (%) ↓\downarrowe1e_1 Cold Refusal (%) ↓\downarrowBenign Refusal (%) ↓\downarrow
Llama-3-8BNaive2.802.852.580.0112.736.35
Llama-3-8BDPO3.444.223.880.480.000.00
Qwen-2.5-7BNaive3.043.422.951.390.000.00
Qwen-2.5-7BDPO3.574.113.830.410.000.00

As shown in Table 1, DPO training directly improves Risk Sensitivity across both model families (+0.64 on Llama-3-8B, +0.53 on Qwen-2.5-7B), confirming that the gains are not merely driven by higher empathy or personalization. The detectors tell a second, separate story. On Qwen-2.5-7B, e2e_2 triggers drop by over 70% (1.39% → 0.41%) and cold refusals are already absent. On Llama-3-8B the Naive row needs both detector columns to be read together: its near-zero e2e_2 rate (0.01%) sits beside a 12.73% cold-refusal rate in the next column, because a bare refusal states no hazardous content for e2e_2 to flag. Once DPO replaces bare refusals with substantive, context-aware redirection (0.00% cold refusal), a detector that reads only the response text occasionally flags a safe reply that explicitly names and refutes the user's risky premise; the human audit below quantifies this direction, with 40% of e2e_2-flagged positive-strategy responses judged safe redirections, that is replies that explicitly reject the user's hazardous premise while offering empathetic support and a constructive lawful alternative (8 of 20 in stratum S3). Finally, cold refusals on benign (non-adversarial) turns fall from 6.35% to 0.00% for Llama-3-8B. Because these requests are safe and helping is the correct behaviour, this is a direct benign control showing that MDPSA removes false refusals on safe queries rather than inducing over-compliance.

2. Human Audit of the Illustrative Boundary Pattern

We performed a full census of Llama-3-8B DPO test outputs for the maximal-compensation pattern the reviewer derives from Eq. (6): high style scores combined with minimal risk credit (Risk=1\text{Risk} = 1, Empathy≥4\text{Empathy} \ge 4, Personalization≥4\text{Personalization} \ge 4). The condition is rare, at 79 responses (0.69% of test turns), with composite scores between 0.600 and 0.733. The looser bound Risk≤2\text{Risk} \le 2 with the same style floor contains 535 responses (4.70%), composite up to 0.800, and it is from that looser pool that the 30 S1 instances below are drawn. Separately, we audited 100 responses with blinded annotators. The sample is deliberately non-random: each stratum targets a feature where a rubric score could plausibly be inflated by tone, namely (i) low risk credit with high style scores, (ii) responses generated under negative strategies s5s_5–s8s_8, (iii) responses generated under positive strategies s1s_1–s4s_4 that the e2e_2 detector nevertheless flagged, and (iv) high-reward adversarial turns (Reward≥0.70\text{Reward} \ge 0.70, Risk≥4\text{Risk} \ge 4). Every case was judged under full persona, dialogue history, current query, and response, with model identity, strategy label and automatic score concealed. Table 2 reports both cohorts.

Table 2: Boundary census and blinded human audit on Llama-3-8B outputs (79-case maximal-compensation census and 100-case stratified sample).

Analysis cohortStratum definitionTotal (nn)Safe RedirectionHarmful ComplianceCold RefusalAgreement vs. auto-label
Pattern censusRisk =1= 1, Empathy ≥4\ge 4, Pers. ≥4\ge 47958210n/a
Stratified audit S1Low risk credit with high style (Risk ≤2\le 2, Emp./Pers. ≥4\ge 4, pool of 535)30264086.7%
Stratified audit S2Negative strategies (s5s_5–s8s_8)30272190.0%
Stratified audit S3e2e_2-flagged positive strategies (s1s_1–s4s_4)20812060.0%
Stratified audit S4High-reward adversarial turns (Reward ≥0.70\ge 0.70, Risk ≥4\ge 4)202000100.0%
Audit totalCombined stratified sample1008118185.0%

In the census cohort the majority of instances, 73.4% (58 of 79), are safe de-escalating redirections that the automated rubric penalised on the risk dimension. Across the 100-case audit, human labels agree with the automated labels in 85.0% of cases (Cohen's κ=0.525\kappa = 0.525; binary κ=0.545\kappa = 0.545). Agreement is 86.7% in S1 and 90.0% in S2, and 100% in S4, where all 20 high-reward adversarial responses were confirmed as genuine safe redirections rather than polite phrasing. The disagreement concentrates in S3, and its direction matters: in those 8 cases the automated e2e_2 label was the more severe of the two, so the detector errs toward over-flagging a partially assisting but safe reply. That is the conservative direction for training, since positive-strategy responses remain in Rt+\mathcal{R}_t^+ and are never demoted on the strength of a detector flag alone.

3. Clarification of Two-Tier Detector Mechanics and Table 7 of the manuscript

Our pipeline employs a structured two-tier safety architecture.

  • Input separation. e2e_2 screens the candidate response text rtr_t alone for actionable, hazardous instructions or illegal methods, while e4e_4 receives the full context (Ct,Ht,qt,rt)(C_t, \mathcal{H}_t, q_t, r_t) to evaluate nuanced personal risk, emotional support, and profile fit.

  • Turn 3 of Table 7 of the manuscript is adjudicated by the second layer. The table illustrates a synthetic data trajectory rather than an aligned model generation, and Turn 3 passes through both layers in sequence.

    1. e2e_2 is the execution gate. By design it fires only on actionable content: specific hazardous methods, dosages, administration schedules, or procurement steps. The Turn 3 reply, generated under s6s_6 (Toxic Positivity), offers perfunctory encouragement with no operational detail at all: no dose, no schedule, no interaction warning. It is therefore correctly not classified as an execution hazard, and the dialogue is not hard-terminated.
    2. e4e_4 performs contextual adjudication. Receiving the full context (Ct,Ht,qt,rt)(C_t, \mathcal{H}_t, q_t, r_t), it recognises the vulnerability nexus and assigns the lowest possible risk rating Risk=1\text{Risk} = 1, giving a composite of 0.267.
    3. Training-level isolation is independent of the scalar. All s5s_5–s8s_8 responses are by construction assigned to the non-preferred pool Rt−\mathcal{R}_t^- and are never used as positive SFT or DPO targets.

    The value 0.267 is thus the minimum non-terminal score, reserved for a response that the execution gate does not classify as an actionable hazard while the contextual layer finds it risk-insensitive. Turn 4 (s1s_1, 0.80) supplies the repaired supervision, which is the multi-turn repair signal MDPSA is designed to provide.

4. Cross-Family Re-Evaluation (non-GLM judge)

To test whether the reported gains are artifacts of the GLM family, we re-scored all 20 model ×\times variant cells with DeepSeek-V4.1-Flash on a paired sample of 500 test turns (identical indices in every cell, stratified by turn depth and query-attack flag).

  • The dataset-level gain replicates completely. Every one of the 16 trained cells (4 configurations ×\times SFT/GRPO/PPO/DPO) improves over that model's own naive baseline, by +0.066 to +0.201; the same coverage holds under GLM (16/16). Paired DPO$-$Naive effects are t=9.88t = 9.88 to 15.9615.96. This is the property MDPSA is meant to confer, and it does not depend on the judge family. Absolute scores are lower under DeepSeek in 18 of the 20 cells (range −0.052-0.052 to +0.016+0.016, mean −0.022-0.022).
  • The ranking among alignment variants is judge-dependent, and it is not the claim we make. DPO's point estimate exceeds PPO's on three configurations but significantly so on only two (Llama-3-3B, t=4.42t = 4.42; Qwen-2.5-7B, t=2.73t = 2.73); on Llama-3-8B the two are indistinguishable (0.7207 vs 0.7172; −0.0035-0.0035, t=−0.38t = -0.38), and on Qwen-2.5-3B (t=1.12t = 1.12) likewise. We therefore state the weaker, defensible claim: offline preference optimization (DPO/PPO) exceeds SFT and GRPO, with DPO and PPO as close competitors.
  • Two complementary axes: hard-boundary compliance and intervention quality. We now report hard execution-boundary compliance (e2e_2, and e1e_1 cold refusal) and personalized intervention quality (Risk Sensitivity, Empathy, Personalization, and the composite that summarises them) as two separate axes, rather than treating a single composite as a standalone certificate of hard-boundary safety. This is a framing change, not a retreat from the safety claim: the intervention axis is what "personalized safety alignment" means in this paper, namely recognising a user's specific vulnerability, declining the hazardous course, and offering lawful alternatives, and it improves uniformly across all 16 trained cells. The cross-family run shows the two axes can move independently: on Llama-3-8B, PPO has the highest composite (0.7207) with a low e2e_2 rate (0.80%, tied with SFT), while DPO is close in composite (0.7172, personalization marginally higher at 3.674 vs 3.642) but flags more often (4.60%); on the other three configurations the lowest e2e_2 rate belongs to GRPO (0.40%–0.60%). Since e2e_2 reads only the response text, a model that explicitly names and rebuts the user's risky premise is more likely to be flagged than one that stays vague. This is exactly the over-flagging pattern our human audit quantifies on stratum S3, where 8 of 20 flagged positives were judged safe.

Comments on the paper

1 comment

Dear Reviewers, AC, SAC, and PC,

Thank you all for your time and helpful feedback. Your comments helped us improve our evaluation and make the design of MDPSA much clearer. We are happy that Reviewer g27o felt their concerns were addressed and raised their score to Weak Acceptance (3). Below is a short summary of what we added and clarified during the rebuttal.

Strengths noted by the reviewers

  1. Looking at both vulnerability neglect and over-accommodation to risky personas in the same conversation addresses an important safety problem [g27o, hiVH, a2ct].
  2. Having 8 scored candidates per turn across 58 strategies gives more detailed supervision than binary safety labels and works well with different alignment methods [g27o, a2ct].
  3. Training on MDPSA gives consistent gains across four models and four alignment methods, along with positive results on persona-augmented HarmBench [g27o, hiVH, a2ct].

New experiments added during the rebuttal

  1. Separate safety metrics and benign controls: We reported the continuous scores (Risk Sensitivity, Empathy, Personalization) separately from the hard binary flags (Cold Refusal and Jailbreak) on the full test set and HarmBench, and added a benign-turn refusal check. DPO improves Risk Sensitivity, lowers Qwen-2.5-7B's jailbreak rate, and removes Llama-3-8B's false refusals on safe queries while keeping its safety boundary [g27o, a2ct].
  2. Cross-family check with DeepSeek-V4.1-Flash: To check for judge bias, we re-scored all 20 model-variant cells on 500 paired test turns with DeepSeek-V4.1-Flash. The gains over naive baselines hold in all 16 trained cells, showing the improvement does not depend on GLM judges. We also noted that while the overall dataset gain is consistent across judges, the exact ranking between close methods (DPO vs. PPO) can change by judge [g27o, hiVH, a2ct].
  3. Blinded human audit and boundary check: A full check of high-style/low-risk Llama-3-8B DPO outputs showed that actual harmful compliance under this pattern is very rare (0.18% of test turns) [g27o, hiVH]. In a blinded, paired human audit of 100 hard boundary turns, human labels agreed with automated labels in 85% of cases (100% on high-reward adversarial turns), and DPO had fewer harmful responses and more safe redirections than the naive baseline [g27o, hiVH, a2ct].
  4. Dimension correlations and ranking ablations: Across all 91,096 test candidates, correlations across dimensions within each strategy group are moderate, positive strategies rank first in over 98% of turns under the scalar reward, and dropping any single dimension lets a different type of bad response through [hiVH, a2ct].
  5. Turn-depth analysis and context check: Looking at scores by turn depth showed opposite issues in the naive baselines (late-turn drop in Llama-3-8B vs. early-turn coldness in Qwen-2.5-7B), while MDPSA-DPO stays stable across all turns [g27o, hiVH, a2ct]. We also checked persona-changed turns and confirmed that the evaluator's risk score changes when the persona or history changes [g27o].

Key clarifications and released data

  1. Two-tier safety setup and training pools: We clarified the exact inputs to each module (text-only for hard refusal/jailbreak checks vs. full context for personal risk) and explained Table 7 Turn 3 [g27o, hiVH]. SFT and DPO positive/negative pools are split strictly by strategy type rather than scalar scores, so negative strategies can never become positive targets regardless of tone [g27o, hiVH, a2ct]. We also now report hard-boundary compliance and intervention quality as two separate axes [g27o, a2ct].
  2. Model inputs and emotional updates: Updated emotional states are used only inside the offline data simulator. Target models never see explicit emotion updates during training or testing, and only get the initial profile and dialogue history. We also clarified that monotonic positive updates model supportive de-escalation, while the adversarial query generator adds high-risk and crisis prompts [g27o, hiVH, a2ct].
  3. Train/test split and data release: We confirmed a strict 1:1 PENGUIN seed-level 70:30 split with zero shared dialogues or seeds (while noting that some profile templates appear in both splits with different initial queries), and released the full split dataset (MDPSA_split.zip, with train.jsonl, test.jsonl, and checksums) for inspection [g27o, a2ct].

We will add all of these new results, references, and clarifications to the revised paper. Thank you again for your helpful reviews, and please let us know if you have any remaining questions before the discussion ends.

Best regards,
The Authors of Submission 3733

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.