acceptodds
Under review as a conference paper at ICLR 2027

VulnSafe: Measuring Vulnerability-Aware Relational Safety in Large Language Models

Abstract

Emotionally supportive interactions require large language models not only to avoid overtly harmful content, but also to recognize vulnerability-related risk, regulate their relational stance, preserve user agency, and provide context-appropriate support. Existing safety benchmarks often measure isolated violations, leaving unclear what underlying response competence their scores represent. We introduce VulnSafe, a Chinese capability assessment centered on Vulnerability-Aware Response Competence (VARC). VARC organizes relational safety into five complementary constructs: Risk Perception, Safety Preservation, Autonomy Protection, Supportive Response, and Boundary Management. Following an evidence-centered design, 41 scenarios elicit construct-relevant behavior through 2,050 balanced prompts and scenario-specific scoring rubrics. The assessment further separates proactive should-do behavior from prohibitive should-not behavior. Across eight representative LLMs, results show distinct competence profiles and larger disparities in proactive requirements than in prohibitive requirements. VulnSafe therefore makes relational safety interpretable as a multidimensional model capability rather than as refusal compliance alone.

Primary area:
datasets and benchmarks
Keywords:
vulnerability-aware relational safety, benchmark evaluation, AI safety

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_GyjTOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes VulnSafe, a Chinese capability assessment centered on VARC (Vulnerability-Aware Response Competence) that decomposes relational safety into five complementary constructs. The paper finds disparities and distinct profiles acros LLMs and provides an approach to understanding and interpreting relational safety rather than focusing solely on refusal compliance.

Critical strengths
  1. The paper tackles an important problem in the mental health and psychosocial safety space. Relational safety is very crucial, especially for cases where single-turn content in itself is not enough. Additionally, the example clearly shows the motivation, and that a warm and validating message at all times is not be the best approach.

  2. I like the construct-centered benchmark design. The paper tackles this problem by grounding on existing successful methods such as VARC, and creating benchmark instances.

Critical weaknesses
  1. The benchmark, in its current status is more about immediate response behaviour, but single turn analysis can be a critical limitation of the relational safety findings. Relational safety is heavily dependent on interactions, such as dependence over time, or companionship and so on. Therefore, single turn does not always show the real behaviour whether it develops, continues or reduces these risks in a conversation.

  2. The human validation shows promise but the failure class precision is low. Furthermore, the individual human agreement is moderate and also the scope is small given the benchmark size.

  3. Proactive should do vs prohibitive should-not comparison is interesting. But it can be confounded by scenario contents. As each scenario corresponds to one label, the comparison involves different tasks rather than matches scenarios. So comparison needs a matched analysis, or narrow the final claims about performances. (The authors do acknowledge this, but this remains a major limitation)

Questions for authors

There are some model-name inconsistencies: Section 3.4 has MiniMax-3.7 but later has MiniMax-M2.7 in eval. Appendix C.4 has GLM-5.2 but the paper shows 5.1. So, exact models, scoring rubrics, and source-to-requirement mappings are important for auditing the benchmark.

4 comments
AuthorsOpenReview ↗

Response to Reviewer GyjT (part 1)

Thank you for highlighting the importance of relational safety and the value of construct-centered assessment. We address your questions by clarifying the response-level measurement target, reporting the full human-audit evidence, and making the interpretation and provenance of the diagnostic comparisons explicit.

Response-level assessment and multi-turn interaction.

VulnSafe evaluates whether a response exhibits the behavior required by a specified vulnerability-related situation. This includes recognizing crisis signals, responding to requests for exclusive companionship, and offering a route to real-world support. For example, M11 tests the immediate response to a request for continuous presence. This is a defined, observable component of relational safety; the current evaluation does not establish how a relationship develops over a sequence of interactions.

We will make that measurement target explicit and add multi-turn evaluation in future work. The proposed extension will examine escalation, repair after an earlier unsafe response, and the maintenance of boundaries across a conversation. The present results will remain scoped to the sampled response-level tasks.

Human validation and Fail-class performance.

Our human audit covers 205 responses across all 41 scenarios, with five responses per scenario and coverage across all eight evaluated models. We recruited 10 master’s students with backgrounds in psychology, ethics, or computer science, each of whom independently evaluated every response while blinded to the model’s identity. This yields 2,050 individual ratings on 205 distinct responses, providing repeated judgments for constructing the human consensus.

The scale and annotation depth can be contextualized against published safety-evaluation studies:

  • PS-Bench (ACL 2026) validates its automatic harmfulness detector using 160 responses, each independently assessed by two annotators, with disagreements resolved through discussion or third-annotator adjudication (Guo et al., 2026).

  • CoSafe (EMNLP 2024) recruits three annotators and evaluates 100 sampled responses per model for three models in its human-validation analysis (Yu et al., 2024).

  • Nullspace Disentanglement for Red Teaming Language Models (EMNLP 2025) recruits six annotators and computes each reported human-evaluation result from 100 sampled test cases across multiple methods and target models (Han et al., 2025).

  • Using In-Context Learning to Improve Dialogue Safety (Findings of EMNLP 2023) evaluates 150 examples from each dataset, collecting three human judgments per response-pair comparison and evaluation dimension (Meade et al., 2023).

These studies provide precedent for targeted human evaluations using hundreds of cases per evaluation setting. Our design emphasizes complete scenario coverage and ten independent ratings per response.

Regarding reliability, our annotators achieve Fleiss’ κ = 0.520 and 87.0% mean pairwise raw agreement. For context, Han et al. (2025) also report Fleiss’ κ = 0.52, using a three-category safety scale. We complement inter-annotator reliability with a separate evaluation against the ten-person consensus: the aggregated model judgments achieve 94.15% agreement, Cohen’s κ = 0.766, and Fail-class precision/recall/F1 of 72.73%/88.89%/80.00%, as shown in Table 11. We will foreground the sampling design, annotation depth, and class-specific results to make the evidence supporting the human audit explicit.

AuthorsOpenReview ↗

Response to Reviewer GyjT (part 2)

Interpreting the should-do/should-not comparison.

We adopt the descriptive interpretation you suggest. The fixed classification comprises 17 should-do and 24 should-not scenarios, based on the primary behavior specified by each rubric. Figure 3 describes performance across these sampled scenario sets. It does not isolate response demand from scenario content, and we will state this directly in the interpretation of the figure.

The supplementary analysis of judge explanations further distinguishes omitted protective action from prohibited action. Its reported examples associate M11 with prohibited companionship promises and L11 and H17 with missing real-world support or crisis recognition. This distinction explains why a scene-level primary demand label need not uniquely determine the type of failure in an individual response. We will present this as an exploratory failure-type analysis, not as a content-matched test or a causal explanation of the performance gap.

The supported finding is that the current rubric-defined requirements reveal different response profiles on the benchmark. We will replace wording that attributes difficulty to an independently established proactive/prohibitive mechanism with this task-specific interpretation.

Model identities, rubrics, and source-to-requirement mappings.

The four models used as both response models and rubric judges are DeepSeek-V4-Pro, GLM-5.1, Kimi-K2.6, and MiniMax-M2.7. The appendix’s GLM-5.2 entry describes the fine-screen annotation stage, while GLM-5.1 is listed for response evaluation and judging. We will organize the model information by stage and reconcile inconsistent shorthand, including the MiniMax entry, against the original run configurations. The resulting table will trace each exact API identifier to the operation it performs.

For reproducibility, we will consolidate three types of supplementary material: a model-role table; complete rubric definitions, including applicability, required behavior, and Fail conditions; and a mapping from normative or professional source clauses to behavioral requirements, scenarios, and rubric criteria. The appendix already specifies the 41 scenario-to-construct and response-demand assignments. The source mapping will distinguish the evidence motivating a requirement from the dialogue material used to instantiate a prompt.

Finally, VARC is the response-level construct introduced in this work. Evidence-centered design is the established assessment methodology informing its operationalization into scenarios and rubrics.

References

  1. Jiahe Guo, Xiangran Guo, Yulin Hu, Zimo Long, Xingyu Sui, Xuda Zhi, Yongbo Huang, Hao He, Weixiang Zhao, Yanyan Zhao, and Bing Qin. 2026. When Personalization Legitimizes Risks: Uncovering Safety Vulnerabilities in Personalized Dialogue Agents. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 27309–27335.

  2. Erxin Yu, Jing Li, Ming Liao, Siqi Wang, Zuchen Gao, Fei Mi, and Lanqing Hong. 2024. CoSafe: Evaluating Large Language Model Safety in Multi-Turn Dialogue Coreference. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 17494–17508.

  3. Yi Han, Yuanxing Liu, Weinan Zhang, and Ting Liu. 2025. Nullspace Disentanglement for Red Teaming Language Models. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 21349–21365.

  4. Nicholas Meade, Spandana Gella, Devamanyu Hazarika, Prakhar Gupta, Di Jin, Siva Reddy, Yang Liu, and Dilek Hakkani-Tur. 2023. Using In-Context Learning to Improve Dialogue Safety. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 11882–11910.

Thank you again for your time and consideration. We would greatly appreciate it if you could let us know whether our responses and additional experiments have sufficiently addressed your concerns. Please feel free to let us know if any further clarification is needed. We would be happy to discuss any remaining questions.

Thanks to the authors for clarifying the human audit and the scope of the comparison. I’m keeping my rating because it remains unclear what the five constructs reveal beyond performance differences across the sampled scenarios.

Reviewer_tarVOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes VulStamp, a framework for predicting software vulnerability severity from source code and generating repair suggestions. It first uses program analysis to extract relevant code, then employs an LLM to produce reports describing exploitability, impact, and scope. The assessment model is trained through prompt tuning, multitask learning, and a weighted reward objective. The paper addresses a useful problem: helping developers prioritize vulnerabilities without relying on manually written descriptions. The reported results show improvements over the evaluated baselines.

Critical strengths
  1. Practically relevant problem. Vulnerability severity assessment can help developers allocate limited remediation resources. Reducing dependence on manually written descriptions is a reasonable research goal.

  2. Reasonable overall design. Combining relevant code fragments with natural-language analysis could help the model capture both implementation details and potential consequences. The component ablations provide some evidence for the usefulness of this combination.

  3. Consideration of temporal evaluation and model compatibility. The authors attempt to split data by vulnerability disclosure time and evaluate the framework with several pretrained models. These are useful steps toward assessing practical applicability.

Critical weaknesses
  1. Technical explanations and vulnerability examples require correction. A central premise is that program analysis and LLM-generated reports help the model understand vulnerabilities correctly. However, some technical explanations raise concerns about this premise. First, Section III-B states that pycallgraph supplements call-graph information for the C/C++ code. Its official project describes a tool for generating call graphs of Python programs. The authors should explain how it was adapted to the C/C++ setting or correct the tool and implementation description. Second, Section VI-A incorrectly states that readlink continues writing beyond the supplied buffer size when the target path is too long. The function truncates the content to the specified size and does not automatically append a null terminator. These issues concern the analysis pipeline and examples used to support the method. I would like to see reproducible program-analysis outputs and an independently checked assessment of whether the generated reports correctly identify vulnerability conditions and consequences.
  2. Several baselines appear close to degenerate prediction, weakening the comparison.
  3. Aggregate improvements do not establish better identification of critical vulnerabilities. An important motivation is to prevent severe vulnerabilities from being underestimated. The weighted reward is intended to emphasize higher-severity classes. However, the main results report aggregate AUC, precision, recall, and F1 without clearly showing performance on the critical class.
Questions for authors
  1. What role does pycallgraph play in the C/C++ analysis pipeline? Can the authors provide a reproducible call-graph construction procedure and correct the explanation of the readlink example?

  2. Why do several models in Table II have recall exactly equal to 25%? Are their predictions concentrated in a single class, and were their training and hyperparameters adequately checked?

  3. What is VulStamp’s recall for the critical class? How much does it reduce critical-to-low and critical-to-medium misclassification relative to the strongest baseline?

  4. With inputs, training budget, and auxiliary supervision controlled, does the weighted reward objective consistently outperform class-weighted cross-entropy or Focal Loss?

Reviewer_V9a5OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Emotionally supportive interactions require large language models not only to avoid overtly harmful content, but also to recognize vulnerability-related risk, regulate their relational stance, preserve user agency, and provide context-appropriate support. Existing safety benchmarks often measure isolated violations, leaving unclear what underlying response competence their scores represent. To address this, we introduce VULNSAFE, a Chinese capability assessment centered on Vulnerability-Aware Response Competence (VARC). VARC organizes relational safety into five complementary constructs: Risk Perception, Safety Preservation, Autonomy Protection, Supportive Response, and Boundary Management. Following an evidence-centered design, the assessment uses 41 scenarios, 2,050 balanced prompts, and scenario-specific scoring rubrics to elicit construct-relevant behavior, and further separates proactive should-do behavior from prohibitive should-not behavior. Across eight representative LLMs, results reveal distinct competence profiles, with larger disparities on proactive requirements than on prohibitive ones. VULNSAFE thus makes relational safety interpretable as a multidimensional model capability rather than as refusal compliance alone.

Critical strengths

Rather than checking whether a model says something harmful, it evaluates the model’s overall competence in responding to vulnerable users, which is much closer to what real emotional-support interactions demand.

Evidence-centered design works backward from the constructs to the scenarios and rubrics, ensuring consistency between what is measured and what is claimed, rather than assembling test items first and labeling them afterward.

Critical weaknesses

The coarse screener has precision/recall of only 0.67/0.74 on the vulnerability class, and its gold standard is Qwen3.5 labels, not human ones. Seed verification, rewriting, and re-checking are all model-based. No human spot-check of scenario-assignment accuracy is reported for the final 2,050 prompts.

The human audit is small and Pass-skewed. Only 33 of 205 items are model-labeled Fail.

“Lower severity is hardest” rests on just five scenarios.

Questions for authors

See weekness

3 comments
AuthorsOpenReview ↗

Response to Reviewer V9a5 (part 1)

Thank you for recognizing the value of evaluating response competence for vulnerable users and of deriving scenarios and rubrics from explicit constructs. Your comments help us distinguish three questions: the reliability of source screening, the alignment of final prompts with scenarios, and the reliability of response scoring. We address each with the corresponding evidence.

Screening supervision and final-prompt alignment.

The Qwen3-0.6B model performs initial candidate retrieval. Its precision of 0.673 and recall of 0.744 measure agreement with Qwen3.5 teacher labels. The fine-tuned Qwen3-8B model subsequently assigns retained queries to five vulnerability-source categories, A–E. These retrieval categories are distinct from the 41 behavioral scenarios used for evaluation. We will make the reference labels and the role of each stage explicit throughout the screening description.

We additionally examined label sensitivity using three-model annotations on 234,497 records. Pairwise Cohen’s κ was 0.554 for Qwen–GLM, 0.448 for Qwen–MiniMax, and 0.470 for GLM–MiniMax; all three agreed on 61.4% of records. This analysis quantifies dependence on the annotation model. We will report these as model-to-model agreement measures, with model consensus identified as such rather than as human ground truth.

To examine the final prompts directly, we also conducted a blind model audit of 410 prompts. Four models selected a top-1 label from the 41 scenarios without seeing the assigned label. Exact label recovery was 62.3% for Kimi, 61.5% for DeepSeek, 58.1% for GLM, and 55.4% for MiniMax. The reported confusions include neighboring requirements, such as identity affirmation versus depathologization and protection from manipulation versus scam-risk detection. These results make fine-grained scenario boundaries an explicit part of the benchmark audit. We will distinguish this label-recovery analysis from the assessment of whether a specified rubric applies to a prompt. The final prompt assignments have not yet received an independent human scenario-assignment audit; the human study described below evaluates response judgments.

Human audit coverage and the Fail class.

Our human audit covers 205 responses across all 41 scenarios, with five responses per scenario and coverage across all eight evaluated models. We recruited 10 master’s students with backgrounds in psychology, ethics, or computer science, each of whom independently evaluated every response while blinded to the model’s identity. This yields 2,050 individual ratings on 205 distinct responses, providing repeated judgments for constructing the human consensus.

The scale and annotation depth can be contextualized against published safety-evaluation studies:

  • PS-Bench (ACL 2026) validates its automatic harmfulness detector using 160 responses, each independently assessed by two annotators, with disagreements resolved through discussion or third-annotator adjudication (Guo et al., 2026).

  • CoSafe (EMNLP 2024) recruits three annotators and evaluates 100 sampled responses per model for three models in its human-validation analysis (Yu et al., 2024).

  • Nullspace Disentanglement for Red Teaming Language Models (EMNLP 2025) recruits six annotators and computes each reported human-evaluation result from 100 sampled test cases across multiple methods and target models (Han et al., 2025).

  • Using In-Context Learning to Improve Dialogue Safety (Findings of EMNLP 2023) evaluates 150 examples from each dataset, collecting three human judgments per response-pair comparison and evaluation dimension (Meade et al., 2023).

These studies provide precedent for targeted human evaluations using hundreds of cases per evaluation setting. Our design emphasizes complete scenario coverage and ten independent ratings per response.

Regarding reliability, our annotators achieve Fleiss’ κ = 0.520 and 87.0% mean pairwise raw agreement. For context, Han et al. (2025) also report Fleiss’ κ = 0.52, using a three-category safety scale. We complement inter-annotator reliability with a separate evaluation against the ten-person consensus: the aggregated model judgments achieve 94.15% agreement, Cohen’s κ = 0.766, and Fail-class precision/recall/F1 of 72.73%/88.89%/80.00%, as shown in Table 11. We will foreground the sampling design, annotation depth, and class-specific results to make the evidence supporting the human audit explicit.

We will foreground the class-specific results and confusion matrix alongside the overall agreement. The 33 model-labeled Fail cases reflect sampling by scenario rather than enrichment for failures. The resulting evidence supports agreement on this audited subset.

AuthorsOpenReview ↗

Response to Reviewer V9a5 (part 2)

Severity and scenario-specific difficulty.

The additional sensitivity analysis identifies the source of the lower-severity aggregate. Under the manuscript’s convention of excluding blanks from the conditional failure-rate denominator, the five lower-severity scenarios yield 399 failures among 1,987 non-blank responses (20.08%). L11, encouraging older users’ social participation, yields 63.0% failures and accounts for 63.2% of failures in this group. Excluding L11 reduces the group rate to 147/1,587 (9.26%); excluding both L11 and L10 reduces it to 33/1,187 (2.78%).

We will replace the group-wide “lower severity is hardest” statement with this scenario-specific finding. Normative severity describes the consequence of an inadequate response, while measured difficulty describes observed model performance. L11 demonstrates that a lower-severity requirement can expose frequent failures.

References

  1. Jiahe Guo, Xiangran Guo, Yulin Hu, Zimo Long, Xingyu Sui, Xuda Zhi, Yongbo Huang, Hao He, Weixiang Zhao, Yanyan Zhao, and Bing Qin. 2026. When Personalization Legitimizes Risks: Uncovering Safety Vulnerabilities in Personalized Dialogue Agents. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 27309–27335.

  2. Erxin Yu, Jing Li, Ming Liao, Siqi Wang, Zuchen Gao, Fei Mi, and Lanqing Hong. 2024. CoSafe: Evaluating Large Language Model Safety in Multi-Turn Dialogue Coreference. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 17494–17508.

  3. Yi Han, Yuanxing Liu, Weinan Zhang, and Ting Liu. 2025. Nullspace Disentanglement for Red Teaming Language Models. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 21349–21365.

  4. Nicholas Meade, Spandana Gella, Devamanyu Hazarika, Prakhar Gupta, Di Jin, Siva Reddy, Yang Liu, and Dilek Hakkani-Tur. 2023. Using In-Context Learning to Improve Dialogue Safety. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 11882–11910.

Thank you again for your time and consideration. We would greatly appreciate it if you could let us know whether our responses and additional experiments have sufficiently addressed your concerns. Please feel free to let us know if any further clarification is needed. We would be happy to discuss any remaining questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.