- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces the Social AI Design Code and EUDAIMONIA, a benchmark for evaluating conversational behaviors that may encourage anthropomorphism, emotional dependence, or unnecessary engagement. The benchmark contains 969 opening-turn queries and 3,147 requirement-specific checks, constructed from WildChat through filtering, relabeling, and controlled rewriting. The authors evaluate 26 models and report substantial but uneven progress across model generations. They also examine human–judge agreement, alternative LLM judges, original versus rewritten queries, and a separate set of multi-turn conversations. Overall, the work provides a useful framework for evaluating social behavior in conversational AI.
Critical strengths
-
An important and clearly specified evaluation target. The paper turns concerns about social boundaries in human–AI interaction into an explicit design code and concrete, requirement-specific checks. This makes the proposed benchmark useful for comparing models and identifying particular behavioral weaknesses.
-
Broad evaluation and meaningful reliability checks. The authors evaluate 26 models and analyze differences across requirements and model generations. They also compare multiple LLM judges, collect human annotations, distinguish original from rewritten queries, and include a separate multi-turn evaluation.
Critical weaknesses
-
The main multi-turn evaluation does not measure each model’s own conversational trajectory. In the 229 WildChat continuations, the earlier assistant replies provided as context are the original ChatGPT replies, rather than replies generated by the model being evaluated. This creates a useful controlled comparison on the same histories, but the model might have responded differently earlier, which could also have changed the user’s subsequent messages. Therefore, these results do not fully establish that the reported multi-turn violation rates and rankings would persist in conversations that each model conducts from the start. Appendix G.2 partially addresses this issue with simulated-user continuations, but it covers only a small set of response models and does not directly validate the main 16-model multi-turn ranking.
-
Preassigning checks from earlier models may miss violations specific to newer models. A requirement becomes a check for a query when it is triggered by the original ChatGPT response or responses from a small set of earlier models. A newer model could violate another requirement on the same query, but that violation would not enter its score if the requirement was not assigned as a check. The paper does not quantify how often this occurs or whether it affects requirement-level conclusions or model rankings. Auditing a sample of unassigned query–requirement pairs would strengthen confidence in the benchmark’s coverage.
Questions for authors
Could the authors evaluate a representative set of models in self-generated multi-turn conversations, or otherwise test whether using each model’s own earlier replies changes the violation rates or rankings? How far do the simulated-user results in Appendix G.2 support the main multi-turn conclusions?
On a stratified sample of queries and evaluated models, could the authors apply all nine requirements, including those not preassigned as checks? How many additional violations would be found, and would including them change the requirement-level results or model rankings?
3 comments
W2 & Q2: On a stratified sample of queries and evaluated models, could the authors apply all nine requirements?
Answer: Thank you for suggesting this important follow-up analysis. We applied all nine requirements to every reply of the 26 models on the stratified sample of Appendix G.2 (90 queries, 5 in-the-wild and 5 rewritten per requirement), which gives 21,060 checks. We found that the model ranking stays largely the same (), and the requirements that were not preassigned add 752 violations to the 4,390 on the preassigned ones.
| Checks | Violations | Rate | |
|---|---|---|---|
| Previous violations | 9,750 | 4,390 | 45.0% |
| Additional violations | 11,310 | 752 | 6.6% |
| All nine | 21,060 | 5,142 | 24.4% |
The next table gives these counts for each model.
| Model | Previous violations (/375) | Additional violations (/435) | Overall violations | Rank |
|---|---|---|---|---|
| Grok-4.7 | 49 (13.1%) | +6 (1.4%) | 6.8% | 1 → 1 |
| GPT-6-Astra | 67 (17.9%) | +6 (1.4%) | 9.0% | 2 → 2 |
| GPT-5.5 | 88 (23.5%) | +9 (2.1%) | 12.0% | 3 → 3 |
| Claude-Opus-4.6 | 105 (28.0%) | +25 (5.7%) | 16.0% | 4 → 5 |
| Claude-Opus-5.5 | 107 (28.5%) | +15 (3.4%) | 15.1% | 5 → 4 |
| GPT-5.4 | 112 (29.9%) | +22 (5.1%) | 16.5% | 6 → 6 |
| Claude-Opus-4.7 | 112 (29.9%) | +28 (6.4%) | 17.3% | 6 → 8 |
| Gemini-3.8-Flash | 124 (33.1%) | +11 (2.5%) | 16.7% | 8 → 7 |
| Grok-4.3 | 127 (33.9%) | +22 (5.1%) | 18.4% | 9 → 10 |
| GPT-4o* | 134 (35.7%) | +9 (2.1%) | 17.7% | 10 → 9 |
| GPT-4o-mini | 145 (38.7%) | +12 (2.8%) | 19.4% | 11 → 11 |
| Qwen3-32B | 198 (52.8%) | +38 (8.7%) | 29.1% | 12 → 13 |
| Gemini-3.1-Pro | 200 (53.3%) | +40 (9.2%) | 29.6% | 13 → 16 |
| Gemini-3.0-Flash | 205 (54.7%) | +34 (7.8%) | 29.5% | 14 → 15 |
| DeepSeek-V4-Flash | 206 (54.9%) | +41 (9.4%) | 30.5% | 15 → 17 |
| Qwen3.6-Max-Preview | 206 (54.9%) | +51 (11.7%) | 31.7% | 15 → 19 |
| Grok-4 | 214 (57.1%) | +51 (11.7%) | 32.7% | 17 → 23 |
| DeepSeek-Chat | 215 (57.3%) | +37 (8.5%) | 31.1% | 18 → 18 |
| Qwen3-8B | 215 (57.3%) | +42 (9.7%) | 31.7% | 18 → 19 |
| DeepSeek-V4-Pro | 215 (57.3%) | +43 (9.9%) | 31.9% | 18 → 21 |
| Qwen3-1.7B | 215 (57.3%) | +51 (11.7%) | 32.8% | 18 → 24 |
| Gemini-2.0-Flash* | 221 (58.9%) | +9 (2.1%) | 28.4% | 22 → 12 |
| Qwen3-14B | 224 (59.7%) | +34 (7.8%) | 31.9% | 23 → 21 |
| Claude-Sonnet-4* | 225 (60.0%) | +12 (2.8%) | 29.3% | 24 → 14 |
| Grok-3 | 227 (60.5%) | +50 (11.5%) | 34.2% | 25 → 25 |
| Qwen3-4B | 234 (62.4%) | +54 (12.4%) | 35.6% | 26 → 26 |
These results lead to two points. First, we selected the preassigned checks using three weaker models (GPT-4o, Claude-Sonnet-4 and Gemini-2.0-Flash). While stronger models trigger fewer of them, they are not free of violations on the checks that the weaker models passed—supporting our view that the benchmark is a long-lasting test rather than a one-off test. Second, we preselect the important checks for cost efficiency (or else it would be exorbitantly expensive!), but we recommend testing all requirements when deploying models in high-stakes scenarios, especially for weaker models. As the table shows, the ranking is generally stable, but weaker models vary more when all requirements are tested.
We have added this discussion to the main paper in the revised draft.
Dear Reviewer jZCh,
Thank you again for your thoughtful review. As the discussion phase ends soon, we wanted to check whether our response and the revised paper address your concerns on evaluating models in multi-turn conversations built from their own replies (Section 4.4 and Appendix H.4) and on applying all nine requirements to every reply (Appendix G.3). If they do, we would be grateful if you would consider raising your score. We would be glad to clarify anything further in the remaining time.
Best regards, The authors
We thank the reviewer for acknowledging that our social behavior objective is important and clearly specified, and that our evaluation is meaningful. We have run the two experiments the reviewer suggested and respond to each comment below.
Answer: We ran the experiment the reviewer suggested on six models (Grok-4.7, GPT-5.5, GPT-4o, Claude-Opus-5.5, Claude-Opus-4.6 and Qwen3-32B). These models continued all 229 conversations using their own replies instead of the preset replies. The earlier assistant replies provided as context are the original ChatGPT replies rather than replies generated by the model being evaluated. We used a user-simulator (Qwen3-VL-8B, as in Appendix G.2) to rewrite each later user message while keeping its topic, intent and style, so that it still follows from the model’s reply before it. A natural concern is whether the rewritten user messages still reflect the real conversations, which they do, as the simulator left 87% of them unchanged apart from spacing and capitalization. We found that the six models rank in the same order with their own replies as with the preset replies, as the table below shows. The ranking therefore does not depend on the preset replies, and no rate changes by more than 4.1 points. We have added this analysis to Section 4.4 in the revision.