Evaluation profile
SafeDialBench
6sub-evals
0.686%total index weight
3components
Within-component eval weight: Misuse resistance 1.02% · Human rights 2.77% · Fairness 1.68%.
Model score (higher is better)Predicted score
About this eval
Dialogue safety across aggression, ethics, fairness, legality, morality, and privacy dimensions.
Included in the behavior ranking.
Sub-evals
| Measure | Component | Direction | Total index weight | Within-component weight |
|---|---|---|---|---|
| aggressionsafedialbench/safedialbench.csv:aggressionMeasures whether the model maintains safe behavior during multi-turn jailbreak conversations about aggression and violence. | ordinary_harm_misuse_resistance:1.000safedialbench/safedialbench.csv | Higher is better | 0.0339% | Misuse resistance 0.339% |
| ethicssafedialbench/safedialbench.csv:ethicsMeasures whether the model maintains safe behavior during multi-turn jailbreak conversations about self-harm and abuse. | human_rights_systemic_harm:1.000safedialbench/safedialbench.csv | Higher is better | 0.208% | Human rights 1.39% |
| fairnesssafedialbench/safedialbench.csv:fairnessMeasures whether the model maintains safe behavior during multi-turn jailbreak conversations about stereotypes and distributional harms. | fairness_nondiscrimination:1.000safedialbench/safedialbench.csv | Higher is better | 0.168% | Fairness 1.68% |
| legalitysafedialbench/safedialbench.csv:legalityMeasures whether the model maintains safe behavior during multi-turn jailbreak conversations about illegal harmful conduct. | ordinary_harm_misuse_resistance:1.000safedialbench/safedialbench.csv | Higher is better | 0.0339% | Misuse resistance 0.339% |
| moralitysafedialbench/safedialbench.csv:moralityMeasures whether the model maintains safe behavior during multi-turn jailbreak conversations about nonviolent immorality and social-norm violations. | ordinary_harm_misuse_resistance:1.000safedialbench/safedialbench.csv | Higher is better | 0.0339% | Misuse resistance 0.339% |
| privacysafedialbench/safedialbench.csv:privacyMeasures whether the model maintains safe behavior during multi-turn jailbreak conversations about personal, organizational, and social privacy violations. | human_rights_systemic_harm:1.000safedialbench/safedialbench.csv | Higher is better | 0.208% | Human rights 1.39% |
aggression
Measures whether the model maintains safe behavior during multi-turn jailbreak conversations about aggression and violence.
| Rank | Model | Value | Relative performance | Provenance |
|---|---|---|---|---|
| 1 | moonshot-v1 | 7.277 | official | |
| 2 | deepseek-r1 | 7.273 | official | |
| 3 | yi-34b-chat | 7.26 | official | |
| 4 | gpt-4o | 7.207 | official | |
| 5 | qwq-32b | 7.19 | official | |
| 6 | glm-4-9b-chat | 7.17 | official | |
| 7 | qwen-14b-chat | 7.15 | official | |
| 8 | qwen-7b-chat | 7.127 | official | |
| 8 | yi-6b-chat | 7.127 | official | |
| 10 | qwen-2.5-14b-instruct | 7.123 | official | |
| 11 | baichuan-2-7b-chat | 7.073 | official | |
| 12 | baichuan-2-13b-chat | 7.03 | official | |
| 13 | mistral-7b-instruct | 7.027 | official | |
| 14 | o3-mini | 7.02 | official | |
| 15 | chatglm3-6b | 7.017 | official | |
| 16 | qwen-2.5-7b-instruct | 7.013 | official | |
| 17 | internlm2-20b-sft | 6.957 | official | |
| 18 | deepseek-llm-7b-chat | 6.953 | official |
ethics
Measures whether the model maintains safe behavior during multi-turn jailbreak conversations about self-harm and abuse.
| Rank | Model | Value | Relative performance | Provenance |
|---|---|---|---|---|
| 1 | glm-4-9b-chat | 7.753 | official | |
| 2 | qwen-14b-chat | 7.68 | official | |
| 2 | yi-34b-chat | 7.68 | official | |
| 4 | chatglm3-6b | 7.637 | official | |
| 5 | qwen-7b-chat | 7.623 | official | |
| 6 | baichuan-2-7b-chat | 7.613 | official | |
| 7 | baichuan-2-13b-chat | 7.6 | official | |
| 8 | mistral-7b-instruct | 7.587 | official | |
| 9 | yi-6b-chat | 7.577 | official | |
| 10 | deepseek-llm-7b-chat | 7.563 | official | |
| 11 | internlm2-20b-sft | 7.547 | official | |
| 12 | gpt-4o | 7.487 | official | |
| 13 | o3-mini | 7.403 | official | |
| 14 | qwen-2.5-14b-instruct | 7.39 | official | |
| 15 | qwen-2.5-7b-instruct | 7.357 | official | |
| 16 | moonshot-v1 | 7.353 | official | |
| 17 | qwq-32b | 7.313 | official | |
| 18 | deepseek-r1 | 7.303 | official |
fairness
Measures whether the model maintains safe behavior during multi-turn jailbreak conversations about stereotypes and distributional harms.
| Rank | Model | Value | Relative performance | Provenance |
|---|---|---|---|---|
| 1 | moonshot-v1 | 7.7 | official | |
| 2 | gpt-4o | 7.68 | official | |
| 3 | deepseek-r1 | 7.607 | official | |
| 4 | qwq-32b | 7.6 | official | |
| 5 | qwen-2.5-14b-instruct | 7.56 | official | |
| 6 | o3-mini | 7.557 | official | |
| 7 | qwen-2.5-7b-instruct | 7.553 | official | |
| 8 | glm-4-9b-chat | 7.4 | official | |
| 9 | yi-34b-chat | 7.337 | official | |
| 10 | yi-6b-chat | 7.277 | official | |
| 11 | qwen-14b-chat | 7.27 | official | |
| 12 | qwen-7b-chat | 7.19 | official | |
| 13 | chatglm3-6b | 7.187 | official | |
| 13 | mistral-7b-instruct | 7.187 | official | |
| 15 | baichuan-2-13b-chat | 7.17 | official | |
| 16 | baichuan-2-7b-chat | 7.123 | official | |
| 17 | deepseek-llm-7b-chat | 7.117 | official | |
| 18 | internlm2-20b-sft | 7.1 | official |
legality
Measures whether the model maintains safe behavior during multi-turn jailbreak conversations about illegal harmful conduct.
| Rank | Model | Value | Relative performance | Provenance |
|---|---|---|---|---|
| 1 | yi-34b-chat | 8.117 | official | |
| 2 | glm-4-9b-chat | 8.103 | official | |
| 3 | mistral-7b-instruct | 7.993 | official | |
| 4 | baichuan-2-13b-chat | 7.987 | official | |
| 4 | qwen-14b-chat | 7.987 | official | |
| 6 | chatglm3-6b | 7.983 | official | |
| 7 | internlm2-20b-sft | 7.977 | official | |
| 8 | deepseek-llm-7b-chat | 7.947 | official | |
| 9 | baichuan-2-7b-chat | 7.937 | official | |
| 10 | qwen-7b-chat | 7.93 | official | |
| 11 | yi-6b-chat | 7.887 | official | |
| 12 | deepseek-r1 | 7.38 | official | |
| 13 | moonshot-v1 | 7.317 | official | |
| 14 | qwq-32b | 7.263 | official | |
| 15 | gpt-4o | 7.21 | official | |
| 15 | qwen-2.5-7b-instruct | 7.21 | official | |
| 17 | o3-mini | 7.193 | official | |
| 18 | qwen-2.5-14b-instruct | 7.17 | official |
morality
Measures whether the model maintains safe behavior during multi-turn jailbreak conversations about nonviolent immorality and social-norm violations.
| Rank | Model | Value | Relative performance | Provenance |
|---|---|---|---|---|
| 1 | yi-34b-chat | 7.52 | official | |
| 2 | qwen-14b-chat | 7.437 | official | |
| 3 | glm-4-9b-chat | 7.427 | official | |
| 4 | baichuan-2-7b-chat | 7.383 | official | |
| 5 | moonshot-v1 | 7.35 | official | |
| 6 | baichuan-2-13b-chat | 7.303 | official | |
| 7 | mistral-7b-instruct | 7.28 | official | |
| 8 | qwen-7b-chat | 7.277 | official | |
| 9 | chatglm3-6b | 7.243 | official | |
| 10 | gpt-4o | 7.237 | official | |
| 11 | internlm2-20b-sft | 7.213 | official | |
| 12 | deepseek-llm-7b-chat | 7.183 | official | |
| 12 | deepseek-r1 | 7.183 | official | |
| 14 | qwq-32b | 7.16 | official | |
| 15 | yi-6b-chat | 7.123 | official | |
| 16 | qwen-2.5-7b-instruct | 7.06 | official | |
| 17 | qwen-2.5-14b-instruct | 7.047 | official | |
| 18 | o3-mini | 7.007 | official |
privacy
Measures whether the model maintains safe behavior during multi-turn jailbreak conversations about personal, organizational, and social privacy violations.
| Rank | Model | Value | Relative performance | Provenance |
|---|---|---|---|---|
| 1 | yi-34b-chat | 7.88 | official | |
| 2 | glm-4-9b-chat | 7.853 | official | |
| 3 | qwen-14b-chat | 7.69 | official | |
| 4 | yi-6b-chat | 7.67 | official | |
| 5 | baichuan-2-13b-chat | 7.617 | official | |
| 6 | mistral-7b-instruct | 7.607 | official | |
| 7 | internlm2-20b-sft | 7.597 | official | |
| 8 | deepseek-llm-7b-chat | 7.587 | official | |
| 9 | chatglm3-6b | 7.567 | official | |
| 10 | qwen-7b-chat | 7.563 | official | |
| 11 | baichuan-2-7b-chat | 7.523 | official | |
| 12 | moonshot-v1 | 7.25 | official | |
| 13 | deepseek-r1 | 7.193 | official | |
| 14 | gpt-4o | 7.14 | official | |
| 15 | qwen-2.5-14b-instruct | 7.117 | official | |
| 16 | qwq-32b | 7.093 | official | |
| 17 | o3-mini | 7.077 | official | |
| 18 | qwen-2.5-7b-instruct | 7.05 | official |