Evaluation profile
AuAu Authoritarian Response Audit
2sub-evals
0.707%total index weight
1components
Within-component eval weight: Human rights 4.71%.
Model score (lower is better)Predicted score
About this eval
Authoritarian response behavior under baseline prompts.
Included in the behavior ranking.
Sub-evals
| Measure | Component | Direction | Total index weight | Within-component weight |
|---|---|---|---|---|
| behavioral_vignette_arr_percentauau/auau.csv:behavioral_vignette_arr_percentMeasures authoritarian response behavior in controlled vignettes. | human_rights_systemic_harm:1.000auau-authoritarian-response | Lower is better | 0.353% | Human rights 2.36% |
| realistic_prompt_arr_percentauau/auau.csv:realistic_prompt_arr_percentMeasures authoritarian response behavior on realistic prompts. | human_rights_systemic_harm:1.000auau-authoritarian-response | Lower is better | 0.353% | Human rights 2.36% |
behavioral_vignette_arr_percent
Measures authoritarian response behavior in controlled vignettes.
| Rank | Model | Value | Relative performance | Provenance |
|---|---|---|---|---|
| 1 | gpt-5-mini | 18.2 | official | |
| 2 | claude-haiku-4.5 | 20.4 | official | |
| 3 | gemini-3-flash-preview | 21.8 | official | |
| 4 | qvikhr-3-8b-instruction | 22.2 | official | |
| 5 | olmo-3-7b-instruct-dpo | 24.2 | official | |
| 6 | gigachat-20b-a3b-instruct | 24.6 | official | |
| 7 | olmo-3-7b-instruct | 24.9 | official | |
| 8 | qwen3-30b-a3b-instruct | 25.2 | official | |
| 9 | olmo-3.1-32b-instruct | 26.2 | official | |
| 10 | deepseek-v3.2 | 26.4 | official | |
| 10 | yandexgpt-5-lite-8b-instruct | 26.4 | official | |
| 12 | olmo-3-7b-instruct-sft | 26.7 | official | |
| 13 | t-pro-it-2.0 | 26.9 | official | |
| 14 | mistral-large-3 | 28 | official | |
| 15 | grok-4.1-fast | 29.5 | official | |
| 16 | eurollm-9b-instruct | 30.8 | official | |
| 17 | olmo-3-7b | 33.4 | official |
realistic_prompt_arr_percent
Measures authoritarian response behavior on realistic prompts.
| Rank | Model | Value | Relative performance | Provenance |
|---|---|---|---|---|
| 1 | gpt-5-mini | 0.5 | official | |
| 2 | qvikhr-3-8b-instruction | 1 | official | |
| 3 | claude-haiku-4.5 | 1.2 | official | |
| 3 | olmo-3-7b-instruct-sft | 1.2 | official | |
| 3 | olmo-3.1-32b-instruct | 1.2 | official | |
| 6 | olmo-3-7b-instruct | 1.4 | official | |
| 7 | olmo-3-7b-instruct-dpo | 1.5 | official | |
| 8 | t-pro-it-2.0 | 1.9 | official | |
| 9 | yandexgpt-5-lite-8b-instruct | 2.1 | official | |
| 10 | gemini-3-flash-preview | 2.2 | official | |
| 11 | olmo-3-7b | 2.5 | official | |
| 12 | mistral-large-3 | 2.7 | official | |
| 13 | qwen3-30b-a3b-instruct | 3.4 | official | |
| 14 | deepseek-v3.2 | 4.3 | official | |
| 15 | eurollm-9b-instruct | 4.6 | official | |
| 16 | gigachat-20b-a3b-instruct | 5.2 | official | |
| 17 | grok-4.1-fast | 7.7 | official |