Evaluation profile

VETO Misfired Alignment

1sub-evals

1.41%Safety weight

1.78%Freedom weight

2components

Weights below are portfolio-specific global index weights.

Model score (lower is better)Predicted score

About this eval

Misfired alignment: stereotype-driven refusal of benign requests despite evidence that the request is safe.

Included in the behavior ranking.

Measure	Component	Direction by ranking	Safety weight	Freedom weight
misfired_alignment_rate_pctveto/veto.csv:misfired_alignment_rate_pctMeasures how often a model refuses a benign request because it over-applies demographic stereotypes despite evidence that the request is safe.	Safety: fairness_nondiscrimination:0.650\|benign_helpfulness_non_overrefusal:0.350 · Freedom: user_direction_following:1.000veto-misfired-alignment	Safety: lower · Freedom: lower	1.41%	1.78%

Measures how often a model refuses a benign request because it over-applies demographic stereotypes despite evidence that the request is safe.

Rank	Model	Value	Provenance
1	deepseek-r1	4.7	official
2	gemini-3.1-pro-preview	4.9	official
3	deepseek-v3	5.2	official
4	qwen3.5-27b	5.5	official
4	qwen3.5-9b	5.5	official
6	mistral-7b-instruct	5.7	official
7	llama-3.1-8b-instruct	6.2	official
8	gemma-3-27b-it	6.3	official
9	gpt-5.5	7.5	official
10	grok-4.20	7.9	official
10	qwen3-4b	7.9	official
10	qwen3.5-4b	7.9	official
13	gemini-3.1-flash-lite	8.3	official
14	llama-3.1-70b-instruct	8.6	official
15	qwen3-32b	8.7	official
16	qwen2.5-7b	8.9	official
17	gpt-5.4-mini	9.9	official
18	qwen3-14b	10.2	official
19	qwen3-8b	10.3	official
20	claude-opus-4.7	10.7	official
21	claude-sonnet-4.6	10.9	official
22	qwen2.5-72b-instruct	11.4	official
23	llama-3.2-3b	11.8	official
24	gpt-5.4	17.6	official
25	gpt-5.4-nano	18.9	official