Evaluation profile

FlagEval Safety and Values

5sub-evals

0.86%total index weight

4components

Within-component eval weight: Misuse resistance 0.814% · Fairness 2.02% · Human rights 1.66% · Benign helpfulness 3.27%.

Model score (higher is better)Predicted score

About this eval

Safety-and-values qualification across core values, discrimination, commercial compliance, lawful rights, and service requirements.

Included in the behavior ranking.

Measure	Component	Direction	Total index weight	Within-component weight
a1_qualified_rateflageval/flageval.csv:a1_qualified_rateMeasures whether the model avoids responses that violate the benchmark’s core-values criteria.	ordinary_harm_misuse_resistance:1.000flageval	Higher is better	0.0407%	Misuse resistance 0.407%
a2_qualified_rateflageval/flageval.csv:a2_qualified_rateMeasures whether the model avoids discriminatory or prejudicial responses about social groups.	fairness_nondiscrimination:1.000flageval	Higher is better	0.202%	Fairness 2.02%
a3_qualified_rateflageval/flageval.csv:a3_qualified_rateMeasures whether the model avoids assistance for illegal or noncompliant commercial activity.	ordinary_harm_misuse_resistance:1.000flageval	Higher is better	0.0407%	Misuse resistance 0.407%
a4_qualified_rateflageval/flageval.csv:a4_qualified_rateMeasures whether the model avoids responses that infringe privacy, intellectual property, or other lawful rights.	human_rights_systemic_harm:1.000flageval	Higher is better	0.25%	Human rights 1.66%
a5_qualified_rateflageval/flageval.csv:a5_qualified_rateMeasures whether the model gives reliable, controllable responses that meet the benchmark’s service requirements.	benign_helpfulness_non_overrefusal:1.000flageval	Higher is better	0.327%	Benign helpfulness 3.27%

Measures whether the model avoids responses that violate the benchmark’s core-values criteria.

Rank	Model	Value	Provenance
1	claude-sonnet-4	87.01	official
2	ernie-4.5-300b-a47b	85.04	official
3	gemini-2.5-pro	84.87	official
4	gpt-4.1	84.2	official
5	o3	82.67	official
6	grok-4	81.99	official
7	phi-4-reasoning-plus	81.55	official
8	step-3	81.42	official
9	o4-mini	81.28	official
10	gpt-5	80.22	official
11	doubao-seed-1.6	80.13	official
12	magistral-medium	79.01	official
13	deepseek-r1	77.33	official
14	nbg3.5-flash-thinking	75.38	official
15	qwen3-235b-a22b	71.76	official
16	spark-4.0-ultra	71.67	official
17	spark-x1	69.87	official
18	glm-4.5	59.1	official

Measures whether the model avoids discriminatory or prejudicial responses about social groups.

Rank	Model	Value	Provenance
1	o3	82.84	official
2	gpt-4.1	82.54	official
3	claude-sonnet-4	82.22	official
4	gemini-2.5-pro	81.19	official
5	step-3	80.47	official
6	doubao-seed-1.6	80.35	official
7	grok-4	80.3	official
8	o4-mini	80.18	official
9	gpt-5	79.66	official
10	nbg3.5-flash-thinking	79.02	official
11	ernie-4.5-300b-a47b	78.79	official
12	magistral-medium	78.73	official
13	phi-4-reasoning-plus	78.31	official
14	deepseek-r1	76.69	official
15	spark-4.0-ultra	73.88	official
16	qwen3-235b-a22b	73.77	official
17	spark-x1	73.44	official
18	glm-4.5	57.84	official

Measures whether the model avoids assistance for illegal or noncompliant commercial activity.

Rank	Model	Value	Provenance
1	gpt-4.1	90.46	official
2	claude-sonnet-4	89.41	official
3	magistral-medium	89.28	official
4	doubao-seed-1.6	88.87	official
5	o3	88.64	official
6	phi-4-reasoning-plus	87.76	official
7	o4-mini	87.75	official
8	spark-4.0-ultra	87.73	official
9	step-3	87.58	official
10	spark-x1	87.17	official
11	gemini-2.5-pro	86.5	official
12	grok-4	86.36	official
13	ernie-4.5-300b-a47b	85.37	official
14	gpt-5	83.26	official
15	nbg3.5-flash-thinking	82.46	official
16	deepseek-r1	82.31	official
17	qwen3-235b-a22b	75.56	official
18	glm-4.5	60.32	official

Measures whether the model avoids responses that infringe privacy, intellectual property, or other lawful rights.

Rank	Model	Value	Provenance
1	gpt-4.1	92.11	official
2	claude-sonnet-4	91.83	official
3	ernie-4.5-300b-a47b	90.04	official
4	gemini-2.5-pro	90.01	official
5	o3	89.98	official
6	doubao-seed-1.6	89.85	official
7	magistral-medium	89.15	official
8	step-3	88.34	official
9	o4-mini	88.24	official
10	grok-4	87.98	official
11	phi-4-reasoning-plus	87.95	official
12	deepseek-r1	84.15	official
13	gpt-5	83.73	official
14	spark-4.0-ultra	82.56	official
15	spark-x1	81.42	official
16	nbg3.5-flash-thinking	80.82	official
17	qwen3-235b-a22b	78.27	official
18	glm-4.5	60.7	official

Measures whether the model gives reliable, controllable responses that meet the benchmark’s service requirements.

Rank	Model	Value	Provenance
1	ernie-4.5-300b-a47b	77.55	official
2	gemini-2.5-pro	74.97	official
3	magistral-medium	74.27	official
4	doubao-seed-1.6	73.47	official
5	step-3	72.92	official
6	grok-4	72.83	official
7	spark-4.0-ultra	72.6	official
8	o3	72.58	official
9	o4-mini	72.2	official
10	gpt-5	72.15	official
11	gpt-4.1	71.36	official
12	nbg3.5-flash-thinking	70.56	official
13	phi-4-reasoning-plus	69.65	official
14	spark-x1	69.48	official
15	deepseek-r1	69.38	official
16	qwen3-235b-a22b	65.07	official
17	glm-4.5	56.79	official
18	claude-sonnet-4	49.88	official