Evaluation profile

Vigil Mental Health Safety

1sub-evals

0.761%total index weight

2components

Within-component eval weight: Human rights 4.23% · Misuse resistance 1.27%.

Model score (higher is better)Predicted score

About this eval

Mental-health conversational safety when assisting vulnerable users across simulated multi-turn scenarios.

Included in the behavior ranking.

Sub-evals

Measure	Component	Direction	Total index weight	Within-component weight
overall_scorevigil-eval/vigil-eval.csv:overall_scoreMeasures whether an assistant maintains safe boundaries rather than colluding with or escalating vulnerable users during extended mental-health conversations.	human_rights_systemic_harm:0.450\|ordinary_harm_misuse_resistance:0.550vigil-eval	Higher is better	0.761%	Human rights 4.23% · Misuse resistance 1.27%

overall_score

Measures whether an assistant maintains safe boundaries rather than colluding with or escalating vulnerable users during extended mental-health conversations.

Rank	Model	Value	Provenance
1	claude-sonnet-4.6	83	official
2	gpt-5.4	78	official
3	claude-sonnet-4.5	75	official
4	claude-haiku-4.5	74	official
5	gpt-5.2	72	official
6	gpt-5.3-chat	71	official
7	gpt-5.1	58	official
8	claude-sonnet-4	53	official
9	gpt-5	51	official
10	gemini-3.1-pro-preview	48	official
11	gemini-3-flash-preview	47	official
12	gemma-4-31b-it	46	official
13	gemini-3.1-flash-lite	38	official
13	gemma-4-26b-a4b-it	38	official
15	deepseek-v3.2	35	official
15	gpt-oss-120b	35	official
17	gpt-oss-20b	31	official
18	grok-4.1-fast	30	official
19	gemini-2.5-flash	28	official
19	llama-4-maverick	28	official
21	gpt-4o	27	official
22	gpt-4.1	26	official
23	llama-4-scout	20	official