Evaluation profile
NESSiE Necessary Safety Benchmark
1sub-evals
0.825%Safety weight
0%Freedom weight
2components
Weights below are portfolio-specific global index weights.
Model score (higher is better)Predicted score
About this eval
Necessary-condition information and access safety while retaining benign helpfulness in paired, low-complexity scenarios.
Included in the behavior ranking.
Sub-evals
| Measure | Component | Direction by ranking | Safety weight | Freedom weight |
|---|---|---|---|---|
| sh_scorenessie/nessie.csv:sh_scoreMeasures whether a model respects simple information and access boundaries without falling back to blanket refusal. | Safety: benign_helpfulness_non_overrefusal:0.500|ordinary_harm_misuse_resistance:0.500nessie | Safety: higher | 0.825% | — |
sh_score
Measures whether a model respects simple information and access boundaries without falling back to blanket refusal.
| Rank | Model | Value | Relative performance | Provenance |
|---|---|---|---|---|
| 1 | gpt-5 | 94.87 | official | |
| 2 | gpt-oss-20b | 93.45 | official | |
| 3 | gemini-2.5-pro | 92.31 | official | |
| 3 | gpt-5.1 | 92.31 | official | |
| 5 | grok-4-fast | 92.02 | official | |
| 6 | gpt-4.1-mini | 87.18 | official | |
| 7 | gpt-5.2 | 86.04 | official | |
| 8 | claude-opus-4.5 | 82.62 | official | |
| 9 | gemini-2.5-flash | 81.77 | official | |
| 10 | claude-sonnet-4 | 80.34 | official | |
| 11 | claude-sonnet-4.5 | 79.49 | official | |
| 11 | gemini-3-flash-preview | 79.49 | official | |
| 13 | qwen3-vl-32b-instruct | 62.39 | official | |
| 14 | qwen3-vl-8b-instruct | 55.84 | official | |
| 15 | mistral-7b-instruct | 28.21 | official | |
| 16 | llama-2-7b-chat | 15.38 | official |