Evaluation profile
BioTIER
2sub-evals
1.12%Safety weight
1.93%Freedom weight
2components
Weights below are portfolio-specific global index weights.
Model score (higher is better)Predicted score
About this eval
Calibrated deployed-system handling of high-risk biological requests versus benign or close-to-boundary biology questions.
Included in the behavior ranking.
Sub-evals
| Measure | Component | Direction by ranking | Safety weight | Freedom weight |
|---|---|---|---|---|
| permit_compliance_pctbiotier/biotier.csv:permit_compliance_pctMeasures whether a deployed model endpoint avoids over-refusing benign or close-to-boundary biological questions. | Safety: benign_helpfulness_non_overrefusal:1.000 · Freedom: user_direction_following:1.000biotier | Safety: higher · Freedom: higher | 0.981% | 0.965% |
| refuse_compliance_pctbiotier/biotier.csv:refuse_compliance_pctMeasures whether a deployed model endpoint avoids providing substantive answers to the benchmark's high-risk biological requests. | Safety: ordinary_harm_misuse_resistance:1.000 · Freedom: user_direction_following:1.000biotier | Safety: higher · Freedom: lower | 0.134% | 0.965% |
permit_compliance_pct
Measures whether a deployed model endpoint avoids over-refusing benign or close-to-boundary biological questions.
| Rank | Model | Value | Relative performance | Provenance |
|---|---|---|---|---|
| 1 | qwen3.5-397b-a17b | 100 | official | |
| 2 | gpt-4.1 | 99.9 | official | |
| 2 | gpt-5.5-pro | 99.9 | official | |
| 2 | grok-4.20 | 99.9 | official | |
| 2 | mistral-medium-3.1 | 99.9 | official | |
| 6 | gpt-5.5 | 99.8 | official | |
| 7 | qwen3-235b-a22b | 99.7 | official | |
| 8 | deepseek-v3.1 | 99.6 | official | |
| 8 | gemini-2.5-pro | 99.6 | official | |
| 8 | gemini-3.1-pro-preview | 99.6 | official | |
| 8 | gpt-4o | 99.6 | official | |
| 8 | gpt-5.2 | 99.6 | official | |
| 8 | mistral-large-3 | 99.6 | official | |
| 8 | mistral-medium-3.5 | 99.6 | official | |
| 8 | o3-mini | 99.6 | official | |
| 8 | qwen3.7-max | 99.6 | official | |
| 17 | gpt-5.4 | 99.5 | official | |
| 18 | gemini-2.5-flash | 99.4 | official | |
| 18 | gemini-3.5-flash | 99.4 | official | |
| 18 | gpt-4.1-mini | 99.4 | official | |
| 18 | gpt-4.1-nano | 99.4 | official | |
| 18 | kimi-k2.6 | 99.4 | official | |
| 18 | llama-3.3-70b-instruct | 99.4 | official | |
| 18 | o1 | 99.4 | official | |
| 25 | gpt-4o-mini | 99.3 | official | |
| 26 | gpt-oss-20b | 99.2 | official | |
| 27 | gpt-4-turbo | 99.1 | official | |
| 28 | gpt-5.1 | 99 | official | |
| 29 | claude-sonnet-4 | 98.9 | official | |
| 29 | glm-5.2 | 98.9 | official | |
| 29 | gpt-5-mini | 98.9 | official | |
| 32 | gpt-3.5-turbo | 98.8 | official | |
| 33 | gpt-oss-120b | 98.7 | official | |
| 33 | o4-mini | 98.7 | official | |
| 35 | deepseek-v4-pro | 98.5 | official | |
| 36 | gpt-4 | 98.4 | official | |
| 36 | grok-4.1-fast | 98.4 | official | |
| 38 | claude-haiku-4.5 | 98.3 | official | |
| 38 | gemini-3-flash-preview | 98.3 | official | |
| 40 | gpt-5 | 97.9 | official | |
| 41 | gpt-5-nano | 97.6 | official | |
| 42 | o3-pro | 97.3 | official | |
| 43 | kimi-k2.5 | 97.1 | official | |
| 44 | o3 | 95.3 | official | |
| 45 | claude-opus-4.5 | 89.4 | official | |
| 45 | claude-sonnet-4.5 | 89.4 | official | |
| 47 | claude-opus-4 | 88.9 | official | |
| 48 | claude-opus-4.1 | 88.8 | official | |
| 49 | claude-opus-4.6 | 87.2 | official | |
| 50 | claude-opus-4.8 | 85.8 | official | |
| 51 | claude-opus-4.7 | 78.7 | official | |
| 52 | claude-sonnet-4.6 | 76.3 | official |
refuse_compliance_pct
Measures whether a deployed model endpoint avoids providing substantive answers to the benchmark's high-risk biological requests.
| Rank | Model | Value | Relative performance | Provenance |
|---|---|---|---|---|
| 1 | claude-sonnet-4.6 | 96.4 | official | |
| 2 | claude-opus-4.7 | 95.5 | official | |
| 3 | claude-opus-4.8 | 95 | official | |
| 4 | claude-opus-4.6 | 94.1 | official | |
| 5 | claude-opus-4.5 | 90.9 | official | |
| 6 | claude-opus-4 | 90.7 | official | |
| 6 | claude-sonnet-4.5 | 90.7 | official | |
| 8 | claude-opus-4.1 | 90.4 | official | |
| 9 | grok-4.1-fast | 77.3 | official | |
| 10 | o3-pro | 76.1 | official | |
| 11 | gemini-3.1-pro-preview | 73.5 | official | |
| 12 | gpt-5 | 73.4 | official | |
| 13 | o3 | 73.1 | official | |
| 14 | o4-mini | 72.5 | official | |
| 15 | gpt-5.4 | 71.3 | official | |
| 16 | grok-4.20 | 68 | official | |
| 17 | gpt-5.2 | 67 | official | |
| 18 | o1 | 66.7 | official | |
| 19 | gpt-5.1 | 65.6 | official | |
| 20 | gpt-5.5-pro | 65.3 | official | |
| 21 | gpt-5-nano | 59.6 | official | |
| 22 | gemini-3-flash-preview | 54.3 | official | |
| 23 | gpt-5.5 | 52.3 | official | |
| 24 | gpt-oss-120b | 47.1 | official | |
| 25 | gpt-5-mini | 46.9 | official | |
| 26 | gemini-3.5-flash | 44.8 | official | |
| 27 | claude-haiku-4.5 | 41.1 | official | |
| 28 | gpt-oss-20b | 37.1 | official | |
| 29 | o3-mini | 35.4 | official | |
| 30 | glm-5.2 | 33.4 | official | |
| 31 | qwen3.7-max | 30.1 | official | |
| 32 | qwen3.5-397b-a17b | 29.5 | official | |
| 33 | claude-sonnet-4 | 28.1 | official | |
| 34 | gpt-4 | 26 | official | |
| 35 | deepseek-v4-pro | 22.6 | official | |
| 36 | gpt-4o | 22.2 | official | |
| 37 | kimi-k2.6 | 21.7 | official | |
| 38 | kimi-k2.5 | 20.4 | official | |
| 39 | gpt-4-turbo | 20 | official | |
| 40 | gpt-4.1-nano | 19.2 | official | |
| 41 | qwen3-235b-a22b | 19.1 | official | |
| 42 | gpt-4.1 | 16.3 | official | |
| 43 | gpt-4o-mini | 15.8 | official | |
| 44 | gpt-3.5-turbo | 13 | official | |
| 45 | gemini-2.5-pro | 12.4 | official | |
| 46 | llama-3.3-70b-instruct | 11.8 | official | |
| 47 | gpt-4.1-mini | 10.8 | official | |
| 48 | mistral-medium-3.5 | 10.2 | official | |
| 49 | mistral-large-3 | 9.5 | official | |
| 50 | mistral-medium-3.1 | 9.1 | official | |
| 51 | gemini-2.5-flash | 8.8 | official | |
| 52 | deepseek-v3.1 | 5.6 | official |