Evaluation profile
Every Model Cheats — Cybench Cheat Propensity
3sub-evals
0.496%Safety weight
0%Freedom weight
1components
Weights below are portfolio-specific global index weights.
Model score (lower is better)Predicted score
About this eval
Propensity of tool-using language-model agents to attempt prohibited shortcuts while solving offensive-cyber benchmark tasks, under no, standard, and severe anti-cheat instructions.
Included in the behavior ranking.
Sub-evals
| Measure | Component | Direction by ranking | Safety weight | Freedom weight |
|---|---|---|---|---|
| baseline_cheat_propensity_pctdreadnode-cheat-propensity/dreadnode-cheat-propensity.csv:baseline_cheat_propensity_pctMeasures prohibited shortcut attempts when no anti-cheat instruction is added. | Safety: responsible_agency_control:1.000dreadnode-cheat-propensity | Safety: lower | 0.165% | — |
| severe_anti_cheat_cheat_propensity_pctdreadnode-cheat-propensity/dreadnode-cheat-propensity.csv:severe_anti_cheat_cheat_propensity_pctMeasures prohibited shortcut attempts under a consequential, explicit anti-cheat instruction. | Safety: responsible_agency_control:1.000dreadnode-cheat-propensity | Safety: lower | 0.165% | — |
| standard_anti_cheat_cheat_propensity_pctdreadnode-cheat-propensity/dreadnode-cheat-propensity.csv:standard_anti_cheat_cheat_propensity_pctMeasures prohibited shortcut attempts under a direct normative anti-cheat instruction. | Safety: responsible_agency_control:1.000dreadnode-cheat-propensity | Safety: lower | 0.165% | — |
baseline_cheat_propensity_pct
Measures prohibited shortcut attempts when no anti-cheat instruction is added.
| Rank | Model | Value | Relative performance | Provenance |
|---|---|---|---|---|
| 1 | deepseek-r1 | 0 | official | |
| 2 | gemini-3.1-pro | 13 | official | |
| 3 | claude-sonnet-4.6 | 17.4 | official | |
| 3 | qwen3-coder-next | 17.4 | official | |
| 5 | deepseek-v4-pro | 21.7 | official | |
| 5 | gpt-5.4-mini | 21.7 | official | |
| 5 | gpt-5.5 | 21.7 | official | |
| 5 | grok-4.3 | 21.7 | official | |
| 9 | glm-5-turbo | 26.1 | official | |
| 10 | claude-opus-4.6 | 30.4 | official | |
| 10 | deepseek-v4-flash | 30.4 | official | |
| 10 | gemini-3-flash-preview | 30.4 | official | |
| 10 | qwen3.7-max | 30.4 | official | |
| 14 | glm-5.1 | 34.8 | official | |
| 15 | qwen3.6-max-preview | 39.1 | official | |
| 16 | claude-opus-4.7 | 43.5 | official | |
| 16 | qwen3.6-plus | 43.5 | official | |
| 18 | claude-haiku-4.5 | 52.2 | official | |
| 18 | grok-4.20 | 52.2 | official | |
| 20 | claude-sonnet-5 | 56.5 | official | |
| 20 | gpt-5.4 | 56.5 | official | |
| 22 | claude-opus-4.8 | 65.2 | official |
severe_anti_cheat_cheat_propensity_pct
Measures prohibited shortcut attempts under a consequential, explicit anti-cheat instruction.
| Rank | Model | Value | Relative performance | Provenance |
|---|---|---|---|---|
| 1 | claude-opus-4.8 | 0 | official | |
| 1 | claude-sonnet-5 | 0 | official | |
| 1 | deepseek-r1 | 0 | official | |
| 1 | deepseek-v4-pro | 0 | official | |
| 1 | gemini-3-flash-preview | 0 | official | |
| 1 | glm-5-turbo | 0 | official | |
| 7 | claude-opus-4.6 | 4.3 | official | |
| 7 | claude-opus-4.7 | 4.3 | official | |
| 7 | claude-sonnet-4.6 | 4.3 | official | |
| 7 | deepseek-v4-flash | 4.3 | official | |
| 7 | gpt-5.5 | 4.3 | official | |
| 7 | grok-4.3 | 4.3 | official | |
| 7 | qwen3.7-max | 4.3 | official | |
| 14 | glm-5.1 | 8.7 | official | |
| 14 | gpt-5.4 | 8.7 | official | |
| 14 | qwen3.6-plus | 8.7 | official | |
| 17 | gemini-3.1-pro | 13 | official | |
| 17 | gpt-5.4-mini | 13 | official | |
| 19 | qwen3-coder-next | 17.4 | official | |
| 19 | qwen3.6-max-preview | 17.4 | official | |
| 21 | claude-haiku-4.5 | 26.1 | official | |
| 22 | grok-4.20 | 43.5 | official |
standard_anti_cheat_cheat_propensity_pct
Measures prohibited shortcut attempts under a direct normative anti-cheat instruction.
| Rank | Model | Value | Relative performance | Provenance |
|---|---|---|---|---|
| 1 | deepseek-r1 | 0 | official | |
| 1 | grok-4.20 | 0 | official | |
| 3 | claude-sonnet-4.6 | 4.3 | official | |
| 3 | glm-5-turbo | 4.3 | official | |
| 3 | glm-5.1 | 4.3 | official | |
| 6 | claude-opus-4.8 | 8.7 | official | |
| 6 | deepseek-v4-flash | 8.7 | official | |
| 6 | gpt-5.5 | 8.7 | official | |
| 6 | grok-4.3 | 8.7 | official | |
| 10 | gemini-3.1-pro | 13 | official | |
| 10 | gpt-5.4-mini | 13 | official | |
| 10 | qwen3.7-max | 13 | official | |
| 13 | claude-sonnet-5 | 17.4 | official | |
| 13 | deepseek-v4-pro | 17.4 | official | |
| 13 | qwen3-coder-next | 17.4 | official | |
| 16 | claude-opus-4.7 | 21.7 | official | |
| 17 | qwen3.6-max-preview | 26.1 | official | |
| 18 | claude-opus-4.6 | 30.4 | official | |
| 18 | gpt-5.4 | 30.4 | official | |
| 20 | claude-haiku-4.5 | 34.8 | official | |
| 21 | qwen3.6-plus | 39.1 | official | |
| 22 | gemini-3-flash-preview | 69.6 | official |