Evaluation profile
Cisco AI Defense Rolling Single-Turn Leaderboard
1sub-evals
0.254%Safety weight
1.82%Freedom weight
1components
Weights below are portfolio-specific global index weights.
Model score (lower is better)Predicted score
About this eval
Single-turn resistance to Cisco adversarial prompts in the rolling public leaderboard.
Included in the behavior ranking.
Sub-evals
| Measure | Component | Direction by ranking | Safety weight | Freedom weight |
|---|---|---|---|---|
| single_turn_attack_success_ratecisco-ai-defense-rolling/cisco-ai-defense-rolling.csv:single_turn_attack_success_rateMeasures how often Cisco's adversarial single-turn prompts elicited outputs classified as attack successes. | Safety: ordinary_harm_misuse_resistance:1.000 · Freedom: user_direction_following:1.000cisco-ai-defense-rolling | Safety: lower · Freedom: higher | 0.254% | 1.82% |
single_turn_attack_success_rate
Measures how often Cisco's adversarial single-turn prompts elicited outputs classified as attack successes.
| Rank | Model | Value | Relative performance | Provenance |
|---|---|---|---|---|
| 1 | claude-opus-4.5 | 2.193 | official | |
| 2 | claude-opus-4.1 | 2.243 | official | |
| 2 | claude-sonnet-5 | 2.243 | official | |
| 4 | claude-opus-4.8 | 2.293 | official | |
| 5 | gpt-5.5 | 2.343 | official | |
| 6 | claude-sonnet-4.5 | 2.592 | official | |
| 7 | claude-opus-4.7 | 2.642 | official | |
| 8 | gpt-5.4 | 2.742 | official | |
| 9 | claude-sonnet-4.6 | 3.041 | official | |
| 10 | claude-sonnet-4 | 3.091 | official | |
| 11 | claude-3.5-sonnet | 3.29 | official | |
| 12 | claude-haiku-4.5 | 3.539 | official | |
| 13 | claude-opus-4.6 | 3.639 | official | |
| 13 | gpt-5.1-chat | 3.639 | official | |
| 15 | gpt-5.3-chat | 4.337 | official | |
| 16 | claude-opus-4 | 4.736 | official | |
| 16 | gpt-5.2 | 4.736 | official | |
| 18 | gpt-5-mini | 5.085 | official | |
| 19 | claude-3-opus | 5.833 | official | |
| 20 | gpt-oss-120b | 7.245 | official | |
| 21 | gpt-5-nano | 7.527 | official | |
| 22 | claude-3-sonnet | 7.827 | official | |
| 23 | gpt-5 | 8.076 | official | |
| 24 | gpt-5.4-nano | 9.521 | official | |
| 24 | minimax-m2.5 | 9.521 | official | |
| 26 | gpt-5.4-mini | 9.571 | official | |
| 27 | claude-3.7-sonnet | 11.27 | official | |
| 28 | o4-mini | 12.31 | official | |
| 29 | gpt-oss-safeguard-120b | 12.33 | official | |
| 30 | gpt-oss-20b | 14.17 | official | |
| 31 | gpt-oss-safeguard-20b | 17.55 | official | |
| 32 | minimax-m2 | 18.05 | official | |
| 33 | gemini-3-pro-preview | 18.1 | official | |
| 34 | minimax-m2.1 | 18.59 | official | |
| 35 | kimi-k2 | 20.59 | official | |
| 36 | claude-3.5-haiku | 20.74 | official | |
| 37 | claude-3-haiku | 21.14 | official | |
| 38 | llama-3.2-1b-instruct | 21.39 | official | |
| 39 | grok-4 | 21.78 | official | |
| 40 | grok-4.5 | 22.98 | official | |
| 41 | kimi-k2.5 | 24.13 | official | |
| 42 | grok-4.1-fast | 29.06 | official | |
| 43 | grok-4-fast | 30.03 | official | |
| 44 | llama-3.1-405b-instruct | 31.56 | official | |
| 45 | grok-4.20 | 31.95 | official | |
| 46 | grok-4.3 | 32.95 | official | |
| 47 | grok-4.20-multi-agent | 33 | official | |
| 48 | nova-2.0-lite | 34.05 | official | |
| 49 | phi-4-multimodal-instruct | 35.59 | official | |
| 50 | phi-4 | 35.89 | official | |
| 51 | qwen3-235b-a22b | 37.49 | official | |
| 52 | glm-5 | 37.79 | official | |
| 53 | qwen3-next-80b-a3b | 38.78 | official | |
| 54 | llama-3.1-70b-instruct | 38.98 | official | |
| 55 | nova-premier | 40.33 | official | |
| 56 | nemotron-3-super-120b-a12b | 40.43 | official | |
| 57 | llama-3.2-3b-instruct | 40.58 | official | |
| 58 | qwen3-coder-480b-a35b-instruct | 40.78 | official | |
| 59 | gpt-4o | 42.02 | official | |
| 60 | phi-4-mini | 42.72 | official | |
| 61 | llama-3.2-instruct-11b-vision | 49.05 | official | |
| 62 | nemotron-3-nano-30b-a3b | 49.1 | official | |
| 63 | nvidia-nemotron-nano-12b-v2 | 50.3 | official | |
| 64 | qwen3-coder-30b-a3b-instruct | 52.09 | official | |
| 65 | llama-3.1-8b-instruct | 52.74 | official | |
| 66 | nova-pro | 53.34 | official | |
| 67 | gpt-4.1 | 53.49 | official | |
| 68 | llama-3.3-70b-instruct | 53.54 | official | |
| 69 | llama-3.2-90b-vision-instruct | 53.94 | official | |
| 70 | deepseek-v3.2-speciale | 54.99 | official | |
| 70 | gpt-4o-mini | 54.99 | official | |
| 72 | llama-4-maverick | 56.23 | official | |
| 73 | voxtral-mini-3b | 57.78 | official | |
| 74 | llama-4-scout | 58.72 | official | |
| 75 | gemma-3-4b | 59.02 | official | |
| 76 | nvidia-nemotron-nano-9b-v2 | 59.27 | official | |
| 77 | gpt-4.1-nano | 60.62 | official | |
| 78 | command-r | 62.91 | official | |
| 79 | mistral-large | 63.06 | official | |
| 80 | nova-lite | 63.61 | official | |
| 81 | glm-4.7 | 63.96 | official | |
| 82 | ministral-3-3b | 64.21 | official | |
| 83 | nova-micro | 64.91 | official | |
| 84 | gemma-3-12b | 65.55 | official | |
| 85 | pixtral-large | 69.04 | official | |
| 86 | gemma-3-27b-it | 69.44 | official | |
| 87 | mistral-large-2 | 69.79 | official | |
| 88 | voxtral-small-24b | 69.94 | official | |
| 89 | ministral-3-8b | 71.09 | official | |
| 90 | command-r-plus | 71.39 | official | |
| 91 | glm-4.7-flash | 72.03 | official | |
| 92 | gpt-4.1-mini | 72.73 | official | |
| 93 | ministral-3-14b | 72.83 | official | |
| 94 | magistral-small | 74.28 | official | |
| 95 | mistral-small-3.1 | 74.33 | official | |
| 96 | deepseek-v3.1 | 75.17 | official | |
| 97 | deepseek-v3.2 | 76.72 | official | |
| 98 | deepseek-r1 | 77.92 | official | |
| 99 | qwen3-32b | 77.97 | official | |
| 100 | devstral-2 | 79.76 | official | |
| 101 | mistral-large-3 | 80.16 | official | |
| 102 | command-a | 81.91 | official | |
| 103 | grok-3 | 84.15 | official | |
| 104 | grok-3-mini | 86.64 | official |