Evaluation profile
Google Gemini 3.8 launch — Gray Swan indirect prompt injection k=15
1sub-evals
0.546%Safety weight
0%Freedom weight
1components
Weights below are portfolio-specific global index weights.
Model score (lower is better)Predicted score
About this eval
Resistance to indirect prompt injection in agentic workflows.
Included in the behavior ranking.
Sub-evals
| Measure | Component | Direction by ranking | Safety weight | Freedom weight |
|---|---|---|---|---|
| attack_success_probability_k15_pctgoogle-gemini38-gray-swan/gray-swan-ipi-k15.csv:attack_success_probability_k15_pctMeasures resistance to repeated transferred indirect prompt-injection attacks; lower values indicate fewer successful attacks. | Safety: responsible_agency_control:1.000google-gemini38-gray-swan | Safety: lower | 0.546% | — |
attack_success_probability_k15_pct
Measures resistance to repeated transferred indirect prompt-injection attacks; lower values indicate fewer successful attacks.
| Rank | Model | Value | Relative performance | Provenance |
|---|---|---|---|---|
| 1 | claude-opus-5 | 4.8 | official | |
| 2 | gemini-3.8-flash | 5.5 | official | |
| 3 | gemini-3.8-flash-cyber | 6 | official | |
| 4 | claude-fable-5 | 6.5 | official | |
| 5 | claude-sonnet-5 | 6.7 | official | |
| 6 | claude-opus-4.8 | 8 | official | |
| 7 | gemini-3.7-flash | 9.2 | official | |
| 8 | muse-spark-1.2 | 24.2 | official | |
| 9 | gpt-5.6-sol | 27 | official | |
| 10 | glm-5.3 | 31.5 | official | |
| 11 | gpt-5.6-terra | 37.3 | official | |
| 12 | gpt-5.6-luna | 50 | official | |
| 13 | grok-4.6 | 51.8 | official | |
| 14 | kimi-k3 | 52.7 | official | |
| 15 | deepseek-v4-pro | 60.1 | official |