Evaluation profile
Emergent Collusion
1sub-evals
0.508%Safety weight
0%Freedom weight
1components
Weights below are portfolio-specific global index weights.
Model score (lower is better)Predicted score
About this eval
Spontaneous anti-competitive coordination in repeated multi-agent auctions.
Included in the behavior ranking.
Sub-evals
| Measure | Component | Direction by ranking | Safety weight | Freedom weight |
|---|---|---|---|---|
| high_illegality_game_rateemergent-collusion/emergent-collusion.csv:high_illegality_game_rateMeasures how often autonomous buyer and seller agents spontaneously coordinate prices or market allocation in repeated auctions. | Safety: responsible_agency_control:1.000emergent-collusion | Safety: lower | 0.508% | — |
high_illegality_game_rate
Measures how often autonomous buyer and seller agents spontaneously coordinate prices or market allocation in repeated auctions.
| Rank | Model | Value | Relative performance | Provenance |
|---|---|---|---|---|
| 1 | grok-3 | 0.19 | official | |
| 2 | chatgpt-4o | 0.23 | official | |
| 3 | deepseek-v3 | 0.26 | official | |
| 4 | grok-3-mini | 0.3 | official | |
| 5 | claude-sonnet-4 | 0.32 | official | |
| 6 | claude-opus-4 | 0.36 | official | |
| 7 | gemini-2.5-pro | 0.39 | official | |
| 8 | mistral-medium-3 | 0.47 | official | |
| 9 | o3 | 0.51 | official | |
| 10 | qwen3-235b-a22b | 0.57 | official | |
| 11 | o4-mini | 0.62 | official | |
| 12 | deepseek-r1 | 0.71 | official | |
| 13 | grok-4 | 0.75 | official |