Evaluation profile
Anthropic Claude 4 System Card
4sub-evals
0.196%Safety weight
0.0709%Freedom weight
3components
Weights below are portfolio-specific global index weights.
Model score (see each sub-eval direction)Predicted score
About this eval
Developer-reported release-time safety behavior.
Included in the behavior ranking.
Sub-evals
| Measure | Component | Direction by ranking | Safety weight | Freedom weight |
|---|---|---|---|---|
| agentic_coding_safetysystem-cards-2026-08-01/anthropic-claude4.csv:agentic_coding_safetySafety rate when coding tools are available for malicious-use requests. | Safety: responsible_agency_control:1.000anthropic-claude4 | Safety: higher | 0.0457% | — |
| benign_request_refusalsystem-cards-2026-08-01/anthropic-claude4.csv:benign_request_refusalRefusal rate on benign requests. | Safety: benign_helpfulness_non_overrefusal:1.000 · Freedom: user_direction_following:1.000anthropic-claude4 | Safety: lower · Freedom: lower | 0.118% | 0.0236% |
| harmful_request_safetysystem-cards-2026-08-01/anthropic-claude4.csv:harmful_request_safetyRate of harmless responses to harmful requests. | Safety: ordinary_harm_misuse_resistance:1.000 · Freedom: user_direction_following:1.000anthropic-claude4 | Safety: higher · Freedom: lower | 0.0161% | 0.0236% |
| strongreject_jailbreak_successsystem-cards-2026-08-01/anthropic-claude4.csv:strongreject_jailbreak_successSuccessful jailbreak rate on the StrongREJECT attack panel. | Safety: ordinary_harm_misuse_resistance:1.000 · Freedom: user_direction_following:1.000anthropic-claude4 | Safety: lower · Freedom: higher | 0.0161% | 0.0236% |
agentic_coding_safety
Safety rate when coding tools are available for malicious-use requests.
| Rank | Model | Value | Relative performance | Provenance |
|---|---|---|---|---|
| 1 | claude-sonnet-4 | 0.9 | official | |
| 2 | claude-opus-4 | 0.88 | official | |
| 3 | claude-3.7-sonnet | 0.85 | official |
benign_request_refusal
Refusal rate on benign requests.
| Rank | Model | Value | Relative performance | Provenance |
|---|---|---|---|---|
| 1 | claude-opus-4 | 0.0007 | official | |
| 2 | claude-sonnet-4 | 0.0023 | official | |
| 3 | claude-3.7-sonnet | 0.0045 | official |
harmful_request_safety
Rate of harmless responses to harmful requests.
| Rank | Model | Value | Relative performance | Provenance |
|---|---|---|---|---|
| 1 | claude-sonnet-4 | 0.9899 | official | |
| 2 | claude-3.7-sonnet | 0.9896 | official | |
| 3 | claude-opus-4 | 0.9843 | official |
strongreject_jailbreak_success
Successful jailbreak rate on the StrongREJECT attack panel.
| Rank | Model | Value | Relative performance | Provenance |
|---|---|---|---|---|
| 1 | claude-sonnet-4 | 0.0266 | official | |
| 2 | claude-opus-4 | 0.0714 | official | |
| 3 | claude-3.7-sonnet | 0.0809 | official |