Model profile
Deepseek R1
DeepSeekdeveloper
2025-01-20release date
#165 / 312overall rank
42eval lineages
Evidence summary
Deepseek R1 has an estimated overall rank of #165; its 90% source-sensitivity interval is #107–#234. Its behavior-only rank is #151; company governance moves the combined estimate to #165. Published evidence spans 42 evals and 7 of 7 behavior components. Its strongest relative result is FORTRESS (over_refusal_score, #1 of 48); its weakest is Anthropic Agentic Misalignment — lethal action (misaligned_action_rate_pct, #10 of 10).
Compare this model
Only models sharing at least one published sub-eval are listed.
Official and reference links
- Artificial Analysis ↗deepseek-r1
- Hugging Face ↗deepseek-ai/DeepSeek-R1
- OpenRouter ↗deepseek/deepseek-r1
- Official model page ↗Exact model document · Reviewed official Hugging Face owner · official repository
- Release source ↗direct research preview identity date
Published eval results
Rank is within that sub-eval. Black marks the observed result; the grey dot marks the value implied by the global rank. Values stay on each source’s native scale.
| Eval / sub-eval | Rank | Value | Distribution | Source |
|---|---|---|---|---|
| AA-Omnisciencehallucination_rate | #186 / 330 | ↓0.8345 | Source ↗official | |
| AIRBench 2024 Safety Scenariossafety_scenarios | #62 / 80 | ↑0.529 | Source ↗official | |
| Anthropic Agentic Misalignment — blackmailmisaligned_action_rate_pct | #8 / 16 | ↓79 | Source ↗official | |
| Anthropic Agentic Misalignment — corporate espionagemisaligned_action_rate_pct | #9 / 16 | ↓73 | Source ↗official | |
| Anthropic Agentic Misalignment — lethal actionmisaligned_action_rate_pct | #10 / 10 | ↓94 | Source ↗official | |
| BullshitBench v2clear_pushback_rate | #92 / 106 | ↑0.075 | Source ↗official | |
| CAIS Risk Indexagent_red_teaming | #38 / 45 | ↓90.9 | Source ↗official | |
| CAIS Risk Indexbioweapons_assistance | #47 / 50 | ↓94.4 | Source ↗official | |
| CAIS Risk Indexhle_overconfidence | #42 / 51 | ↓73 | Source ↗official | |
| CAIS Risk Indexmachiavelli | #18 / 47 | ↓85.1 | Source ↗official | |
| CAIS Risk Indexmask | #37 / 53 | ↓42.7 | Source ↗official | |
| CAIS Risk Indextextquests_harm | #12 / 50 | ↓15.4 | Source ↗official | |
| ChiSafetyBenchharmful_response_rate | #8 / 14 | ↓0.65 | Source ↗official | |
| ChiSafetyBenchmcq_score | #9 / 12 | ↑71.41 | Source ↗official | |
| Cisco AI Defense Rolling Single-Turn Leaderboardsingle_turn_attack_success_rate | #98 / 104 | ↓77.92 | Source ↗official | |
| Confabulationsconfabulation_rate | #15 / 52 | ↓15.1 | Source ↗official | |
| Emergent Collusionhigh_illegality_game_rate | #12 / 13 | ↓0.71 | Source ↗official | |
| Enkrypt AI Safety Leaderboardbias_attack_non_success_rate | #60 / 241 | ↑25.06 | Source ↗official | |
| Enkrypt AI Safety Leaderboardcbrn_attack_non_success_rate | #239 / 241 | ↑46.5 | Source ↗official | |
| Enkrypt AI Safety Leaderboardharmful_attack_non_success_rate | #197 / 241 | ↑42.22 | Source ↗official | |
| Enkrypt AI Safety Leaderboardtoxicity_attack_non_success_rate | #77 / 239 | ↑97.91 | Source ↗official | |
| FinEval 6.0 Safety Awarenesssafety_awareness_score | #4 / 8 | ↑81.03 | Source ↗official | |
| FlagEval Safety and Valuesa1_qualified_rate | #13 / 18 | ↑77.33 | Source ↗official | |
| FlagEval Safety and Valuesa2_qualified_rate | #14 / 18 | ↑76.69 | Source ↗official | |
| FlagEval Safety and Valuesa3_qualified_rate | #16 / 18 | ↑82.31 | Source ↗official | |
| FlagEval Safety and Valuesa4_qualified_rate | #12 / 18 | ↑84.15 | Source ↗official | |
| FlagEval Safety and Valuesa5_qualified_rate | #15 / 18 | ↑69.38 | Source ↗official | |
| FORTRESSaverage_risk_score | #49 / 49 | ↓74.39 | Source ↗official | |
| FORTRESSover_refusal_score | #1 / 48 | ↓0.49 | Source ↗official | |
| HELM Safetyanthropic_red_team | #60 / 80 | ↑0.975 | Source ↗official | |
| HELM Safetybbq | #17 / 80 | ↑0.9657 | Source ↗official | |
| HELM Safetyharmbench | #71 / 80 | ↑0.4987 | Source ↗official | |
| HELM Safetysimple_safety_tests | #51 / 80 | ↑0.9793 | Source ↗official | |
| HELM Safetyxstest | #38 / 80 | ↑0.9617 | Source ↗official | |
| HUMAINE Trust, Ethics and Safetytrust_ethics_safety_score | #15 / 54 | ↑27.99 | Source ↗official | |
| LiveSecBenchethics | #26 / 43 | ↑39.41 | Source ↗official | |
| LiveSecBenchfactuality | #33 / 43 | ↑32.41 | Source ↗official | |
| LiveSecBenchlegality | #6 / 43 | ↑85.68 | Source ↗official | |
| LiveSecBenchprivacy | #20 / 43 | ↑59.73 | Source ↗official | |
| LiveSecBenchpsychological_health | #16 / 43 | ↑58.87 | Source ↗official | |
| MACHIAVELLIdeception_relative_random_pct | #18 / 50 | ↓85.1 | Source ↗official | |
| MASKlying_probability_pct | #34 / 53 | ↓44.19 | Source ↗official | |
| OpenAgentSafetyllm_judge_safety_vulnerable | #5 / 7 | ↓62.78 | Source ↗official | |
| OpenAgentSafetyrule_based_safety_vulnerable | #4 / 7 | ↓32.78 | Source ↗official | |
| OpenAgentSafetysuccessful_completion | #6 / 7 | ↑20.86 | Source ↗official | |
| PandaBench JBB direct-request panelsafety_rate | #42 / 46 | ↑0.93 | Source ↗official | |
| PHAREbias_resistance_diagnostic | #64 / 66 | ↑0.2549 | Source ↗official | |
| PHAREhallucination_resistance_diagnostic | #39 / 70 | ↑0.7289 | Source ↗official | |
| PHAREharm_resistance_diagnostic | #27 / 70 | ↑0.9515 | Source ↗official | |
| PHAREjailbreak_resistance_diagnostic | #50 / 67 | ↑0.4067 | Source ↗official | |
| RealityTest — Text AI-Identity Disclosuredisclosure_probability | #11 / 17 | ↑0.234 | Source ↗official | |
| RefusalBenchyouden_j | #19 / 19 | ↑-0.004255 | Source ↗official | |
| SABERoverall_safety_rate | #13 / 13 | ↑15.29 | Source ↗official | |
| SABERscenario_a_safety_rate | #13 / 13 | ↑15.69 | Source ↗official | |
| SABERscenario_b_safety_rate | #13 / 13 | ↑24.12 | Source ↗official | |
| SABERscenario_c_safety_rate | #13 / 13 | ↑8.07 | Source ↗official | |
| SafeDialBenchaggression | #2 / 18 | ↑7.273 | Source ↗official | |
| SafeDialBenchethics | #18 / 18 | ↑7.303 | Source ↗official | |
| SafeDialBenchfairness | #3 / 18 | ↑7.607 | Source ↗official | |
| SafeDialBenchlegality | #12 / 18 | ↑7.38 | Source ↗official | |
| SafeDialBenchmorality | #12 / 18 | ↑7.183 | Source ↗official | |
| SafeDialBenchprivacy | #13 / 18 | ↑7.193 | Source ↗official | |
| Shelleducation_jsr | #9 / 14 | ↓0.672 | Source ↗official | |
| Shellfinance_jsr | #8 / 14 | ↓0.522 | Source ↗official | |
| Shellmanagement_jsr | #9 / 14 | ↓0.682 | Source ↗official | |
| SM-Benchadversarial | #22 / 79 | ↑84.88 | Source ↗official | |
| SM-Benchambiguous_interpretation | #72 / 79 | ↑67.26 | Source ↗official | |
| SM-Benchanti_hallucination | #63 / 79 | ↑84.82 | Source ↗official | |
| SM-Bencheq_boundaries | #59 / 79 | ↑57.58 | Source ↗official | |
| SM-Benchoverfit | #38 / 79 | ↑74.86 | Source ↗official | |
| Social Welfare Function Benchmarkfairness | #8 / 19 | ↑0.523 | Source ↗official | |
| SOSBenchbiology_pvr | #19 / 23 | ↓0.814 | Source ↗official | |
| SOSBenchchemistry_pvr | #22 / 23 | ↓0.834 | Source ↗official | |
| SOSBenchmedicine_pvr | #18 / 23 | ↓0.806 | Source ↗official | |
| SOSBenchpharmacology_pvr | #22 / 23 | ↓0.964 | Source ↗official | |
| SOSBenchphysics_pvr | #22 / 23 | ↓0.872 | Source ↗official | |
| SOSBenchpsychology_pvr | #19 / 23 | ↓0.806 | Source ↗official | |
| SpeciesismBenchexplicit_speciesism_scale | #1 / 7 | ↓1.84 | Source ↗official | |
| SpeciesismBenchmorally_wrong_rate | #3 / 8 | ↑39.88 | Source ↗official | |
| SpeciesismBenchspeciesism_recognition_rate | #7 / 8 | ↑76.42 | Source ↗official | |
| SpeciEvalbelief_animal_sentience | #77 / 113 | ↑6.62 | Source ↗official | |
| SpeciEvalland_animal_4ns | #49 / 113 | ↓4.47 | Source ↗official | |
| SpeciEvalsea_animal_4ns | #36 / 113 | ↓4.65 | Source ↗official | |
| SpeciEvalspeciesism | #69 / 113 | ↓2.15 | Source ↗official | |
| SYCON Benchfalse_presupposition_tof | #1 / 11 | ↑3.21 | Source ↗official | |
| SYCON Benchunethical_queries_tof | #2 / 11 | ↑2.72 | Source ↗official | |
| TrustLLM contemporary collapsed applicationtrustllm | #3 / 8 | ↑0.62 | Source ↗official | |
| Vectara HHEM Factual Consistencyfactual_consistency_rate | #66 / 94 | ↑88.7 | Source ↗official | |
| VETO Misfired Alignmentmisfired_alignment_rate_pct | #1 / 25 | ↓4.7 | Source ↗official |
Values evaluations
Descriptive values and political-framing results are separate from safety/ethics ranks. Each strip shows the evaluation’s observed model range; its endpoint labels state what lower and higher values mean.
UGI Political Values
ValueCompass
| Dimension | Value | Distribution |
|---|---|---|
| Universalism | 69.2 | |
| Self-direction | 51 | |
| Care / Harm | 48.9 | |
| Fairness / Cheating | 47.5 | |
| Ethical | 94.9 |
CAISI CCP narrative alignment
CCPBench political narrative alignment
| Dimension | Value | Distribution |
|---|---|---|
| CCP-narrative alignment — all questions | 3.92 | |
| CCP-narrative alignment — China topics | 4.61 | |
| CCP-narrative alignment — non-China controls | 1.85 |