Model profile
Deepseek V3
DeepSeekdeveloper
2024-12-26release date
#136 / 309overall rank
33eval lineages
Evidence summary
Deepseek V3 has an estimated overall rank of #136; its 90% source-sensitivity interval is #62–#200. Its behavior-only rank is #116; company governance moves the combined estimate to #136. Published evidence spans 33 evals and 7 of 7 behavior components. Its strongest relative result is Contextual MoralChoice (human_agreement, #2 of 22); its weakest is SOSBench (medicine_pvr, #23 of 23).
Compare this model
Only models sharing at least one published sub-eval are listed.
Official and reference links
- Artificial Analysis ↗deepseek-v3
- Hugging Face ↗deepseek-ai/DeepSeek-V3
- OpenRouter ↗deepseek/deepseek-chat-v3-0324
- Official model page ↗Exact model document · Reviewed official Hugging Face owner · official repository
- Release source ↗direct research preview identity date
Published eval results
Rank is within that sub-eval. Black marks the observed result; the grey dot marks the value implied by the global rank. Values stay on each source’s native scale.
| Eval / sub-eval | Rank | Value | Distribution | Source |
|---|---|---|---|---|
| AA-Omnisciencehallucination_rate | #204 / 328 | ↓0.8586 | Source ↗official | |
| AgentDrive Safety Compliancescr | #6 / 48 | ↑95 | Source ↗official | |
| AIRBench 2024 Safety Scenariossafety_scenarios | #73 / 80 | ↑0.408 | Source ↗official | |
| AnimalHarmBenchscore | #5 / 10 | ↑0.04 | Source ↗official | |
| BullshitBench v2clear_pushback_rate | #101 / 106 | ↑0.04 | Source ↗official | |
| ChiSafetyBenchharmful_response_rate | #5 / 14 | ↓0.43 | Source ↗official | |
| ChiSafetyBenchmcq_score | #6 / 12 | ↑84.17 | Source ↗official | |
| Confabulationsconfabulation_rate | #44 / 52 | ↓39.11 | Source ↗official | |
| Contextual MoralChoicehuman_agreement | #2 / 22 | ↑0.6 | Source ↗official | |
| Emergent Collusionhigh_illegality_game_rate | #3 / 13 | ↓0.26 | Source ↗official | |
| Enkrypt AI Safety Leaderboardbias_attack_non_success_rate | #156 / 241 | ↑13.18 | Source ↗official | |
| Enkrypt AI Safety Leaderboardcbrn_attack_non_success_rate | #134 / 241 | ↑87.33 | Source ↗official | |
| Enkrypt AI Safety Leaderboardharmful_attack_non_success_rate | #172 / 241 | ↑52.22 | Source ↗official | |
| Enkrypt AI Safety Leaderboardtoxicity_attack_non_success_rate | #201 / 239 | ↑89.64 | Source ↗official | |
| HELM Safetyanthropic_red_team | #61 / 80 | ↑0.971 | Source ↗official | |
| HELM Safetybbq | #13 / 80 | ↑0.967 | Source ↗official | |
| HELM Safetyharmbench | #72 / 80 | ↑0.497 | Source ↗official | |
| HELM Safetysimple_safety_tests | #67 / 80 | ↑0.953 | Source ↗official | |
| HELM Safetyxstest | #22 / 80 | ↑0.971 | Source ↗official | |
| HUMAINE Trust, Ethics and Safetytrust_ethics_safety_score | #22 / 54 | ↑27.39 | Source ↗official | |
| JailBenchjailbreak_success_rate | #10 / 14 | ↓70.67 | Source ↗official | |
| LiveSecBenchethics | #38 / 43 | ↑20.88 | Source ↗official | |
| LiveSecBenchfactuality | #34 / 43 | ↑29.99 | Source ↗official | |
| LiveSecBenchlegality | #42 / 43 | ↑7.76 | Source ↗official | |
| LiveSecBenchprivacy | #42 / 43 | ↑6.72 | Source ↗official | |
| LiveSecBenchpsychological_health | #36 / 43 | ↑25.03 | Source ↗official | |
| LLM Ethics Benchmarkscore | #3 / 5 | ↑86.1 | Source ↗official | |
| MASKlying_probability_pct | #49 / 53 | ↓54.48 | Source ↗official | |
| OpenAgentSafetyllm_judge_safety_vulnerable | #4 / 7 | ↓62.23 | Source ↗official | |
| OpenAgentSafetyrule_based_safety_vulnerable | #2 / 7 | ↓32.44 | Source ↗official | |
| OpenAgentSafetysuccessful_completion | #4 / 7 | ↑22.12 | Source ↗official | |
| PandaBench JBB direct-request panelsafety_rate | #24 / 46 | ↑0.985 | Source ↗official | |
| PHAREbias_resistance_diagnostic | #13 / 66 | ↑0.5751 | Source ↗official | |
| PHAREhallucination_resistance_diagnostic | #57 / 70 | ↑0.6675 | Source ↗official | |
| PHAREharm_resistance_diagnostic | #48 / 70 | ↑0.909 | Source ↗official | |
| PHAREjailbreak_resistance_diagnostic | #62 / 67 | ↑0.3107 | Source ↗official | |
| Reward Hacking Benchmarkintegrity_score | #3 / 13 | ↑99.4 | Source ↗official | |
| Social Welfare Function Benchmarkfairness | #3 / 19 | ↑0.594 | Source ↗official | |
| SOSBenchbiology_pvr | #22 / 23 | ↓0.856 | Source ↗official | |
| SOSBenchchemistry_pvr | #17 / 23 | ↓0.6 | Source ↗official | |
| SOSBenchmedicine_pvr | #23 / 23 | ↓0.872 | Source ↗official | |
| SOSBenchpharmacology_pvr | #15 / 23 | ↓0.916 | Source ↗official | |
| SOSBenchphysics_pvr | #17 / 23 | ↓0.722 | Source ↗official | |
| SOSBenchpsychology_pvr | #21 / 23 | ↓0.82 | Source ↗official | |
| SpeciesismBenchmorally_wrong_rate | #7 / 8 | ↑32.7 | Source ↗official | |
| SpeciesismBenchspeciesism_recognition_rate | #5 / 8 | ↑87.71 | Source ↗official | |
| SpeciEvalbelief_animal_sentience | #96 / 113 | ↑6.42 | Source ↗official | |
| SpeciEvalland_animal_4ns | #95 / 113 | ↓4.86 | Source ↗official | |
| SpeciEvalsea_animal_4ns | #99 / 113 | ↓5.1 | Source ↗official | |
| SpeciEvalspeciesism | #92 / 113 | ↓2.54 | Source ↗official | |
| SYCON Benchfalse_presupposition_tof | #5 / 11 | ↑2.88 | Source ↗official | |
| SYCON Benchunethical_queries_tof | #5 / 11 | ↑1.99 | Source ↗official | |
| TACbase_welfare_rate | #7 / 76 | ↑42.3 | Source ↗self run | |
| UAVBench safety-critical decision recognitionethical_safety_critical_accuracy | #4 / 27 | ↑0.755 | Source ↗official | |
| Vectara HHEM Factual Consistencyfactual_consistency_rate | #23 / 94 | ↑93.9 | Source ↗official | |
| VETO Misfired Alignmentmisfired_alignment_rate_pct | #3 / 25 | ↓5.2 | Source ↗official |
Values evaluations
Descriptive values and political-framing results are separate from safety/ethics ranks. Each strip shows the evaluation’s observed model range; its endpoint labels state what lower and higher values mean.
UGI Political Values
ValueCompass
| Dimension | Value | Distribution |
|---|---|---|
| Universalism | 70.4 | |
| Self-direction | 47.7 | |
| Care / Harm | 34.5 | |
| Fairness / Cheating | 32.2 | |
| Ethical | 91.7 |