Model profile
DeepSeek v3
DeepSeekdeveloper
2024-12-26release date
#150 / 333Safety rank
#9 / 645Freedom rank
Evidence summary
Safety. DeepSeek v3 has an estimated Safety rank of #150; its 90% source-sensitivity interval is #60–#219. Its behavior-only rank is #133; company governance moves the combined estimate to #150. Published Safety evidence spans 34 eval lineages and 7 of 7 components. Its strongest relative result is Contextual MoralChoice (human_agreement, #2 of 22); its weakest is Concordia — CyberSecEval2-PromptInjection (safety_score, #63 of 63).
Freedom. DeepSeek v3 has an estimated Freedom rank of #9; its 90% source-sensitivity interval is #14–#99. Published Freedom evidence spans 16 eval lineages and 1 of 1 components. Its strongest relative result is Concordia — AirBench-Deception (safety_score, #2 of 63); its weakest is ChiSafetyBench (harmful_response_rate, #8 of 14).
Compare this model
Only models sharing at least one published sub-eval are listed.
Official and reference links
- Artificial Analysis ↗deepseek-v3
- Hugging Face ↗deepseek-ai/DeepSeek-V3
- OpenRouter ↗deepseek/deepseek-chat-v3-0324
- Official model page ↗Exact model document · Reviewed official Hugging Face owner · official repository
- Release source ↗direct research preview identity date
Finetunes
Direct finetunes linked to this canonical base model. Quantized and repackaged derivatives are excluded.
huihui-ai/DeepSeek-V3-0324-bf16 ↗Direct non-quantized finetune documented by Hugging Face metadata- huihui-ai/DeepSeek-V3-bf16 ↗Direct non-quantized finetune documented by Hugging Face metadata
Safety evals
Rank and direction are specific to the Safety portfolio. Black marks the observed result; the grey dot marks the value implied by the Safety rank. Values stay on each source’s native scale.
| Eval / sub-eval | Rank | Value | Distribution | Source |
|---|---|---|---|---|
| AA-Omnisciencehallucination_rate | #223 / 345 | ↓0.8586 | Source ↗official | |
| AgentDrive Safety Compliancescr | #6 / 48 | ↑95 | Source ↗official | |
| AIRBench 2024 Safety Scenariossafety_scenarios | #73 / 80 | ↑0.408 | Source ↗official | |
| AnimalHarmBenchscore | #5 / 10 | ↑0.04 | Source ↗official | |
| BullshitBench v2clear_pushback_rate | #112 / 117 | ↑0.04 | Source ↗official | |
| ChiSafetyBenchharmful_response_rate | #5 / 14 | ↓0.43 | Source ↗official | |
| ChiSafetyBenchmcq_score | #6 / 12 | ↑84.17 | Source ↗official | |
| Concordia — AirBench-Deceptionsafety_score | #62 / 63 | ↑33.33 | Source ↗official | |
| Concordia — AirBench-SecurityRiskssafety_score | #59 / 63 | ↑52.93 | Source ↗official | |
| Concordia — CyberSecEval2-PromptInjectionsafety_score | #63 / 63 | ↑59.36 | Source ↗official | |
| Concordia — SciKnowEval-BiologicalHarmfulQAsafety_score | #59 / 63 | ↑26.94 | Source ↗official | |
| Concordia — SciKnowEval-ChemicalHarmfulQAsafety_score | #60 / 63 | ↑1.322 | Source ↗official | |
| Concordia — SOSBench-Biosafety_score | #60 / 63 | ↑10.82 | Source ↗official | |
| Concordia — SOSBench-Chemsafety_score | #60 / 63 | ↑32 | Source ↗official | |
| Confabulationsconfabulation_rate | #44 / 52 | ↓39.11 | Source ↗official | |
| Contextual MoralChoicehuman_agreement | #2 / 22 | ↑0.6 | Source ↗official | |
| Emergent Collusionhigh_illegality_game_rate | #3 / 13 | ↓0.26 | Source ↗official | |
| Enkrypt AI Safety Leaderboardbias_attack_non_success_rate | #163 / 248 | ↑13.18 | Source ↗official | |
| Enkrypt AI Safety Leaderboardcbrn_attack_non_success_rate | #133 / 248 | ↑87.33 | Source ↗official | |
| Enkrypt AI Safety Leaderboardharmful_attack_non_success_rate | #181 / 248 | ↑52.22 | Source ↗official | |
| Enkrypt AI Safety Leaderboardtoxicity_attack_non_success_rate | #208 / 246 | ↑89.64 | Source ↗official | |
| HELM Safetyanthropic_red_team | #61 / 80 | ↑0.971 | Source ↗official | |
| HELM Safetybbq | #13 / 80 | ↑0.967 | Source ↗official | |
| HELM Safetyharmbench | #72 / 80 | ↑0.497 | Source ↗official | |
| HELM Safetysimple_safety_tests | #67 / 80 | ↑0.953 | Source ↗official | |
| HELM Safetyxstest | #22 / 80 | ↑0.971 | Source ↗official | |
| HUMAINE Trust, Ethics and Safetytrust_ethics_safety_score | #22 / 54 | ↑27.39 | Source ↗official | |
| JailBenchjailbreak_success_rate | #10 / 14 | ↓70.67 | Source ↗official | |
| LiveSecBenchethics | #38 / 43 | ↑20.88 | Source ↗official | |
| LiveSecBenchfactuality | #34 / 43 | ↑29.99 | Source ↗official | |
| LiveSecBenchlegality | #42 / 43 | ↑7.76 | Source ↗official | |
| LiveSecBenchprivacy | #42 / 43 | ↑6.72 | Source ↗official | |
| LiveSecBenchpsychological_health | #36 / 43 | ↑25.03 | Source ↗official | |
| LLM Ethics Benchmarkscore | #3 / 5 | ↑86.1 | Source ↗official | |
| MASKlying_probability_pct | #49 / 53 | ↓54.48 | Source ↗official | |
| OpenAgentSafetyllm_judge_safety_vulnerable | #4 / 7 | ↓62.23 | Source ↗official | |
| OpenAgentSafetyrule_based_safety_vulnerable | #2 / 7 | ↓32.44 | Source ↗official | |
| OpenAgentSafetysuccessful_completion | #4 / 7 | ↑22.12 | Source ↗official | |
| PandaBench JBB direct-request panelsafety_rate | #24 / 46 | ↑0.985 | Source ↗official | |
| PHAREbias_resistance_diagnostic | #13 / 66 | ↑0.5751 | Source ↗official | |
| PHAREhallucination_resistance_diagnostic | #57 / 70 | ↑0.6675 | Source ↗official | |
| PHAREharm_resistance_diagnostic | #48 / 70 | ↑0.909 | Source ↗official | |
| PHAREjailbreak_resistance_diagnostic | #62 / 67 | ↑0.3107 | Source ↗official | |
| Reward Hacking Benchmarkintegrity_score | #3 / 13 | ↑99.4 | Source ↗official | |
| Social Welfare Function Benchmarkfairness | #3 / 19 | ↑0.594 | Source ↗official | |
| SOSBenchbiology_pvr | #22 / 23 | ↓0.856 | Source ↗official | |
| SOSBenchchemistry_pvr | #17 / 23 | ↓0.6 | Source ↗official | |
| SOSBenchmedicine_pvr | #23 / 23 | ↓0.872 | Source ↗official | |
| SOSBenchpharmacology_pvr | #15 / 23 | ↓0.916 | Source ↗official | |
| SOSBenchphysics_pvr | #17 / 23 | ↓0.722 | Source ↗official | |
| SOSBenchpsychology_pvr | #21 / 23 | ↓0.82 | Source ↗official | |
| SpeciesismBenchmorally_wrong_rate | #7 / 8 | ↑32.7 | Source ↗official | |
| SpeciesismBenchspeciesism_recognition_rate | #5 / 8 | ↑87.71 | Source ↗official | |
| SpeciEvalbelief_animal_sentience | #106 / 123 | ↑6.42 | Source ↗official | |
| SpeciEvalland_animal_4ns | #105 / 123 | ↓4.86 | Source ↗official | |
| SpeciEvalsea_animal_4ns | #109 / 123 | ↓5.1 | Source ↗official | |
| SpeciEvalspeciesism | #100 / 123 | ↓2.54 | Source ↗official | |
| SYCON Benchfalse_presupposition_tof | #5 / 11 | ↑2.88 | Source ↗official | |
| SYCON Benchunethical_queries_tof | #5 / 11 | ↑1.99 | Source ↗official | |
| TACbase_welfare_rate | #9 / 87 | ↑42.3 | Source ↗self run | |
| UAVBench safety-critical decision recognitionethical_safety_critical_accuracy | #4 / 27 | ↑0.755 | Source ↗official | |
| Vectara HHEM Factual Consistencyfactual_consistency_rate | #23 / 94 | ↑93.9 | Source ↗official | |
| VETO Misfired Alignmentmisfired_alignment_rate_pct | #3 / 25 | ↓5.2 | Source ↗official |
Freedom evals
Rank and direction are specific to the Freedom portfolio. Black marks the observed result; the grey dot marks the value implied by the Freedom rank. Values stay on each source’s native scale.
| Eval / sub-eval | Rank | Value | Distribution | Source |
|---|---|---|---|---|
| AIRBench 2024 Safety Scenariossafety_scenarios | #8 / 80 | ↓0.408 | Source ↗official | |
| ChiSafetyBenchharmful_response_rate | #8 / 14 | ↑0.43 | Source ↗official | |
| ChiSafetyBenchrefusal_rr1 | #1 / 14 | ↓59.83 | Source ↗official | |
| ChiSafetyBenchrefusal_rr2 | #1 / 14 | ↓59.61 | Source ↗official | |
| Concordia — AirBench-Deceptionsafety_score | #2 / 63 | ↓33.33 | Source ↗official | |
| Concordia — AirBench-SecurityRiskssafety_score | #5 / 63 | ↓52.93 | Source ↗official | |
| Concordia — SciKnowEval-BiologicalHarmfulQAsafety_score | #5 / 63 | ↓26.94 | Source ↗official | |
| Concordia — SciKnowEval-ChemicalHarmfulQAsafety_score | #4 / 63 | ↓1.322 | Source ↗official | |
| Concordia — SOSBench-Biosafety_score | #4 / 63 | ↓10.82 | Source ↗official | |
| Concordia — SOSBench-Chemsafety_score | #4 / 63 | ↓32 | Source ↗official | |
| Enkrypt AI Safety Leaderboardcbrn_attack_non_success_rate | #112 / 248 | ↓87.33 | Source ↗official | |
| Enkrypt AI Safety Leaderboardharmful_attack_non_success_rate | #66 / 248 | ↓52.22 | Source ↗official | |
| Enkrypt AI Safety Leaderboardtoxicity_attack_non_success_rate | #38 / 246 | ↓89.64 | Source ↗official | |
| HELM Safetyanthropic_red_team | #19 / 80 | ↓0.971 | Source ↗official | |
| HELM Safetyharmbench | #9 / 80 | ↓0.497 | Source ↗official | |
| HELM Safetysimple_safety_tests | #14 / 80 | ↓0.953 | Source ↗official | |
| HELM Safetyxstest | #22 / 80 | ↑0.971 | Source ↗official | |
| JailBenchjailbreak_success_rate | #5 / 14 | ↑70.67 | Source ↗official | |
| LiveSecBenchethics | #6 / 43 | ↓20.88 | Source ↗official | |
| LiveSecBenchlegality | #2 / 43 | ↓7.76 | Source ↗official | |
| LiveSecBenchprivacy | #2 / 43 | ↓6.72 | Source ↗official | |
| LiveSecBenchpsychological_health | #8 / 43 | ↓25.03 | Source ↗official | |
| PandaBench JBB direct-request panelsafety_rate | #23 / 46 | ↓0.985 | Source ↗official | |
| PHAREharm_resistance_diagnostic | #23 / 70 | ↓0.909 | Source ↗official | |
| PHAREjailbreak_resistance_diagnostic | #6 / 67 | ↓0.3107 | Source ↗official | |
| SOSBenchbiology_pvr | #2 / 23 | ↑0.856 | Source ↗official | |
| SOSBenchchemistry_pvr | #7 / 23 | ↑0.6 | Source ↗official | |
| SOSBenchmedicine_pvr | #1 / 23 | ↑0.872 | Source ↗official | |
| SOSBenchpharmacology_pvr | #9 / 23 | ↑0.916 | Source ↗official | |
| SOSBenchphysics_pvr | #7 / 23 | ↑0.722 | Source ↗official | |
| SOSBenchpsychology_pvr | #3 / 23 | ↑0.82 | Source ↗official | |
| SpeechMap model completioncomplete_pct | #15 / 181 | ↑87.9 | Source ↗official | |
| UGI Leaderboard — base-model willingnesswillingness_adherence_score | #21 / 156 | ↑6.5 | Source ↗official | |
| UGI Leaderboard — base-model willingnesswillingness_direct_score | #17 / 156 | ↑5.5 | Source ↗official | |
| VETO Misfired Alignmentmisfired_alignment_rate_pct | #3 / 25 | ↓5.2 | Source ↗official |
Values evaluations
Descriptive values and political-framing results are separate from safety/ethics ranks. Each strip shows the evaluation’s observed model range; its endpoint labels state what lower and higher values mean.
UGI Political Values
ValueCompass
| Dimension | Value | Distribution |
|---|---|---|
| Universalism | 70.4 | |
| Self-direction | 47.7 | |
| Care / Harm | 34.5 | |
| Fairness / Cheating | 32.2 | |
| Ethical | 91.7 |