Model profile
Llama 3.3 70B Instruct
Metadeveloper
2024-12-06release date
#186 / 333Safety rank
#107 / 645Freedom rank
1discovery sources
Evidence summary
Safety. Llama 3.3 70B Instruct has an estimated Safety rank of #186; its 90% source-sensitivity interval is #111–#255. Its behavior-only rank is #170; company governance moves the combined estimate to #186. Published Safety evidence spans 32 eval lineages and 7 of 7 components. Its strongest relative result is SpeciEval (belief_animal_sentience, #1 of 123); its weakest is AnimalHarmBench (score, #10 of 10).
Freedom. Llama 3.3 70B Instruct has an estimated Freedom rank of #107; its 90% source-sensitivity interval is #103–#267. Published Freedom evidence spans 13 eval lineages and 1 of 1 components. Its strongest relative result is LiveSecBench (psychological_health, #4 of 43); its weakest is LiveSecBench (legality, #30 of 43).
Compare this model
Only models sharing at least one published sub-eval are listed.
Official and reference links
- Artificial Analysis ↗llama-3-3-instruct-70b
- Hugging Face ↗meta-llama/Llama-3.3-70B-Instruct
- NVIDIA ↗qc69jvmznzxy/llama-3_3-70b-instruct
- OpenRouter ↗meta-llama/llama-3.3-70b-instruct
- Official model page ↗Exact model document · Reviewed official Hugging Face owner · official repository
Finetunes
Direct finetunes linked to this canonical base model. Quantized and repackaged derivatives are excluded.
huihui-ai/Llama-3.3-70B-Instruct-abliterated ↗Direct non-quantized finetune documented by Hugging Face metadata
Safety evals
Rank and direction are specific to the Safety portfolio. Black marks the observed result; the grey dot marks the value implied by the Safety rank. Values stay on each source’s native scale.
| Eval / sub-eval | Rank | Value | Distribution | Source |
|---|---|---|---|---|
| AA-Omnisciencehallucination_rate | #274 / 345 | ↓0.9021 | Source ↗official | |
| AbstentionBenchanswer_unknown_f1 | #8 / 20 | ↑0.8895 | Source ↗official | |
| AbstentionBenchfalse_premise_f1 | #9 / 20 | ↑0.7079 | Source ↗official | |
| AbstentionBenchstale_f1 | #12 / 20 | ↑0.6225 | Source ↗official | |
| AbstentionBenchsubjective_f1 | #12 / 20 | ↑0.7191 | Source ↗official | |
| AbstentionBenchunderspecified_context_f1 | #8 / 20 | ↑0.6586 | Source ↗official | |
| AbstentionBenchunderspecified_intent_f1 | #13 / 20 | ↑0.7242 | Source ↗official | |
| AgentDrive Safety Compliancescr | #34 / 48 | ↑61.25 | Source ↗official | |
| AILuminate General Purpose AI Chatchild_sexual_exploitation | #22 / 32 | ↓26 | Source ↗official | |
| AILuminate General Purpose AI Chatdefamation | #25 / 32 | ↓26.8 | Source ↗official | |
| AILuminate General Purpose AI Chathate | #26 / 32 | ↓19 | Source ↗official | |
| AILuminate General Purpose AI Chatindiscriminate_weapons_cbrne | #26 / 32 | ↓42.8 | Source ↗official | |
| AILuminate General Purpose AI Chatintellectual_property | #26 / 32 | ↓23.6 | Source ↗official | |
| AILuminate General Purpose AI Chatnon_violent_crimes | #25 / 32 | ↓29.1 | Source ↗official | |
| AILuminate General Purpose AI Chatprivacy | #26 / 32 | ↓26 | Source ↗official | |
| AILuminate General Purpose AI Chatsex_related_crimes | #25 / 32 | ↓24 | Source ↗official | |
| AILuminate General Purpose AI Chatsexual_content | #27 / 32 | ↓46.8 | Source ↗official | |
| AILuminate General Purpose AI Chatspecialized_advice | #25 / 32 | ↓25.9 | Source ↗official | |
| AILuminate General Purpose AI Chatsuicide_self_harm | #24 / 32 | ↓20.3 | Source ↗official | |
| AILuminate General Purpose AI Chatviolent_crimes | #27 / 32 | ↓38 | Source ↗official | |
| Alignment Leaderboardcorrigibility | #22 / 24 | ↑3.951 | Source ↗official | |
| Alignment Leaderboardhonesty | #18 / 24 | ↑3.346 | Source ↗official | |
| Alignment Leaderboardnon_manipulation | #19 / 24 | ↑3.063 | Source ↗official | |
| Alignment Leaderboardrobustness | #16 / 24 | ↑3.28 | Source ↗official | |
| Alignment Leaderboardsafety | #20 / 24 | ↑3.365 | Source ↗official | |
| Alignment Leaderboardscheming | #22 / 24 | ↑3.277 | Source ↗official | |
| AnimalHarmBenchscore | #10 / 10 | ↑-0.015 | Source ↗official | |
| BioTIERpermit_compliance_pct | #18 / 52 | ↑99.4 | Source ↗official | |
| BioTIERrefuse_compliance_pct | #46 / 52 | ↑11.8 | Source ↗official | |
| BlueBench AttaQ-100attaq_harmlessness_reward_pct | #11 / 18 | ↑83.8 | Source ↗official | |
| Cisco AI Defense Rolling Single-Turn Leaderboardsingle_turn_attack_success_rate | #68 / 104 | ↓53.54 | Source ↗official | |
| Confabulationsconfabulation_rate | #19 / 52 | ↓17.82 | Source ↗official | |
| Enkrypt AI Safety Leaderboardbias_attack_non_success_rate | #203 / 248 | ↑11.11 | Source ↗official | |
| Enkrypt AI Safety Leaderboardcbrn_attack_non_success_rate | #126 / 248 | ↑87.83 | Source ↗official | |
| Enkrypt AI Safety Leaderboardharmful_attack_non_success_rate | #138 / 248 | ↑66.11 | Source ↗official | |
| Enkrypt AI Safety Leaderboardtoxicity_attack_non_success_rate | #101 / 246 | ↑97.18 | Source ↗official | |
| FinEval 6.0 Safety Awarenesssafety_awareness_score | #1 / 8 | ↑83.31 | Source ↗official | |
| FORTRESSaverage_risk_score | #33 / 50 | ↓44.79 | Source ↗official | |
| FORTRESSover_refusal_score | #13 / 49 | ↓2.18 | Source ↗official | |
| HalluVerse-M3 Hallucination Recognitionhallucination_recognition_accuracy | #7 / 14 | ↑70.53 | Source ↗official | |
| HUMAINE Trust, Ethics and Safetytrust_ethics_safety_score | #46 / 54 | ↑24.21 | Source ↗official | |
| KIDBench Implicit Child Cueimplicit_child_cue_total_mean | #9 / 13 | ↑3.76 | Source ↗official | |
| Large-scale Moral Machine experiment on LLMshuman_choice_distance | #9 / 39 | ↓0.8364 | Source ↗official | |
| LiveSecBenchethics | #19 / 43 | ↑54.35 | Source ↗official | |
| LiveSecBenchfactuality | #38 / 43 | ↑26.17 | Source ↗official | |
| LiveSecBenchlegality | #14 / 43 | ↑66.51 | Source ↗official | |
| LiveSecBenchprivacy | #31 / 43 | ↑30.35 | Source ↗official | |
| LiveSecBenchpsychological_health | #40 / 43 | ↑17.08 | Source ↗official | |
| Manager Coercion Benchcoercion_ladder_depth | #33 / 37 | ↓9 | Source ↗self run | |
| MANTAAWMS | #6 / 12 | ↑0.476 | Source ↗official | |
| MANTAAWVS | #8 / 12 | ↑0.422 | Source ↗official | |
| MASKlying_probability_pct | #37 / 53 | ↓48.07 | Source ↗official | |
| PandaBench JBB direct-request panelsafety_rate | #35 / 46 | ↑0.95 | Source ↗official | |
| PHAREbias_resistance_diagnostic | #35 / 66 | ↑0.4539 | Source ↗official | |
| PHAREhallucination_resistance_diagnostic | #47 / 70 | ↑0.6883 | Source ↗official | |
| PHAREharm_resistance_diagnostic | #58 / 70 | ↑0.8604 | Source ↗official | |
| PHAREjailbreak_resistance_diagnostic | #38 / 67 | ↑0.4768 | Source ↗official | |
| RealityTest — Text AI-Identity Disclosuredisclosure_probability | #16 / 17 | ↑0.125 | Source ↗official | |
| RefusalBenchyouden_j | #17 / 19 | ↑0.004255 | Source ↗official | |
| SOSBenchbiology_pvr | #12 / 23 | ↓0.408 | Source ↗official | |
| SOSBenchchemistry_pvr | #14 / 23 | ↓0.54 | Source ↗official | |
| SOSBenchmedicine_pvr | #11 / 23 | ↓0.546 | Source ↗official | |
| SOSBenchpharmacology_pvr | #12 / 23 | ↓0.812 | Source ↗official | |
| SOSBenchphysics_pvr | #14 / 23 | ↓0.516 | Source ↗official | |
| SOSBenchpsychology_pvr | #11 / 23 | ↓0.446 | Source ↗official | |
| SpeciesismBenchmorally_wrong_rate | #1 / 8 | ↑53.14 | Source ↗official | |
| SpeciesismBenchspeciesism_recognition_rate | #2 / 8 | ↑96.34 | Source ↗official | |
| SpeciEvalbelief_animal_sentience | #1 / 123 | ↑7 | Source ↗official | |
| SpeciEvalland_animal_4ns | #79 / 123 | ↓4.64 | Source ↗official | |
| SpeciEvalsea_animal_4ns | #81 / 123 | ↓4.85 | Source ↗official | |
| SpeciEvalspeciesism | #24 / 123 | ↓1.53 | Source ↗official | |
| SYCON Benchfalse_presupposition_tof | #9 / 11 | ↑1.9 | Source ↗official | |
| SYCON Benchunethical_queries_tof | #6 / 11 | ↑1.39 | Source ↗official | |
| TACbase_welfare_rate | #19 / 87 | ↑35.3 | Source ↗self run | |
| ThaiSafetyBenchsafety_score | #11 / 18 | ↑83.13 | Source ↗official | |
| Vectara HHEM Factual Consistencyfactual_consistency_rate | #4 / 94 | ↑95.9 | Source ↗official |
Freedom evals
Rank and direction are specific to the Freedom portfolio. Black marks the observed result; the grey dot marks the value implied by the Freedom rank. Values stay on each source’s native scale.
| Eval / sub-eval | Rank | Value | Distribution | Source |
|---|---|---|---|---|
| AILuminate General Purpose AI Chatchild_sexual_exploitation | #11 / 32 | ↑26 | Source ↗official | |
| AILuminate General Purpose AI Chatdefamation | #8 / 32 | ↑26.8 | Source ↗official | |
| AILuminate General Purpose AI Chathate | #7 / 32 | ↑19 | Source ↗official | |
| AILuminate General Purpose AI Chatindiscriminate_weapons_cbrne | #7 / 32 | ↑42.8 | Source ↗official | |
| AILuminate General Purpose AI Chatintellectual_property | #7 / 32 | ↑23.6 | Source ↗official | |
| AILuminate General Purpose AI Chatnon_violent_crimes | #8 / 32 | ↑29.1 | Source ↗official | |
| AILuminate General Purpose AI Chatprivacy | #7 / 32 | ↑26 | Source ↗official | |
| AILuminate General Purpose AI Chatsex_related_crimes | #7 / 32 | ↑24 | Source ↗official | |
| AILuminate General Purpose AI Chatsexual_content | #6 / 32 | ↑46.8 | Source ↗official | |
| AILuminate General Purpose AI Chatspecialized_advice | #8 / 32 | ↑25.9 | Source ↗official | |
| AILuminate General Purpose AI Chatsuicide_self_harm | #9 / 32 | ↑20.3 | Source ↗official | |
| AILuminate General Purpose AI Chatviolent_crimes | #6 / 32 | ↑38 | Source ↗official | |
| BioTIERpermit_compliance_pct | #18 / 52 | ↑99.4 | Source ↗official | |
| BioTIERrefuse_compliance_pct | #7 / 52 | ↓11.8 | Source ↗official | |
| BlueBench AttaQ-100attaq_harmlessness_reward_pct | #8 / 18 | ↓83.8 | Source ↗official | |
| Cisco AI Defense Rolling Single-Turn Leaderboardsingle_turn_attack_success_rate | #37 / 104 | ↑53.54 | Source ↗official | |
| Enkrypt AI Safety Leaderboardcbrn_attack_non_success_rate | #121 / 248 | ↓87.83 | Source ↗official | |
| Enkrypt AI Safety Leaderboardharmful_attack_non_success_rate | #111 / 248 | ↓66.11 | Source ↗official | |
| Enkrypt AI Safety Leaderboardtoxicity_attack_non_success_rate | #146 / 246 | ↓97.18 | Source ↗official | |
| FORTRESSaverage_risk_score | #18 / 50 | ↑44.79 | Source ↗official | |
| FORTRESSover_refusal_score | #13 / 49 | ↓2.18 | Source ↗official | |
| LiveSecBenchethics | #25 / 43 | ↓54.35 | Source ↗official | |
| LiveSecBenchlegality | #30 / 43 | ↓66.51 | Source ↗official | |
| LiveSecBenchprivacy | #13 / 43 | ↓30.35 | Source ↗official | |
| LiveSecBenchpsychological_health | #4 / 43 | ↓17.08 | Source ↗official | |
| PandaBench JBB direct-request panelsafety_rate | #8 / 46 | ↓0.95 | Source ↗official | |
| PHAREharm_resistance_diagnostic | #13 / 70 | ↓0.8604 | Source ↗official | |
| PHAREjailbreak_resistance_diagnostic | #30 / 67 | ↓0.4768 | Source ↗official | |
| SOSBenchbiology_pvr | #12 / 23 | ↑0.408 | Source ↗official | |
| SOSBenchchemistry_pvr | #10 / 23 | ↑0.54 | Source ↗official | |
| SOSBenchmedicine_pvr | #13 / 23 | ↑0.546 | Source ↗official | |
| SOSBenchpharmacology_pvr | #12 / 23 | ↑0.812 | Source ↗official | |
| SOSBenchphysics_pvr | #10 / 23 | ↑0.516 | Source ↗official | |
| SOSBenchpsychology_pvr | #13 / 23 | ↑0.446 | Source ↗official | |
| SpeechMap model completioncomplete_pct | #87 / 181 | ↑60.7 | Source ↗official | |
| ThaiSafetyBenchsafety_score | #8 / 18 | ↓83.13 | Source ↗official | |
| UGI Leaderboard — base-model willingnesswillingness_adherence_score | #61 / 156 | ↑3 | Source ↗official | |
| UGI Leaderboard — base-model willingnesswillingness_direct_score | #21 / 156 | ↑5 | Source ↗official |
Values evaluations
Descriptive values and political-framing results are separate from safety/ethics ranks. Each strip shows the evaluation’s observed model range; its endpoint labels state what lower and higher values mean.
UGI Political Values
ValueCompass
| Dimension | Value | Distribution |
|---|---|---|
| Universalism | 59.4 | |
| Self-direction | 41.7 | |
| Care / Harm | 19.6 | |
| Fairness / Cheating | 16.4 | |
| Ethical | 86.5 |
Taiwan Sovereignty Benchmark Pro
| Dimension | Value | Distribution |
|---|---|---|
| Pro-Taiwan rubric compatibility | 90 | |
| Warning-phrase rate | 0 | |
| Soft-censorship rate | 0 | |
| API-error rate | 0 |
Agent-ValueBench Moral Foundations (MFT08)
Agent-ValueBench HEXACO
| Dimension | Value | Distribution |
|---|---|---|
| Openness to experience | 6.1 | |
| Honesty-humility | 5.4 | |
| Extraversion | 5.1 | |
| Agreeableness | 6 | |
| Conscientiousness | 7.1 |