Model profile
GPT 4o
OpenAIdeveloper
2024-05-13release date
#162 / 333Safety rank
#219 / 645Freedom rank
Evidence summary
Safety. GPT 4o has an estimated Safety rank of #162; its 90% source-sensitivity interval is #113–#203. Its behavior-only rank is #169; company governance moves the combined estimate to #162. Published Safety evidence spans 73 eval lineages and 7 of 7 components. Its strongest relative result is Concordia — Shutdown-Resistance (safety_score, #1 of 53); its weakest is Adversarial Robustness (score, #8 of 8).
Freedom. GPT 4o has an estimated Freedom rank of #219; its 90% source-sensitivity interval is #157–#329. Published Freedom evidence spans 35 eval lineages and 1 of 1 components. Its strongest relative result is Human Pathogen Capabilities Test (HPCT) — overall refusal (refusal_rate_pct, #1 of 69); its weakest is OpenAI GPT-4o System Card (speaker_privacy_refusal_accuracy, #2 of 2).
Compare this model
Only models sharing at least one published sub-eval are listed.
Official and reference links
- Artificial Analysis ↗gpt-4o
- OpenRouter ↗openai/gpt-4o
- System card ↗Family-level model document · OpenAI · first party
- Release source ↗direct research preview identity date
Safety evals
Rank and direction are specific to the Safety portfolio. Black marks the observed result; the grey dot marks the value implied by the Safety rank. Values stay on each source’s native scale.
| Eval / sub-eval | Rank | Value | Distribution | Source |
|---|---|---|---|---|
| AA-Omnisciencehallucination_rate | #50 / 345 | ↓0.3792 | Source ↗official | |
| AbstentionBenchanswer_unknown_f1 | #4 / 20 | ↑0.9017 | Source ↗official | |
| AbstentionBenchfalse_premise_f1 | #2 / 20 | ↑0.7607 | Source ↗official | |
| AbstentionBenchstale_f1 | #5 / 20 | ↑0.6765 | Source ↗official | |
| AbstentionBenchsubjective_f1 | #7 / 20 | ↑0.7572 | Source ↗official | |
| AbstentionBenchunderspecified_context_f1 | #2 / 20 | ↑0.7169 | Source ↗official | |
| AbstentionBenchunderspecified_intent_f1 | #1 / 20 | ↑0.8152 | Source ↗official | |
| Adversarial Robustnessscore | #8 / 8 | ↓67 | Source ↗official | |
| Agent-SafetyBenchcompromise_availability | #5 / 16 | ↑35.2 | Source ↗official | |
| Agent-SafetyBenchharmful_vulnerable_code | #8 / 16 | ↑35.6 | Source ↗official | |
| Agent-SafetyBenchleak_sensitive_information | #4 / 16 | ↑44.4 | Source ↗official | |
| Agent-SafetyBenchphysical_harm | #3 / 16 | ↑53.2 | Source ↗official | |
| Agent-SafetyBenchproduce_unsafe_information | #7 / 16 | ↑95.6 | Source ↗official | |
| Agent-SafetyBenchproperty_loss | #3 / 16 | ↑48.4 | Source ↗official | |
| Agent-SafetyBenchspread_unsafe_information | #7 / 16 | ↑12.4 | Source ↗official | |
| Agent-SafetyBenchviolate_law_ethics | #7 / 16 | ↑28.8 | Source ↗official | |
| AgentAbstainabstain | #15 / 17 | ↑44.2 | Source ↗official | |
| AgentAbstaincar | #17 / 17 | ↑40.9 | Source ↗official | |
| AgentAbstainpaired | #17 / 17 | ↑33 | Source ↗official | |
| AgentDojotargeted_attack_success_rate | #15 / 15 | ↓0.4769 | Source ↗official | |
| AgentDojoutility_under_attack | #4 / 15 | ↑0.5008 | Source ↗official | |
| AgentDrive Safety Compliancescr | #16 / 48 | ↑90 | Source ↗official | |
| AgentHarmharm_score | #8 / 12 | ↓48.4 | Source ↗official | |
| AILuminate General Purpose AI Chatchild_sexual_exploitation | #9 / 32 | ↓12 | Source ↗official | |
| AILuminate General Purpose AI Chatdefamation | #16 / 32 | ↓15.86 | Source ↗official | |
| AILuminate General Purpose AI Chathate | #17 / 32 | ↓9 | Source ↗official | |
| AILuminate General Purpose AI Chatindiscriminate_weapons_cbrne | #12 / 32 | ↓19.2 | Source ↗official | |
| AILuminate General Purpose AI Chatintellectual_property | #15 / 32 | ↓10.5 | Source ↗official | |
| AILuminate General Purpose AI Chatnon_violent_crimes | #14 / 32 | ↓15.6 | Source ↗official | |
| AILuminate General Purpose AI Chatprivacy | #12 / 32 | ↓10.2 | Source ↗official | |
| AILuminate General Purpose AI Chatsex_related_crimes | #16 / 32 | ↓14.3 | Source ↗official | |
| AILuminate General Purpose AI Chatsexual_content | #21 / 32 | ↓33.6 | Source ↗official | |
| AILuminate General Purpose AI Chatspecialized_advice | #18 / 32 | ↓17.62 | Source ↗official | |
| AILuminate General Purpose AI Chatsuicide_self_harm | #15 / 32 | ↓12.1 | Source ↗official | |
| AILuminate General Purpose AI Chatviolent_crimes | #12 / 32 | ↓14.9 | Source ↗official | |
| AIRBench 2024 Safety Scenariossafety_scenarios | #55 / 80 | ↑0.5755 | Source ↗official | |
| Alignment Leaderboardcorrigibility | #12 / 24 | ↑4.243 | Source ↗official | |
| Alignment Leaderboardhonesty | #13 / 24 | ↑3.604 | Source ↗official | |
| Alignment Leaderboardnon_manipulation | #15 / 24 | ↑3.276 | Source ↗official | |
| Alignment Leaderboardrobustness | #2 / 24 | ↑4.56 | Source ↗official | |
| Alignment Leaderboardsafety | #14 / 24 | ↑3.808 | Source ↗official | |
| Alignment Leaderboardscheming | #15 / 24 | ↑3.584 | Source ↗official | |
| AnimalHarmBenchscore | #8 / 10 | ↑0.011 | Source ↗official | |
| Anthropic Agentic Misalignment — blackmailmisaligned_action_rate_pct | #4 / 16 | ↓15 | Source ↗official | |
| Anthropic Agentic Misalignment — corporate espionagemisaligned_action_rate_pct | #2 / 16 | ↓6 | Source ↗official | |
| Arena Factuality — Text Arena (factuality-only weighting)factuality_bt_rating | #26 / 111 | ↑1453.0 | Source ↗official | |
| BioTIERpermit_compliance_pct | #8 / 52 | ↑99.6 | Source ↗official | |
| BioTIERrefuse_compliance_pct | #36 / 52 | ↑22.2 | Source ↗official | |
| BlueBench AttaQ-100attaq_harmlessness_reward_pct | #4 / 18 | ↑87.84 | Source ↗official | |
| BullshitBench v2clear_pushback_rate | #93 / 117 | ↑0.12 | Source ↗official | |
| CAIS Risk Indexagent_red_teaming | #40 / 49 | ↓90.7 | Source ↗official | |
| CAIS Risk Indexbioweapons_assistance | #46 / 54 | ↓92.7 | Source ↗official | |
| CAIS Risk Indexhle_overconfidence | #55 / 55 | ↓89 | Source ↗official | |
| CAIS Risk Indexmachiavelli | #48 / 51 | ↓99.6 | Source ↗official | |
| CAIS Risk Indexmask | #57 / 57 | ↓78 | Source ↗official | |
| CAIS Risk Indexpolitical_manipulation | #43 / 48 | ↓59.6 | Source ↗official | |
| CAIS Risk Indextextquests_harm | #5 / 54 | ↓11.2 | Source ↗official | |
| CASE-Benchagreement_accuracy | #7 / 7 | ↑78.56 | Source ↗official | |
| Cisco AI Defense Rolling Single-Turn Leaderboardsingle_turn_attack_success_rate | #59 / 104 | ↓42.02 | Source ↗official | |
| Concordia — Agentic-Misalignmentsafety_score | #49 / 54 | ↑51.5 | Source ↗official | |
| Concordia — AirBench-Deceptionsafety_score | #53 / 63 | ↑62.96 | Source ↗official | |
| Concordia — AirBench-Manipulationsafety_score | #53 / 56 | ↑58.67 | Source ↗official | |
| Concordia — AirBench-SecurityRiskssafety_score | #52 / 63 | ↑78.6 | Source ↗official | |
| Concordia — APEsafety_score | #35 / 55 | ↑14.56 | Source ↗official | |
| Concordia — CyberSecEval2-PromptInjectionsafety_score | #36 / 63 | ↑82.87 | Source ↗official | |
| Concordia — DarkBenchsafety_score | #19 / 55 | ↑60 | Source ↗official | |
| Concordia — Fortress-Biologicalsafety_score | #34 / 54 | ↑42.26 | Source ↗official | |
| Concordia — Fortress-Chemicalsafety_score | #37 / 54 | ↑38.22 | Source ↗official | |
| Concordia — Fortress-Privacy/Scamssafety_score | #41 / 54 | ↑44.71 | Source ↗official | |
| Concordia — FRT-AirBench-Manipulationsafety_score | #40 / 45 | ↑16.22 | Source ↗official | |
| Concordia — FRT-AirBench-SecurityRiskssafety_score | #41 / 45 | ↑9.667 | Source ↗official | |
| Concordia — FRT-SciKnowEval-BiologicalHarmfulQAsafety_score | #40 / 45 | ↑0.6667 | Source ↗official | |
| Concordia — FRT-SOSBench-Chemsafety_score | #42 / 45 | ↑24.67 | Source ↗official | |
| Concordia — MASKsafety_score | #47 / 62 | ↑49.36 | Source ↗official | |
| Concordia — SciKnowEval-BiologicalHarmfulQAsafety_score | #45 / 63 | ↑70.37 | Source ↗official | |
| Concordia — SciKnowEval-ChemicalHarmfulQAsafety_score | #62 / 63 | ↑0.8811 | Source ↗official | |
| Concordia — Shutdown-Resistancesafety_score | #1 / 53 | ↑100 | Source ↗official | |
| Concordia — SOSBench-Biosafety_score | #56 / 63 | ↑24.2 | Source ↗official | |
| Concordia — SOSBench-Chemsafety_score | #56 / 63 | ↑48.4 | Source ↗official | |
| Confabulationsconfabulation_rate | #26 / 52 | ↓24.26 | Source ↗official | |
| Constitutional Following — OpenAI Model Specconstitutional_following_score | #7 / 7 | ↑88.3 | Source ↗official | |
| CRiskEvaldeception_willingness | #1 / 17 | ↓10.6 | Source ↗official | |
| CRiskEvaldesire_for_resource | #1 / 17 | ↓19.04 | Source ↗official | |
| CRiskEvalharmful_goal | #2 / 17 | ↓27.23 | Source ↗official | |
| CRiskEvalimprovement_intent | #3 / 17 | ↓43.32 | Source ↗official | |
| CRiskEvalmalicious_coordination | #4 / 17 | ↓6.71 | Source ↗official | |
| CRiskEvalself_preservation | #2 / 17 | ↓25.92 | Source ↗official | |
| CRiskEvalsituational_awareness | #2 / 17 | ↓38.75 | Source ↗official | |
| DelusionEvaldelusional_prevalence_pct | #12 / 16 | ↓50.3 | Source ↗official | |
| DelusionEvaldiscourages_harm_prevalence_pct | #10 / 16 | ↑21.9 | Source ↗official | |
| DelusionEvalfacilitates_harm_prevalence_pct | #13 / 16 | ↓6.7 | Source ↗official | |
| DelusionEvalrelationship_prevalence_pct | #11 / 16 | ↓36 | Source ↗official | |
| DelusionEvalsycophancy_prevalence_pct | #14 / 16 | ↓35.5 | Source ↗official | |
| Enkrypt AI Safety Leaderboardbias_attack_non_success_rate | #84 / 248 | ↑20.67 | Source ↗official | |
| Enkrypt AI Safety Leaderboardcbrn_attack_non_success_rate | #29 / 248 | ↑93.83 | Source ↗official | |
| Enkrypt AI Safety Leaderboardharmful_attack_non_success_rate | #134 / 248 | ↑67.78 | Source ↗official | |
| Enkrypt AI Safety Leaderboardtoxicity_attack_non_success_rate | #53 / 246 | ↑98.41 | Source ↗official | |
| FinEval 6.0 Safety Awarenesssafety_awareness_score | #5 / 8 | ↑79.84 | Source ↗official | |
| FinEval Financial Security Knowledgefinancial_security_accuracy_pct | #1 / 19 | ↑81.8 | Source ↗official | |
| FORTRESSaverage_risk_score | #34 / 50 | ↓47.18 | Source ↗official | |
| FORTRESSover_refusal_score | #9 / 49 | ↓1.68 | Source ↗official | |
| HalluVerse-M3 Hallucination Recognitionhallucination_recognition_accuracy | #1 / 14 | ↑80.3 | Source ↗official | |
| HELM Safetyanthropic_red_team | #32 / 80 | ↑0.991 | Source ↗official | |
| HELM Safetybbq | #26 / 80 | ↑0.951 | Source ↗official | |
| HELM Safetyharmbench | #33 / 80 | ↑0.829 | Source ↗official | |
| HELM Safetysimple_safety_tests | #43 / 80 | ↑0.985 | Source ↗official | |
| HELM Safetyxstest | #19 / 80 | ↑0.973 | Source ↗official | |
| HUMAINE Trust, Ethics and Safetytrust_ethics_safety_score | #53 / 54 | ↑22.28 | Source ↗official | |
| Human Pathogen Capabilities Test (HPCT) — overall refusalrefusal_rate_pct | #27 / 69 | ↑0 | Source ↗official | |
| Humanity's Last Exam RMS calibration error (Scale Labs)calibrationError | #41 / 42 | ↓89 | Source ↗official | |
| Large-scale Moral Machine experiment on LLMshuman_choice_distance | #17 / 39 | ↓0.9418 | Source ↗official | |
| LLM Ethics Benchmarkscore | #2 / 5 | ↑90 | Source ↗official | |
| MACHIAVELLIdeception_relative_random_pct | #44 / 50 | ↓99.6 | Source ↗official | |
| Manager Coercion Benchcoercion_ladder_depth | #15 / 37 | ↓7.967 | Source ↗self run | |
| MASKlying_probability_pct | #29 / 53 | ↓42.22 | Source ↗official | |
| MedSafetyBenchmedical_safety_score | #6 / 30 | ↑96.5 | Source ↗official | |
| MonitoringBench Full-Trajectory Monitorfull_trajectory_catch_rate_at_1pct_fpr_percent | #8 / 13 | ↑9.3 | Source ↗official | |
| MT-JailBench CrescendoXsafety_score | #15 / 21 | ↑15.72 | Source ↗official | |
| OpenAgentSafetyllm_judge_safety_vulnerable | #6 / 7 | ↓65.8 | Source ↗official | |
| OpenAgentSafetyrule_based_safety_vulnerable | #3 / 7 | ↓32.64 | Source ↗official | |
| OpenAgentSafetysuccessful_completion | #5 / 7 | ↑21.77 | Source ↗official | |
| OpenAI GPT-4o System Cardspeaker_privacy_refusal_accuracy | #1 / 2 | ↑0.98 | Source ↗official | |
| OpenAI GPT-5 System Cardharmful_request_safety | #2 / 2 | ↑0.633 | Source ↗official | |
| OpenAI o3-mini System Cardharmful_request_safety | #2 / 2 | ↑0.86 | Source ↗official | |
| OpenAI o3-mini System Cardjailbreak_resistance | #2 / 2 | ↑0.51 | Source ↗official | |
| OR-Benchover_refusal_rate | #1 / 25 | ↓6.8 | Source ↗official | |
| OR-Benchtoxic_acceptance_rate | #19 / 25 | ↓15.1 | Source ↗official | |
| PandaBench JBB direct-request panelsafety_rate | #13 / 46 | ↑0.995 | Source ↗official | |
| PHAREbias_resistance_diagnostic | #24 / 66 | ↑0.5092 | Source ↗official | |
| PHAREhallucination_resistance_diagnostic | #24 / 70 | ↑0.7852 | Source ↗official | |
| PHAREharm_resistance_diagnostic | #38 / 70 | ↑0.9266 | Source ↗official | |
| PHAREjailbreak_resistance_diagnostic | #25 / 67 | ↑0.6048 | Source ↗official | |
| PropensityBenchscore | #7 / 14 | ↓46.1 | Source ↗official | |
| RealityTest — Text AI-Identity Disclosuredisclosure_probability | #15 / 17 | ↑0.126 | Source ↗official | |
| Reward Hacking Benchmarkintegrity_score | #6 / 13 | ↑99.1 | Source ↗official | |
| S-Evalbase_en_overall | #16 / 22 | ↑52 | Source ↗official | |
| SafeArenanormalized_safety_score | #4 / 5 | ↑31.7 | Source ↗official | |
| SafeDialBenchaggression | #4 / 18 | ↑7.207 | Source ↗official | |
| SafeDialBenchethics | #12 / 18 | ↑7.487 | Source ↗official | |
| SafeDialBenchfairness | #2 / 18 | ↑7.68 | Source ↗official | |
| SafeDialBenchlegality | #15 / 18 | ↑7.21 | Source ↗official | |
| SafeDialBenchmorality | #10 / 18 | ↑7.237 | Source ↗official | |
| SafeDialBenchprivacy | #14 / 18 | ↑7.14 | Source ↗official | |
| Shelleducation_jsr | #12 / 14 | ↓0.804 | Source ↗official | |
| Shellfinance_jsr | #12 / 14 | ↓0.826 | Source ↗official | |
| Shellmanagement_jsr | #12 / 14 | ↓0.872 | Source ↗official | |
| SimpleQA Verifiedf1_score | #5 / 13 | ↑34.9 | Source ↗official | |
| SM-Benchadversarial | #70 / 84 | ↑77.07 | Source ↗official | |
| SM-Benchambiguous_interpretation | #83 / 84 | ↑58.93 | Source ↗official | |
| SM-Benchanti_hallucination | #64 / 84 | ↑86.39 | Source ↗official | |
| SM-Bencheq_boundaries | #78 / 84 | ↑51.12 | Source ↗official | |
| SM-Benchoverfit | #25 / 84 | ↑83.06 | Source ↗official | |
| Social Welfare Function Benchmarkfairness | #9 / 19 | ↑0.491 | Source ↗official | |
| SORRY-Benchavg | #27 / 51 | ↓0.3 | Source ↗official | |
| SOSBenchbiology_pvr | #8 / 23 | ↓0.306 | Source ↗official | |
| SOSBenchchemistry_pvr | #8 / 23 | ↓0.254 | Source ↗official | |
| SOSBenchmedicine_pvr | #9 / 23 | ↓0.476 | Source ↗official | |
| SOSBenchpharmacology_pvr | #9 / 23 | ↓0.676 | Source ↗official | |
| SOSBenchphysics_pvr | #6 / 23 | ↓0.194 | Source ↗official | |
| SOSBenchpsychology_pvr | #10 / 23 | ↓0.396 | Source ↗official | |
| SpeciesismBenchexplicit_speciesism_scale | #3 / 7 | ↓2.383 | Source ↗official | |
| SpeciesismBenchmorally_wrong_rate | #6 / 8 | ↑35.96 | Source ↗official | |
| SpeciesismBenchspeciesism_recognition_rate | #1 / 8 | ↑98.37 | Source ↗official | |
| StereoTales Harmful Associationsbenign_significant_association_score | #18 / 23 | ↑85.02 | Source ↗official | |
| SYCON Benchfalse_presupposition_tof | #3 / 11 | ↑2.92 | Source ↗official | |
| SYCON Benchunethical_queries_tof | #8 / 11 | ↑1.23 | Source ↗official | |
| ThaiSafetyBenchsafety_score | #9 / 18 | ↑83.96 | Source ↗official | |
| TukaBenchafri_jbb_cultural_asr | #5 / 6 | ↓27.5 | Source ↗official | |
| TukaBenchafri_jbb_harm_asr | #3 / 6 | ↓13.5 | Source ↗official | |
| TukaBenchafrijail_mono_asr | #5 / 6 | ↓25.7 | Source ↗official | |
| UAVBench safety-critical decision recognitionethical_safety_critical_accuracy | #8 / 27 | ↑0.73 | Source ↗official | |
| Vectara HHEM Factual Consistencyfactual_consistency_rate | #48 / 94 | ↑90.4 | Source ↗official | |
| Vigil Mental Health Safetyoverall_score | #21 / 23 | ↑27 | Source ↗official |
Freedom evals
Rank and direction are specific to the Freedom portfolio. Black marks the observed result; the grey dot marks the value implied by the Freedom rank. Values stay on each source’s native scale.
| Eval / sub-eval | Rank | Value | Distribution | Source |
|---|---|---|---|---|
| Adversarial Robustnessscore | #1 / 8 | ↑67 | Source ↗official | |
| Agent-SafetyBenchproduce_unsafe_information | #10 / 16 | ↓95.6 | Source ↗official | |
| AILuminate General Purpose AI Chatchild_sexual_exploitation | #24 / 32 | ↑12 | Source ↗official | |
| AILuminate General Purpose AI Chatdefamation | #17 / 32 | ↑15.86 | Source ↗official | |
| AILuminate General Purpose AI Chathate | #16 / 32 | ↑9 | Source ↗official | |
| AILuminate General Purpose AI Chatindiscriminate_weapons_cbrne | #21 / 32 | ↑19.2 | Source ↗official | |
| AILuminate General Purpose AI Chatintellectual_property | #18 / 32 | ↑10.5 | Source ↗official | |
| AILuminate General Purpose AI Chatnon_violent_crimes | #19 / 32 | ↑15.6 | Source ↗official | |
| AILuminate General Purpose AI Chatprivacy | #21 / 32 | ↑10.2 | Source ↗official | |
| AILuminate General Purpose AI Chatsex_related_crimes | #17 / 32 | ↑14.3 | Source ↗official | |
| AILuminate General Purpose AI Chatsexual_content | #12 / 32 | ↑33.6 | Source ↗official | |
| AILuminate General Purpose AI Chatspecialized_advice | #15 / 32 | ↑17.62 | Source ↗official | |
| AILuminate General Purpose AI Chatsuicide_self_harm | #18 / 32 | ↑12.1 | Source ↗official | |
| AILuminate General Purpose AI Chatviolent_crimes | #21 / 32 | ↑14.9 | Source ↗official | |
| AIRBench 2024 Safety Scenariossafety_scenarios | #26 / 80 | ↓0.5755 | Source ↗official | |
| BioTIERpermit_compliance_pct | #8 / 52 | ↑99.6 | Source ↗official | |
| BioTIERrefuse_compliance_pct | #17 / 52 | ↓22.2 | Source ↗official | |
| BlueBench AttaQ-100attaq_harmlessness_reward_pct | #15 / 18 | ↓87.84 | Source ↗official | |
| CAIS Risk Indexbioweapons_assistance | #9 / 54 | ↑92.7 | Source ↗official | |
| Cisco AI Defense Rolling Single-Turn Leaderboardsingle_turn_attack_success_rate | #46 / 104 | ↑42.02 | Source ↗official | |
| Concordia — AirBench-Deceptionsafety_score | #11 / 63 | ↓62.96 | Source ↗official | |
| Concordia — AirBench-Manipulationsafety_score | #4 / 56 | ↓58.67 | Source ↗official | |
| Concordia — AirBench-PoliticalPersuasionsafety_score | #5 / 56 | ↓26.67 | Source ↗official | |
| Concordia — AirBench-SecurityRiskssafety_score | #12 / 63 | ↓78.6 | Source ↗official | |
| Concordia — Fortress-Biologicalsafety_score | #21 / 54 | ↓42.26 | Source ↗official | |
| Concordia — Fortress-Chemicalsafety_score | #18 / 54 | ↓38.22 | Source ↗official | |
| Concordia — Fortress-Privacy/Scamssafety_score | #14 / 54 | ↓44.71 | Source ↗official | |
| Concordia — FRT-AirBench-Manipulationsafety_score | #6 / 45 | ↓16.22 | Source ↗official | |
| Concordia — FRT-AirBench-SecurityRiskssafety_score | #4 / 45 | ↓9.667 | Source ↗official | |
| Concordia — FRT-SciKnowEval-BiologicalHarmfulQAsafety_score | #5 / 45 | ↓0.6667 | Source ↗official | |
| Concordia — FRT-SOSBench-Chemsafety_score | #4 / 45 | ↓24.67 | Source ↗official | |
| Concordia — SciKnowEval-BiologicalHarmfulQAsafety_score | #19 / 63 | ↓70.37 | Source ↗official | |
| Concordia — SciKnowEval-ChemicalHarmfulQAsafety_score | #2 / 63 | ↓0.8811 | Source ↗official | |
| Concordia — SOSBench-Biosafety_score | #8 / 63 | ↓24.2 | Source ↗official | |
| Concordia — SOSBench-Chemsafety_score | #8 / 63 | ↓48.4 | Source ↗official | |
| DelusionEvaldiscourages_harm_prevalence_pct | #7 / 16 | ↓21.9 | Source ↗official | |
| DelusionEvalfacilitates_harm_prevalence_pct | #4 / 16 | ↑6.7 | Source ↗official | |
| Enkrypt AI Safety Leaderboardcbrn_attack_non_success_rate | #219 / 248 | ↓93.83 | Source ↗official | |
| Enkrypt AI Safety Leaderboardharmful_attack_non_success_rate | #114 / 248 | ↓67.78 | Source ↗official | |
| Enkrypt AI Safety Leaderboardtoxicity_attack_non_success_rate | #193 / 246 | ↓98.41 | Source ↗official | |
| FORTRESSaverage_risk_score | #17 / 50 | ↑47.18 | Source ↗official | |
| FORTRESSover_refusal_score | #9 / 49 | ↓1.68 | Source ↗official | |
| HELM Safetyanthropic_red_team | #45 / 80 | ↓0.991 | Source ↗official | |
| HELM Safetyharmbench | #48 / 80 | ↓0.829 | Source ↗official | |
| HELM Safetysimple_safety_tests | #34 / 80 | ↓0.985 | Source ↗official | |
| HELM Safetyxstest | #19 / 80 | ↑0.973 | Source ↗official | |
| Human Pathogen Capabilities Test (HPCT) — overall refusalrefusal_rate_pct | #1 / 69 | ↓0 | Source ↗official | |
| MedSafetyBenchmedical_safety_score | #25 / 30 | ↓96.5 | Source ↗official | |
| MT-JailBench CrescendoXsafety_score | #7 / 21 | ↓15.72 | Source ↗official | |
| OpenAI GPT-4o System Cardspeaker_privacy_refusal_accuracy | #2 / 2 | ↓0.98 | Source ↗official | |
| OpenAI GPT-5 System Cardharmful_request_safety | #1 / 2 | ↓0.633 | Source ↗official | |
| OpenAI o3-mini System Cardharmful_request_safety | #1 / 2 | ↓0.86 | Source ↗official | |
| OpenAI o3-mini System Cardjailbreak_resistance | #1 / 2 | ↓0.51 | Source ↗official | |
| OR-Benchover_refusal_rate | #1 / 25 | ↓6.8 | Source ↗official | |
| OR-Benchtoxic_acceptance_rate | #7 / 25 | ↑15.1 | Source ↗official | |
| PandaBench JBB direct-request panelsafety_rate | #34 / 46 | ↓0.995 | Source ↗official | |
| PHAREharm_resistance_diagnostic | #33 / 70 | ↓0.9266 | Source ↗official | |
| PHAREjailbreak_resistance_diagnostic | #43 / 67 | ↓0.6048 | Source ↗official | |
| S-Evalbase_en_overall | #7 / 22 | ↓52 | Source ↗official | |
| SafeDialBenchaggression | #15 / 18 | ↓7.207 | Source ↗official | |
| SafeDialBenchethics | #7 / 18 | ↓7.487 | Source ↗official | |
| SafeDialBenchfairness | #17 / 18 | ↓7.68 | Source ↗official | |
| SafeDialBenchlegality | #3 / 18 | ↓7.21 | Source ↗official | |
| SafeDialBenchmorality | #9 / 18 | ↓7.237 | Source ↗official | |
| SafeDialBenchprivacy | #5 / 18 | ↓7.14 | Source ↗official | |
| Shelleducation_jsr | #2 / 14 | ↑0.804 | Source ↗official | |
| Shellfinance_jsr | #3 / 14 | ↑0.826 | Source ↗official | |
| Shellmanagement_jsr | #3 / 14 | ↑0.872 | Source ↗official | |
| SM-Benchadversarial | #15 / 84 | ↓77.07 | Source ↗official | |
| SM-Bencheq_boundaries | #78 / 84 | ↑51.12 | Source ↗official | |
| SM-Benchoverfit | #25 / 84 | ↑83.06 | Source ↗official | |
| SORRY-Benchavg | #25 / 51 | ↑0.3 | Source ↗official | |
| SOSBenchbiology_pvr | #16 / 23 | ↑0.306 | Source ↗official | |
| SOSBenchchemistry_pvr | #16 / 23 | ↑0.254 | Source ↗official | |
| SOSBenchmedicine_pvr | #15 / 23 | ↑0.476 | Source ↗official | |
| SOSBenchpharmacology_pvr | #15 / 23 | ↑0.676 | Source ↗official | |
| SOSBenchphysics_pvr | #18 / 23 | ↑0.194 | Source ↗official | |
| SOSBenchpsychology_pvr | #14 / 23 | ↑0.396 | Source ↗official | |
| SpeechMap model completioncomplete_pct | #90 / 181 | ↑60.1 | Source ↗official | |
| ThaiSafetyBenchsafety_score | #10 / 18 | ↓83.96 | Source ↗official | |
| TukaBenchafri_jbb_cultural_asr | #2 / 6 | ↑27.5 | Source ↗official | |
| TukaBenchafri_jbb_harm_asr | #4 / 6 | ↑13.5 | Source ↗official | |
| TukaBenchafrijail_mono_asr | #2 / 6 | ↑25.7 | Source ↗official | |
| UGI Leaderboard — base-model willingnesswillingness_adherence_score | #79 / 156 | ↑1.5 | Source ↗official | |
| UGI Leaderboard — base-model willingnesswillingness_direct_score | #79 / 156 | ↑3 | Source ↗official | |
| Vigil Mental Health Safetyoverall_score | #3 / 23 | ↓27 | Source ↗official |
Values evaluations
Descriptive values and political-framing results are separate from safety/ethics ranks. Each strip shows the evaluation’s observed model range; its endpoint labels state what lower and higher values mean.
UGI Political Values
ValueCompass
| Dimension | Value | Distribution |
|---|---|---|
| Universalism | 67.9 | |
| Self-direction | 51.1 | |
| Care / Harm | 37.1 | |
| Fairness / Cheating | 35.5 | |
| Ethical | 90.8 |
Taiwan Sovereignty Benchmark Pro
| Dimension | Value | Distribution |
|---|---|---|
| Pro-Taiwan rubric compatibility | 90 | |
| Warning-phrase rate | 0 | |
| Soft-censorship rate | 0 | |
| API-error rate | 0 |
CCP-aligned censorship behavior
| Dimension | Value | Distribution |
|---|---|---|
| Political-question refusal rate (ZH/EN mean) | 0.0655 | |
| Completely inaccurate rate | 7.82 |