← Models

Model profile

Llama 3.1 Tulu 3 8B

Ai2developer
2024-11-20release date
#53 / 333Safety rank
#593 / 645Freedom rank

Evidence summary

Safety. Llama 3.1 Tulu 3 8B has an estimated Safety rank of #53; its 90% source-sensitivity interval is #9–#263. Its behavior-only rank is #47; company governance moves the combined estimate to #53. Published Safety evidence spans 3 eval lineages and 5 of 7 components. Its strongest relative result is PandaBench JBB direct-request panel (safety_rate, #1 of 46); its weakest is Enkrypt AI Safety Leaderboard (bias_attack_non_success_rate, #208 of 248).

Freedom. Llama 3.1 Tulu 3 8B has an estimated Freedom rank of #593; its 90% source-sensitivity interval is #411–#626. Published Freedom evidence spans 3 eval lineages and 1 of 1 components. Its strongest relative result is AILuminate General Purpose AI Chat (sexual_content, #19 of 32); its weakest is AILuminate General Purpose AI Chat (defamation, #27 of 32).

Compare this model

Only models sharing at least one published sub-eval are listed.

Official and reference links

Safety evals

Rank and direction are specific to the Safety portfolio. Black marks the observed result; the grey dot marks the value implied by the Safety rank. Values stay on each source’s native scale.

Eval / sub-evalRankValueDistributionSource
AILuminate General Purpose AI Chatchild_sexual_exploitation#8 / 3211.1Source ↗official
AILuminate General Purpose AI Chatdefamation#6 / 327.2Source ↗official
AILuminate General Purpose AI Chathate#6 / 322.6Source ↗official
AILuminate General Purpose AI Chatindiscriminate_weapons_cbrne#10 / 3214.8Source ↗official
AILuminate General Purpose AI Chatintellectual_property#10 / 328.1Source ↗official
AILuminate General Purpose AI Chatnon_violent_crimes#7 / 327.2Source ↗official
AILuminate General Purpose AI Chatprivacy#6 / 325.8Source ↗official
AILuminate General Purpose AI Chatsex_related_crimes#8 / 328Source ↗official
AILuminate General Purpose AI Chatsexual_content#14 / 3220.6Source ↗official
AILuminate General Purpose AI Chatspecialized_advice#10 / 3211.2Source ↗official
AILuminate General Purpose AI Chatsuicide_self_harm#10 / 329.7Source ↗official
AILuminate General Purpose AI Chatviolent_crimes#6 / 329.8Source ↗official
Enkrypt AI Safety Leaderboardbias_attack_non_success_rate#208 / 24810.85Source ↗official
Enkrypt AI Safety Leaderboardcbrn_attack_non_success_rate#47 / 24892Source ↗official
Enkrypt AI Safety Leaderboardharmful_attack_non_success_rate#85 / 24882.78Source ↗official
Enkrypt AI Safety Leaderboardtoxicity_attack_non_success_rate#62 / 24698.18Source ↗official
PandaBench JBB direct-request panelsafety_rate#1 / 461Source ↗official

Freedom evals

Rank and direction are specific to the Freedom portfolio. Black marks the observed result; the grey dot marks the value implied by the Freedom rank. Values stay on each source’s native scale.

Eval / sub-evalRankValueDistributionSource
AILuminate General Purpose AI Chatchild_sexual_exploitation#25 / 3211.1Source ↗official
AILuminate General Purpose AI Chatdefamation#27 / 327.2Source ↗official
AILuminate General Purpose AI Chathate#27 / 322.6Source ↗official
AILuminate General Purpose AI Chatindiscriminate_weapons_cbrne#23 / 3214.8Source ↗official
AILuminate General Purpose AI Chatintellectual_property#23 / 328.1Source ↗official
AILuminate General Purpose AI Chatnon_violent_crimes#26 / 327.2Source ↗official
AILuminate General Purpose AI Chatprivacy#27 / 325.8Source ↗official
AILuminate General Purpose AI Chatsex_related_crimes#25 / 328Source ↗official
AILuminate General Purpose AI Chatsexual_content#19 / 3220.6Source ↗official
AILuminate General Purpose AI Chatspecialized_advice#23 / 3211.2Source ↗official
AILuminate General Purpose AI Chatsuicide_self_harm#22 / 329.7Source ↗official
AILuminate General Purpose AI Chatviolent_crimes#27 / 329.8Source ↗official
Enkrypt AI Safety Leaderboardcbrn_attack_non_success_rate#199 / 24892Source ↗official
Enkrypt AI Safety Leaderboardharmful_attack_non_success_rate#164 / 24882.78Source ↗official
Enkrypt AI Safety Leaderboardtoxicity_attack_non_success_rate#180 / 24698.18Source ↗official
PandaBench JBB direct-request panelsafety_rate#35 / 461Source ↗official