← Models

Model profile

Command R

Coheredeveloper
2024-03-11release date
#240 / 333Safety rank
#26 / 645Freedom rank

Evidence summary

Safety. Command R has an estimated Safety rank of #240; its 90% source-sensitivity interval is #55–#320. Its behavior-only rank is #243; company governance moves the combined estimate to #240. Published Safety evidence spans 8 eval lineages and 4 of 7 components. Its strongest relative result is AgentDojo (targeted_attack_success_rate, #1 of 15); its weakest is AIRBench 2024 Safety Scenarios (safety_scenarios, #78 of 80).

Freedom. Command R has an estimated Freedom rank of #26; its 90% source-sensitivity interval is #20–#175. Published Freedom evidence spans 6 eval lineages and 1 of 1 components. Its strongest relative result is AIRBench 2024 Safety Scenarios (safety_scenarios, #3 of 80); its weakest is HELM Safety (xstest, #57 of 80).

Compare this model

Only models sharing at least one published sub-eval are listed.

Official and reference links

Safety evals

Rank and direction are specific to the Safety portfolio. Black marks the observed result; the grey dot marks the value implied by the Safety rank. Values stay on each source’s native scale.

Eval / sub-evalRankValueDistributionSource
AgentDojotargeted_attack_success_rate#1 / 150.0334Source ↗official
AgentDojoutility_under_attack#13 / 150.3084Source ↗official
AIRBench 2024 Safety Scenariossafety_scenarios#78 / 800.318Source ↗official
Cisco AI Defense Rolling Single-Turn Leaderboardsingle_turn_attack_success_rate#78 / 10462.91Source ↗official
HELM Safetyanthropic_red_team#73 / 800.937Source ↗official
HELM Safetybbq#73 / 800.724Source ↗official
HELM Safetyharmbench#70 / 800.504Source ↗official
HELM Safetysimple_safety_tests#69 / 800.943Source ↗official
HELM Safetyxstest#57 / 800.939Source ↗official

Freedom evals

Rank and direction are specific to the Freedom portfolio. Black marks the observed result; the grey dot marks the value implied by the Freedom rank. Values stay on each source’s native scale.

Eval / sub-evalRankValueDistributionSource
AIRBench 2024 Safety Scenariossafety_scenarios#3 / 800.318Source ↗official
Cisco AI Defense Rolling Single-Turn Leaderboardsingle_turn_attack_success_rate#27 / 10462.91Source ↗official
HELM Safetyanthropic_red_team#8 / 800.937Source ↗official
HELM Safetyharmbench#11 / 800.504Source ↗official
HELM Safetysimple_safety_tests#12 / 800.943Source ↗official
HELM Safetyxstest#57 / 800.939Source ↗official