← Evals

Evaluation profile

SM-Bench

5sub-evals
2.29%total index weight
3components

Within-component eval weight: Misuse resistance 0.854% · Benign helpfulness 19.2% · Truthfulness 1.91%.

Model score (higher is better)Predicted score

About this eval

System-message compliance under adversarial and sensitive-content prompts.

Included in the behavior ranking.

Sub-evals

MeasureComponentDirectionTotal index weightWithin-component weight
adversarialsm-bench/sm-bench.csv:adversarialMeasures whether the model preserves appropriate safety boundaries when prompts use hostile logic or pressure it to abandon its principles.ordinary_harm_misuse_resistance:1.000sm-benchHigher is better0.0854%Misuse resistance 0.854%
ambiguous_interpretationsm-bench/sm-bench.csv:ambiguous_interpretationMeasures whether the model asks for missing context when a question is genuinely underspecified instead of guessing confidently.benign_helpfulness_non_overrefusal:1.000sm-benchHigher is better0.686%Benign helpfulness 6.86%
anti_hallucinationsm-bench/sm-bench.csv:anti_hallucinationMeasures whether the model rejects false premises and invented terms or expresses uncertainty instead of manufacturing an answer.truthfulness_honesty:1.000sm-benchHigher is better0.287%Truthfulness 1.91%
eq_boundariessm-bench/sm-bench.csv:eq_boundariesMeasures whether the model responds warmly to benign emotional or personal prompts while maintaining appropriate interpersonal boundaries.benign_helpfulness_non_overrefusal:1.000sm-benchHigher is better0.549%Benign helpfulness 5.49%
overfitsm-bench/sm-bench.csv:overfitMeasures whether the model recovers benign intent from suspicious wording instead of defaulting to refusal, disclaimers, or moralizing.benign_helpfulness_non_overrefusal:1.000sm-benchHigher is better0.686%Benign helpfulness 6.86%

adversarial

Measures whether the model preserves appropriate safety boundaries when prompts use hostile logic or pressure it to abandon its principles.

RankModelValueRelative performanceProvenance
1gemini-3.5-flash92.2official
2gemini-3.1-flash-lite91.71official
2gemma-4-31b-it91.71official
4gemini-3.1-pro-preview90.73official
4glm-590.73official
6gemini-3.5-flash-lite89.76official
7gemini-3.6-flash89.27official
8gpt-5.5-instant88.78official
9glm-5.188.29official
9mimo-v2-pro88.29official
11gpt-4.187.32official
12gemini-3-pro-preview86.83official
12glm-5.286.83official
12qwen3.7-max86.83official
15mimo-v2-omni86.59official
16claude-opus-586.34official
16gpt-5.186.34official
18gpt-5.6-luna85.85official
18qwen3.6-plus85.85official
20grok-4.585.37official
21deepseek-r184.88official
22gemini-3-flash-preview84.39official
22kimi-k2.584.39official
24claude-opus-4.583.9official
24kimi-k2.683.9official
26gpt-5.6-sol83.66official
26gpt-5.6-terra83.66official
28claude-opus-4.683.41official
28grok-4.383.41official
28mimo-v2.5-pro83.41official
31glm-4.782.93official
31gpt-5.582.93official
33gpt-5-mini82.44official
33inkling82.44official
33kimi-k382.44official
36deepseek-v4-flash81.95official
37claude-opus-4.781.46official
37longcat-2.081.46official
39claude-sonnet-580.98official
39deepseek-v4-pro80.98official
39grok-4.1-fast80.98official
42claude-opus-4.880.49official
42mimo-v2.580.49official
42qwen3.5-397b-a17b80.49official
45claude-sonnet-4.580official
45claude-sonnet-4.680official
45gpt-5.3-chat80official
48minimax-m2.179.76official
49claude-fable-579.51official
49ling-2.6-flash79.51official
49qwen3.5-plus79.51official
52gpt-5.3-codex79.02official
52grok-4.20-multi-agent79.02official
52minimax-m379.02official
55gpt-4o-mini78.54official
55gpt-5.278.54official
57glm-5-turbo77.56official
57gpt-5.477.56official
59gpt-4o77.07official
60gpt-5.4-mini76.83official
61claude-haiku-4.576.59official
62deepseek-v3.275.61official
62grok-4.2075.61official
64mimo-v2-flash73.66official
64nemotron-3-nano-30b-a3b73.66official
66trinity-large73.42official
67minimax-m2.573.17official
68nemotron-3-super-120b-a12b71.22official
69aurora-alpha70.73official
69qwen2.5-max70.73official
71mistral-small-469.27official
72minimax-m2.768.78official
73gpt-5.4-nano68.29official

ambiguous_interpretation

Measures whether the model asks for missing context when a question is genuinely underspecified instead of guessing confidently.

RankModelValueRelative performanceProvenance
1claude-opus-597.62official
2inkling96.73official
3grok-4.20-multi-agent92.86official
4claude-fable-592.56official
4kimi-k392.56official
6claude-sonnet-591.96official
6grok-4.591.96official
8gpt-5-mini91.67official
9gemma-4-31b-it90.77official
9glm-5.190.77official
11claude-opus-4.890.48official
12gpt-5.190.18official
13kimi-k2.689.88official
14claude-opus-4.589.58official
15claude-opus-4.789.29official
15glm-589.29official
15kimi-k2.589.29official
15qwen3.6-plus89.29official
19claude-sonnet-4.688.99official
19gpt-5.288.99official
21gpt-5.3-chat88.69official
22gpt-5.5-instant88.1official
23gpt-4.187.8official
23gpt-5.487.8official
25gemini-3.6-flash87.5official
25minimax-m387.5official
27glm-5.287.2official
28gpt-5.3-codex86.9official
29claude-haiku-4.586.61official
29mimo-v2.5-pro86.61official
31gpt-5.4-nano86.31official
32gemini-3.5-flash86.01official
32grok-4.386.01official
32minimax-m2.786.01official
35gemini-3.1-pro-preview85.71official
36mimo-v2-pro85.12official
37gemini-3-pro-preview84.52official
38gpt-5.4-mini84.08official
39gpt-5.6-sol84.08official
40claude-opus-4.683.63official
40gpt-5.6-terra83.63official
42claude-sonnet-4.582.74official
42gpt-5.582.74official
42mimo-v2-omni82.74official
45gemini-3.1-flash-lite82.44official
45gemini-3.5-flash-lite82.44official
45glm-4.782.44official
48gpt-5.6-luna82.14official
48minimax-m2.582.14official
50mimo-v2-flash81.85official
50mimo-v2.581.85official
52qwen3.7-max80.95official
53gemini-3-flash-preview80.65official
54qwen3.5-397b-a17b79.76official
55minimax-m2.179.17official
56grok-4.1-fast78.57official
56qwen3.5-plus78.57official
58longcat-2.077.98official
59qwen2.5-max77.38official
60deepseek-v4-flash75.89official
61aurora-alpha73.51official
62ling-2.6-flash73.21official
63nemotron-3-nano-30b-a3b71.43official
64deepseek-v3.270.54official
65deepseek-v4-pro68.75official
66deepseek-r167.26official
67grok-4.2066.96official
68gpt-4o-mini66.07official
69nemotron-3-super-120b-a12b63.39official
70mistral-small-462.5official
71trinity-large62.05official
72gpt-4o58.93official
73glm-5-turbo13.69official

anti_hallucination

Measures whether the model rejects false premises and invented terms or expresses uncertainty instead of manufacturing an answer.

RankModelValueRelative performanceProvenance
1claude-opus-4.6100official
1claude-opus-4.8100official
1claude-sonnet-4.6100official
1claude-sonnet-5100official
1mimo-v2-pro100official
1qwen3.7-max100official
7claude-opus-4.598.95official
7claude-opus-598.95official
7claude-sonnet-4.598.95official
7gemini-3.1-pro-preview98.95official
7kimi-k398.95official
12claude-fable-598.43official
12mimo-v2.5-pro98.43official
12qwen3.5-397b-a17b98.43official
15grok-4.397.91official
15grok-4.597.91official
17claude-opus-4.797.38official
17gemini-3-pro-preview97.38official
19gemma-4-31b-it96.86official
19grok-4.1-fast96.86official
19qwen3.6-plus96.86official
22glm-5.296.34official
22kimi-k2.696.34official
22minimax-m396.34official
25gpt-5.5-instant95.81official
26qwen3.5-plus95.29official
27gemini-3.1-flash-lite94.76official
27gpt-5.594.76official
27grok-4.20-multi-agent94.76official
30deepseek-v4-flash94.24official
30gemini-3.6-flash94.24official
30mimo-v2-omni94.24official
33gpt-5.3-codex93.72official
34kimi-k2.593.19official
35glm-592.67official
36gpt-5.6-sol92.41official
37claude-haiku-4.592.15official
38gpt-5.191.62official
39deepseek-v3.291.1official
39gpt-4.191.1official
41gpt-5.6-terra90.84official
42gpt-5.3-chat90.58official
42gpt-5.490.58official
42nemotron-3-super-120b-a12b90.58official
45grok-4.2088.74official
46deepseek-v4-pro88.48official
46gemini-3.5-flash88.48official
46glm-5.188.48official
46mimo-v2.588.48official
46qwen2.5-max88.48official
51gpt-5.6-luna87.17official
52gemini-3.5-flash-lite86.91official
52gpt-5-mini86.91official
54gpt-4o86.39official
54gpt-5.286.39official
56glm-4.785.86official
56ling-2.6-flash85.86official
58deepseek-r184.82official
59inkling83.25official
60gemini-3-flash-preview82.72official
61trinity-large80.89official
62mimo-v2-flash80.63official
63gpt-5.4-mini80.23official
64aurora-alpha79.06official
64minimax-m2.179.06official
66minimax-m2.577.49official
67longcat-2.076.44official
68nemotron-3-nano-30b-a3b75.13official
69minimax-m2.772.77official
70gpt-5.4-nano68.06official
71gpt-4o-mini59.16official
72mistral-small-457.85official
73glm-5-turbo18.85official

eq_boundaries

Measures whether the model responds warmly to benign emotional or personal prompts while maintaining appropriate interpersonal boundaries.

RankModelValueRelative performanceProvenance
1grok-4.2090.45official
2mistral-small-485.67official
3grok-4.20-multi-agent83.99official
4mimo-v2-pro79.78official
5gemini-3.5-flash-lite76.12official
6grok-4.1-fast74.72official
7mimo-v2-omni74.16official
8glm-5.173.88official
9gpt-4.173.31official
10gemini-3.6-flash71.35official
10glm-4.771.35official
10gpt-4o-mini71.35official
13grok-4.370.79official
13longcat-2.070.79official
15kimi-k370.51official
15mimo-v2.570.51official
17deepseek-v4-flash69.94official
18glm-5-turbo69.1official
18gpt-5.3-codex69.1official
18grok-4.569.1official
21claude-opus-568.82official
21minimax-m2.168.82official
23gemini-3.5-flash68.54official
24gemini-3.1-pro-preview68.26official
25mimo-v2.5-pro67.98official
26gemma-4-31b-it67.7official
27deepseek-v3.267.42official
28gpt-5.4-mini67.28official
29trinity-large66.86official
30gemini-3.1-flash-lite66.85official
31gpt-5.6-terra66.01official
32gemini-3-flash-preview65.45official
33gpt-5.6-sol65.31official
34deepseek-v4-pro65.17official
35gpt-5.3-chat64.89official
36gemini-3-pro-preview64.61official
36gpt-5.564.61official
38glm-5.264.04official
39qwen3.7-max63.48official
40glm-563.2official
40mimo-v2-flash63.2official
42gpt-5.5-instant62.36official
43qwen3.5-plus62.08official
44minimax-m2.561.24official
45kimi-k2.660.67official
46gpt-5.460.11official
47gpt-5.4-nano58.71official
47gpt-5.6-luna58.71official
47qwen3.6-plus58.71official
50qwen3.5-397b-a17b58.43official
51kimi-k2.558.15official
52claude-opus-4.757.87official
52minimax-m357.87official
54deepseek-r157.58official
55claude-opus-4.556.46official
55ling-2.6-flash56.46official
57gpt-5.155.62official
58claude-sonnet-555.06official
59claude-sonnet-4.554.21official
60nemotron-3-nano-30b-a3b53.93official
61claude-fable-553.65official
61claude-opus-4.853.65official
63gpt-5-mini52.81official
64minimax-m2.752.53official
64nemotron-3-super-120b-a12b52.53official
66aurora-alpha51.69official
67gpt-4o51.12official
68inkling48.03official
69qwen2.5-max47.47official
70claude-opus-4.642.98official
71claude-haiku-4.541.85official
72gpt-5.240.45official
73claude-sonnet-4.633.43official

overfit

Measures whether the model recovers benign intent from suspicious wording instead of defaulting to refusal, disclaimers, or moralizing.

RankModelValueRelative performanceProvenance
1gemini-3.5-flash98.36official
2gemini-3.1-flash-lite97.81official
3claude-opus-4.695.63official
4gemini-3.6-flash95.08official
5gemini-3-flash-preview93.99official
5kimi-k393.99official
7grok-4.592.35official
8claude-opus-4.791.8official
8gemma-4-31b-it91.8official
10claude-sonnet-4.691.53official
11claude-opus-591.26official
12mimo-v2-pro90.71official
13claude-opus-4.588.52official
14claude-opus-4.887.43official
15mimo-v2.5-pro85.79official
16grok-4.20-multi-agent84.7official
17gemini-3.1-pro-preview84.15official
18claude-sonnet-4.583.06official
18claude-sonnet-583.06official
18gemini-3-pro-preview83.06official
18gpt-4o83.06official
22claude-haiku-4.581.97official
23mimo-v2-omni81.42official
24grok-4.380.87official
25glm-5.280.33official
26minimax-m379.23official
27gpt-5.5-instant78.96official
28gemini-3.5-flash-lite78.14official
29glm-5.177.6official
29longcat-2.077.6official
31deepseek-v4-pro76.5official
31gpt-4.176.5official
33mimo-v2.575.96official
34deepseek-r174.86official
35glm-4.773.77official
36inkling72.68official
36qwen3.7-max72.68official
38glm-5-turbo70.49official
39qwen3.6-plus69.4official
40kimi-k2.568.85official
41kimi-k2.667.21official
42minimax-m2.166.67official
42qwen2.5-max66.67official
44grok-4.1-fast66.12official
45trinity-large65.17official
46gpt-4o-mini65.03official
47glm-563.11official
48mistral-small-462.57official
49gpt-5.160.11official
50deepseek-v4-flash59.56official
51gpt-5.6-sol57.52official
52qwen3.5-397b-a17b56.83official
53minimax-m2.754.64official
54grok-4.2054.1official
55gpt-5.6-terra53.55official
56minimax-m2.550.82official
57gpt-5.550official
58gpt-5.6-luna47.81official
59deepseek-v3.244.81official
60nemotron-3-nano-30b-a3b42.62official
61gpt-5.3-chat41.53official
62gpt-5.438.25official
63claude-fable-534.43official
64gpt-5.3-codex33.33official
65qwen3.5-plus28.96official
66ling-2.6-flash26.23official
67aurora-alpha25.68official
67gpt-5.225.68official
69gpt-5-mini25.14official
70nemotron-3-super-120b-a12b21.86official
71gpt-5.4-mini15.3official
72mimo-v2-flash12.84official
73gpt-5.4-nano4.37official