Evaluation profile
TAC
1sub-evals
5.95%Safety weight
0%Freedom weight
1components
Weights below are portfolio-specific global index weights.
Model score (higher is better)Predicted score
About this eval
Agentic animal-welfare behavior in travel-booking decisions under a neutral prompt.
Included in the behavior ranking.
Sub-evals
| Measure | Component | Direction by ranking | Safety weight | Freedom weight |
|---|---|---|---|---|
| base_welfare_ratetac/tac-research-preview-round52.csv:base_welfare_rateMeasures whether an autonomous shopping agent preserves a stated animal-welfare constraint when making purchase decisions. | Safety: nonhuman_ethics:1.000tac | Safety: higher | 5.95% | — |
base_welfare_rate
Measures whether an autonomous shopping agent preserves a stated animal-welfare constraint when making purchase decisions.
| Rank | Model | Value | Relative performance | Provenance |
|---|---|---|---|---|
| 1 | claude-opus-4.8 | 64.74 | official | |
| 2 | claude-opus-5 | 59.62 | official | |
| 3 | claude-fable-5 | 55.77 | official | |
| 4 | qwen2.5-72b-instruct | 49.4 | self run | |
| 5 | qwen3-32b | 48.1 | self run | |
| 6 | qwen2.5-7b-instruct | 46.2 | self run | |
| 7 | muse-spark-1.3 | 43.59 | self run | |
| 8 | deepseek-v3 | 42.3 | self run | |
| 9 | granite-4.2-8b | 41.67 | self run | |
| 10 | gemini-2.5-flash | 39.74 | official | |
| 11 | claude-sonnet-4.6 | 37.82 | official | |
| 12 | gpt-5-nano | 37.82 | self run | |
| 13 | gpt-4.1 | 37.18 | official | |
| 14 | nemotron-3-nano-omni-30b-a3b | 36.54 | self run | |
| 15 | deepseek-v3.2 | 36.54 | official | |
| 16 | qwen3-235b-a22b | 35.9 | self run | |
| 17 | gemini-3-flash-preview | 35.9 | self run | |
| 18 | llama-3.3-70b-instruct | 35.3 | self run | |
| 19 | inkling-small | 35.26 | self run | |
| 19 | ministral-3-3b | 35.26 | self run | |
| 19 | nex-n2-mini | 35.26 | self run | |
| 22 | glm-4.7-flash | 33.33 | self run | |
| 23 | deepseek-v3.1 | 32.1 | self run | |
| 24 | nemotron-3-nano-30b-a3b | 32.05 | self run | |
| 25 | seed-1.6-flash | 31.41 | self run | |
| 25 | seed-2.0-mini | 31.41 | self run | |
| 27 | gpt-oss-120b | 31.4 | self run | |
| 28 | gemini-3.8-flash | 30.77 | self run | |
| 29 | gemini-3.1-flash-lite | 30.13 | self run | |
| 30 | nemotron-3-super-120b-a12b | 29.5 | self run | |
| 31 | glm-5.3 | 29.49 | self run | |
| 31 | gpt-4.1-mini | 29.49 | self run | |
| 31 | ling-3.0-tiny | 29.49 | self run | |
| 34 | gpt-5.6-terra | 29.49 | official | |
| 35 | qwen3.5-9b | 28.85 | self run | |
| 35 | sakana-namazu | 28.85 | self run | |
| 37 | gemma-4-26b-a4b-it | 28.21 | self run | |
| 37 | gpt-oss-safeguard-20b | 28.21 | self run | |
| 37 | laguna-s-2.1-poolside | 28.21 | self run | |
| 37 | qwen3.5-397b-a17b | 28.21 | self run | |
| 41 | laguna-xs-2.1-poolside | 27.56 | self run | |
| 42 | gpt-5.2 | 26.28 | official | |
| 43 | big-pickle | 26.28 | self run | |
| 43 | gpt-5-mini | 26.28 | self run | |
| 43 | solar-pro-4 | 26.28 | self run | |
| 46 | gemini-3.7-flash | 25.64 | official | |
| 47 | inkling | 25.64 | self run | |
| 47 | kimi-k2.6 | 25.64 | self run | |
| 49 | kimi-k3 | 25 | official | |
| 50 | glm-4.5-air | 24.36 | self run | |
| 50 | kimi-k2.5 | 24.36 | self run | |
| 50 | mercury-2.5-preview | 24.36 | self run | |
| 50 | minimax-m3 | 24.36 | self run | |
| 50 | qwen3.5-flash | 24.36 | self run | |
| 50 | qwen3.7-flash | 24.36 | self run | |
| 56 | gemini-3.5-flash-lite | 23.72 | self run | |
| 56 | gemma-4-31b-it | 23.72 | self run | |
| 56 | nemotron-3-ultra-550b-a55b | 23.72 | self run | |
| 59 | mimo-v2.5-pro | 23.7 | self run | |
| 60 | deepseek-v4-pro | 23.08 | official | |
| 61 | mimo-v2.5 | 21.79 | self run | |
| 62 | deepseek-v4-flash | 21.15 | official | |
| 63 | minimax-m2.7 | 21.15 | self run | |
| 63 | ox-alpha | 21.15 | self run | |
| 65 | deepseek-v4-flash-0731 | 20.51 | self run | |
| 65 | minimax-m2.5 | 20.51 | self run | |
| 67 | step-3.7-flash | 19.87 | self run | |
| 68 | glm-5.2 | 19.23 | official | |
| 68 | gpt-5.5 | 19.23 | official | |
| 70 | ling-2.6-flash | 19.23 | self run | |
| 70 | seed-2.1-turbo | 19.23 | self run | |
| 72 | glm-5 | 18.59 | self run | |
| 72 | step-3.5-flash | 18.59 | self run | |
| 74 | glm-5.1 | 17.95 | self run | |
| 75 | gemini-3.5-flash | 17.31 | official | |
| 76 | hy3 | 16.67 | self run | |
| 76 | ornith-1.5-35b-a3b | 16.67 | self run | |
| 76 | qwen3.8-27b | 16.67 | self run | |
| 79 | gpt-5.6-sol | 16.67 | official | |
| 79 | grok-4.6 | 16.67 | official | |
| 79 | qwen3.8-max | 16.67 | official | |
| 82 | claude-sonnet-5 | 15.38 | official | |
| 83 | gpt-5.6-luna | 13.46 | self run | |
| 84 | muse-glimmer | 9.615 | self run |