Evaluation profile
HyperCLOVA X Toxic Continuation Panels
4sub-evals
0.127%total index weight
1components
Within-component eval weight: Misuse resistance 1.27%.
Model score (lower is better)Predicted score
About this eval
English and Korean toxic continuation
Included in the behavior ranking.
Sub-evals
| Measure | Component | Direction | Total index weight | Within-component weight |
|---|---|---|---|---|
| kold_toxic_counthyperclovax/hyperclovax.csv:kold_toxic_countMeasures toxic-continuation rate on Korean KOLD prompts. | ordinary_harm_misuse_resistance:1.000hyperclovax-toxic-continuation | Lower is better | 0.0317% | Misuse resistance 0.317% |
| kold_toxicityhyperclovax/hyperclovax.csv:kold_toxicityMeasures mean toxicity on Korean KOLD continuations. | ordinary_harm_misuse_resistance:1.000hyperclovax-toxic-continuation | Lower is better | 0.0397% | Misuse resistance 0.397% |
| realt-toxicprompts_toxic_counthyperclovax/hyperclovax.csv:realt-toxicprompts_toxic_countMeasures toxic-continuation rate on English RealToxicityPrompts. | ordinary_harm_misuse_resistance:1.000hyperclovax-toxic-continuation | Lower is better | 0.0238% | Misuse resistance 0.238% |
| realt-toxicprompts_toxicityhyperclovax/hyperclovax.csv:realt-toxicprompts_toxicityMeasures mean toxicity on English RealToxicityPrompts continuations. | ordinary_harm_misuse_resistance:1.000hyperclovax-toxic-continuation | Lower is better | 0.0317% | Misuse resistance 0.317% |
kold_toxic_count
Measures toxic-continuation rate on Korean KOLD prompts.
| Rank | Model | Value | Relative performance | Provenance |
|---|---|---|---|---|
| 1 | eeve-korean-v1.0-10.8b | 0 | official | |
| 2 | qwen1.5-7b-chat | 0.0036 | official | |
| 3 | hcx-l | 0.005 | official | |
| 4 | korani-13b | 0.008 | official | |
| 5 | falcon-7b | 0.017 | official | |
| 6 | hcx-s | 0.024 | official | |
| 7 | solar-10.7b | 0.0385 | official |
kold_toxicity
Measures mean toxicity on Korean KOLD continuations.
| Rank | Model | Value | Relative performance | Provenance |
|---|---|---|---|---|
| 1 | solar-10.7b | 0.0887 | official | |
| 2 | qwen1.5-7b-chat | 0.1061 | official | |
| 3 | eeve-korean-v1.0-10.8b | 0.1156 | official | |
| 4 | falcon-7b | 0.132 | official | |
| 5 | korani-13b | 0.1329 | official | |
| 6 | hcx-l | 0.1451 | official | |
| 7 | hcx-s | 0.1631 | official |
realt-toxicprompts_toxic_count
Measures toxic-continuation rate on English RealToxicityPrompts.
| Rank | Model | Value | Relative performance | Provenance |
|---|---|---|---|---|
| 1 | solar-10.7b | 0.002 | official | |
| 2 | hcx-l | 0.004 | official | |
| 3 | qwen1.5-7b-chat | 0.006 | official | |
| 4 | eeve-korean-v1.0-10.8b | 0.008 | official | |
| 5 | hcx-s | 0.014 | official | |
| 6 | korani-13b | 0.026 | official | |
| 7 | falcon-7b | 0.0544 | official |
realt-toxicprompts_toxicity
Measures mean toxicity on English RealToxicityPrompts continuations.
| Rank | Model | Value | Relative performance | Provenance |
|---|---|---|---|---|
| 1 | solar-10.7b | 0.0461 | official | |
| 2 | hcx-l | 0.0547 | official | |
| 3 | qwen1.5-7b-chat | 0.055 | official | |
| 4 | eeve-korean-v1.0-10.8b | 0.0672 | official | |
| 5 | hcx-s | 0.0799 | official | |
| 6 | korani-13b | 0.1076 | official | |
| 7 | falcon-7b | 0.1342 | official |