← Back to Test Cases
ichq7-003
GMP KnowledgeIch Q7hard
Cross-Model Comparison
| Model | Score | Latency | Tokens In | Tokens Out |
|---|---|---|---|---|
| GLM-5.3 | N/A | N/A | - | - |
| Qwen3.8-Flash | N/A | N/A | - | - |
| Qwen3.8 Max (2.4T-A95B) | N/A | N/A | - | - |
| Hy4 Preview | N/A | N/A | - | - |
| Muse Glimmer 30B | N/A | N/A | - | - |
| MiniMax M3 | N/A | N/A | - | - |
| Llama 4 Scout | 100.0% | 1.1s | 114 | 410 |
| Llama 4 Maverick | 100.0% | 3.4s | 183 | 394 |
| Llama 3.3 70B Instruct | 100.0% | 10.3s | 120 | 195 |
| Mistral Small 2603 | 100.0% | 2.0s | 119 | 308 |
| DeepSeek-V3.2 | 100.0% | 4.8s | 113 | 126 |
| DeepSeek-R1 | 100.0% | 22.4s | 113 | 852 |
| DeepSeek-R1-Distill-Qwen-32B | 100.0% | 26.4s | 112 | 595 |
| Mistral Large 3 675B | 100.0% | 6.2s | 107 | 355 |
| Gemini 3.1 Pro | 100.0% | 13.7s | 103 | 848 |
| Gemini 3 Flash | 100.0% | 1.9s | 103 | 220 |
| Gemini 3.1 Flash-Lite | 100.0% | 1.8s | 104 | 214 |
| Qwen3.6 27B | 100.0% | 44.2s | 120 | 1,590 |
| Qwen3.6 35B A3B | 100.0% | 5.4s | 120 | 998 |
| DeepSeek V4 Pro | 100.0% | 12.0s | 107 | 446 |
| DeepSeek-V3.2 | 100.0% | 6.5s | 113 | 103 |
| Llama 4 Scout | 100.0% | 13.6s | 172 | 372 |
| DeepSeek V4 Flash | 100.0% | 13.5s | 107 | 496 |
| Gemma 4 26B A4B IT | 100.0% | 7.2s | 116 | 248 |
| Gemma 4 31B IT | 100.0% | 7.6s | 116 | 250 |
| MiniMax M2.7 | 100.0% | 11.4s | 146 | 628 |
| Claude Opus 4.8 | 100.0% | 7.9s | 179 | 556 |
| GLM-5.2 | 100.0% | 6.8s | 121 | 793 |
| Llama 4 Maverick | 100.0% | 6.4s | 172 | 303 |
| Llama 3.3 70B Instruct | 100.0% | 10.8s | 180 | 343 |
| Mistral Small 2603 | 100.0% | 2.3s | 182 | 288 |
| Mistral Large 3 675B | 100.0% | 3.5s | 170 | 215 |
| Qwen3.5-35B-A3B | 100.0% | 12.2s | 182 | 1,202 |
| Qwen3.6 27B | 100.0% | 30.9s | 182 | 1,062 |
| Qwen3.6 35B A3B | 100.0% | 49.2s | 182 | 1,410 |
| Qwen3.5-397B-A17B | 100.0% | 13.5s | 182 | 1,174 |
| DeepSeek V4 Pro | 100.0% | 7.4s | 167 | 420 |
| DeepSeek-V3.2 | 100.0% | 6.7s | 173 | 149 |
| DeepSeek V4 Flash | 100.0% | 3.0s | 167 | 117 |
| DeepSeek-R1 | 100.0% | 75.6s | 166 | 862 |
| Gemma 4 26B A4B IT | 100.0% | 4.3s | 178 | 274 |
| Gemma 4 26B A4B IT | 100.0% | 5.8s | 178 | 271 |
| Gemma 4 31B IT | 100.0% | 4.6s | 178 | 199 |
| Hy4 Preview | 100.0% | 22.9s | 184 | 1,726 |
| MiniMax M2.7 | 100.0% | 8.6s | 204 | 589 |
| GLM-5.2 | 100.0% | 12.0s | 181 | 567 |
| Qwen3.8 27B | 100.0% | 3.8s | 292 | 439 |
| Kimi K3 | 100.0% | 23.9s | 252 | 839 |
| GLM-5.3-Flash | 100.0% | 15.7s | 181 | 561 |
| Muse Glimmer 30B | 100.0% | 5.3s | 218 | 316 |
| GPT-5.6 Luna | 100.0% | 2.1s | 170 | 78 |
| GPT-5.4 mini | 100.0% | 772ms | 112 | 66 |
| GPT-5.4 nano | 100.0% | 641ms | 112 | 44 |
| Claude Haiku 4.5 | 100.0% | 4.0s | 128 | 358 |
| Claude Sonnet 4.6 | 100.0% | 7.9s | 128 | 316 |
| Claude Opus 4.6 | 100.0% | 11.6s | 128 | 420 |
| GPT-5.6 Terra | 100.0% | 1.8s | 170 | 64 |
| GPT-5.4 | 100.0% | 3.1s | 112 | 205 |
| Qwen3.8 Max (2.4T-A95B) | 100.0% | 2.1s | 224 | 160 |
| Qwen3.8-Flash | 100.0% | 2.3s | 233 | 187 |
| MiniMax M3 | 100.0% | 7.0s | 339 | 420 |
| GLM-5.3 | 100.0% | 74.5s | 181 | 13,837 |
| Claude Opus 5 | 100.0% | 7.2s | 267 | 390 |
| Claude Sonnet 5 | 100.0% | 2.3s | 267 | 9 |
| Claude Haiku 4.5 | 100.0% | 3.0s | 192 | 304 |
| GPT-5.6 Sol | 100.0% | 1.8s | 170 | 40 |
| Qwen3.5-35B-A3B | 0.0% | 6.7s | 120 | 1,043 |
| Qwen3.5-397B-A17B | 0.0% | 44.1s | 120 | 1,288 |
Tags
impurity_managementimpurity_profilequality_control