← Back to Test Cases
ichq7-002
GMP KnowledgeIch Q7medium
Cross-Model Comparison
| Model | Score | Latency | Tokens In | Tokens Out |
|---|---|---|---|---|
| Qwen3.8-Flash | N/A | N/A | - | - |
| GLM-5.3 | N/A | N/A | - | - |
| MiniMax M3 | N/A | N/A | - | - |
| Qwen3.8 Max (2.4T-A95B) | N/A | N/A | - | - |
| Hy4 Preview | N/A | N/A | - | - |
| Muse Glimmer 30B | N/A | N/A | - | - |
| Llama 4 Scout | 100.0% | 1.6s | 97 | 429 |
| Llama 4 Maverick | 100.0% | 13.4s | 98 | 414 |
| Mistral Small 2603 | 100.0% | 2.4s | 102 | 373 |
| Qwen3.5-397B-A17B | 100.0% | 33.3s | 98 | 1,189 |
| DeepSeek-V3.2 | 100.0% | 21.9s | 91 | 174 |
| DeepSeek-R1 | 100.0% | 21.3s | 97 | 792 |
| DeepSeek-R1-Distill-Qwen-32B | 100.0% | 27.2s | 91 | 625 |
| Mistral Large 3 675B | 100.0% | 5.9s | 90 | 382 |
| Gemini 3.1 Pro | 100.0% | 9.2s | 88 | 871 |
| Gemini 3 Flash | 100.0% | 2.6s | 88 | 237 |
| Gemini 3.1 Flash-Lite | 100.0% | 2.0s | 88 | 243 |
| Qwen3.6 27B | 100.0% | 12.9s | 98 | 1,053 |
| Qwen3.6 35B A3B | 100.0% | 5.1s | 98 | 924 |
| DeepSeek V4 Pro | 100.0% | 16.9s | 91 | 928 |
| DeepSeek-V3.2 | 100.0% | 8.1s | 91 | 153 |
| Llama 4 Scout | 100.0% | 8.2s | 155 | 326 |
| DeepSeek V4 Flash | 100.0% | 6.9s | 91 | 219 |
| Gemma 4 26B A4B IT | 100.0% | 4.0s | 101 | 277 |
| Gemma 4 31B IT | 100.0% | 8.7s | 101 | 208 |
| MiniMax M2.7 | 100.0% | 7.2s | 128 | 355 |
| Claude Opus 4.8 | 100.0% | 9.4s | 150 | 678 |
| GLM-5.2 | 100.0% | 5.2s | 100 | 423 |
| Llama 4 Maverick | 100.0% | 20.7s | 155 | 365 |
| Llama 3.3 70B Instruct | 100.0% | 7.5s | 183 | 299 |
| Qwen3.8 27B | 100.0% | 2.5s | 202 | 109 |
| Mistral Small 2603 | 100.0% | 2.2s | 165 | 287 |
| Mistral Large 3 675B | 100.0% | 3.7s | 153 | 313 |
| Qwen3.5-35B-A3B | 100.0% | 5.9s | 160 | 1,178 |
| Qwen3.6 27B | 100.0% | 44.6s | 160 | 1,972 |
| Qwen3.6 35B A3B | 100.0% | 6.2s | 160 | 1,235 |
| Qwen3.5-397B-A17B | 100.0% | 27.9s | 160 | 2,120 |
| DeepSeek V4 Pro | 100.0% | 5.2s | 151 | 248 |
| DeepSeek-V3.2 | 100.0% | 5.8s | 157 | 139 |
| DeepSeek V4 Flash | 100.0% | 3.9s | 151 | 275 |
| DeepSeek-R1 | 100.0% | 27.5s | 150 | 887 |
| Gemma 4 26B A4B IT | 100.0% | 7.2s | 163 | 311 |
| Gemma 4 26B A4B IT | 100.0% | 9.4s | 163 | 210 |
| Gemma 4 31B IT | 100.0% | 14.5s | 163 | 229 |
| Hy4 Preview | 100.0% | 50.2s | 168 | 4,200 |
| MiniMax M2.7 | 100.0% | 6.3s | 188 | 314 |
| GLM-5.2 | 100.0% | 13.0s | 160 | 707 |
| Kimi K3 | 100.0% | 16.0s | 232 | 567 |
| GLM-5.3-Flash | 100.0% | 33.1s | 160 | 1,053 |
| GPT-5.6 Luna | 100.0% | 2.1s | 151 | 101 |
| GPT-5.4 mini | 100.0% | 1.0s | 93 | 67 |
| GPT-5.4 nano | 100.0% | 1.9s | 93 | 60 |
| Claude Haiku 4.5 | 100.0% | 3.5s | 98 | 304 |
| Claude Sonnet 4.6 | 100.0% | 9.0s | 98 | 392 |
| Claude Opus 4.6 | 100.0% | 11.1s | 98 | 417 |
| GPT-5.6 Terra | 100.0% | 2.4s | 151 | 79 |
| GPT-5.4 | 100.0% | 2.6s | 93 | 154 |
| Muse Glimmer 30B | 100.0% | 6.0s | 201 | 363 |
| Qwen3.8 Max (2.4T-A95B) | 100.0% | 2.3s | 202 | 192 |
| Qwen3.8-Flash | 100.0% | 3.4s | 211 | 283 |
| MiniMax M3 | 100.0% | 6.9s | 310 | 221 |
| GLM-5.3 | 100.0% | 7.8s | 160 | 1,135 |
| Claude Opus 5 | 100.0% | 12.1s | 238 | 788 |
| Claude Sonnet 5 | 100.0% | 1.4s | 238 | 9 |
| Claude Haiku 4.5 | 100.0% | 3.0s | 162 | 289 |
| GPT-5.6 Sol | 100.0% | 1.6s | 151 | 45 |
| Qwen3.5-35B-A3B | 0.0% | 8.7s | 98 | 1,468 |
| Llama 3.3 70B Instruct | 0.0% | 1800.5s | 99 | 4 |
Tags
process_validationdocumented_evidence