← Back to Test Cases
21cfr11-001
GMP Knowledge21cfr11medium
Cross-Model Comparison
| Model | Score | Latency | Tokens In | Tokens Out |
|---|---|---|---|---|
| GLM-5.3 | N/A | N/A | - | - |
| MiniMax M3 | N/A | N/A | - | - |
| Hy4 Preview | N/A | N/A | - | - |
| Muse Glimmer 30B | N/A | N/A | - | - |
| Qwen3.8 Max (2.4T-A95B) | N/A | N/A | - | - |
| Qwen3.8-Flash | N/A | N/A | - | - |
| Llama 4 Scout | 100.0% | 4.7s | 77 | 387 |
| Llama 4 Maverick | 100.0% | 18.5s | 77 | 438 |
| Llama 3.3 70B Instruct | 100.0% | 6.0s | 78 | 185 |
| Mistral Small 2603 | 100.0% | 1.7s | 85 | 241 |
| DeepSeek-V3.2 | 100.0% | 5.2s | 71 | 169 |
| DeepSeek-R1 | 100.0% | 10.0s | 77 | 310 |
| DeepSeek-R1-Distill-Qwen-32B | 100.0% | 39.0s | 72 | 500 |
| Mistral Large 3 675B | 100.0% | 4.2s | 73 | 243 |
| Qwen3.5-35B-A3B | 100.0% | 9.5s | 80 | 1,354 |
| Gemini 3.1 Pro | 100.0% | 12.7s | 70 | 1,020 |
| Gemini 3 Flash | 100.0% | 10.2s | 70 | 249 |
| Gemini 3.1 Flash-Lite | 100.0% | 2.4s | 70 | 296 |
| Qwen3.6 27B | 100.0% | 65.0s | 80 | 1,184 |
| Qwen3.6 35B A3B | 100.0% | 11.3s | 80 | 2,172 |
| DeepSeek V4 Pro | 100.0% | 2.2s | 71 | 98 |
| DeepSeek-V3.2 | 100.0% | 5.8s | 71 | 154 |
| DeepSeek V4 Flash | 100.0% | 7.1s | 71 | 390 |
| Gemma 4 26B A4B IT | 100.0% | 10.3s | 83 | 328 |
| Gemma 4 31B IT | 100.0% | 12.3s | 83 | 262 |
| MiniMax M2.7 | 100.0% | 9.4s | 105 | 539 |
| Llama 4 Maverick | 100.0% | 4.1s | 136 | 378 |
| Claude Opus 4.8 | 100.0% | 6.9s | 110 | 506 |
| GLM-5.2 | 100.0% | 4.0s | 79 | 402 |
| Llama 4 Scout | 100.0% | 10.8s | 135 | 352 |
| Llama 3.3 70B Instruct | 100.0% | 15.7s | 137 | 317 |
| Mistral Small 2603 | 100.0% | 3.0s | 148 | 329 |
| Qwen3.8 27B | 100.0% | 3.1s | 184 | 174 |
| Mistral Large 3 675B | 100.0% | 12.6s | 136 | 251 |
| Qwen3.5-35B-A3B | 100.0% | 12.1s | 142 | 978 |
| Qwen3.6 27B | 100.0% | 64.0s | 142 | 1,822 |
| Qwen3.6 35B A3B | 100.0% | 11.0s | 142 | 1,801 |
| Qwen3.5-397B-A17B | 100.0% | 5.8s | 142 | 1,038 |
| DeepSeek V4 Pro | 100.0% | 11.8s | 131 | 370 |
| DeepSeek-V3.2 | 100.0% | 2.9s | 131 | 90 |
| DeepSeek V4 Flash | 100.0% | 4.4s | 131 | 356 |
| DeepSeek-R1 | 100.0% | 69.1s | 132 | 1,097 |
| Gemma 4 26B A4B IT | 100.0% | 6.8s | 145 | 320 |
| Qwen3.8 Max (2.4T-A95B) | 100.0% | 3.8s | 184 | 339 |
| Gemma 4 31B IT | 100.0% | 9.7s | 145 | 320 |
| MiniMax M2.7 | 100.0% | 14.6s | 168 | 1,053 |
| GLM-5.2 | 100.0% | 18.6s | 139 | 461 |
| Muse Glimmer 30B | 100.0% | 6.4s | 181 | 324 |
| Kimi K3 | 100.0% | 13.1s | 212 | 519 |
| GLM-5.3-Flash | 100.0% | 24.9s | 139 | 155 |
| Hy4 Preview | 100.0% | 12.8s | 148 | 798 |
| GPT-5.6 Luna | 100.0% | 1.9s | 131 | 91 |
| GPT-5.4 mini | 100.0% | 813ms | 73 | 69 |
| GPT-5.4 nano | 100.0% | 1.1s | 73 | 145 |
| Claude Haiku 4.5 | 100.0% | 3.4s | 79 | 292 |
| Claude Sonnet 4.6 | 100.0% | 7.9s | 79 | 308 |
| Claude Opus 4.6 | 100.0% | 9.1s | 79 | 372 |
| GPT-5.4 | 100.0% | 2.2s | 73 | 69 |
| Qwen3.8-Flash | 100.0% | 9.0s | 193 | 304 |
| MiniMax M3 | 100.0% | 4.2s | 303 | 323 |
| GLM-5.3 | 100.0% | 2.8s | 140 | 204 |
| Claude Sonnet 5 | 100.0% | 1.9s | 198 | 9 |
| Claude Opus 5 | 100.0% | 6.1s | 198 | 392 |
| Claude Haiku 4.5 | 100.0% | 2.6s | 143 | 252 |
| GPT-5.6 Sol | 100.0% | 2.8s | 131 | 46 |
| GPT-5.6 Terra | 100.0% | 2.8s | 131 | 86 |
| Qwen3.5-397B-A17B | 0.0% | 25.9s | 80 | 1,066 |
Tags
electronic_recordscompliance