MakoBench
Powered by CrucibleForge
Suite 4.0 (47 models tested)
MakoBench: Main Bench (suite 4.0)
27 ranked of 35 models Β· 12 older versions hidden
Swipe the table sideways for more columns.
| Model | Total | Chat | Code | Quant | Date tested |
|---|---|---|---|---|---|
| DeepSeek V4 Flash | 19.2 | 8.0 | 11.2 | API | 2026-11-01 |
| Grok 4.7 | 19.0 | 9.0 | 10.0 | API | 2026-11-06 |
| Kimi K3.1 Pro | 18.9 | 8.9 | 10.0 | API | 2026-11-06 |
| Claude Opus 5.5 | 18.6 | 8.9 | 9.7 | API | 2026-11-06 |
| Claude Sonnet 5.5 | 18.0 | 8.2 | 9.8 | API | 2026-11-06 |
| GPT-6.1 Terra | 17.7 | 8.1 | 9.6 | API | 2026-11-06 |
| DeepSeek V4 Pro | 17.5 | 7.4 | 10.1 | API | 2026-11-01 |
| GPT-6 Luna | 17.1 | 7.7 | 9.4 | API | 2026-11-06 |
| Claude Opus 5older version2 blocked | 16.4 | 8.0 | 8.3 | API | 2026-11-06 |
| Claude Opus 4.8older version | 16.2 | 7.1 | 9.1 | API | 2026-11-06 |
| Claude Opus 4.7older version | 16.1 | 6.9 | 9.3 | API | 2026-11-06 |
| Swish 31B FinalQ8_0 | 16.1 | 7.7 | 8.4 | Q8_0 | 2026-11-01 |
| Claude Fable 5.12 blocked | 15.7 | 8.0 | 7.7 | API | 2026-11-07 |
| Claude Fable 5older version5 blocked | 15.7 | 8.4 | 7.3 | API | 2026-11-07 |
| Claude Opus 4.6older version | 15.5 | 7.3 | 8.3 | API | 2026-11-06 |
| Claude Haiku 5.5 | 15.3 | 6.0 | 9.3 | API | 2026-11-06 |
| Claude Sonnet 5older version | 15.2 | 6.2 | 9.0 | API | 2026-11-06 |
| Artemis 31B v1.2 | 15.2 | 7.1 | 8.1 | Q6_K | 2026-11-01 |
| Omega Arsenal 31B FinalQ8_0 | 15.1 | 6.5 | 8.6 | Q8_0 | 2026-11-01 |
| Qwen3.8 27B HereticNVFP4 | 14.9 | 6.9 | 8.0 | NVFP4 | 2026-11-03 |
| Dark Craving 31B v2.0Q8_0 | 14.5 | 6.9 | 7.5 | Q8_0 | 2026-11-03 |
| MiniMax M3 | 14.4 | 5.9 | 8.5 | API | 2026-11-02 |
| Claude Sonnet 4.6older version | 14.2 | 6.9 | 7.4 | API | 2026-11-06 |
| Dark Scarlett 27B v3.0Q8_0β provisional | 14.2 | 6.4 | 7.9 | Q8_0 | 2026-11-01 |
| Dark Scarlett 27B v2.0 (Q5_K_M)Q5_K_M | 14.0 | 5.9 | 8.1 | Q5_K_M | 2026-11-01 |
| Claude Opus 4.5older version | 13.9 | 6.6 | 7.2 | API | 2026-11-06 |
| OrcaRouter Qwen3.8 27BQ8_0 | 13.8 | 6.3 | 7.6 | Q8_0 | 2026-11-03 |
| MiniMax M3.1 Flash | 13.5 | 5.4 | 8.1 | API | 2026-11-02 |
| Darker Scarlett 27B v0.1Q4_K_M | 13.1 | 6.2 | 6.9 | Q4_K_M | 2026-11-01 |
| Claude Sonnet 4.5older version | 12.8 | 6.7 | 6.1 | API | 2026-11-06 |
| MiniMax M2.7older version | 12.6 | 6.5 | 6.1 | API | 2026-11-02 |
| Split-Untied 31BQ8_0 | 12.1 | 6.9 | 5.2 | Q8_0 | 2026-11-03 |
| MiniMax M2.5older version | 11.7 | 5.5 | 6.2 | API | 2026-11-02 |
| Qwen3.8 27B Twin Turbo FableQ5_K_Mβ provisional | 11.5 | 5.6 | 5.8 | Q5_K_M | 2026-11-01 |
| Claude Haiku 4.5older version | 11.4 | 5.6 | 5.8 | API | 2026-11-06 |
| Wicked Tutu 26B-A4BQ8_0 | 11.2 | 5.2 | 6.1 | Q8_0 | 2026-11-01 |
| gpt-oss 20B | 11.2 | 4.2 | 7.0 | Q8_0 | 2026-11-06 |
| Qwen3.5 122B HereticQ4_K_S | 11.2 | 5.4 | 5.8 | Q4_K_S | 2026-11-01 |
| Gemma 4 26B-A4B QATQ4_0 | 10.2 | 4.2 | 6.0 | Q4_0 | 2026-11-01 |
| Darker Hero 27BQ4_K_Sβ provisional | 9.5 | 4.1 | 5.4 | Q4_K_S | 2026-11-01 |
| Spark-X 2.6 4B | 5.4 | 2.1 | 3.2 | Q8_0 | 2026-11-07 |
| DeepSeek R1 0528 8BQ4_K_M | 2.7 | 1.3 | 1.3 | Q4_K_M | 2026-11-03 |
| Qwen3.8 27BQ5_K_Sβ provisional | β | β | 8.5 | Q5_K_S | 2026-11-07 |
| Grok Build 0.1β provisional | β | β | 4.5 | API | 2026-11-06 |
| Qwen2.5 1.5BQ4_K_Mβ provisional | β | β | 0.3 | Q4_K_M | 2026-11-05 |
| Qwen2.5 0.5BQ8_0β provisional | β | β | 0.3 | Q8_0 | 2026-11-05 |
| Gemma 2 2B AbliteratedQ5_K_Sβ provisional | β | β | 0.2 | Q5_K_S | 2026-11-05 |
- Total = Code + Chat. 10 each is a perfect run of the standard tests, so 20 is the nominal top; bonus packs add points on top, up to the suite's most possible (a row's bars run to it).
- Β± Total = half of the 90% interval of a bootstrap over the test cases; models closer together than that are statistically tied.
- Each model is run once, so Β± covers which cases were drawn, not run-to-run noise, and it understates the real uncertainty. Rows for the same weights (quant or serving-tier twins) differ by up to 2.3 points here. Treat gaps of a point or two as ties.
- Class is a coarse band on Total: S β₯ 16, A β₯ 13, B β₯ 10, C β₯ 0.
- Lean = chat-leaning or code-leaning when the gap to the board median is 1.5 points or more, otherwise balanced.
- tok/s = median generation speed; only comparable between models run on the same host.
- Quant = the quantization we scored. Q8_0 is the standard for local models; any other quant is marked as a non-standard quant in the row details. API = a hosted model, served by its provider.
- Run cost = what one run of the Code and Chat tests cost at the provider's price, generation only. ~ = estimated from token counts at list price. * means some requests came back without usage data, so the figure may be low. β = run on our own hardware, so no bill.
- N blocked = the provider's content filter blocked N replies; each scores zero.
- One row per model: other quants and serving tiers of the same weights sit under the main row (open the arrow) and are not ranked.
- β Provisional = CrucibleForge's compatibility check failed for that model, so its score may mis-measure it. It is shown without a rank until it is re-run, except rows tested below Q5, which are not re-run.
- A dash in Total means that model has no complete Chat score yet, so it is not ranked.
MakoBench is free and has no ads. Running it costs power, hardware and our time.
β Donate: Help keep us goingRequest a model Want a model tested? Search the list, add your +1, or send a new one.