MakoBench: Main Bench

• Total = Code + Chat. 10 each is a perfect run of the standard tests, so 20 is the nominal top; bonus sets add points on top, with no ceiling.
• ± Total = half of the 90% interval of a bootstrap over the test cases; models closer together than that are statistically tied.
• Each model is run once, so ± covers which cases were drawn, not run-to-run noise, and it understates the real uncertainty. Rows for the same weights (quant or serving-tier twins) differ by up to 2.3 points here. Treat gaps of a point or two as ties.
• Class is a coarse band on Total: S ≥ 16, A ≥ 13, B ≥ 10, C ≥ 0.
• Lean = chat-leaning or code-leaning when the gap to the board median is 1.5 points or more, otherwise balanced.
• tok/s = median generation speed; only comparable between models run on the same host.
• Quant = the quantization we scored. Q8_0 is the standard for local models; any other quant is marked as a non-standard quant in the row details. API = a hosted model, served by its provider.
• Run cost = what one run of the Code and Chat tests cost at the provider's price, generation only. ~ = estimated from token counts at list price. * means some requests came back without usage data, so the figure may be low. — = run on our own hardware, so no bill.
• N blocked = the provider's content filter blocked N replies; each scores zero.
• One row per model: other quants and serving tiers of the same weights sit under the main row (open the arrow) and are not ranked.
• ⚠ Provisional = CrucibleForge's compatibility check failed for that model, so its score may mis-measure it. It is shown without a rank until it is re-run, except rows tested below Q5, which are not re-run.
• A dash in Total means that model has no complete Chat score yet, so it is not ranked.

New here? See what the tests measure.