MakoBench Adds a Run Cost Column
- Category
- AI & Local LLM
- Posted
- November 7, 2026
- Updated
- November 7, 2026
- By
- Jacob Lloyd — written with AI assistance, post-project
- Read time
- 2 min read
In plain terms: MakoBench is our public board of language models. Hosted models now carry a Run cost: what one run of the standard Code and Chat tests cost at the provider's published price. Local models show a dash because they run on our own hardware. When a provider's content filter blocks a reply, the row says so and the reply scores zero.
The board has a new column, Run cost. For a hosted model it is what one full run of the standard Code and Chat tests cost at the provider's published price, adding up the tokens sent in and the tokens written back, thinking included.
- What is left out. Bonus sets, retries, the judge (it runs on our own hardware) and any cache or off-peak discount. The figure is close to what you would pay at list price.
- ~ before a number. The provider did not bill per token (a flat plan), so we priced the token counts at list price.
- * after a number. Some requests came back without usage data, so the figure may be low.
- A dash. The model ran on our own hardware. There is no bill.
The same release marks blocked replies. Some providers run a content filter that can refuse to return a reply. A row where that happened carries a small "blocked" mark and a note in its details. A blocked reply scores zero, the same as a failed answer. CrucibleForge finds them in the transcripts on every refresh, so future runs are marked automatically.
Two board updates
- Latest versions only. A tick box, on by default, hides a model when a newer version of it is on the board, so Claude Opus 5 steps aside for 5.5. It never hides a fine-tune. Each row also has a Built on tag, the product line for hosted models and the base weights for local ones, and the filter matches it.
- MAX. A small MAX mark shows when a model has reached the top of every set in a part of the tests. The tests cannot tell such models apart, so a harder set is due for that part.
The table is also easier to read on a phone. Model names now show in full, with short names in place of long file names, and a hint says when the table scrolls sideways.
Details are in the documentation and in How MakoBench works.