Local coding models on real hardware. Monthly. Reproducible.
Leaderboards measure models on hardware you don’t own, on tasks you don’t do. This report measures coding task classes — bug fixes, small features, refactors, test generation, documentation, SQL migrations, repository Q&A — on the GPUs people actually have, and publishes the harness settings so you can re-run it.
Methodology, pinned
Task classes drawn from real agentic work; a public benchmark subset for external credibility plus a curated suite per class; results recorded per run: success, cost, latency, retries, escalations, tokens by tier. Where humans are in the loop, acceptance is recorded too. The full protocol is versioned and published with the harness.
What we don’t publish: anything we haven’t run.
Latest report
report-index
[Report #1 — lands launch week]
Local Coding Models on Real Hardware, No. 1,
covering [N] models across [N] GPU classes.
Reports publish in the Benchmarks & Receipts category and surface here automatically once the first one lands.
Get the next one
One email a month: the hardware benchmark report and the routing changelog. No product nagging.
newsletter-form
The newsletter form mounts here. It is one email field and one consent checkbox, double opt-in, and it is not wired up yet.
Your hardware, specifically
Reports cover GPU classes. For your exact machine: