The Qwen models seem to be the best currently in my experience.
But if you're thinking about buying new hardware just for local inference: don't.
You'll spend more upfront for rapidly out of date and depreciating hardware while running inferior models compared to the latest from Anthropic and OpenAI.
Unless your requirement is prompt privacy, or you have idle hardware sitting around, the ROI just isn't there compared to a monthly subscription.
Would use a capability based routers so you can use a blend of OSS models. I.e. use the least capable model per prompt that includes the appropriate tooling capability, etc. Can even include a frontier provider subscription and get almost as many tokens at very close to the benchmarking on a $20/mo subscription as a $200/mo subscription. Easier with Claude's bearer token setup but I have seen people do it with OpenAI subscriptions as well.
OpenCode + vllm, model will depend on your hardware, but OpenCode also has a killer $10/m plan with quotas for some top tier open weight models.
I'm using qwen3.6 on a DGX spark, llama-cpp has prompt cache bugs for qwen/gemma models (among more being reported). Using my OpenCode-go sub when I want a bigger / more capable model
But if you're thinking about buying new hardware just for local inference: don't.
You'll spend more upfront for rapidly out of date and depreciating hardware while running inferior models compared to the latest from Anthropic and OpenAI.
Unless your requirement is prompt privacy, or you have idle hardware sitting around, the ROI just isn't there compared to a monthly subscription.
reply