Owned vs rented intelligence: open weights and local LLM hardware
Post731 words
Argues rented AI can be switched off, so run open weights locally. The frontier leads by about 11 to 22 points on SWE-bench Pro, but open models closed the gap elsewhere, and the play is a 4-bit small-active-param MoE on a $3-4k box.
Key claims
Frontier gap is roughly 11 to 22 points on SWE-bench Pro
4-bit quantization shrinks a 30B model from ~60GB to ~18GB with minor quality loss
GX10 (128GB, 273 GB/s) runs 4-bit MoEs at ~20-55 tok/s
Claude Fable 5, the most capable model ever released to the public, lasted three days before a US export-control order pulled it offline.
But you know the story. That's why I'm here to tell you about the part that REALLY matters:
Anything you "rent" can be switched off by someone who isn't you.
Sounds logical, no?
For me, THIS is the whole case for owning your stack, and here is where open weights actually stand today:
The frontier still wins on the hard stuff. Fable 5 (and even Opus 4.8) lead on the measured, long-horizon agentic coding benchmarks, the contamination-resistant ones that actually predict production reliability.
That gap is real, roughly 11 to 22 points on SWE-bench Pro.
But on everything else it has basically closed:
- GLM-5 beat the then-current Opus 4.6 on MATH-500 head to head (97.4 vs 96.4).
- Kimi K2.6 hits 96.4% on AIME 2026.
These ship under MIT and Modified-MIT licenses, which means you download the weights. They are yours. Forever. No kill switch.
Yet, it doesn't end there, bc the Chinese AI labs (obviously) used the opportunity that the Trump admin handed them on a silver platter to ship their own new flagship models:
- GLM-5.2 (Jun 13)
- Kimi K2.7 (Jun 12)
And do not sleep on these two:
- K2.7 is a 1T MoE (32B active) that cut reasoning tokens ~30% over K2.6, so it's cheaper and faster per task, and Moonshot actually benched it head to head against Opus 4.8 and GPT-5.5, not just against itself. It even edges Opus 4.8 on some agentic tool-use rows.
- GLM-5.2 ships a 1M context, runs ~3x faster than 5.1 (Z . ai's number) at roughly a tenth of frontier pricing, and already took #1 on BridgeBench's reasoning board, ahead of Fable 5.
One widely shared hands-on ranking slots both right around Opus 4.8, with K2.7 nosing just ahead and GLM-5.2 right alongside it (even tho SWE-bench Pro, LiveCodeBench or DeepSWE haven't reported on them yet).
So treat the launch numbers as early, not locked, and run your own evals on your own tasks.
But as impressive as that is, lets get back to the part that actually matters for people like us: running this stuff on your own hardware.
Spoiler: Unfortunately, you can't run the 744B and 1T flagships at home. And definitely not on the mac minis or gx10 devices that CT has been selling you since the OpenClaw-mania days.
Yet, local LLMs are more powerful than ever and there is a way of how you can make stuff work.
The play is the small-active-param MoE tier on a 3 to 4k USD box, which is where quantization comes in, bc it is how you can make efficiency happen.
Raw weights ship in 16-bit. Quantizing rounds them down to 8, 4, sometimes 2 bits.
WTF that means?
In short, it shrinks the model AND, more importantly, shrinks how many bytes you move per token.
On a bandwidth-bound box, that is what sets your speed, and 4-bit is the sweet spot.
A 30B model drops from ~60GB to ~18GB and runs several times faster, for a quality hit you mostly never notice (and which at that point you can even run on a good mac mini).
8-bit is near lossless if you have the memory.
Go below 4-bit and it falls apart, which is exactly why cramming a 744B model into 2-bit just to make it "fit" is a trap (aka you lose way too much performance).
So, TL;DR:
The move is not to buy the cheapest hardware or go for the maximum possible and spill $10k+ before you even have a use case.
It's to get the right model at 4-bit with some memory headroom for context.
For example: On my ASUS GX10 (128GB, 273 GB/s), here is what actually fits and runs at 4-bit:
- Qwen3-30B-A3B
- GLM-4.7-Flash
- Kimi-Linear-48B
- GLM-4.5-Air
- Qwen3.5-122B-A10B
Roughly 20 to 55 tok/s depending on active params and engine, north of 100 with an NVFP4 build.
Sure, none of these can compete with Fable 5.
But all of them are private, free per token, run offline, and cannot be revoked by anyone. Ever.
Borrowed intelligence is smarter today.
Owned intelligence is reliable forever. The capability gap is shrinking. The sovereignty gap is widening.
For anything that has to keep working tomorrow, build the boring local box.
2 images on X