The community is a compression algorithm

On June 13, Z.ai released GLM-5.2: a frontier-class model with 753 billion parameters, open weights, MIT license. You could download the exact same intelligence that powers their paid API and, legally speaking, do whatever you wanted with it.

There was just one problem. You could not run it.

The wall

Local inference lives and dies by one equation: generation speed is memory bandwidth divided by the bytes you read per token. No clever software escapes it. Your machine has to hold the model, and it has to stream the active part of it through the processor for every single token it writes.

GLM-5.2 ships in BF16 at 753 billion parameters: a 1.51 TB file. Nothing with a keyboard holds that, so the community’s first move is always to quantize. Even at 4-bit precision it is still a 400 GB file. It activates about 40 billion parameters per token, which means reading roughly 21.5 GB from memory for every token generated. Want a comfortable 30 tokens per second? That is 645 GB/s of sustained reads, on a machine that also holds 400 GB in memory. The best measurement I found on real, buyable hardware was 17.7 tokens per second on a 512 GB Mac Studio, and the arithmetic behind that number is exactly what the equation predicts.

Here is the twist that makes the story very 2026: that Mac Studio configuration no longer exists. When RAM prices exploded this spring, Apple quietly removed the 256 GB and 512 GB memory options. The one consumer machine that could hold the model left the catalog while the model was going viral.

So the honest summary, four weeks ago: a frontier model, free to use, that almost nobody outside a datacenter could actually use. An MIT license you could admire but not exercise.

The clock starts

I have seen this movie before, and so have you if you have followed local AI for a while.

In March 2023, Meta’s LLaMA weights were state of the art and required serious datacenter hardware. Within weeks, one developer wrote llama.cpp so it would run on his MacBook. Quantization formats followed, each generation squeezing more quality into fewer bits. Then Mixture-of-Experts models arrived, and the community worked out expert offloading, the trick that lets my own 8 GB gaming GPU run a 30B model at 41 tokens per second, which I wrote about in a 30B model on an 8 GB GPU.

None of those advances came from new hardware. They came from open weights plus time plus a crowd of motivated people.

GLM-5.2 restarted that clock on June 13. Within days, contributors were patching llama.cpp to load its unusual attention layout. And about four weeks in, the first serious compression landed.

Cutting the sleeping half

A Mixture-of-Experts model is mostly asleep. Each layer holds a pool of experts, small specialist networks, and a router wakes only a handful of them per token. Run the model on real text and you discover something useful: the router has favorites. Some experts fire constantly, others almost never.

REAP pruning, a technique published by Cerebras, turns that observation into a knife. Score every expert by how often the router actually uses it and how much its output matters. Then delete the bottom half. The router is rewired to the survivors, and the model keeps working.

Community builders applied it to GLM-5.2 and stacked aggressive 2-bit quantization on top. The result: 753B parameters became 381B, and two-bit quantization took the model down to a 139 GB file, an elevenfold cut from the 1.51 TB the weights shipped as.

Run the equation again with the new numbers. 139 GB fits in hardware that ordinary people own or can buy used. The bytes read per token drop from 21.5 to about 14.6, and suddenly a used big-memory Mac measures around 20 tokens per second, while a stack of used 24 GB gaming cards, the eternal kings of memory-per-dollar, crosses 30 for well under $10k. A month earlier, that hardware could not even load the model.

I want to be fair about the cost, because the model card is refreshingly blunt about it: this first prune loses real quality. Perplexity is up about 37%, greedy decoding can collapse into repetition, and you need specific sampler settings to keep it stable. This is a proof of direction, not a daily driver. But the direction is the point: a gentler 25% prune already sits within a few percent of the original, and a non-uniform 504B variant reportedly holds parity. The knife is getting sharper by the week.

Release day is the starting price

Here is the mental model I take away from this, and it applies well beyond GLM-5.2.

When a closed model ships, its cost of use is fixed by the vendor. When an open-weights model ships, its cost of use is just the opening bid. From that moment, thousands of people with strong incentives start shaving it down: quantizers, pruners, kernel writers, one person with a patch that makes an odd attention scheme load at all. Each contribution compounds with the others, because it all accumulates in shared, public tooling.

The practical consequence is easy to miss if you only read release-day coverage. The question “can we run this ourselves?” has a date attached. For GLM-5.2 the answer was no in June, is “yes, with caveats” in July, and will plausibly be a boring yes by autumn, on hardware that costs less than a mid-range company car.

If you work somewhere data cannot leave the building, this is the trend worth watching. On-premise frontier AI keeps looking like a thought experiment on the day each new model drops, and keeps quietly becoming an option a quarter later. Nobody makes the hardware cheaper. The community makes the model smaller.

Time, plus open weights, plus a crowd that wants it to work: that is a compression algorithm. And its ratio improves every year.

Further reading

• GLM-5.2 announcement from Z.ai, the model this whole story orbits.
• Unsloth’s GLM-5.2 GGUF quants, the 4-bit and 2-bit builds of the full model.
• GLM-5.2 REAP-50 Q2, the 139 GB community prune, with an honest model card about its limits.
• REAP: Router-weighted Expert Activation Pruning from Cerebras Research, the pruning method behind it.
• llama.cpp, the project that has been turning datacenter models into desktop ones since 2023.