How a 744B-parameter AI model is running on a normal laptop

A 744 billion parameter AI model sounds like something that should live in a datacenter, not on a regular laptop that struggles with a dozen Chrome tabs. Yet that’s exactly what’s happening with GLM 5.2, one of the strongest open models available today, thanks to a tiny C project called Colibri.

With no GPU and just 25–32 GB of RAM, Colibri can run GLM 5.2 locally by trading raw speed for clever systems engineering. It’s slow, but it works—and it completely changes how we think about who can access frontier AI models.

Why a 744B model doesn’t actually need 744B in RAM

On paper, GLM 5.2 has 744 billion parameters. If you take that number at face value, the official deployment guides make sense: they tell you to run it on an eight-node H200 GPU cluster with a huge context window and massive memory.

But those headline parameter counts are misleading. GLM 5.2 is a Mixture-of-Experts (MoE) model, which means only a small part of the network is active for each token. In practice:

• Only about 40 billion parameters are used per token.
• Of those, roughly 11 GB of data actually change from one token to the next.

So while the full model file is huge, most of it is “asleep” at any given moment. That’s the key insight that makes running GLM 5.2 on a laptop possible.

Mixture-of-Experts in plain language

To understand why this works, it helps to know what a Mixture-of-Experts model actually is.

In a standard dense model, every neuron participates in every forward pass. Ask it a question, and the entire network wakes up to help answer it. That’s simple, but extremely inefficient.

In a Mixture-of-Experts model like GLM 5.2, the network is split into many specialized “experts.” When you feed in a token, a router decides which experts are relevant and only activates those. The rest stay inactive.

Think of it like a giant office building: instead of dragging every employee into every meeting, you only invite the people who actually know something about the topic. For GLM 5.2, that means:

• Around 40B parameters are active per token.
• The remaining ~700B parameters sit idle unless they’re specifically needed.

Because of this, loading the entire 744B parameter set into RAM is unnecessary. The total size becomes more of a storage problem than a live memory problem.

Colibri: a tiny bare-metal engine for GLM 5.2

Colibri is a minimalist inference engine written in about 1,300 lines of pure C. There’s no Python, no PyTorch, no CUDA—just a low-level program that reads bytes from disk and does the math.

To make GLM 5.2 fit on a normal machine, Colibri does two big things:

1. 4-bit quantization: The model weights are converted into a compact 4-bit integer format (int4). This shrinks the dense “core” of the model—things like attention layers, embeddings, and shared experts that always run—down to about 9.9 GB.

2. Split core vs. experts: Colibri loads that 9.9 GB dense core into RAM and keeps it there. The rest of the routed experts—around 370 GB worth—stay on your NVMe SSD.

When the router decides a specific expert is needed for the next token, Colibri streams only that expert from disk into memory on demand. Instead of buying more VRAM or RAM, you’re effectively swapping memory for SSD read speed.

The brutal trade-off: speed vs. memory

This design has a very real downside: SSDs are fast, but they’re nowhere near as fast as VRAM or even main memory.

On a cold cache—when none of the needed experts are already in RAM—Colibri may need to read about 11 GB of fresh expert data from the SSD just to generate a single token. That leads to generation speeds around 0.05–0.1 tokens per second on typical hardware.

In practice, that means:

• You don’t see a smooth streaming response like a cloud API.
• You type a prompt, hit enter, and wait—sometimes minutes—for a short paragraph.
• It feels more like running a heavy batch job than chatting with an assistant.

This sparked debate among developers: is something this slow actually useful, or is it just a cool demo? The answer depends on your expectations and your hardware.

Fighting the SSD bottleneck with multi-token prediction

Colibri doesn’t just accept the disk bottleneck—it tries to work around it using Multi Token Prediction (MTP), a form of speculative decoding.

Instead of generating one token, fetching new experts, then repeating, MTP lets the model:

• Guess several future tokens ahead.
• Check if the experts needed for those tokens are already warm in cache.
• Verify multiple tokens in a single forward pass when possible.

With a warm cache, Colibri can validate around 2.2–2.8 tokens per pass, significantly improving effective throughput.

However, MTP is a double-edged sword. On a completely cold cache, speculating ahead can require loading even more experts from disk, which can make things slower. The payoff gets much better on machines with very fast internal storage.

Real-world performance: when does this become usable?

On modest hardware, Colibri is slow but functional. It’s best suited for:

• Long, complex reasoning tasks where latency matters less than capability.
• Offline experiments, research, or hobby projects where cloud costs are a concern.

On faster machines, the story changes. Tests on an Apple M5 Max with high-speed internal storage report around 1.06 tokens per second. That’s still slower than a cloud API, but it crosses an important threshold: it feels like a usable tool instead of a science fair project.

At that speed, you can kick off a non-trivial coding or reasoning task, grab a coffee, and come back to a high-quality answer from a frontier-level model running entirely on your own device.

How strong is GLM 5.2, really?

GLM 5.2 isn’t just big—it’s genuinely competitive with top-tier proprietary models, especially for coding and reasoning.

Key points about the model:

• Trained on roughly 28.5 trillion tokens (original GLM 5, released February 2026).
• Updated through GLM 5.1 and then GLM 5.2, released June 13 under an unrestricted MIT license.
• Aggressively tuned for code and complex reasoning.

On benchmarks:

• It scores 51 on the Artificial Analysis Intelligence Index v4.1, reportedly the highest among open-weight models.
• On Terminal Bench 2.1 (long-horizon coding), it scores 81.0, reportedly beating OpenAI’s GPT 5.5 on those specific developer tasks.

It also offers:

• Two reasoning modes (High and Max).
• A 1 million token context window (though the long-context sparse attention needed to fully exploit this isn’t yet fully wired up in Colibri).

If you want a deeper comparison of how GLM 5.2 stacks up against other frontier models, see our breakdown in GLM 5.2 vs Opus 4.8 vs GPT 5.5.

What about quality loss from 4-bit quantization?

To make GLM 5.2 fit into 9.9 GB of RAM for the dense core, Colibri uses 4-bit quantization. This is a very aggressive compression, and it inevitably raises a key question: how much quality does the model lose?

At the time of writing, the community is still actively benchmarking the 4-bit Colibri variant. Early signs suggest that, while there is some degradation compared to full-precision weights, the model remains extremely capable for coding and reasoning tasks—especially considering it’s running on commodity hardware with no GPU.

For more background on where GLM 5.2 came from and why it’s such a big deal in the open ecosystem, you can check our earlier coverage in our GLM 5.2 launch roundup.

Why this matters: frontier AI without the datacenter

For years, big AI labs have pushed the idea that true “frontier” intelligence lives behind APIs, paywalls, and GPU clusters. If you wanted top-tier performance, you were expected to rent it from the cloud and swallow the cost.

Colibri and GLM 5.2 challenge that assumption. They show that:

• You don’t need a datacenter to experiment with frontier weights.
• You can trade speed for memory and run a state-of-the-art model on a normal laptop.
• Access to cutting-edge AI can be a systems engineering problem, not just a budget problem.

It’s not about matching cloud-level latency. It’s about proving that these models can exist outside corporate infrastructure, on machines that individuals actually own.

Who is this setup for?

Running GLM 5.2 via Colibri is not for everyone. It’s best suited for:

• Developers and researchers who care more about capability and control than speed.
• Hobbyists who want to build complex local pipelines—agents, code tools, or research assistants—without paying heavy API bills.
• Systems engineers interested in pushing the limits of what’s possible on commodity hardware.

If your primary goal is a snappy chat assistant, a smaller local model or a cloud API will feel much better. But if you want GPT 5.5-level coding ability running directly on your machine, fully under your control, this is one of the most exciting paths available right now.

A shift in mindset: from consumer product to computer science

The most important part of this story isn’t the exact token-per-second number. It’s the mindset shift.

Instead of treating AI as a black-box subscription service, Colibri treats it as a computer science challenge: how do you make an enormous, sparsely activated model work on limited hardware, even if it’s slow?

By leaning on Mixture-of-Experts, 4-bit quantization, disk streaming, and speculative decoding, it shows that the barrier to entry for frontier AI is lower than we’ve been led to believe. If you’re willing to accept latency and roll up your sleeves, the “impossible” suddenly becomes a weekend project on the laptop you already own.

That’s not just a clever hack—it’s a glimpse of a future where cutting-edge models are tools you can run, inspect, and modify yourself, not just services you rent.