How Gemma 4 Coder brings Fable 5‑level reasoning to your laptop

Local coding models have quietly leveled up. One of the most interesting new arrivals is a community fine-tune of Google’s Gemma 4 called Gemma 4 12B Coder Fable 5 Composer 2.5—or simply, Gemma Coder. It runs fully offline on consumer hardware, yet carries the reasoning style of Anthropic’s Fable 5, a model many people can’t even access anymore.

Why Gemma 4 12B is such a strong base

Gemma 4 has become one of the most downloaded open model families, with over 150 million pulls across its variants. The 12B version is the sweet spot for local use: it’s small enough to run on a typical laptop with 16 GB of RAM, but still powerful enough for serious coding and reasoning work.

Unlike many multimodal models, Gemma 4 doesn’t bolt on separate vision or audio encoders. Instead, it projects raw image patches and audio waveforms directly into the language model. That encoder-free design keeps the model compact while still handling text, images, and audio in a single architecture. If you want a deeper dive into why this matters, check out this breakdown of the encoder-free Gemma 4 12B design.

On benchmarks, Gemma 4 12B is more than just efficient. It scores around 77% on MMLU Pro, a broad reasoning and knowledge test. That’s a big jump over last year’s Gemma 3 27B, which landed closer to 67%, despite being more than twice the size. In practice, the newer 12B model rivals Google’s own 26B model on many tasks while using less than half the memory.

Other key specs:

• 256,000-token context window for long conversations and large code files
• Support for roughly 140 languages
• Apache 2 license, so it’s safe for commercial use, modification, and redistribution
• Training data up to January 2025

That Apache 2 license is a quiet but important upgrade. Earlier Gemma releases had custom terms that made some companies nervous about shipping products on top of them. With Apache 2, those concerns largely disappear.

What Gemma Coder actually is

Gemma Coder is a community fine-tune built on top of the instruction-tuned Gemma 4 12B base model. The goal: turn an already strong general model into a focused, reliable Python coding assistant that you can run locally.

Despite being uploaded as a hobby project, it quickly climbed Hugging Face’s trending charts and racked up thousands of downloads. The appeal is simple: it combines Gemma 4’s efficiency with the distilled reasoning of much larger, closed models.

Training on only code that actually works

Most coding models are trained on huge piles of scraped code. A lot of that code is buggy, incomplete, or never even run. The model learns patterns, but not necessarily correct solutions.

Gemma Coder takes a different approach. Every training example had to pass its own tests before it was allowed into the dataset. If the reasoning didn’t produce code that ran successfully, it was thrown out.

For a relatively small 12B model, this kind of clean, verified training signal makes a noticeable difference. You’re not just getting plausible-looking code—you’re getting patterns that have already been validated against real test cases.

How Fable 5’s reasoning lives on inside a local model

The long model name, “Fable 5 Composer 2.5,” hints at the two main reasoning sources used to train Gemma Coder:

• Composer 2.5 – the primary source of solutions. Only answers where the generated code passed its tests were kept.
• Fable 5 – Anthropic’s high-end model, used as a second-pass solver for the hardest problems.

The training pipeline worked like this:

1. Composer 2.5 attempted a coding problem.
2. If its solution failed the tests, that problem was handed to Fable 5.
3. Fable 5 solved it again from scratch, with fresh reasoning.
4. Only passing, fully tested solutions were kept for training Gemma Coder.

In other words, this small 12B local model has been distilled from the reasoning traces of a much more powerful system. Fable 5 itself briefly launched in early June before access was suspended under export control rules. Many people can’t touch the hosted version anymore—but its “thinking style” survives inside this open, offline model.

That’s why people jokingly call Gemma Coder the “ghost of Fable 5” in open source. If you’re curious about Fable 5’s original capabilities, there’s a detailed hands-on review in this week-long test of Fable 5 as a coding assistant.

Context window, config bug, and version updates

When Gemma Coder first appeared, it was listed with a 131,000-token context window. A community member noticed something was off. It turned out to be a bug in Google’s original config file, not in the fine-tune itself.

The underlying weights always supported the full 256,000-token context. Once the issue was spotted, the creator repatched every quantized build to expose the correct context length. If you grabbed an early copy, it’s worth re-downloading the latest version.

The builder has also released a version 2 that leans harder into agentic behavior and tool use, plus a full-precision “master” checkpoint if you want to create your own custom quantizations or fine-tunes.

Running Gemma Coder locally with Ollama

One of the nicest parts of Gemma Coder is how easy it is to run on your own machine. The simplest path is through Ollama, which handles downloads, quantization, and serving for you.

Before you start, make sure you’re on a recent Ollama version—Gemma 4 uses a newer architecture, and older builds may refuse to load it.

The recommended quantization is:

• Q4_K_M (~6.5 GB) – best balance of quality and memory for most GPUs

Other options usually available on the model page:

• ~4.5 GB quant – for very tight VRAM situations
• Q6 / Q8 – larger, higher-quality quantizations if you have more memory

Once you pull the model through Ollama, it will download directly from Hugging Face and be ready to use with a single command. For a more detailed walkthrough of running Gemma 4 models locally with Ollama, you can also refer to this step-by-step Gemma 4 + Ollama guide.

Using LM Studio and other local tools

If you prefer a graphical interface, LM Studio is another easy option. The flow is straightforward:

• Download the Gemma Coder model file (in your preferred quantization).
• Import it into LM Studio.
• Set the recommended sampling parameters (more on those below).
• Chat with it like any other local assistant.

Other alternatives include:

• Jan – a desktop app for running local models
• llama.cpp – for those who want low-level control and custom setups
• Newer agent frameworks that can plug into Gemma Coder as a backend

For most users, though, Ollama or LM Studio will cover everything you need.

Recommended settings for best results

The model creator suggests these sampling settings as a good default:

• Temperature: 1.0
• Top-p: 0.95
• Top-k: 64
• Repetition penalty: 1.1

The repetition penalty is particularly important. Without it, the model can sometimes get stuck and start repeating strings (for example, a long sequence of zeros). Setting a penalty around 1.1 keeps outputs more stable.

If you’re doing pure coding and want highly predictable answers, you can drop the temperature down to 0.0 for more deterministic behavior. For brainstorming or exploring multiple approaches, a temperature of 0.7–1.0 works well.

What coding tasks Gemma Coder is good at

Gemma Coder is tuned primarily for Python and algorithmic problems. In practice, it shines on tasks like:

• Implementing functions and utilities (e.g., “find the longest palindrome in a string”)
• Refactoring messy loops or deeply nested logic into clearer code
• Explaining step-by-step reasoning before producing the final answer
• Working through edge cases and test-driven solutions

One key behavior you’ll notice: the model tends to reason first, then code. It will often explain its approach, walk through edge cases, and only then output the final function or script. That’s not just a stylistic choice—it reflects how it was trained. If you try to suppress or strip out that reasoning step, performance on harder problems can actually drop.

For focused, self-contained coding tasks, it performs surprisingly well for a 12B model running entirely on your own hardware.

Where it struggles: agents and large codebases

There are limits. The first release of Gemma Coder can get shaky when you ask it to:

• Read and reason over many files at once
• Chain a long series of tool calls or actions
• Act as a fully autonomous agent over a large, complex codebase

It can handle these workflows in short bursts, but it’s not something you’d want to leave running unattended on a big project. That’s exactly what the version 2 fine-tune is trying to improve: more robust agentic behavior and better tool use.

Safety, scope, and when to use something else

Two important caveats if you’re thinking about using Gemma Coder in anything serious:

• De-refused safety: The fine-tune has had many of the original safety guardrails stripped away to keep it task-focused. It may answer questions or follow instructions that the base model would decline. If you’re putting this anywhere near production or end users, you’ll need to add your own safety filters and guardrails.
• Narrow focus: It’s centered on Python and English. It’s not meant to be a general-purpose chatbot or a source of factual world knowledge. Always double-check non-code answers and treat it primarily as a coding assistant.

If you need broad-domain knowledge, multilingual support, or very strong agent capabilities across huge codebases, a larger cloud model will still serve you better. Gemma Coder isn’t trying to beat those—it’s trying to bring a lot of that reasoning quality down to hardware you already own.

Why this model matters for local AI

Gemma Coder doesn’t dethrone the biggest proprietary models, and it doesn’t have to. Its importance is in what it represents:

• A 12B-parameter model that runs on an 8 GB GPU or a typical laptop
• A 256k context window for long code and conversations
• Apache 2 licensing for commercial and open use
• Reasoning distilled from systems that are now heavily restricted or offline

Put together, it points to a clear trend: smaller models, better reasoning, no cloud, no monthly bills. If you want a private, capable Python helper that never sends your code to a server, Gemma Coder is well worth trying.

Spin it up through Ollama or LM Studio, throw your own real-world prompts at it, and see how far a modern 12B local model can go on your actual work.