Knowledge Base · Local AI

How much VRAM to run Gemma 4 locally?

Updated November 2026. Sizes below come from the official Ollama downloads, checked in November 2026.

As an Amazon Associate I earn from qualifying purchases. The graphics card links below are affiliate links.

To run Gemma 4 locally at 4-bit, you need about 8 GB of VRAM for the small E4B, 10 GB for the 12B, and 20 to 22 GB for the 26B or 31B. A 12 GB card like the RTX 3060 12GB runs the Gemma 4 12B fully on the graphics card, and a 24 GB card runs the biggest Gemma 4 models.

Gemma 4 is Google's open model family, free to use under the Apache 2.0 license, and it comes in five sizes: E2B, E4B, 12B, 26B A4B and 31B. The frustrating part is the names. "E4B" and "26B A4B" don't tell you how much memory they take, so the table below does.

Words you'll see in this guide

VRAM: the memory on your graphics card. AI models load into it, so more VRAM means you can run bigger models.
Parameters (7B, 14B, 70B): a rough measure of a model's size. "8B" means 8 billion. Bigger models are usually smarter and need more memory.
Quantization: shrinking a model by storing its numbers less precisely, so it fits in less memory with a small drop in quality. Think of saving a photo as a smaller JPEG. "4-bit" (Q4) is the common setting.
Mixture-of-Experts (MoE): a model design that only uses part of itself for each word it writes, so it runs faster than its total size suggests. The whole model still has to be loaded.
Context: how much text the model can keep in mind at once, measured in tokens. Longer context uses more memory.
Offloading: running part of a model on the graphics card and the rest on the processor and system RAM. It works, but slowly.

See the full glossary

How much VRAM does each Gemma 4 model need?

The download sizes are Ollama's 4-bit (Q4_K_M) versions. Add about 1 to 2 GB on top for the conversation at Ollama's default settings. That's the "VRAM needed" column.

Model	Download (4-bit)	VRAM needed	Runs on
Gemma 4 E2B	4.6 GB	About 6 GB	Any modern mini PC on the processor alone
Gemma 4 E4B (the default)	6.6 GB	About 8 GB	An 8 GB card, or a 16 GB mini PC
Gemma 4 12B (best balance)	8.0 GB	About 10 GB	A 12 GB card like the RTX 3060 12GB
Gemma 4 26B A4B (MoE)	18 GB	About 20 GB	A 24 GB card, or a 12 to 16 GB card with the rest in system RAM
Gemma 4 31B	20 GB	About 22 GB	A 24 GB card like the RTX 3090, or a Mac with 32 GB or more

Download sizes from the Ollama library, checked November 2026. "E" sizes count the core model only; the files also carry the parts that handle images and audio, which is why E4B downloads bigger than its name suggests.

Which Gemma 4 to pick:

No graphics card (mini PC or laptop): E4B with 16 GB of RAM, or E2B with 8 GB. Expect it to be slow but usable for short jobs.
8 GB card: E4B fits fully. The 12B mostly fits, with a small part in system RAM.
12 GB card: the 12B. This is the sweet spot, and why I keep pointing people at a used 3060.
16 GB card: the 12B with long conversations, or the 26B A4B with some of it offloaded.
24 GB card: the 31B for the best answers, or the 26B A4B if you want speed.

Is the Gemma 4 26B or 31B better for a 24 GB card?

Pick the Gemma 4 31B if you care most about answer quality, and the 26B A4B if you care most about speed. The 31B is a dense model, so all 30.7B parameters work on every word. The 26B A4B is a mixture-of-experts model with only 3.8B parameters active at a time, so it writes much faster even though both need about the same memory.

The 26B A4B has one more trick. Because so little of it works at once, it stays usable when it doesn't fully fit, so a 12 or 16 GB card with 32 GB of system RAM can run it with the overflow in RAM. The 31B slows to a crawl when you try that.

Can Gemma 4 12B run on 12 GB of VRAM?

Yes. The Gemma 4 12B 4-bit download is 8.0 GB, which leaves room for the conversation on a 12 GB card like the RTX 3060 12GB. Google says the 12B runs with 16 GB of VRAM or unified memory, which gives more room for long chats, but at 4-bit a 12 GB card holds it fully.

Gemma 4 12B came out on June 3, 2026, two months after the rest of the family. Google says it scores close to the 26B A4B at less than half the memory, and it takes images and audio as input. I haven't run it myself yet, but on paper it's the Gemma 4 model most people should start with.

Does 8-bit Gemma 4 fit?

8-bit keeps a little more quality but needs almost twice the memory. Ollama's 8-bit (Q8_0) downloads:

E4B: 9.0 GB
12B: 13 GB, fits a 16 GB card like the RTX 5060 Ti 16GB
26B A4B: 28 GB, too big for any single consumer card
31B: 34 GB, a 48 GB setup or a Mac or 128 GB mini PC

Ollama also offers "QAT" versions (for example gemma4:12b-it-qat), which are built to lose less quality at 4-bit and are slightly smaller. On a tight card, they're the better 4-bit download.

How to run Gemma 4 with Ollama

Not sure what your PC can handle? The Server Calculator gives you a straight answer. If you're new, start with the Ollama and Open WebUI setup guide. Then:

Install or update Ollama from ollama.com/download. Gemma 4 needs a recent version, so update even if you already have it.
Download and start the size that fits your card. For a 12 GB card:
```
ollama run gemma4:12b
```
You should see: a >>> prompt once the download finishes.
Type a question. Leave it running.
Open a second terminal and check where the model loaded:
```
ollama ps
```
You should see: 100% GPU in the PROCESSOR column. A CPU/GPU split means part of it spilled into system RAM, and a smaller size will run faster.

The other tags are gemma4:e2b, gemma4:e4b, gemma4:26b and gemma4:31b. Plain gemma4 gets you the E4B. Prefer a desktop app? LM Studio runs Gemma 4 too.

Why does Gemma 4 use more VRAM in long chats?

Every token of conversation the model holds in mind takes memory. Ollama sets the context by how much VRAM you have: 4K tokens under 24 GB, 32K tokens from 24 to 48 GB, and 256K at 48 GB or more, according to the Ollama context length docs. Gemma 4 12B, 26B and 31B can hold up to 256K tokens, and E2B and E4B up to 128K, but you rarely have the memory for all of it.

If you need longer chats, for example to paste in a long document, set it when you start Ollama:

OLLAMA_CONTEXT_LENGTH=32000 ollama serve

Then run ollama ps again. If it no longer says 100% GPU, drop the number or step down a size.

Who Gemma 4 is NOT for

You want ChatGPT-level answers on a mini PC. E2B and E4B are good at summaries, short drafts and Home Assistant style commands. They aren't general geniuses, and that's normal for small models.
You only have an 8 GB card and want the 31B. It won't run at a useful speed. Run the 12B, or look at the best GPU for local AI under $500.
You mainly write code. Gemma 4 is a capable all-rounder, but compare it with the Qwen models first. See VRAM for Qwen 3.

Quick answers

How much VRAM do I need to run Gemma 4 locally?

At 4-bit, Gemma 4 E4B needs about 8 GB of VRAM, Gemma 4 12B about 10 GB, the 26B A4B about 20 GB, and the 31B about 22 GB. E2B runs on a processor with 8 GB of system RAM.

Which Gemma 4 model should I run on an RTX 3060 12GB?

Gemma 4 12B. Its 4-bit download is 8.0 GB, so it fits fully on a 12 GB card with room for the conversation. The 26B A4B also works with part of it in system RAM, just slower.

Can Gemma 4 run without a graphics card?

Yes. Gemma 4 E2B and E4B run on a mini PC or laptop processor with 8 to 16 GB of RAM. They're slower than on a graphics card but fine for short tasks.

What does the "A4B" in Gemma 4 26B A4B mean?

It means about 4 billion parameters are active per word. Gemma 4 26B A4B is a mixture-of-experts model with 25.2B parameters in total and 3.8B active, so it runs fast, but it still needs memory for all 25.2B.

Your next step

Check what your own hardware can run with the Server Calculator, then pull the Gemma 4 size it points to. If you're shopping, compare what a 16 GB vs 24 GB card runs or browse the Shop page.

Affiliate disclosure: The graphics card links on this page are affiliate links. If you buy through them, I may earn a small commission at no extra cost to you. As an Amazon Associate I earn from qualifying purchases.