Knowledge Base · Local AI

How much VRAM do you need to run Llama 3 70B locally?

By Tiny · how I pick

Updated October 2026. This is the model that forces the hardest hardware choice in home AI, so here's the number, the math behind it, and every realistic way to run it.

As an Amazon Associate I earn from qualifying purchases. The hardware links below are affiliate links.

To run Llama 3 70B at the usual 4-bit setting, you need roughly 40 to 48 GB of memory for the model, plus a few GB more for the conversation. No single consumer graphics card has that much. That's why this model pushes people toward two graphics cards or a Mac with a lot of memory.

Words you'll see in this guide
  • VRAM: the memory on your graphics card. AI models load into it, so more VRAM means you can run bigger models.
  • Quantization (Q4): shrinking a model by storing its numbers less precisely, so it fits in less memory with a small drop in quality. Think of saving a photo as a smaller JPEG. "4-bit" (Q4) is the common setting.
  • KV cache: the model's working memory for your conversation. It grows the longer the conversation gets.
  • Offloading: running part of a model on the graphics card and the rest on the processor and system RAM. It works, but slowly.
  • Unified memory: Apple's design where the processor and graphics share one fast pool of memory, so a Mac's RAM works like VRAM.

See the full glossary

Where the number comes from

At 4-bit, a model needs about 0.55 to 0.6 GB of memory per billion parameters. For 70 billion parameters, that's about 42 GB just for the model itself. Community measurements of Llama 3 70B at Q4_K_M land right there, around 40 to 43 GB.

On top of that comes the KV cache, which grows with the length of your conversation. A short chat adds a couple of GB. A long 32K-token conversation can add more than 10 GB. To run it comfortably with real conversations, aim for about 48 GB.

In one line: about 42 GB for the model, about 48 GB to use it comfortably. Since no single consumer card has that, running 70B means two cards, a Mac with a lot of memory, or a slower setup that leans on the processor.

The realistic ways to run it

The usual home lab answer

Two used RTX 3090s (24 GB each, 48 GB total)

Two RTX 3090s pool to 48 GB, which fits Llama 3 70B at Q4 with room for conversation, and runs it fast. The catches:

  • Your motherboard needs two suitable PCIe slots.
  • You need a big power supply.
  • Your case and cooling have to handle two large cards.
  • Used 3090 prices climbed all year, to roughly $1,200 each on eBay as of October 2026, so this pair costs far more than it did a year ago.
The quiet, simple answer

An Apple Silicon Mac with 64 GB or more

A Mac's processor and graphics share one pool of memory, so a 64 GB Mac Studio, MacBook, or Mac mini can load a 70B model at Q4 with no second card and no power supply math. It's nearly silent and uses little power. It won't match two 3090s on speed, but it's the least hassle way to run 70B at home, and it doubles as your everyday computer.

If you already have one card

One 24 GB card plus system RAM

With one 24 GB card like a 3090 or 4090, you can load as much of the model as fits on the card and run the rest on the processor and system RAM. It works, and it's cheap if you already own the parts. But the parts running on the processor slow everything down a lot. Expect a few tokens per second. That's fine for experimenting and painful for daily use.

Do you actually need 70B?

Ask yourself this before you spend anything. For most home uses (chat, summaries, drafts, help with code), a good 8B to 14B model runs fast on one mainstream graphics card, or even a mini PC, and does the job. Moving up to 70B gets you better reasoning and fewer mistakes, at several times the cost and complexity.

Try a smaller model first. Move to 70B once you hit a limit you can actually name.

Before you buy two graphics cards: plenty of people spend four figures chasing 70B and then find a 14B model did everything they needed. Rent an hour of a big model in the cloud first, and see whether the difference is worth it to you.

Work out your own numbers

The Server Calculator shows exactly what your current or planned hardware can run, from 8B up to 70B and beyond, with a speed estimate for each.

Related guides

Affiliate disclosure: The hardware links on this page are affiliate links. If you buy through them, I may earn a small commission at no extra cost to you. As an Amazon Associate I earn from qualifying purchases.