← Interactive LLM VRAM Calculator — try your own context length & quantization
How much VRAM to run Llama 3.1 8B? (FP16/Q8/Q6/Q4 requirements)
Architecture: 32 layers, GQA with 8 KV heads. Figures are approximations for weights + an 8k-token FP16 KV cache + ~1 GB runtime overhead, based on typical published GGUF sizes. Actual usage varies by runtime and settings.
| Quantization | Weights | Total VRAM (8k context) | Fits on (example) |
|---|---|---|---|
| FP16 full precision (16-bit) | ~16 GB | ~18 GB | RTX 3090 / RTX 4090 (24 GB) |
| Q8_0 8-bit | ~8.5 GB | ~11 GB | RTX 3060 12 GB |
| Q6_K ~6.6-bit | ~6.6 GB | ~8.7 GB | RTX 3060 12 GB |
| Q5_K_M ~5.7-bit | ~5.7 GB | ~7.8 GB | RTX 3060 Ti / RX 7600 (8 GB) |
| Q4_K_M ~4.8-bit | ~4.9 GB | ~7.0 GB | RTX 3060 Ti / RX 7600 (8 GB) |
How much VRAM do I need to run Llama 3.1 8B?
You need roughly ~7.0 GB of VRAM to run Llama 3.1 8B at Q4_K_M with an 8k-token context — that fits on RTX 3060 Ti / RX 7600 (8 GB). At Q8_0 plan for about ~11 GB, and full FP16 needs around ~18 GB. Llama 3.1 8B is one of the easiest capable models to run locally — Q4_K_M fits on common 8 GB GPUs, and even full FP16 fits on a 24 GB card.
What affects these numbers?
Three things: the quantization (bytes per parameter), the KV cache (grows linearly with context length — the table assumes 8k tokens), and runtime overhead. Longer contexts need more memory; quantizing the KV cache (e.g. q8_0 cache in llama.cpp) roughly halves that part. Use the interactive calculator to plug in your exact context length and quantization.