← Interactive LLM VRAM Calculator — try your own context length & quantization
How much VRAM to run DeepSeek-R1 70B (Llama distill)? (FP16/Q8/Q6/Q4 requirements)
Architecture: Llama 3.3 70B architecture (80 layers, GQA with 8 KV heads) fine-tuned on DeepSeek-R1 outputs. Figures are approximations for weights + an 8k-token FP16 KV cache + ~1 GB runtime overhead, based on typical published GGUF sizes. Actual usage varies by runtime and settings.
| Quantization | Weights | Total VRAM (8k context) | Fits on (example) |
|---|---|---|---|
| FP16 full precision (16-bit) | ~141 GB | ~145 GB | 2×A100 80 GB (160 GB) |
| Q8_0 8-bit | ~75 GB | ~79 GB | A100 / H100 (80 GB) |
| Q6_K ~6.6-bit | ~58 GB | ~62 GB | 2×RTX 5090 (64 GB) |
| Q5_K_M ~5.7-bit | ~50 GB | ~54 GB | 2×RTX 5090 (64 GB) |
| Q4_K_M ~4.8-bit | ~43 GB | ~47 GB | 2×RTX 3090 or RTX A6000 (48 GB) |
How much VRAM do I need to run DeepSeek-R1 70B (Llama distill)?
You need roughly ~47 GB of VRAM to run DeepSeek-R1 70B (Llama distill) at Q4_K_M with an 8k-token context — that fits on 2×RTX 3090 or RTX A6000 (48 GB). At Q8_0 plan for about ~79 GB, and full FP16 needs around ~145 GB. DeepSeek-R1-Distill-Llama-70B is a fine-tune of Llama 3.3 70B, so its memory footprint is identical to Llama 70B: Q4_K_M needs about 48 GB of VRAM. Reasoning models also generate long chains of thought, so budget for larger context if you can.
What affects these numbers?
Three things: the quantization (bytes per parameter), the KV cache (grows linearly with context length — the table assumes 8k tokens), and runtime overhead. Longer contexts need more memory; quantizing the KV cache (e.g. q8_0 cache in llama.cpp) roughly halves that part. Use the interactive calculator to plug in your exact context length and quantization.