Meta · Dense · GQA
Llama 3.3 70B Instruct: VRAM by quantization and context
70.6B parameters, 80 layers, 8 KV heads. Here is what that means in gigabytes.
Architecture
| Parameters | 70.6B | Vendor model card |
|---|---|---|
| Layers | 80 | config.json |
| Hidden size | 8192 | config.json |
| Attention heads | 64 | config.json |
| KV heads | 8 | config.json (GQA) |
| Head dim | 128 | config.json |
| Max context | 131072 tokens | config.json |
Source: Meta Llama 3 architecture (published); repo gated
VRAM by quantization
Total includes 8K of KV cache and 1.5 GB runtime overhead.
| Quantization | Weights | KV @ 8K | Total needed | Runs on |
|---|---|---|---|---|
| FP16 / BF16 2.0 B/param | 141.2 GB | 2.7 GB | 145.4 GB | No |
| INT8 / Q8_0 1.06 B/param | 74.8 GB | 2.7 GB | 79.0 GB | Mac Studio M4 Max 128GB |
| Q4_K_M (4-bit) 0.6 B/param | 42.4 GB | 2.7 GB | 46.5 GB | Mac Studio M4 Max 128GB |
| Q5_K_M (5-bit) 0.72 B/param | 50.8 GB | 2.7 GB | 55.0 GB | Mac Studio M4 Max 128GB |
KV cache by context length
Q4_K_M weights plus 1.5 GB overhead. KV cache at FP16.
| Context | KV cache | Total needed | Runs on |
|---|---|---|---|
| 4k | 1.3 GB | 45.2 GB | Mac Studio M4 Max 128GB |
| 8k | 2.7 GB | 46.5 GB | Mac Studio M4 Max 128GB |
| 16k | 5.4 GB | 49.2 GB | Mac Studio M4 Max 128GB |
| 32k | 10.7 GB | 54.6 GB | Mac Studio M4 Max 128GB |
| 64k | 21.5 GB | 65.3 GB | Mac Studio M4 Max 128GB |
| 128k | 42.9 GB | 86.8 GB | Mac Studio M4 Max 128GB |
Verdict
At Q4_K_M the weights alone are about 42.4 GB. With 8K of context and 1.5 GB of runtime overhead it lands near 46.5 GB. It fits: Mac Studio M4 Max 128GB.
Back to calculator → · Open the TCO calculator → · Why 70B won't fit in 24 GB →
All throughput figures are estimates derived from memory bandwidth and published measurements. VRAM figures are computed from model architecture and quantization assumptions. Verify against your own runtime before buying hardware.