Meta · Dense · GQA
Llama 3.1 8B Instruct：各量化与上下文下的显存
8.03B 参数、32 层、8 个 KV 头。换算成 GB 是这个结果。
架构参数
| Parameters | 8.03B | Vendor model card |
|---|---|---|
| Layers | 32 | config.json |
| Hidden size | 4096 | config.json |
| Attention heads | 32 | config.json |
| KV heads | 8 | config.json (GQA) |
| Head dim | 128 | config.json |
| Max context | 131072 tokens | config.json |
Source: Meta Llama 3 architecture (published); repo gated
各量化精度下的显存
Total includes 8K of KV cache and 1.5 GB runtime overhead.
| 量化 | Weights | KV @ 8K | 总需求 | 可运行于 |
|---|---|---|---|---|
| FP16 / BF16 2.0 B/param | 16.1 GB | 1.1 GB | 18.6 GB | Mac Studio M4 Max 128GB |
| INT8 / Q8_0 1.06 B/param | 8.5 GB | 1.1 GB | 11.1 GB | Mac Studio M4 Max 128GB |
| Q4_K_M (4-bit) 0.6 B/param | 4.8 GB | 1.1 GB | 7.4 GB | Mac Studio M4 Max 128GB |
| Q5_K_M (5-bit) 0.72 B/param | 5.8 GB | 1.1 GB | 8.4 GB | Mac Studio M4 Max 128GB |
各上下文长度下的 KV 缓存
Q4_K_M weights plus 1.5 GB overhead. KV cache at FP16.
| 上下文 | KV 缓存 | 总需求 | 可运行于 |
|---|---|---|---|
| 4k | 0.5 GB | 6.9 GB | Mac Studio M4 Max 128GB |
| 8k | 1.1 GB | 7.4 GB | Mac Studio M4 Max 128GB |
| 16k | 2.1 GB | 8.5 GB | Mac Studio M4 Max 128GB |
| 32k | 4.3 GB | 10.6 GB | Mac Studio M4 Max 128GB |
| 64k | 8.6 GB | 14.9 GB | Mac Studio M4 Max 128GB |
| 128k | 17.2 GB | 23.5 GB | Mac Studio M4 Max 128GB |
结论
At Q4_K_M the weights alone are about 4.8 GB. With 8K of context and 1.5 GB of runtime overhead it lands near 7.4 GB. It fits: RTX 4090, RTX 3090, RTX 5090, Mac Studio M4 Max 128GB.
所有吞吐数字均为基于内存带宽与公开实测的估算值，显存数字按模型架构与量化假设推算。购买硬件前请以你自己的运行环境为准。