AI
LocalAI Calc
VRAM + cost per token
Runs in your browser · no signup · no data sent

How much VRAM does your local LLM actually need?

Most calculators stop at memory. This one also tells you what the tokens cost — hardware, electricity and depreciation — so you can see whether local inference beats a cloud API.

6 models 4 GPUs Formulas shown Updated 2026-11-10
VRAM & throughput calculator
Updates as you type. Nothing leaves your browser.
Q4_K_M measures ~0.60 bytes per parameter, not the theoretical 0.50.
8k
KV cache grows linearly with context. This is what breaks long-context runs.
Usable memory is capped below the sticker number: 90% on NVIDIA, 85% on a 128GB Mac.
4h
0.12
Real usage, not 24/7. Most people run a few hours a day.
Model weights—
KV cache—
Runtime overhead1.5 GB
Total required—
Usable budget—
Headroom—
Weights KV cache Overhead
Estimated speed
—
Cost / 1M tokens
—
Max context that fits
—
Power draw
—
Which hardware runs this
Same model, same quantization, same context — compared across GPUs and Apple silicon.
RTX 4090NVIDIA · 24 GB GDDR6X
1008 GB/s
Fits
Check price →
RTX 3090NVIDIA · 24 GB GDDR6X
936 GB/s
Fits
Check price →
RTX 5090NVIDIA · 32 GB GDDR7
1792 GB/s
Fits
Check price →
Mac Studio M4 Max 128GBApple · 128 GB Unified memory
546 GB/s
Fits
Check price →
Model × hardware matrix
Q4_K_M weights plus 8K context. Green fits, amber is tight, red needs offload.
ModelWeights (Q4)KV @ 8KTotal needed RTX 4090 24GBRTX 3090 24GBRTX 5090 32GBMac Studio 128GB
Llama 3.3 70B
Meta		42.4 GB2.7 GB46.5 GBNoNoNoFits
Qwen3 32B
Alibaba		19.7 GB2.1 GB23.3 GBTightTightFitsFits
Mistral Small 24B
Mistral AI		14.2 GB1.3 GB17.0 GBFitsFitsFitsFits
Qwen3 14B
Alibaba		8.9 GB1.3 GB11.7 GBFitsFitsFitsFits
Qwen3 8B
Alibaba		4.9 GB1.2 GB7.6 GBFitsFitsFitsFits
Llama 3.1 8B
Meta		4.8 GB1.1 GB7.4 GBFitsFitsFitsFits
How the math works

Weights come from parameter count times bytes per parameter. KV cache comes from layers, KV heads and head dimension. Runtime overhead is treated as a fixed 1.5GB. Usable memory is a percentage of the card's capacity, because the driver and framebuffers take a cut.

Cost per million tokens adds electricity and depreciation, then divides by tokens produced per year at the estimated speed. Full formulas are on the TCO page.

Open the TCO calculator →  ·  Read the hardware guide →

Questions
Why does Q4 need 0.60 bytes per parameter, not 0.50?
Q4_K_M stores most weights at 4 bits but keeps scales, zero-points and some layers at higher precision. Measured files land around 0.60 bytes per parameter — Qwen3-32B is 19.8GB for 32.8B parameters. Using 0.50 underestimates memory by about 20% and is the main reason people buy a card that turns out too small.
Why calculate KV cache separately?
KV cache scales with context length, not with model size alone. Qwen3-32B needs 256 KiB per token at FP16 — 8GB at 32K context. A 24GB card holding 19.5GB of weights has roughly 3GB left, which caps you near 8K. That is why a model can load fine and then fail the moment you paste in a long document.
Are the tokens/sec numbers measured?
No. They are estimates derived from memory bandwidth, using a calibration factor of 0.29 taken from published RTX 4090 measurements on Llama 3 8B Q4 (about 62 tok/s against a 214 tok/s bandwidth ceiling). Real throughput varies with batch size, backend and quantization. Treat them as a planning number, not a benchmark.
Does a 70B model really not run on 24GB?
At Q4_K_M, 70B weights alone are about 42GB. On a 24GB card the remainder has to sit in system RAM and stream over PCIe, which drops generation to roughly 1–3 tok/s. It technically runs; it is not usable for interactive work. See the dedicated guide for the numbers.

All throughput figures are estimates derived from memory bandwidth and published measurements. VRAM figures are computed from model architecture and quantization assumptions. Verify against your own runtime before buying hardware.