LLM VRAM Calculator
Enter a model size, quantization and context length to see the memory it needs.
Runs in your browser. Nothing is uploaded.
For mixture-of-experts models, enter the total parameters: every expert has to sit in memory.
Enter the number of parameters in billions, for example 8 or 70.
Bits per weight include the scales stored with each block: . GPTQ and AWQ 4-bit files land close to Q4.
Enter a context length in tokens, for example 8192 or 32k.
Architecture and KV cache
Defaults come from a typical model of this size
(). Take exact values from the model's config.json if you have it.
Grouped-query attention: KV heads share query heads, which keeps the cache small. No grouped-query attention: every head keeps its own keys and values, so the cache is large. For a model without GQA, set KV heads equal to attention heads.
Splitting a model across cards adds a little overhead on each one, so leave some headroom.
macOS keeps part of unified memory for the system: by default about two thirds on Macs up to 36 GB and three quarters above. The limit can be raised, but the system still needs room.
Estimated memory needed
≈ 6.7 GB
on :
The model fits with room to spare. At this quantization the longest context that fits is about tokens per request, if the model supports that much.
It fits, but with less than 10% headroom. Close other apps that use the GPU or shorten the context. ( GB) leaves comfortable room.
You need about more. ( GB) fits at this context. No quantization in this list fits at this context; try a smaller model or a shorter context.
With partial offload, about of the layers stay on the GPU and the rest run from system RAM, which is several times slower.
How this is calculated
Estimates in GB of 1024³ bytes, the unit GPU memory is sold in. Real use varies by runtime (llama.cpp, Ollama, LM Studio, vLLM), batch size and drivers.
Memory for popular model sizes
Total GB at tokens of context, KV cache in . Colours show the fit on : fits tight does not fit
| Size | FP16 | Q8 | Q6 | Q5 | Q4 |
|---|---|---|---|---|---|
| 1B | |||||
| 3B | |||||
| 7B–8B | |||||
| 13B–14B | |||||
| 24B | |||||
| 32B | |||||
| 70B |
Checking how long your prompt is? Paste it into the Token Counter to see how much context it takes.
Embed this calculator
Paste this code into your site. The calculator stays free and up to date.
How to use the calculator
- Enter the model size in billions of parameters, or pick a common size. The number is usually in the model's name: an 8B model has about 8 billion parameters.
- Choose the quantization of the file you plan to download. Q4 is the usual starting point for running models locally; FP16 is the unquantized original.
- Set the context length you actually need. A chat with a few pages of history fits in 8K tokens; long documents and coding agents often want 32K or more.
- Pick your graphics card or Mac, or enter your memory by hand. The verdict tells you whether the model fits, and the table shows the same estimate for other popular sizes.
How the estimate works
A running model keeps three things in memory.
- Weights. Parameters × bits per weight ÷ 8. Quantized formats store small scaling factors next to the weights, so Q4_K_M takes about 4.85 bits per weight rather than 4, and Q8_0 takes 8.5.
- KV cache. For every token in the context, each layer keeps a key and a value vector: 2 × layers × context length × KV heads × head dimension × bytes per value. Models with grouped-query attention (GQA) share each key and value across several query heads, so their cache is several times smaller. Open the "Architecture and KV cache" section to enter exact values from a model's
config.json. - Overhead. The runtime, compute buffers and memory fragmentation. We add 1 GB plus 5% of the weights, which errs on the safe side for llama.cpp, Ollama and LM Studio.
A model fits comfortably when the total stays under 90% of usable memory. Between 90% and 100% it usually loads, but there is little room for a longer context or for other programs that use the GPU.
Memory for common model sizes
Total GB at an 8,192-token context with an FP16 KV cache, for typical architectures:
| Model size | FP16 | Q8 | Q6 | Q5 | Q4 |
|---|---|---|---|---|---|
| 1B | 3.2 | 2.3 | 2.1 | 1.9 | 1.8 |
| 3B | 7.7 | 5.0 | 4.3 | 4.0 | 3.7 |
| 7B–8B | 17.6 | 10.3 | 8.4 | 7.6 | 6.7 |
| 13B–14B | 29.9 | 17.0 | 13.7 | 12.2 | 10.8 |
| 24B | 49.5 | 27.5 | 21.8 | 19.3 | 16.8 |
| 32B | 65.6 | 36.2 | 28.7 | 25.3 | 22.0 |
| 70B | 140.4 | 76.2 | 59.6 | 52.2 | 45.0 |
Apple Silicon Macs
Macs share one pool of memory between the CPU and the GPU. By default macOS lets the GPU use roughly two thirds of it on machines with up to 36 GB and about three quarters on larger ones, so a 64 GB Mac has around 48 GB for a model. You can change the share in the calculator if you have raised the limit yourself.
Looking for software to run models with? Browse AI models and developer tools in our directory.
How much VRAM do I need for a 70B model?
About 45 GB at Q4 with an 8K context, so two 24 GB cards or a Mac with 64 GB or more. At FP16 the same model needs around 140 GB, which means several data-center GPUs.
Can I run an 8B model on an 8 GB card?
At Q4 an 8B model needs about 6.7 GB at an 8K context, so it fits on an 8 GB card with little to spare. Shorten the context or use a smaller quantization if the runtime reports out-of-memory errors.
Why does the context length change the result so much?
The KV cache grows in a straight line with the context. For a 70B model it is about 2.5 GB at 8K tokens and 40 GB at 128K. Quantizing the cache to 8 bits roughly halves it.
Is Q4 good enough?
For chat and most everyday tasks, Q4_K_M is the common choice because the quality loss is small for the memory it saves. Q5 and Q6 are closer to the original; formats below 4 bits save more memory but degrade output more noticeably.
What if the model does not fit?
Runtimes such as llama.cpp can keep some layers in system RAM and run them on the CPU. The model then works, but generation slows down a lot, because system memory is much slower than VRAM.
Does the calculator estimate speed?
No. Speed depends mostly on memory bandwidth and on how much of the model sits on the GPU, and it varies too much between setups to predict well from these inputs.
AI models tools from our directory
Browse: AI models, Developer tools
Terminal-based browser testing with automated pass-fail proof
All-in-one solution for mobile data management and recovery.
Advanced AI for training and deploying machine learning models.
Automated solution for bypassing online user verification challenges.
Stay informed about the latest AI products and trends.
