How Much VRAM Does a 1B, 3B or 8B Model Need?
Weights alone: a 1B model needs 2 GB at FP16, a 3B needs 6 GB, an 8B needs 16 GB, and each halves per precision step down. Add the KV cache and serving overhead, and the practical minimum card is roughly one tier above whatever the weights suggest.
How much memory does each size need?
Two terms: weights, which are fixed, and cache, which grows with context and batch size. This table covers the first and adds a working allowance for the second.
| Model size | Precision | Weights | Practical minimum card |
|---|---|---|---|
| 1B | FP16 | 2.0 GB | 4 GB |
| 1B | Q8 | 1.0 GB | 4 GB |
| 1B | Q4 | 0.5 GB | CPU or any GPU |
| 3B | FP16 | 6.0 GB | 12 GB |
| 3B | Q8 | 3.0 GB | 8 GB |
| 3B | Q4 | 1.5 GB | 4 GB |
| 8B | FP16 | 16.0 GB | 24 GB |
| 8B | Q8 | 8.0 GB | 16 GB |
| 8B | Q4 | 4.0 GB | 8 GB |
If you are coming from the other direction — you have a card and want to know what fits — that calculation is done model by model in which SLM fits in 4GB of VRAM. This page does the reverse: you have picked a size and want to know what to buy or provision.
Where does that arithmetic come from?
weight bytes = parameters × bytes per parameter, and nothing else.
FP16 and BF16 are 2 bytes, INT8 is 1, 4-bit is about 0.5. So 8B × 2 = 16 GB, 3B × 1 = 3 GB, 1B × 0.5 = 0.5 GB. Every weight figure above is that multiplication. None of them is a measurement, and real files run slightly above the 4-bit figure because quantization blocks carry scales — see what is quantization.
The “practical minimum card” column adds a margin, and the margin is where the judgement is. It is not arbitrary: it is roughly weights plus a cache allowance plus a gigabyte or so of runtime overhead — the CUDA context, the framework, and the allocator’s headroom. The one measured anchor we have is from our inference-cost benchmark, where a fine-tuned 4B model served at BF16 through vLLM occupied 7.6 GiB of GPU memory in total, against 8.0 GB of weights by the arithmetic above. Weights dominate, but not by as much as you would like.
Note the unit convention. GB here is 10⁹ bytes; tools that report GiB (2³⁰ bytes) will show the same weights as roughly 7% smaller.
How much does the KV cache add?
Enough to change the answer, and it is computable from the model’s published attention config rather than guessed.
Per token, the cache is:
2 × layers × kv_heads × head_dim × bytes_per_elementThe leading 2 is for keys and values. Plugging in the published configs for three Qwen3 students, at BF16:
| Model | Layers | KV heads | Head dim | Cache per token | At 4K context | At 32K context |
|---|---|---|---|---|---|---|
| Qwen3-1.7B | 28 | 8 | 128 | 0.115 MB | 0.47 GB | 3.76 GB |
| Qwen3-4B-Instruct-2507 | 36 | 8 | 128 | 0.147 MB | 0.60 GB | 4.83 GB |
| Qwen3-8B | 36 | 8 | 128 | 0.147 MB | 0.60 GB | 4.83 GB |
Read the last column. At a 32K context, Qwen3-1.7B’s cache is 3.76 GB against 3.4 GB of BF16 weights — the cache is larger than the model. And because Qwen3-4B and Qwen3-8B share a layer count, KV head count and head dimension, they have identical cache cost per token despite one being twice the size of the other.
Three consequences follow. Quantizing the weights does not shrink the cache. Multiply the cache figure by batch size for a concurrent server. And grouped-query attention is doing most of the work here already — all three models use 8 KV heads against far more query heads, which is why these numbers are as small as they are.
Which GPU does that imply?
Match the card to the row you landed on, then check the context length before you order anything.
An 8 GB card comfortably serves anything up to 3B at Q8 or 8B at Q4, at modest context. A 12–16 GB card covers 3B at FP16 and 8B at Q8 with real context headroom. 24 GB is the first tier where an 8B model at FP16 is genuinely comfortable at long context and small batch. Above that you are buying throughput rather than capacity — an 80 GB H100 was overkill on memory for the 4B model in our benchmark, and we used it because our tasks were prefill-heavy and it was the FLOPs that paid, not the bytes.
If none of these rows is affordable, the honest answer is often a smaller model rather than a bigger card. On a bounded task, a fine-tuned 1B can land where a 3B base does — we measured exactly that on Siemens PLC documentation, where a distilled Llama 1B reached 61.1% against a Llama 3B base at 60.4%. What size model do you need is the decision that precedes this page.
What makes these numbers wrong?
Four things, all of which push the requirement up rather than down.
Batch size. The cache table is for one sequence. A server handling 16 concurrent requests at 4K context needs 16 times the cache figure.
Long contexts you did not plan for. Retrieval pipelines quietly grow prompts. A model sized at 2K that ships with 16K contexts will fail in production and not in testing.
Fragmentation and allocator behaviour. Frameworks reserve pools; the memory a process holds is not the memory a model needs.
Assuming FP16 is the deployment precision. Most SLM deployments are not FP16, and the honest comparison of the three options is in Q4 vs Q8 vs FP16 for your SLM. If the answer to this page is “a card I do not have”, quantization is the first lever and CPU deployment is the second.